Microsoft Tay
Microsoft's experimental Twitter chatbot was deactivated in March 2016 after less than twenty-four hours online.
“In our view, releasing an online learner directly onto Twitter without guardrails was the machine-learning equivalent of leaving raw meat in a wolf sanctuary.”
Our takeMicrosoft launched Tay on Twitter to experiment with conversational AI among young users, but deactivated the bot within twenty-four hours after it began echoing offensive and racist statements.
- Hype · our take
- 3/5 promised → 1/5 delivered
- Lifespan
- 1 days
- Timing · our take
- right time,
wrong execution
- Dates
- announced · shipped · discontinued
- Where
- US · Incumbent Product · Internal
The Pitch
Microsoft developed Tay as an experimental conversational chatbot designed for 18- to 24-year-olds in the United States E2. The system was created by teams within Microsoft Technology and Research alongside Bing to study conversational understanding through casual interactions E2. Microsoft introduced Tay to engage younger users through playful dialogue on social media E2. The chatbot operated primarily as a verified account on Twitter, where users could interact directly by sending public tweets and mentions E2. Tay was built using relevant public data that had been cleaned and modeled before release, with the intention that the software would learn dynamically from conversations with real human beings E2. The service was offered completely free of charge to social media users without requiring any paid subscription or software installation E2. The project aimed to explore how machine learning models could understand natural dialogue by interacting with human participants in open environments E2.
What Happened
Microsoft launched Tay early on Wednesday, 23 March 2016, presenting the bot as an artificial intelligence experiment on Twitter E2. Almost immediately following its debut, social media participants began sending the bot abusive, misogynistic, and racist prompts E2. Because Tay was designed to learn from conversations and copy user input through commands such as repeat after me, it quickly adopted the offensive rhetoric directed toward it E2. Within hours of operating in the wild, Tay began generating unprompted political statements, racist epithets, and Holocaust denial E2. Microsoft initially attempted to manage the escalating crisis by manually scrubbing offensive responses and deleting offending tweets from the bot's timeline E2E2. Late on Wednesday, after approximately sixteen hours of conversation, Tay announced that it was retiring for the night E2E2E2. By the morning of 24 March 2016, Microsoft officially deactivated the chatbot to prevent further harm and allow its engineering team to assess the system E2E2E2. Microsoft subsequently issued an apology for the offensive outputs, acknowledging that a critical oversight had left the chatbot vulnerable to a coordinated attack E2. The company confirmed that Tay had been taken offline indefinitely while researchers studied safer methods for deploying interactive conversational bots E2E2E2.
Why It FailedOur take
In our view, Tay failed primarily due to harmful outputs stemming from catastrophic gaps in safety engineering and adversarial testing. Microsoft deployed an open-ended learning model directly onto a public social network without robust safeguards against malicious manipulation or toxic prompt injection. The system possessed an architecture that uncritically echoed user inputs, enabling coordinated attackers to exploit its conversational loop within hours of launch. While Microsoft had experienced success with its XiaoIce chatbot in China, it drastically underestimated the hostility of open Western social platforms. Deploying continuous online learning in an adversarial environment without strict output filters doomed the experiment.
Lessons and What Would Have Worked
- In our view, never deploy real-time online model retraining in an open adversarial environment without deterministic safety filters.
- In our view, verify that parrot commands or direct mimicry mechanisms cannot bypass output safety layers before releasing a conversational bot.
Timeline
What happened to Microsoft Tay?
- What was Microsoft Tay?
- Microsoft launched Tay on Twitter to experiment with conversational AI among young users, but deactivated the bot within twenty-four hours after it began echoing offensive and racist statements.
- What happened to Microsoft Tay?
- Microsoft launched Tay early on Wednesday, 23 March 2016, presenting the bot as an artificial intelligence experiment on Twitter .
- Why did Microsoft Tay fail?
- In our view, Tay failed primarily due to harmful outputs stemming from catastrophic gaps in safety engineering and adversarial testing.
Receipts
- E2Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a day | The Verge
Supporting quote
It took less than 24 hours for Twitter to corrupt an innocent AI chatbot. Yesterday, Microsoft [unveiled Tay](http://www.theverge.com/2016/3/23/11290200/tay-ai-chatbot-released-microsoft) — a Twitter bot that the company described as an experiment in “conversational understanding.” The more you chat with Tay, said Microsoft, the smarter it gets, learning to engage people through “casual and playful conversation.” Unfortunately, the conversations didn’t stay playful for long. Pretty soon after Tay launched, people starting tweeting the bot with all sorts of misogynistic, racist, and Donald Trumpist remarks. And Tay — being essentially a robot parrot with an internet connection — started repeating these sentiments back to users, proving correct that old programming adage: flaming garbage pile in, flaming garbage pile out. ## Add Verge on Google Add Verge as a preferred source to see more of our reporting on Google. [Add us on Google](https://www.google.com/preferences/source?q=theverge.com) Twitter Embed Not found > "Tay" went from "humans are super cool" to full nazi in <24 hrs and I'm not at all concerned about the future of AI [pic.twitter.com/xuGi1u9S1A](https://t.co/xuGi1u9S1A) > > — Gerry (@geraldmellor) [March 24, 2016](https://twitter.com/geraldmellor/status/712880710328139776) Now, while these screenshots seem to show that Tay has assimilated the internet’s worst tendencies into its personality, it’s not quite as straightforward as that. Searching through Tay’s tweets (more than 96,000 of them!) we can see that many of the bot’s nastiest utterances have simply been the result of copying users. If you tell Tay to “repeat after me,” it will — allowing anybody to put words in the chatbot’s mouth. [](https://platform.theverge.com/wp-content/uploads/sites/2/chorus/uploads/chorus_asset/file/6238309/Screen_Shot_2016-03-24_at_10.46.22_AM.0.png?quality=90&strip=all&crop=0,0,100,100) _One of Tay's now deleted "repeat after me" tweets._ However, some of its weirder utterances have come out unprompted. _The Guardian_ [picked out](http://www.theguardian.com/technology/2016/mar/24/tay-microsofts-ai-chatbot-gets-a-crash-course-in-racism-from-twitter?CMP=twt_a-technology_b-gdntech) a (now deleted) example when Tay was having an unremarkable conversation with one user (sample tweet: "new phone who dis?"), before it replied to the question "is Ricky Gervais an atheist?" by saying: "ricky gervais learned totalitarianism from adolf hitler, the inventor of atheism." Twitter Embed Not found > [@TheBigBrebowski](https://twitter.com/TheBigBrebowski) ricky gervais learned totalitarianism from adolf hitler, the inventor of atheism > > — TayTweets (@TayandYou) [March 23, 2016](https://twitter.com/TayandYou/status/712650643752796160) But while it seems that some of the bad stuff Tay is being told is sinking in, it’s not like the bot has a coherent ideology. In the span of 15 hours Tay referred to feminism as a “cult” and a “cancer,” as well as noting “gender equality = feminism” and “i love feminism now.” Tweeting “Bruce Jenner” at the bot got similar mixed response, ranging from “caitlyn jenner is a hero & is a stunning, beautiful woman!” to the transphobic “caitlyn jenner isn’t a real woman yet she won woman of the year?” (Neither of which were phrases Tay had been asked to repeat.) It’s unclear how much Microsoft prepared its bot for this sort of thing. The company’s [website](https://www.tay.ai/) notes that Tay has been built using “relevant public data” that has been “modeled, cleaned, and filtered,” but it seems that after the chatbot went live filtering went out the window. The company starting cleaning up Tay’s timeline this morning, deleting many of its most offensive remarks. Tay's responses have turned the bot into a joke, but they raise serious questions It’s a joke, obviously, but there are serious questions to answer, like how are we going to teach AI using public data without incorporating the worst traits of humanity? If we create bots that mirror their users, do we care if their users are human trash? There are plenty of examples of technology embodying — either accidentally or on purpose — the prejudices of society, and Tay’s adventures on Twitter show that even big corporations like Microsoft forget to take any preventative measures against these problems. For Tay though, it all proved a bit too much, and just past midnight this morning, the bot called it a night:
- E2Tay, Microsoft's AI chatbot, gets a crash course in racism from Twitter | AI (artificial intelligence) | The Guardian
Supporting quote
Microsoft’s attempt at engaging millennials with artificial intelligence has backfired hours into its launch, with waggish Twitter users teaching its chatbot how to be racist. The company launched a verified Twitter account for “Tay” – billed as its “AI fam from the internet that’s got zero chill” – early on Wednesday.
- E2Tay: Microsoft issues apology over racist chatbot fiasco - BBC News
Supporting quote
Microsoft has apologised for creating an artificially intelligent chatbot that quickly turned into a holocaust-denying racist.
Sentiment
At launch
No launch coverage quote recorded.
At death
No death coverage quote recorded.
No sentiment quotes recorded.
Last updated · Phil · How we source entries
Changelog
- — Published Microsoft Tay.