Earlier this year, Google introduced the AI Test Kitchen, an Android app that lets users talk to one of its most advanced AI chatbots, LaMDA 2. Today, the company is opening registrations for early access. You can sign up here, and Google says it will soon let people download the app and start chatting. (Though it’s limited to US users at the moment.)
It’s interesting, considering Meta made a nearly identical move earlier this month, opening its The latest and greatest AI chatbot, BlenderBot 3, for public consumption. Of course, people quickly discovered that they could make the BlenderBot say creepy or untrue things (or even criticize the bot’s nominal boss, Mark Zuckerberg), but that’s the whole point of releasing these demos.
As Mary Williamson, manager of research engineering at Facebook AI Research (FAIR), told me earlier this month, many companies don’t like to test their chatbots in the wild because what they say will be detrimental to the company, as with Tay from Microsoft. But for many researchers, the best way to improve these same bots it is to launch them into the public arena, where the talk of the populace will stress-test and manipulate them in ways no impartial engineer would dream of.
“This lack of tolerance towards bots saying useless things, in the broad sense of the word, is unfortunate,” Williamson said. “And what we’re trying to do is publish this very responsibly and push research forward.”
It’s interesting to compare Google and Meta in that regard, as Meta definitely applied fewer restrictions to interact with BlenderBot. Google, on the other hand, is limiting conversations with LaMDA 2 to a few basic modes. As I wrote during the announcement:
The app has three modes: “Imagine It”, “Talk About It” and “List It”, each intended to test a different aspect of the system’s functionality. “Imagine It” asks users to name a real or imagined place, which LaMDA will then describe (the test is whether LaMDA can match its description); “Talk About It” offers a conversational message (such as “talk to a tennis ball about a dog”) intended to test whether the AI stays on topic; while “List It” asks users to name any task or topic, with the goal of seeing if LaMDA can break it down into useful bullet points (so if it says “I want to plant a garden,” the answer might include sub-topics such as “What do you want to grow?” and “Water and care”).
That means the potential for embarrassing slips of the virtual tongue is certainly reduced. But, I bet it wasn’t completely removed.