LaMDA
Developer(s) | Google Brain |
---|---|
Successor | PaLM |
Available in | English |
Type | Large language model |
License | Proprietary |
LaMDA (Language Model for Dialogue Applications) is a family of conversational
History
Background
On January 28, 2020,
First generation
Google announced the LaMDA conversational
Second generation
On May 11, 2022, Google unveiled LaMDA 2, the successor to LaMDA, during the 2022 Google I/O keynote. The new incarnation of the model draws examples of text from numerous sources, using it to formulate unique "natural conversations" on topics that it may not have been trained to respond to.[12]
Sentience claims
On June 11, 2022,
Lemoine's claims were widely pushed back by the scientific community.
Products
AI Test Kitchen
With the unveiling of LaMDA 2 in May 2022, Google also launched the AI Test Kitchen, a
Bard
On February 6, 2023, Google announced Bard, a conversational AI chatbot powered by LaMDA, in response to the unexpected popularity of OpenAI's ChatGPT chatbot.[38][39][40] Google positions the chatbot as a "collaborative AI service" rather than a search engine.[41][42] Bard became available for early access on March 21.[43][44][45]
Other products
In addition to Bard, Pichai also unveiled the company's Generative Language API, an
Method
LaMDA uses a decoder-only transformer language model.[46] It is pre-trained on a text corpus that includes both documents and dialogs consisting of 1.56 trillion words,[47] and is then trained with fine-tuning data generated by manually annotated responses for sensibleness, interestingness, and safety.[48] Tests by Google indicated that LaMDA surpassed human responses in the area of interestingness.[49] The LaMDA transformer model and an external information retrieval system interact to improve the accuracy of facts provided to the user.[50]
Three different models were tested, with the largest having 137 billion non-embedding parameters:[51]
Parameters | Layers | Units (dmodel) | Heads |
---|---|---|---|
2B | 10 | 2560 | 40 |
8B | 16 | 4096 | 64 |
137B | 64 | 8192 | 128 |
See also
- BERT (language model)
- Chinese room
- Ethics of artificial intelligence
- Gemini (language model)
- Natural language processing
- Philosophy of artificial intelligence
- Prompt engineering
References
General
- Thoppilan, Romal; De Freitas, Daniel; Hall, Jamie; Shazeer, Noam; Kulshreshtha, Apoorv; Cheng, Heng-Tze; Jin, Alicia; Bos, Taylor; Baker, Leslie; Du, Yu; Li, YaGuang; Lee, Hongrae; Zheng, Huaixiu Steven; Ghafouri, Amin; Menegali, Marcelo; Huang, Yanping; Krikun, Maxim; Lepikhin, Dmitry; Qin, James; Chen, Dehao; Xu, Yuanzhong; Chen, Zhifeng; Roberts, Adam; Bosma, Maarten; Zhao, Vincent; Zhou, Yanqi; Chang, Chung-Ching; Krivokon, Igor; Rusch, Will; Pickett, Marc; Srinivasan, Pranesh; Man, Laichee; Meier-Hellstern, Kathleen; Ringel Morris, Meredith; Doshi, Tulsee; Delos Santos, Renelito; Duke, Toju; Soraker, Johnny; Zevenbergen, Ben; Prabhakaran, Vinodkumar; Diaz, Mark; Hutchinson, Ben; Olson, Kristen; Molina, Alejandra; Hoffman-John, Erin; Lee, Josh; Aroyo, Lora; Rajakumar, Ravi; Butryna, Alena; Lamm, Matthew; Kuzmina, Viktoriya; Fenton, Joe; Cohen; Aaron; Bernstein, Rachel; Kurzweil, Ray; Aguera-Arcas, Blaise; Cui, Claire; Croak, Marian; Chi, Ed; Le, Quoc (January 20, 2022). "LaMDA: Language Models for Dialog Applications". arXiv:2201.08239 [cs.CL].
Citations
- ^ Johnson, Khari (January 28, 2020). "Meena is Google's attempt at making true conversational AI". VentureBeat. Archived from the original on October 1, 2022. Retrieved March 11, 2023.
- ^ Heaven, William Douglas (January 30, 2020). "Google says its new chatbot Meena is the best in the world". MIT Technology Review. Archived from the original on August 2, 2020. Retrieved March 11, 2023.
- ^ Popper, Ben (May 27, 2016). "Ray Kurzweil is building a chatbot for Google". The Verge. Archived from the original on May 27, 2016. Retrieved March 11, 2023.
- ^ from the original on March 7, 2023. Retrieved March 9, 2023.
- ^ a b Condon, Stephanie (May 18, 2021). "Google I/O 2021: Google unveils new conversational language model, LaMDA". ZDNET. Archived from the original on May 18, 2021. Retrieved June 12, 2022.
- ^ Roth, Emma (March 5, 2023). "Meet the companies trying to keep up with ChatGPT". The Verge. Archived from the original on March 5, 2023. Retrieved March 9, 2023.
- from the original on March 21, 2023. Retrieved October 16, 2023.
- from the original on June 9, 2022. Retrieved June 12, 2022.
- ^ Cheng, Heng-Tze; Thoppilan, Romal (January 21, 2022). "LaMDA: Towards Safe, Grounded, and High-Quality Dialog Models for Everything". Google AI. Archived from the original on March 25, 2022. Retrieved June 12, 2022.
- ^ Thoppilan et al. 2022, p. 6.
- ^ Thoppilan et al. 2022, pp. 5–6.
- ^ Wiggers, Kyle (May 11, 2022). "Google details its latest language model and AI Test Kitchen, a showcase for AI research". TechCrunch. Archived from the original on May 11, 2022. Retrieved June 12, 2022.
- ^ a b c Khan, Jeremy (June 13, 2022). "A.I. experts say the Google researcher's claim that his chatbot became 'sentient' is ridiculous—but also highlights big problems in the field". Fortune. Archived from the original on June 13, 2022. Retrieved June 18, 2022.
- from the original on June 11, 2022. Retrieved June 12, 2022.
- from the original on June 12, 2022. Retrieved June 18, 2022.
- ^ Vlamis, Kelsey (June 12, 2022). "Read the conversations that helped convince a Google engineer an artificial intelligence chatbot had become sentient: 'I am often trying to figure out who and what I am'". Business Insider. Archived from the original on June 12, 2022. Retrieved June 12, 2022.
- ^ Levy, Steven (June 17, 2022). "Blake Lemoine Says Google's LaMDA AI Faces 'Bigotry'". Wired. Archived from the original on June 18, 2022. Retrieved June 18, 2022.
- ^ Nguyen, Britney (June 23, 2022). "Suspended Google engineer says the AI he believes to be sentient hired a lawyer". Business Insider. Archived from the original on June 23, 2022. Retrieved June 29, 2022.
- ^ Khushi, Akanksha (July 23, 2022). "Google fires software engineer who claimed its AI chatbot is sentient". Reuters. Archived from the original on July 23, 2022. Retrieved July 23, 2022.
- ^ Clark, Mitchell (July 22, 2022). "The engineer who claimed a Google AI is sentient has been fired". The Verge. Archived from the original on July 23, 2022. Retrieved July 24, 2022.
- ^ Metz, Rachel (June 13, 2022). "No, Google's AI is not sentient". CNN Business. Archived from the original on June 15, 2022. Retrieved June 19, 2022.
- ^ Sparkles, Matthew (June 13, 2022). "Has Google's LaMDA artificial intelligence really achieved sentience?". New Scientist. Archived from the original on June 13, 2022. Retrieved June 20, 2022.
- from the original on June 12, 2022. Retrieved June 18, 2022.
- ^ Alba, Davey (June 14, 2022). "Google Debate Over 'Sentient' Bots Overshadows Deeper AI Issues". Bloomberg News. Archived from the original on June 14, 2022. Retrieved June 19, 2022.
- ^ Goldman, Sharon (June 16, 2022). "AI Weekly: LaMDA's 'sentient' AI debate triggers memories of IBM Watson". VentureBeat. Archived from the original on June 19, 2022. Retrieved June 19, 2022.
- ^ Johnson, Khari (June 14, 2022). "LaMDA and the Sentient AI Trap". Wired. Archived from the original on June 14, 2022. Retrieved June 18, 2022.
- from the original on June 18, 2022. Retrieved June 19, 2022.
- ^ Christian, Brian (June 21, 2022). "How a Google Employee Fell for the Eliza Effect". The Atlantic. Archived from the original on June 21, 2022. Retrieved February 8, 2023.
- ^ Low, Cherlynn (May 11, 2022). "Google's AI Test Kitchen lets you experiment with its natural language model". Engadget. Archived from the original on May 11, 2022. Retrieved June 12, 2022.
- ^ Vincent, James (May 11, 2022). "Google is Beta Testing Its AI Future". The Verge. Archived from the original on May 11, 2022. Retrieved June 12, 2022.
- ^ Bhattacharya, Ananya (May 11, 2022). "Google is so nervous about what its newest bot will say, it made the app invitation-only". Quartz. Archived from the original on May 12, 2022. Retrieved June 12, 2022.
- ^ Vincent, James (August 25, 2022). "Google has opened up the waitlist to talk to its experimental AI chatbot". The Verge. Archived from the original on August 25, 2022. Retrieved August 27, 2022.
- ^ Vincent, James (November 2, 2022). "Google's text-to-image AI model Imagen is getting its first (very limited) public outing". The Verge. Archived from the original on November 10, 2022. Retrieved February 15, 2023.
- from the original on January 20, 2023. Retrieved February 6, 2023.
- ^ Wiggers, Kyle (May 11, 2023). "Hands on with Google's AI-powered music generator". TechCrunch. Archived from the original on May 11, 2023. Retrieved June 10, 2023.
- ^ Millman, Ethan (May 11, 2023). "We've Heard the Future of Music. So Far, It Sounds Terrible". Rolling Stone. Archived from the original on May 11, 2023. Retrieved June 10, 2023.
- 9to5Google. Archivedfrom the original on August 2, 2023. Retrieved October 11, 2023.
- ^ from the original on February 6, 2023. Retrieved February 6, 2023.
- from the original on February 6, 2023. Retrieved February 6, 2023.
- Forbes. Archivedfrom the original on February 7, 2023. Retrieved February 6, 2023.
- ^ Mollman, Steve (March 3, 2023). "Google's head of ChatGPT rival Bard reassures employees it's 'a collaborative A.I. service' and 'not search'". Fortune. Archived from the original on March 4, 2023. Retrieved March 9, 2023.
- ^ Elias, Jennifer (March 3, 2023). "Google execs tell employees in testy all-hands meeting that Bard A.I. isn't just about search". CNBC. Archived from the original on March 4, 2023. Retrieved March 11, 2023.
- from the original on March 21, 2023. Retrieved March 21, 2023.
- from the original on March 21, 2023. Retrieved March 21, 2023.
- ^ Vincent, James (March 21, 2023). "Google opens early access to its ChatGPT rival Bard — here are our first impressions". The Verge. Archived from the original on March 21, 2023. Retrieved March 21, 2023.
- ^ Thoppilan et al. 2022, section 3.
- ^ Thoppilan et al. 2022, section 3 and appendix E.
- ^ Thoppilan et al. 2022, section 5 and 6.
- ^ Hager, Ryne (June 16, 2022). "How Google's LaMDA AI works, and why it seems so much smarter than it is". Android Police. Archived from the original on June 16, 2022. Retrieved June 19, 2022.
- ^ Thoppilan et al. 2022, section 6.2.
- ^ Thoppilan et al. 2022, section 3 and appendix D.