Meta, the parent company of Facebook, has announced the launch of a new AI-based large-language model focused on research community use. The move puts the company in competition with others racing to develop AI technology.
Also Read: Meta Copies Twitter, Rolls Out Blue Tick Subscription Service
AI has become the most popular buzzword this year, with major tech companies such as Microsoft, Google, Baidu, Alibaba, and now Meta embracing it. The battle to become the forerunner has already kicked off in the technology space, with big companies dropping metaverse projects to pump up the AI hype.
Meta's AI, specifically LLaMA-13B seems exciting, but again I don't think any of these tools are ready for broad public adoption. If the company makes the models and the input weights publicly knowable that would be a huge positive.
— . (@masonpelt) February 25, 2023
Meta announced the new Large Language Model Meta AI (LLaMA) “as part of Meta’s commitment to open science,” and it will be available under a non-commercial license to researchers and entities affiliated with government, civil society, and academia.
“(LLaMA) is designed to help researcher advance their work in the subfield of AI. Smaller, more performant models such as LLaMA enable others in the research community who don’t have access to large amounts of infrastructure to study these models, further democratizing access in this important, fast-changing field,” stated Meta in a blog post.
The model is trained on 20 languages with a particular emphasis on those utilizing Latin and Cyrillic alphabets, and it requires significantly less computing power than their previous offerings, said Meta.
Is LLaMA an experiment?
The hype of generative AI has been at its peak since the launch of ChatGPT last November, but Meta remained silent while other rival tech giants dove into it. However, nearly three months later, Meta overheard for the first time discussing generative AI, which could be a test for future products.
“Meta’s announcement today appears to be a step in testing their generative AI capabilities so they can implement them into their products in the future,” said Gil Luria, senior software analyst at D.A. Davidson.
The tech industry discovered a fertile investment sector in AI and Meta, attempting to figure out a unique way to incorporate it into its products.
Today we release LLaMA, 4 foundation models ranging from 7B to 65B parameters.
LLaMA-13B outperforms OPT and GPT-3 175B on most benchmarks. LLaMA-65B is competitive with Chinchilla 70B and PaLM 540B.
The weights for all models are open and available at https://t.co/q51f2oPZlE
1/n pic.twitter.com/DPyJFBfWEq— Guillaume Lample (@GuillaumeLample) February 24, 2023
“Generative AI is a new application of AI that Meta has less experience with, but is clearly important for the future of their business,” said Luria.
The company acknowledged that more research is necessary to tackle the risks associated with bias, toxic comments, and hallucinations in large language models, and that LLaMA is no exception to these challenges.
“As a foundation model, LLaMA is designed to be versatile and can be applied to many different use cases, versus a fine-tuned model that is designed for a specific task,” stated Meta.
Meta expects to outperform GPT-3
Meta believes that the LLaMA could outperform competitors that examine more parameters or variables that the algorithm takes into account.
The LLaMA language model has a 13-billion-parameter version that outperforms GPT-3, the model used to build ChatGPT. The LLaMA model, with 65 billion parameters, is considered competitive with Google’s Chinchilla70B and PaLM-540B models, which are even larger than the model that powered Google’s Bard chat search.
Meta wants in on the #ChatGPT hype. Today they release LLaMA – their foundational large language model with 65 billion parameters. 🤯@OpenAI led the way and every single.l big tech company wants a slice of the piece. https://t.co/E0FZYVlPgQ
— Nina Schick (@NinaDSchick) February 24, 2023
“Our smallest model, LLaMA 7B, is trained on one trillion tokens,” said Meta.
Last year, Meta also released a large-language model, OPT-175B, aiming to assist researchers, on which its new chatbot, BlenderBot, is based.
Last year, Meta launched Galactica, which claimed to be capable of writing scientific articles and solving math problems. However, the AI generated authoritative-sounding but inaccurate responses, and as a result, Meta quickly pulled it down.
and then