Meta has confirmed that it is releasing its new open-source large language model (LLM) within the next month, according to reports.
TechCrunch reported that at an event in London on Tuesday (March 10), Meta’s president of global affairs, Nick Clegg, said, “Within the next month, actually less, hopefully in a very short period of time, we hope to start rolling out our new suite of next-generation foundation models, Llama 3.
“There will be a number of different models with different capabilities, different versatilities [released] during the course of this year, starting really very soon,” he added.
The Information originally claimed that a smaller version of Llama 3 would be released as early as next week, with the complete open-source model still set to launch in July. This version will be capable of competing with Claude 3 and GPT-4.
Chris Cox, Meta’s Chief Product Officer, added that the plan involves using Llama 3 to power a variety of Meta’s products.
The announcement comes as Meta attempts to catch up in the hyper-competitive world of generative AI. Its predecessor, Llama 2, was released in July 2023 and faced criticism for its limitations.
Joelle Pineau, Vice President of AI Research, stated, “Our objective is to evolve a Llama-powered Meta AI into the world’s most useful assistant. However, there’s still a significant amount of work needed to achieve this goal.”
However, Meta’s chief AI scientist Yann LeCun believes that Joint Embedding Predictive Architecture (JEPA) is the answer rather than generative AI. According to Meta, unlike generative approaches that try to fill in every missing pixel, JEPA has the flexibility to discard unpredictable information, which leads to improved training and sample efficiency.
Right. That's why we need JEPA world models and use them for planning to fulfill a specified goal. https://t.co/8TI2NGRPvj
— Yann LeCun (@ylecun) April 7, 2024
What is Llama 3?
Llama 3, a large language model, will be available in sizes ranging from the very small, designed to compete with models such as Claude Haiku or Gemini Nano, to larger versions fully equipped for responses and reasoning, similar to GPT-4 or Claude Opus.
Details about Llama 3 remain scarce, however, it is expected that it will follow the previous version’s footsteps by being open source and is anticipated to be multimodal, understanding both visual and text inputs.
It is thought that Llama 3 will come in various versions and sizes, with the smallest having 7 billion parameters and the largest having around 140 billion parameters. However, this still falls short of the trillion-plus parameters that were used to train GPT-4.
Featured image: Canva
