Meta, the parent company of Facebook and Instagram, is in the process of developing a new artificial intelligence system with the aim of matching the capabilities of OpenAI’s most advanced model, which was responsible for creating ChatGPT and is backed by Microsoft. This information comes from sources familiar with the matter. Meta’s objective is to make this new AI model, anticipated to be ready by the next year, significantly more robust than their recently released model, known as Llama 2, which was launched just two months ago.
The specifics of this planned system may evolve, but its purpose is to assist other companies in creating services that can generate sophisticated text, perform analysis, and produce other types of output. A team, initiated by Meta’s CEO Mark Zuckerberg earlier this year, is dedicated to accelerating the development of generative AI tools capable of mimicking human expressions. Training for this new large language model is expected to commence in early 2024, as per some individuals familiar with the matter.
These plans, which have not been previously reported, align with Mark Zuckerberg’s efforts to establish Meta as a significant player in the AI field after trailing behind competitors. Competition in the AI sector has intensified considerably this year, sparking debates on various topics such as the optimal business models and regulatory frameworks for AI technology.
Meta is presently expanding its data centers to support this endeavor and acquiring additional H100 chips from Nvidia, which are essential for AI training. Although Meta collaborated with Microsoft to make Llama 2 available on Microsoft’s Azure cloud-computing platform, it intends to train the new model using its own infrastructure, according to some sources.
Mark Zuckerberg is advocating for the new model, like Meta’s previous AI offerings, to be open-sourced, making it freely accessible for other companies to develop AI-powered tools.
Zuckerberg will be part of a group of prominent tech executives attending a summit organized by Senate Majority Leader Chuck Schumer to discuss the handling of AI. Other attendees include Sam Altman, CEO of OpenAI, and Sundar Pichai, CEO of Google.
It’s worth noting that the model currently in development may not necessarily catch up with Meta’s competitors. Meta hopes it will be roughly as capable as GPT-4, launched by OpenAI in March, which underpins OpenAI’s revenue-generating initiatives, including the recently introduced ChatGPT for Business. Meta’s new model is also expected to debut after Google’s anticipated launch of Gemini, an advanced large language model.
Meta’s open-source approach has its advantages, primarily lower costs and adaptability. However, some legal experts raise concerns about potential downsides, such as the misuse of potentially copyright-protected information and the broader accessibility of a powerful tool that could be exploited for generating and spreading disinformation or other malicious activities. Meta’s legal team is currently reviewing these concerns as part of the company’s planning process.
Large language models typically become more potent with larger datasets. While Meta’s most powerful version of Llama 2 was trained on 70 billion parameters, the size of GPT-4, developed by OpenAI, has not been disclosed but is estimated to be approximately 20 times larger, at 1.5 trillion parameters. Some AI experts believe there may be alternative methods to achieve GPT-4’s capabilities without necessarily increasing its size to that extent.