Meta execs obsessed over beating OpenAI’s GPT-4 internally 🤖

“Honestly… Our goal needs to be GPT-4,” said Meta’s VP of Generative AI, Ahmad Al-Dahle, in an October 2023 message to researcher Hugo Touvron. “We have 64k GPUs coming! We need to learn how to build frontier and win this race.”

While Meta publicly releases open AI models, internal discussions show their leadership was more focused on outpacing competitors like OpenAI and Anthropic, which do not release model weights but gate access via APIs. Models like Anthropic’s Claude and OpenAI’s GPT-4 were seen as the standard to surpass.

The French AI startup Mistral, considered a major open competitor, was also mentioned but with a dismissive tone. “Mistral is peanuts for us,” Al-Dahle remarked in one exchange. “We should be able to do better,” he added.

The intense competition within the tech industry to lead in advanced AI is evident in the messages. Meta’s AI leaders were described as “very aggressive” in acquiring data to train Llama, with one executive admitting, “Llama 3 is literally all I care about.” However, prosecutors allege that shortcuts were occasionally taken, including the use of copyrighted materials for training.

In one exchange, Touvron acknowledged the need for better datasets for Llama 3 and discussed leveraging the LibGen dataset, which includes copyrighted materials. “Do we have the right datasets in there?” Al-Dahle asked. “Is there anything you wanted to use but couldn’t for some stupid reason?”

Meta CEO Mark Zuckerberg previously emphasized closing the gap between Llama models and proprietary models from competitors like OpenAI and Google. In a July 2024 letter, he claimed, “This year, Llama 3 is competitive with the most advanced models and leading in some areas. Starting next year, we expect future Llama models to become the most advanced in the industry.”

When Llama 3 was released in April 2024, it proved competitive with top proprietary models and outperformed other open options, including those from Mistral. However, lawsuits questioning the legality of the training data used continue to pose challenges for Meta.

Powered by Markelitics.com

‍