Mistral AI bucks release trend by dropping torrent link to new open source LLM

A Reddit post described the Mistral LLM as a “scaled-down GPT-4” that appears to be “a MoE consisting of 8 7B experts.” For the inference of each token, only 2 experts are used, the post continued, adding that “from GPT-4 leaks, we can speculate that GPT-4 is a MoE model with 8 experts, each with 111B parameters of their own and 55B shared attention parameters (166B parameters per model). For the inference of each token, also only 2 experts are used.”

Uri Eliabayev, an AI consultant and founder of the “Machine & Deep learning Israel” community, told VentureBeat in a message that Mistral was “well-known” for this kind of release, “without any paper, blog, code or press release.” And open source AI advocate Jay Scambler messaged that the release was “definitely unusual, but it has generated quite a bit of buzz which I think is the point.”

The guerrilla move was immediately hailed by many in the AI community. For example, entrepreneur George Hotz posted:

And Eric Jang,  vice president of AI, 1X Technologies and former research scientist in robotics at Google, posted that Mistral’s brand “is already becoming one of my favorites in the AI space.”

Mistral, a Paris-based startup that just secured a $2 billion valuation in a blockbuster funding round led by Andreessen Horowitz, was already well-known its a record-setting $118 million seed round — reportedly the largest seed in the history of Europe — and its first large language AI model, Mistral 7B, launched in September.

The company also has been in the center of the debate around the EU AI Act, after it was reported to be lobbying the European Parliament’s for less regulation on open source AI.