CroissantLLM

CroissantLLM is a 1.3-billion-parameter language model pretrained on 3 trillion tokens with a strict 1:1 English-to-French ratio, built by researchers centered at CentraleSupélec. It was designed as a statement against English-centric training: a custom bilingual tokenizer, a manually curated French data split, and bilingual fine-tuning datasets, small enough to run on consumer hardware. The team also released dozens of intermediate checkpoints, the training code, and CroissantLLMChat, a chat-tuned variant.

To evaluate French properly the authors created FrenchBench, a suite of classification and generation tasks in French, and scored their transparency practices at 81 percent of the Foundation Model Transparency Index criteria. Released under Apache 2.0 in early 2024, it became a reference point for sovereign-language modeling in France, with the French government experimenting with browser-based deployments of the chat variant.