T0pp

T0pp (T Zero Plus Plus) is the flagship 11-billion-parameter checkpoint of BigScience's T0 family, released in 2022. It fine-tunes the T5-XXL backbone on P3, a massive collection of prompted datasets, and was the model the authors recommended as the best all-round performer. T0pp demonstrated that large-scale multitask prompted training unlocks strong zero-shot performance on unseen tasks, a direct precursor to the instruction-tuning era.

The model is fully open on Hugging Face (bigscience/T0pp) under Apache 2.0. As an encoder-decoder it is suited to understanding and generation tasks with explicit prompts rather than open-ended chat. Release date is era-level (late 2021 paper, 2022 public checkpoints).