UL2-20B

UL2-20B is Google Research's 20-billion-parameter encoder-decoder model, released October 2022 alongside the paper Unifying Language Learning Paradigms. Its innovation is the mixture-of-denoisers objective, which blends span corruption, prefix language modeling, and other pretraining paradigms with mode switching, so one model performs well at both fine-tuning and few-shot prompting. Google reported it beating GPT-3 175B and T5 11B on many evaluations.

Google released the 20B checkpoints openly under Apache 2.0. It is the base model behind FLAN-UL2 and remains a useful research artifact for studying unified pretraining objectives.