GatorTron

GatorTron is a family of clinical transformer models developed by the University of Florida and NVIDIA, published in npj Digital Medicine in 2022. The large variant scales to 8.9 billion parameters, trained on more than 90 billion words including de-identified UF Health clinical notes, PubMed, and Wikipedia, using 992 A100 GPUs on the HiPerGator-AI supercomputer.

At release it was the largest clinical-domain transformer and achieved state-of-the-art results on five clinical NLP tasks including concept extraction, relation extraction, and medical question answering. The models are encoder-only, suited to understanding rather than generation, with weights distributed through NVIDIA's model catalog and Hugging Face.