GPT-Neo 2.7B

GPT-Neo 2.7B was EleutherAI's 2021 replication of the GPT-3 architecture at the 2.7B scale, trained on the Pile, the group's 825GB curated pretraining dataset. Released under the Apache 2.0 license, it was one of the first serious open alternatives to proprietary GPT-3-class APIs and became a widely used base for early open-source fine-tuning experiments.

It belongs to the GPT-Neo family alongside the 1.3B variant and preceded EleutherAI's larger GPT-J 6B. While far behind modern models in capability, it is a landmark release in the history of open language models.