OpenELM

OpenELM (Open-source Efficient Language Models) is Apple's family of small language models released in April 2024, designed to run on-device on Apple silicon rather than in the cloud. Eight models ship in four sizes, 270M, 450M, 1.1B, and 3B, each in pretrained and instruction-tuned variants.

The key innovation is layer-wise scaling, which allocates parameters unevenly across transformer layers for better accuracy at a given size: the 1.1B model beats OLMo by 2.36% while using half the pretraining tokens. Apple published the complete training and evaluation framework, training logs, checkpoints, and pre-training configurations, an unusually transparent release from the company.