LitGPT

LitGPT is Lightning AI's open-source project offering 20+ hackable, from-scratch large language model implementations with ready-made recipes for pretraining, finetuning, and deployment. The emphasis is on readability over abstraction: minimal code you can actually read and modify, with flash attention, FSDP, LoRA, QLoRA, adapters, and reduced-precision training (fp4/8/16/32) built in, scaling from a single GPU to 1000+ GPUs and TPUs.

It is aimed at practitioners who want to understand and customize every layer of the training stack rather than treat training as a black box, and YAML training recipes make runs reproducible. The project is Apache 2.0 licensed and widely used as a study and experimentation baseline.

Honest limitations: from-scratch reimplementation means it can lag the bleeding edge of newly released architectures until contributors add them. It is a framework for people comfortable reading training code, not a no-code finetuning service, so beginners should expect a real learning curve before productive runs.

Clone the GitHub repo or pip install, pick a model recipe, and run the pretrain or finetune script with your config.