CLIP

Introduced in January 2021 in the paper 'Learning Transferable Visual Models From Natural Language Supervision' (arXiv:2103.00020), CLIP (Contrastive Language-Image Pre-training) learns a shared embedding space for images and text by training on ~400 million image-caption pairs from the internet. Matching pairs are pulled together, mismatched pairs pushed apart — no curated class labels required.

CLIP matched a supervised ResNet-50 on ImageNet zero-shot, without ever seeing ImageNet's 1.28M labeled examples, and became the default visual encoder for a generation of multimodal systems (it's inside Stable Diffusion, LLaVA, Flamingo-era pipelines, and countless retrieval products). The code is MIT-licensed; the weights are released for research use under OpenAI's terms, so check the terms before commercial deployment.