Mask2Former
A single transformer architecture that handles panoptic, instance and semantic segmentation with masked attention.
About
Mask2Former came out of Meta's FAIR lab in late 2021 as a follow-up to MaskFormer, and it answered a simple question: can one architecture do every image segmentation task? Instead of building separate networks for panoptic, instance and semantic segmentation, it uses a masked-attention decoder that only lets the model attend inside predicted mask regions. That one change made training easier and beat the specialized state of the art on all four standard datasets.
The numbers the authors reported were striking at the time: 57.8 PQ on COCO panoptic, 50.1 AP on COCO instance, and 57.7 mIoU on ADE20K semantic segmentation, all from the same model. It shipped in Detectron2 and later in Hugging Face Transformers, so it quietly became the default baseline that newer segmentation papers compare against — and it still gets used in production pipelines where a single model needs to cover multiple segmentation tasks.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- Paper-reported (arXiv 2112.01527): 57.8 PQ on COCO panoptic, 50.1 AP on COCO instance, 57.7 mIoU on ADE20K semantic segmentation (Swin-L backbone)
- Community-compiled comparisons show later unified models (OneFormer, OMG-Seg) reporting marginally higher scores
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 8/10
Scored 8/10 (TIER verified, metrics-v1). Evidence: peer-reviewed CVPR 2022 paper (arXiv 2112.01527) with COCO/ADE20K/Cityscapes evals; integrated into Hugging Face Transformers docs; Meta Research project page live. Strengths: widely cited, genuinely open weights + code, still a standard baseline. Docked for: (1) headline numbers are author-reported on standard academic sets; (2) no hosted API — self-hosting required. License: no SPDX identifier declared in the original repo — verify before commercial use.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 9.0/10
- Maintenance · recency, releases, issue load
- 8.0/10
- Openness · license + docs
- 9.0/10
- Manual review · sanity read + red-flag check
- 8.0/10
- · CVPR 2022 paper arXiv 2112.01527; 57.8 PQ COCO panoptic, 50.1 AP COCO instance, 57.7 mIoU ADE20K (Swin-L)
- · Hugging Face Transformers + Detectron2 integration; Meta Research project page live; still a standard baseline
- · Headline numbers author-reported; no hosted API — self-hosting required