LISA
Reasoning segmentation: an LLM that thinks through complex queries and outputs the mask for what it concludes.
About
LISA (Large Language Instructed Segmentation Assistant) from CUHK, SmartMore, and Microsoft Research Asia introduced the reasoning segmentation task. Instead of 'segment the red car', the query can be something like 'the object the person would use to cut the cake', and the model has to reason first, then segment. It adds a SEG token to an LLaVA-style multimodal LLM and uses an embedding-as-mask paradigm, where the token's hidden embedding is decoded into a mask by a SAM-style head.
The paper shows LISA-7B hitting 74.1, 76.5, and 71.1 cIoU on refCOCO val, testA, and testB, and outperforming prior methods by over 20 points of gIoU on the new ReasonSeg benchmark of implicit queries, with strong zero-shot behavior even before fine-tuning on reasoning data. Code and weights are Apache-2.0, which spawned a family of follow-ups (LISA++, LaSagnA). The 13B model needs serious VRAM, so most people run the 7B.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
Author-reported (CVPR 2024): LISA-7B 74.1/76.5/71.1 cIoU on refCOCO val/testA/testB; 44.4 gIoU on ReasonSeg; +20 gIoU over prior methods
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (verified tier, metrics-v1). Evidence: CVPR 2024 paper (arXiv:2308.00692) with refCOCO and ReasonSeg numbers confirmed via the paper tables; Apache-2.0 confirmed via ecosyste.ms repo metadata. Strengths: created the reasoning-segmentation task, strong zero-shot reasoning, spawned an active follow-up line. Docked for: (1) heavy inference cost (7B/13B LLM plus SAM head); (2) repo last pushed in 2024, maintenance has slowed. License: Apache-2.0.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.0/10
- Maintenance · recency, releases, issue load
- 5.0/10
- Openness · license + docs
- 9.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · CVPR 2024 (arXiv 2308.00692, Aug 2023)
- · LISA-7B 74.1/76.5/71.1 cIoU on refCOCO val/testA/testB; 44.4 gIoU on ReasonSeg, author-reported
- · Created the reasoning-segmentation task; follow-ups LISA++, LaSagnA