腾讯微信视觉团队开源 WeVisOne-VL 视觉语言模型，统一检测、OCR、分割与抠图

Tencent’s WeChat Vision team has released WeVisOne-VL, an open vision language model in 2B and 4B sizes. The WeVisOne-VL vision language model handles detection, referring, counting, pointing, OCR, segmentation and alpha matting in one model. Pixel outputs come from a 9.6M-parameter decoder, so no separate SAM is needed. It also answers “None” when a queried object is absent. For developers, that can replace a detector, an OCR engine and a segmenter with one model.

TL;DR

• Size: 2B (2.44B params) and 4B (4.83B params) on Hugging Face, and a 9.6M-parameter pixel decoder. Trained with up to 8,192 tokens of context; deployment context length is not disclosed.
• Runs on: A single GPU through Transformers or vLLM in BF16. The package also lists NPU and CPU devices. VRAM needs are not disclosed. A community GGUF ran object-level tasks on a 6 GB RTX 2060.
• Performance: Beats the same-scale VLM baselines in Tencent’s tables on most of 30+ benchmarks (self-reported).
• Best: 71.3 F1 Mean on Dense200 dense detection, versus 58.7 for LocateAnything-3B.
• Worst: 52.6% rejection rate on PR-Bench, so many absent-target queries still get a box.

What is WeVisOne-VL?

WeVisOne-VL (“WeChat Vision All-in-One”) is a perception-focused model from Tencent built on Qwen3-VL. It targets fine-grained localization rather than open-ended chat. Users choose the output granularity: points, boxes, binary masks or continuous alpha mattes.

How does WeVisOne-VL work?

Every task is cast as structured next-token prediction. The model adds coordinate tokens <0> to <1000> for normalized boxes and points. An absent target produces None instead of a guessed box.

For pixel tasks, the model emits an <OBJ> token after each predicted box. Its hidden state becomes a query for the pixel decoder, according to the model card. The decoder has 3 parts:

• Visual Feature Adapter: reuses multi-level Qwen3-VL ViT features instead of a second image encoder.
• Box-Guided Query Decoder: 6 deformable cross-attention blocks seeded by the <OBJ> state and the box.
• Dense Prediction Head: refines output at S/4, S/2 and S/1 into a full-resolution mask or matte.

How was it trained?

A knowledge-graph-driven data engine annotated over 100M images with about 3.6B boxes. SAM3, Rex-Omni and an in-house detector proposed regions, and Qwen3-VL verified them.

Training used 3 SFT stages. Stage 1 aligned output formats on 90M examples. Stage 2 raised input to 1,960 visual tokens and up to 500 boxes per response. Stage 3 trained the pixel decoder, then specialized it for matting on 130M samples. A final GRPO stage used 178K uncertainty-filtered examples, as described in the research paper.

How does WeVisOne-VL perform on benchmarks?

All numbers below are self-reported, using greedy decoding and at most 1,960 visual tokens per image. Detection, referring and OCR use F1 averaged over IoU 0.5 to 0.95.

• Detection: 55.3 F1 on LVIS and 71.3 on Dense200 for the 4B model.
• Referring: 81.8 F1 on RefCOCOg test, versus 77.6 for LocateAnything-3B.
• Rejection: 96.4% on Tencent’s in-house WeRef-Bench (2,880 queries), versus 73.0%.
• Counting: 89.6% exact-match on PixMo-Count test, versus 82.8%.
• OCR: 52.0 F1 on TotalText, ahead of the PP-OCRv5 specialist at 25.7.
• Segmentation: 71.2 gIoU on ReasonSeg val, versus 65.0 for LISA-13B.
• Matting: 7.3 MAD on P3M-500-NP box prompts, versus 15.4 for SAM2-Matting (SAM3). Lower is better.

The 2B model trails the 4B model on most tasks. It still scores better box-prompted matting on P3M-500-NP, at 6.6 MAD.

WeVisOne-VL vs Qwen3-VL, Rex-Omni and LocateAnything

How do you run WeVisOne-VL?

Weights for the 2B and 4B models are on Hugging Face. The wevisone_vl package wraps Transformers and vLLM backends. Tasks include detection, referring, point, count_box, ocr, segmentation and matting.

from wevisone_vl import WeVisOneVL
model = WeVisOneVL("tencent/WeVisOne-VL-4B", backend="transformers")  # or "vllm"
result = model.predict(image, task="matting", query="person")

Hosted demos are live for 2B and 4B. Tencent also open-sourced Mask2Alpha, the matting annotator from its data engine. Fine-tuning code is still on the TODO list. GGUF builds cannot produce masks, since the decoder needs native visual features.

Key Takeaways

• 1 model covers detection, OCR, counting, pointing, segmentation and matting.
• A 9.6M-parameter decoder replaces SAM for masks and alpha mattes.
• 4B scores 71.3 F1 on Dense200, 12.6 points above LocateAnything-3B.
• Apache 2.0 weights, unlike non-commercial LocateAnything-3B.

Check out the Paper, GitHub repo, Model weights and Project page. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us

The post Tencent WeVisOne-VL Unifies Detection, OCR, Segmentation and Matting appeared first on MarkTechPost.