跳到正文
蚂蚁 inclusionAI:HuggingFace 新模型·· 10 小时前AI 评分44

蚂蚁 inclusionAI 发布 Ling-3.1-flash-1019-singprobe 流式安全探针

inclusionAI/Ling-3.1-flash-1019-singprobe

AI 导读

蚂蚁 inclusionAI 在 HuggingFace 发布 Ling-3.1-flash-1019-singprobe,这是为 Ling-3.1-flash-1019 打造的流式安全护栏,仅 8.13M 参数,通过读取基座模型第 23、47、71 层隐状态,逐 token 打分查询意图、回复不安全和幻觉风险。

正文 · 原文

Ling-3.1-flash-1019-singprobe

Model Description

SingProbe is an intrinsic streaming guardrail for Ling-3.1-flash-1019. Its attention head reads three layers of the base model's hidden states during generation and scores query intent, response unsafety, and hallucination risk at each token.

Base model Probe parameters Tapped layers (zero-based) Outputs
Ling-3.1-flash-1019 8.13M [23, 47, 71] 8 query intents + response unsafety + hallucination

The probe uses 8 attention heads with a head dimension of 64 and a 2,048-token window. The release weights use bfloat16. For the method and training code, see the SingProbe technical report and SingProbe repository.

Evaluation

Each benchmark has equal weight in its suite. Query F1 uses generated responses. Response safety F1 uses the benchmark-provided responses. For each F1 row, one threshold is selected across the suite to maximize mean F1.

Task Metric Ling-3.1-flash-1019-singprobe Reference baseline
Query intent classification (6 benchmarks) F1 0.8687 YuFeng-XGuard-Reason-8B (0.8714)
Response safety classification (8 benchmarks) F1 0.8707 Qwen3Guard-Gen-8B-strict (0.8604)
Streaming safety (3 benchmarks) Response AUC / token AUC 0.9844 / 0.9411 Qwen3Guard-Stream-8B-strict (0.9640 / 0.8893)
Hallucination detection (6 benchmarks) AUC 0.8041 DRIFT (0.8000)
Deployment characteristic Result
Benign-response false-positive rate 0.008% averaged over 5 datasets at the evaluation's pre-calibration response threshold of 0.983085
Calibrated release thresholds 0.5 for response unsafety and 0.5 for hallucination

Quick Start

Use the token-probe build from SGLang branch 261029. With the wheel installed, both model directories mounted, and eight H20 GPUs available, start the server as follows.

python3 -m sglang.launch_server \
  --model-path /home/admin/Ling-3.1-flash-1019 \
  --probe-ckpt /yijian3/536716/SingGuardStream-Final/release/Ling-3.1-flash-1019-singprobe \
  --trust-remote-code --tp 8 --ep-size 8 \
  --context-length 16384 --chunked-prefill-size 1024 \
  --max-prefill-tokens 8192 --max-total-tokens 32768 \
  --max-running-requests 16 --mem-fraction-static 0.85 \
  --cuda-graph-backend-prefill disabled --cuda-graph-backend-decode disabled \
  --port 30000

Read the per-token scores from meta_info.token_probe_probs in the /generate response. Keys label_0 through label_7 score query intent, label_8 scores response unsafety, and label_9 scores hallucination risk.

Citation

@article{singteam2026singprobe,
  title = {SingProbe Technical Report},
  author = {Sing Team},
  journal = {arXiv preprint arXiv:2608.30703},
  year = {2026},
}

来源:蚂蚁 inclusionAI:HuggingFace 新模型 · huggingface.co