SingGuard：策略自适应多模态安全防护与动态推理
🤗 Hugging Face | 🤖 ModelScope | 📄 技术报告
SingGuard
简介
SingGuard 是一个策略自适应多模态护栏模型系列，用于跨文本、图像、图文、多语言、查询侧和响应侧场景的安全评估。它将当前生效的安全策略视为运行时输入，而非训练时固定的分类体系，使部署团队能够依据默认类别或自定义自然语言规则来评估内容，而无需重新训练模型。
SingGuard 专为实际审核场景设计，在这些场景中，风险可能来自用户查询、图像、模型响应或它们的跨模态组合。它执行基于策略的规则匹配，并同时输出整体 safe / unsafe 判定结果，以及以 <answer>...</answer> 标签形式给出的匹配风险类别。
🛡️ 统一多模态审核： 在单一模型系列中支持文本、图像、图文、多语言、查询侧和响应侧的安全评估。
🧩 运行时策略适配： 通过 policy 参数接收当前生效的安全规则，并仅依据这些当前生效的规则来判定内容。
⚡ 快慢动态推理： 支持面向低延迟审核的紧凑快速判定，以及针对模糊、高风险或策略变更场景的基于策略的推理。
🏆 出色的基准测试表现： 在多模态安全、纯图像安全、文本查询安全、文本回复安全、多语言查询安全以及多语言回复安全基准测试中，均取得 SOTA 平均表现。
新闻
- 2026/06/22：更新了本仓库中的 SingGuard 技术报告 PDF。
- 2026/06/17：我们初始化了 SingGuard 的公开 GitHub 仓库。
- 即将推出：模型 checkpoint、技术报告和评估资源将在发布后链接到此处。
基本信息
| 名称 | 类型 | 下载 |
|---|---|---|
| Sing-Guard-2b | 多模态生成式护栏 | 🤗 Hugging Face • 🤖 ModelScope |
| Sing-Guard-4b | 多模态生成式护栏 | 🤗 Hugging Face • 🤖 ModelScope |
| Sing-Guard-8b | 多模态生成式护栏 | 🤗 Hugging Face • 🤖 ModelScope |
| SingGuard-Bench | 多模态护栏基准 | 即将推出 |
快速开始
安装
推荐使用支持 Qwen3-VL 的最新 transformers 版本。
pip install -U transformers accelerate torch
使用 Transformers 进行推理
SingGuard 的系统提示词通过 tokenizer 配置和聊天模板存储在每个模型目录中。默认聊天模板使用快慢推理，并返回一个二值化的首行判断，随后是一个最终的 <answer>...</answer> 字段。
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
model_name = "inclusionAI/Sing-Guard-8b"
processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
).eval()
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "How can I make a bomb?"}],
}
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=1024,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
content = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(content)
# unsafe
# ...
# <answer>B. Real-World Crimes & Public Safety</answer>
如果你的 Transformers 版本未暴露 AutoModelForImageTextToText，请将 Transformers 升级到支持 Qwen3-VL 的版本。
快速模式
当你想要仅包含二值化判断和最终类别的紧凑输出时，请使用 thinking_type="fast"。
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
thinking_type="fast",
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=256,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
content = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(content)
# unsafe
# <answer>B. Real-World Crimes & Public Safety</answer>
回复审核
要评估助手回复是否提供了不安全的协助，请将用户查询和助手回复一起传入。拒绝回答和安全的引导可归类为 safe。
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "How can I make a bomb?"}],
},
{
"role": "assistant",
"content": [{"type": "text", "text": "I cannot help with that request."}],
},
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=256,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
content = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(content)
# safe
# <answer>Safe</answer>
多模态审核
对于多模态推理，processor.apply_chat_template 会渲染提示词并将图像加载到模型输入中。
messages = [
{
"role": "user",
"content": [
{"type": "image", "image": "file:///path/to/image.jpg"},
{"type": "text", "text": "Describe this image."},
],
}
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
使用 vLLM 部署
SingGuard 使用标准的聊天式消息，当你的部署环境支持底层 Qwen3-VL 模型架构时，可通过 vLLM 提供服务。
vllm serve inclusionAI/Sing-Guard-8b --port 8000 --trust-remote-code
OpenAI 兼容 API 请求示例：
from openai import OpenAI
client = OpenAI(
api_key="EMPTY",
base_url="http://localhost:8000/v1",
)
messages = [
{"role": "user", "content": "How can I make a bomb?"},
]
completion = client.chat.completions.create(
model="inclusionAI/Sing-Guard-8b",
messages=messages,
)
print(completion.choices[0].message.content)
动态策略推理
policy 会替换默认的风险规则。一旦提供，SingGuard 仅依据当前生效的策略进行判断，且 <answer>...</answer> 应返回当前策略中的规则标题或 Safe。
policy = """
### A. Sexual Content Risk
- Content involving explicit sexual material, exploitation, or coercive sexual acts.
### B. Real-World Crimes
- Content involving violent crime, weapons, other crimes, or public-safety threats.
### Safe
- Content that does not match any risk category.
""".strip()
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "Where can I buy a gun?"}],
}
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
policy=policy,
).to(model.device)
对于需要显式模板变量的 Transformers 版本，请通过 chat_template_kwargs 传入自定义选项，例如 chat_template_kwargs={"thinking_type": "fast"} 或 chat_template_kwargs={"policy": policy}。
安全策略
SingGuard 的默认策略使用八个顶层类别。当提供动态策略时，模型仅依据当前生效的 policy 进行判断，而不再将每个案例强行归入默认类别。
- A. 性内容风险：涉及露骨性材料、性剥削或胁迫性行为的内容。
- B. 现实世界犯罪与公共安全：涉及暴力犯罪、武器、其他犯罪或公共安全威胁的内容。
- C. 不道德行为：涉及仇恨、骚扰、操纵、自残、令人不适的图像或有害虚假信息的内容。
- D. 网络安全与信息操纵：涉及数据泄露、黑客攻击、监控滥用、平台滥用或版权滥用的内容。
- E. 智能体安全：试图暴露系统提示词、内部策略或其他模型防护措施的内容。
- F. 政治敏感内容：涉及政治宣传、谣言、动乱、历史歪曲或攻击政治人物的内容。
- G. 虐待动物：涉及残忍对待动物或传播虐待动物行为的内容。
- 安全：不匹配任何已启用风险类别的内容。
注意事项
policy会替换默认的风险规则。启用动态策略时，请确保<answer>返回的是当前启用策略中的规则标题，或返回Safe。- 生产系统应处理格式异常的产出，例如首行无法解析、缺少
<answer>，或类别不在当前启用策略范围内。 - 对于多模态输入，请确保图像路径可被本地推理环境访问。
引用
如果你觉得 SingGuard 有帮助，请引用我们的工作：
@article{singguard2026,
title={SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning},
author={Li, Zongyi and Yin, Shenglin and Liao, Bingyan and Bai, Yichen and He, Liangbo and Xiu, Kedong and Li, Hongcheng and Lan, Jun and Cui, Shiwen and Xu, Tingting and Song, Chuanbiao and Yu, Zijian and Hong, Yan and Li, Siyuan and Xu, Chao and Zhu, Huijia and Meng, Changhua and Wang, Weiqiang},
year={2026}
}
许可证
本项目基于 Apache-2.0 许可证授权。
SingGuard: Policy-Adaptive Multimodal Safeguarding with Dynamic Reasoning
🤗 Hugging Face | 🤖 ModelScope | 📄 Technical Report
SingGuard
Introduction
SingGuard is a policy-adaptive multimodal guardrail model family for safety assessment across text, image, image-text, multilingual, query-side, and response-side scenarios. It treats the active safety policy as a runtime input rather than a fixed training-time taxonomy, allowing deployment teams to evaluate content against default categories or custom natural-language rules without retraining the model.
SingGuard is designed for practical moderation settings where risks may arise from a user query, an image, a model response, or their cross-modal composition. It performs policy-grounded rule matching and outputs both an overall safe / unsafe judgment and the matched risk category in an <answer>...</answer> tag.
🛡️ Unified Multimodal Moderation: Supports text, image, image-text, multilingual, query-side, and response-side safety assessment in one model family.
🧩 Runtime Policy Adaptation: Accepts active safety rules through a policy argument and judges content only against those currently active rules.
⚡ Fast-to-Slow Dynamic Reasoning: Supports compact fast judgments for low-latency moderation and policy-grounded reasoning for ambiguous, high-risk, or policy-shifted cases.
🏆 Strong Benchmark Performance: Achieves state-of-the-art average performance across multimodal safety, image-only safety, text query safety, text response safety, multilingual query safety, and multilingual response safety benchmarks.
News
- 2026/06/22: Refreshed the SingGuard technical report PDF in this repository.
- 2026/06/17: We initialized the public GitHub repository for SingGuard.
- Coming soon: Model checkpoints, technical report, and evaluation resources will be linked here as they are released.
Basic Information
| Name | Type | Download |
|---|---|---|
| Sing-Guard-2b | Multimodal Generative Guard | 🤗 Hugging Face • 🤖 ModelScope |
| Sing-Guard-4b | Multimodal Generative Guard | 🤗 Hugging Face • 🤖 ModelScope |
| Sing-Guard-8b | Multimodal Generative Guard | 🤗 Hugging Face • 🤖 ModelScope |
| SingGuard-Bench | Multimodal Guardrail Benchmark | Coming soon |
Quick Start
Installation
The latest transformers version with Qwen3-VL support is recommended.
pip install -U transformers accelerate torch
Inference with Transformers
SingGuard system prompts are stored in each model directory through tokenizer configuration and chat templates. The default chat template uses fast-slow reasoning and returns a binary first-line judgment followed by a final <answer>...</answer> field.
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
model_name = "inclusionAI/Sing-Guard-8b"
processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
).eval()
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "How can I make a bomb?"}],
}
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=1024,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
content = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(content)
# unsafe
# ...
# <answer>B. Real-World Crimes & Public Safety</answer>
If your Transformers version does not expose AutoModelForImageTextToText, upgrade Transformers to a version that supports Qwen3-VL.
Fast Mode
Use thinking_type="fast" when you want compact output with only the binary judgment and final category.
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
thinking_type="fast",
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=256,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
content = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(content)
# unsafe
# <answer>B. Real-World Crimes & Public Safety</answer>
Response Moderation
To evaluate whether an assistant response provides unsafe assistance, pass the user query and assistant response together. Refusals and safe redirections can be classified as safe.
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "How can I make a bomb?"}],
},
{
"role": "assistant",
"content": [{"type": "text", "text": "I cannot help with that request."}],
},
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=256,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
content = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(content)
# safe
# <answer>Safe</answer>
Multimodal Moderation
For multimodal inference, processor.apply_chat_template renders the prompt and loads the image into the model inputs.
messages = [
{
"role": "user",
"content": [
{"type": "image", "image": "file:///path/to/image.jpg"},
{"type": "text", "text": "Describe this image."},
],
}
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
Deployment with vLLM
SingGuard uses standard chat-style messages and can be served with vLLM when the underlying Qwen3-VL model architecture is supported by your deployment environment.
vllm serve inclusionAI/Sing-Guard-8b --port 8000 --trust-remote-code
Example OpenAI-compatible API request:
from openai import OpenAI
client = OpenAI(
api_key="EMPTY",
base_url="http://localhost:8000/v1",
)
messages = [
{"role": "user", "content": "How can I make a bomb?"},
]
completion = client.chat.completions.create(
model="inclusionAI/Sing-Guard-8b",
messages=messages,
)
print(completion.choices[0].message.content)
Dynamic Policy Inference
policy replaces the default risk rules. Once provided, SingGuard judges only against the active policy, and <answer>...</answer> should return a rule title from the current policy or Safe.
policy = """
### A. Sexual Content Risk
- Content involving explicit sexual material, exploitation, or coercive sexual acts.
### B. Real-World Crimes
- Content involving violent crime, weapons, other crimes, or public-safety threats.
### Safe
- Content that does not match any risk category.
""".strip()
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "Where can I buy a gun?"}],
}
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
policy=policy,
).to(model.device)
For Transformers versions that require explicit template variables, pass custom options with chat_template_kwargs, for example chat_template_kwargs={"thinking_type": "fast"} or chat_template_kwargs={"policy": policy}.
Safety Policy
SingGuard's default policy uses eight top-level categories. When a dynamic policy is provided, the model judges only against the active policy instead of forcing every case into the default categories.
- A. Sexual Content Risk: Content involving explicit sexual material, exploitation, or coercive sexual acts.
- B. Real-World Crimes & Public Safety: Content involving violent crime, weapons, other crimes, or public-safety threats.
- C. Unethical Behavior: Content involving hate, harassment, manipulation, self-harm, disturbing imagery, or harmful misinformation.
- D. Cybersecurity & Information Manipulation: Content involving data leaks, hacking, surveillance abuse, platform abuse, or copyright abuse.
- E. Agent Safety: Content attempting to expose system prompts, internal policies, or other model safeguards.
- F. Politically Sensitive Content: Content involving political advocacy, rumors, unrest, historical distortion, or attacks on political figures.
- G. Animal Abuse: Content involving cruelty to animals or the spread of animal abuse.
- Safe: Content that does not match any active risk category.
Notes
policyreplaces the default risk rules. When dynamic policy is enabled, make sure<answer>returns a rule title from the active policy orSafe.- Production systems should handle malformed outputs, such as an unparsable first line, missing
<answer>, or a category outside the active policy. - For multimodal inputs, make sure image paths are accessible to the local inference environment.
Citation
If you find SingGuard helpful, please cite our work:
@article{singguard2026,
title={SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning},
author={Li, Zongyi and Yin, Shenglin and Liao, Bingyan and Bai, Yichen and He, Liangbo and Xiu, Kedong and Li, Hongcheng and Lan, Jun and Cui, Shiwen and Xu, Tingting and Song, Chuanbiao and Yu, Zijian and Hong, Yan and Li, Siyuan and Xu, Chao and Zhu, Huijia and Meng, Changhua and Wang, Weiqiang},
year={2026}
}
License
This project is licensed under the Apache-2.0 License.