蚂蚁 inclusionAI 开源 Ming-3.0-asr-tiny 语音识别模型

Context-aware speech recognition for Chinese, English, and dialects.

English · 简体中文

GitHub code and cookbook · Quick Start · Evaluation

Ming-3.0-asr-tiny combines the Ling-tiny-v3 language backbone with an audio encoder. It transcribes single-turn Chinese and English speech, covers regional dialects, and can use hotwords or preceding dialogue to improve recognition of domain-specific terms. This repository includes the model weights, tokenizer, custom Transformers implementation, and runnable inference example.

Capabilities

• Chinese and English ASR: speech-to-text transcription through one model interface.
• Dialect recognition: evaluated on 15 regional dialect test sets, including Cantonese, Minnan, and Shanghainese.
• Context ASR: accepts relevant hotwords or dialogue context as part of the transcription prompt.
• Challenging speech: evaluated on noisy speech and five application domains.

Architecture

Quick Start

Download and install

Use Python 3.12 on a Linux machine with a compatible CUDA GPU. Download the complete checkpoint and install its dependencies, including PyTorch and torchaudio:

python -m pip install "huggingface_hub[cli]"
hf download inclusionAI/Ming-3.0-asr-tiny --local-dir Ming-3.0-asr-tiny
cd Ming-3.0-asr-tiny
python -m pip install -r requirements.txt

Run the PyTorch / Transformers example

python examples/transcribe.py \
  --model . \
  --audio /path/to/speech.wav

Add --language-hint zh or --language-hint en when the language is known. The example uses CUDA, BF16, SDPA, greedy decoding, and up to 256 new tokens; it loads the bundled model code with trust_remote_code=True. The dependency versions are listed in requirements.txt, including Transformers 5.12.1, fla-core 0.5.2, and PyTorch 2.11.0. For another CUDA version, first install matching PyTorch and torchaudio 2.11.0 wheels using the official PyTorch instructions, then run pip install -r requirements.txt.

PyTorch / Transformers Python API

Run this directly from the downloaded model directory with your own speech.wav. It is the same inference path as the bundled executable example:

import soundfile as sf
import torch
from transformers import AutoModelForCausalLM, AutoProcessor
from asr_request import decode_transcript, prepare_asr_request

processor = AutoProcessor.from_pretrained(".", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    ".", trust_remote_code=True, dtype=torch.bfloat16,
    device_map="cuda:0", attn_implementation="sdpa",
).eval()
waveform, rate = sf.read("speech.wav", dtype="float32", always_2d=True)
inputs = prepare_asr_request(
    processor, (waveform.T, rate), return_tensors="pt",
).to(model.device)
eos = model.config.text_config.eos_token_id
with torch.inference_mode():
    output = model.generate(
        **inputs, max_new_tokens=256, do_sample=False,
        eos_token_id=eos, pad_token_id=eos, no_repeat_ngram_size=10,
    )
print(decode_transcript(
    processor.tokenizer, output[0, inputs.input_ids.shape[1]:].tolist(),
))

Context ASR

Ordinary transcription and Context ASR use the same input format. Append the topic and hotwords directly to the base instruction, without a newline. For known-language clips up to 60 seconds, --context-chunks selects 3000 mel frames up to 30 seconds and 6000 beyond 30 seconds:

python examples/transcribe.py --model . --audio /path/to/speech.wav \
  --language-hint zh --context-chunks \
  --prompt $'Please recognize the language of this speech and transcribe it, placing a `\t` between the identified language and the transcribe.This is an audio about 电影蜥蜴.This audio may contains the following words or phrases:蜥蜴,姜知垠,曹承佑,姜惠贞,八音盒.'

For English, use --language-hint en. The topic and hotwords above are illustrative. Dialogue context can also be included in the prompt text. For Python calls, pass the full prompt through prepare_asr_request(..., prompt=...).

SGLang serving

For GPU serving, see the Ming SGLang adapter and the ASR serving guide, which includes pinned adapter patches, environment setup, a server launcher, and client examples. Once the server is running, transcribe a WAV file through its audio endpoint:

curl --fail-with-body --max-time 600 \
  http://127.0.0.1:8000/v1/audio/transcriptions \
  -F 'file=@/path/to/speech.wav' -F 'model=ming-asr' \
  -F 'response_format=json' -F 'temperature=0'

The response contains a text field. The cookbook also covers dataset evaluation settings.

Evaluation

ASR Overview

Watch the Ming-3.0-asr-tiny release film

Chinese and English ASR

Dialects and Domains

Context ASR

Usage Notes

• Output is transcription text, without word-level timestamps or speaker diarization.
• The included example does not support streaming input; image and video inputs are unsupported.
• Long audio can be encoded in chunks with --audio-chunk-frames, subject to the model context limit.
• Hotwords are hints, not a guarantee that a term will appear in the transcript.

For local Transformers inference, run the bundled PyTorch example.

License

The model weights and original project code are released under the MIT License, following the Ming project. Bundled Apache-2.0 source files retain their own headers and license text. modeling_whisper_encoder.py retains the MIT notice for code adapted from OpenAI Whisper.

Citation

If you use Ming-3.0-asr-tiny, cite the GitHub repository. Citation metadata is also available in CITATION.cff.

@misc{ming_3_0_asr_tiny_2026,
  author = {{inclusionAI}},
  title = {Ming-3.0-asr-tiny: Context-Aware Speech Recognition},
  year = {2026},
  url = {https://github.com/inclusionAI/Ming-3.0-asr}
}