Qwen3‑LiveTranslate: Real‑Time Multimodal Interpretation — See It, Hear It, Speak It！

DASHSCOPE

HUGGING FACE DEMO

Qwen3‑LiveTranslate‑Flash delivers high‑precision, lightning‑fast and ultra‑reliable real‑time multilingual audio and video interpretation. With the extensive capabilities of Qwen3‑Omni and training on millions of hours of multimodal data, it enables both offline and live translation in 18 languages, making cross‑language communication seamless.

Key Features:

• Multilingual and Dialect Coverage: Supports major official languages including Chinese, English, French, German, Russian, Italian, Spanish, Portuguese, Japanese, Korean, Indonesian, Thai, Vietnamese, Arabic, Hindi, Greek, Turkish; as well as dialect and accent translation for Mandarin, Cantonese, Beijing, Wu, Sichuan, and Tianjin dialects.

• Vision‑Enhanced Comprehension: For the first time, Qwen3‑LiveTranslate‑Flash incorporates visual context augmentation, enabling it to not only understand what it hears but also understand what it sees. By detecting and interpreting lip movements, gestures, on‑screen text, and real‑world entities, the system robustly handles noisy audio environments and resolves ambiguities in terms with multiple meanings.

• 3s Latency: A lightweight mixture‑of‑experts architecture, coupled with dynamic sampling, enables simultaneous interpretation with latency as low as three seconds.

• Lossless Interpretation: Utilizes semantic unit prediction to mitigate cross‑lingual reordering challenges in translation, achieving real‑time translation quality that is close to offline translation.

• Natural Voice Quality: With training on massive speech datasets, the model delivers lifelike voices whose tone and expressiveness naturally follow the meaning of the source speech.

Performance#

Qwen3‑LiveTranslate‑Flash achieves significantly higher accuracy than strong large-scale models, including Gemini‑2.5‑Flash, GPT‑4o‑Audio‑Preview, and Voxtral Small‑24B, on public benchmarks for Chinese, English and multilingual speech translation.

Qwen3‑LiveTranslate‑Flash consistently delivers leading translation performance across different domains and under challenging acoustic conditions.

Semantic unit prediction technology alleviates cross-lingual reordering issues, enabling real-time simultaneous interpretation to significantly reduce latency while maintaining over 94% of the accuracy achieved by non-real-time translation.

Visual enhancement technology further improves Qwen3-LiveTranslate-Flash’s translation precision in challenging scenarios such as noisy audio, ambiguous word meanings, and proper noun translation. In real-time settings, visual information compensates for missing speech context, making its advantages even more pronounced.

Examples#

Speech‑to‑Speech Simultaneous Translation

• Local API Test: real‑time interpretation | English → Chinese
  • Audio Source: Alibaba 2023 Q4 (December) Earnings Conference Call
  • Input Audio:*
    

    在原站查看媒体

在原站查看媒体

Vision‑Enhanced Speech Translation

• Example 1 – Homophones / Ambiguous Terms | English → Chinese

在原站查看媒体

• English Audio Input: What is mask? This is mask. This is mask. This is mask. This is Musk.

• Audio Only Output: 什么是口罩？这是口罩，这是口罩，这是口罩，这是口罩。

• Vision Enhanced Output: 什么是口罩？这是面膜，这是口罩，这是面具，这是马斯克。

• Example 2 – Low‑Frequency Proper Noun Recognition (Person Names) | Thai → Chinese

• Video Source: https://www.youtube.com/watch?v=YgGLuKdQUYk

在原站查看媒体

• Thai Audio Input: สวัสดีค่ะploy imodนะคะ พบกันเช้าวันที่17มีนาคม2024กับสรุปข่าวประจำ สัปดาห์นะคะ

• Audio Only Output: 大家好，我是Ploy Aimod。我们又见面了，今天是2024年3月17日的早晨，一起来看看本周的新闻摘要。

• Vision Enhanced Output: 大家好，我是Ploy iMod。我们于2024年3月17日早上见面，带来本周的新闻摘要。

Supported Languages#

Supported Voices#

What's Next#

We will keep advancing the accuracy, naturalness, and emotional fidelity of our speech translation; extend coverage to more languages; and reinforce its robustness across varied and challenging acoustic environments. Our goal is to bridge linguistic divides, enabling conversations to flow as smoothly and naturally as if speaking face to face.