Qwen3-VL: Sharper Vision, Deeper Thought, Broader Action

文章目录

QWEN CHAT

GITHUB

HUGGING FACE

MODELSCOPE

DISCORD

Today, we officially launch the all-new Qwen3-VL series — the most powerful vision-language model in the Qwen family to date. In this generation, we’ve made major improvements across multiple dimensions: whether it’s understanding and generating text, perceiving and reasoning about visual content, supporting longer context lengths, understanding spatial relationships and dynamic videos, or interacting with AI agents — Qwen3-VL shows clear and significant progress in every area.

Firstly, we are open-sourcing the flagship model of this series: Qwen3-VL-235B-A22B, available in both Instruct and Thinking versions. The Instruct version matches or even exceeds Gemini 2.5 Pro in major visual perception benchmarks. The Thinking version achieves state-of-the-art results across many multimodal reasoning benchmarks.


The goal of Qwen3-VL is not just to "see" images or videos — but to truly understand the world, interpret events, and take action. To achieve this, we’ve systematically upgraded key capabilities, pushing visual models from simple “perception” toward deeper "cognition", and from basic "recognition" to advanced “reasoning and execution.”

Key Highlights:#

  • Visual Agent Capabilities: Qwen3-VL can operate computer and mobile interfaces — recognize GUI elements, understand button functions, call tools, and complete tasks. It achieves top global performance on benchmarks like OS World, and using tools significantly improves its performance on fine-grained perception tasks.

  • Superior Text-Centric Performance: Qwen3-VL employs early-stage joint pretraining of text and visual modalities, continuously strengthening its language capabilities. Its performance on text-based tasks matches that of Qwen3-235B-A22B-2507 — the flagship language model — making it a truly “text-grounded, multimodal powerhouse” for the next generation of vision-language models.

  • Greatly Improved Visual Coding: It can now generate code from images or videos — for example, turning a design mockup into Draw.io, HTML, CSS, or JavaScript code — making “what you see is what you get” visual programming a reality.

  • Much Better Spatial Understanding: 2D grounding from absolute coordinates to relative coordinates. It can judge object positions, viewpoint changes, and occlusion relationships. It supports 3D grounding, laying the foundation for complex spatial reasoning and embodied AI applications.

  • Long Context & Long Video Understanding: All models natively support 256K tokens of context, expandable up to 1 million tokens. This means you can input hundreds of pages of technical documents, entire textbooks, or even two-hour videos — and the model will remember everything and retrieve details accurately, down to the exact second in videos.

  • Stronger Multimodal Reasoning (Thinking Version): The Thinking model is specially optimized for STEM and math reasoning. When facing complex subject questions, it can notice fine details, break down problems step by step, analyze cause and effect, and give logical, evidence-based answers. It achieves strong performance on reasoning benchmarks like MathVision, MMMU, and MathVista.

  • Upgraded Visual Perception & Recognition: By improving the quality and diversity of pre-training data, the model can now recognize a much wider range of objects — from celebrities, anime characters, products, and landmarks, to animals and plants — covering both everyday life and professional “recognize anything” needs.

  • Better OCR Across More Languages & Complex Scenes: OCR now supports 32 languages (up from 10), covering more countries and regions. It performs more reliably under challenging real-world conditions like poor lighting, blur, or tilted text. Recognition accuracy for rare characters, ancient scripts, and technical terms has also improved significantly. Its ability to understand long documents and reconstruct fine structures is further enhanced.


Model Performance#

We comprehensively evaluated the model's visual capabilities across ten dimensions, covering university-level comprehensive problems, mathematical and scientific reasoning, logic puzzles, general visual question answering, subjective experience and instruction following, multilingual text recognition and chart/document parsing, 2D/3D object grounding, multi-image understanding, embodied and spatial perception, video understanding, agent task execution, and code generation. Overall, Qwen3-VL-235B-A22B-Instruct achieves top performance on most metrics among non-reasoning models, significantly outperforming closed-source models such as Gemini 2.5 Pro and GPT-5, while also setting new state-of-the-art results among open-source multimodal models, demonstrating its strong generalization ability and comprehensive performance on complex visual tasks.

In terms of reasoning models, Qwen3-VL-235B-A22B-Thinking also sets new records on most metrics among open-source multimodal models, showing competitive performance against top closed-source models like Gemini 2.5 Pro and GPT-5. Notably, on complex multimodal math problems such as Mathvision, it even outperforms Gemini 2.5 Pro. Although it still lags behind closed-source SOTA models in multidisciplinary problems, visual reasoning, and video understanding, it demonstrates clear advantages in agent capabilities, document understanding, and 2D/3D grounding tasks.

On text-centric tasks, both Qwen3-VL-235B-A22B-Instruct and Qwen3-VL-235B-A22B-Thinking demonstrate strong performance, comparable to the language model Qwen3-235B-A22B-2507.

Additionally, Qwen3-VL-235B-A22B-Instruct supports image-based reasoning with tool use. We tested this on four fine-grained perception and embodied interaction tasks. Results show consistent performance gains across all four benchmarks — proving that combining “image analysis + tool calling” is key to boosting visual perception.

In long-context testing, we ran a “needle-in-a-haystack” experiment using ultra-long videos. Results show 100% accuracy with 256K context length. Even when expanded to 1M tokens, equivalent to approximately two hours of continuous video, accuracy remains at 99.5% — demonstrating extremely strong long-sequence modeling ability.

To test multilingual text recognition beyond Chinese and English, we built a test set with 39 languages. The model achieves over 70% accuracy in 32 of them — reaching practical usability and showing strong multilingual generalization.

Model Updates#

In terms of architecture, we still adopt the native dynamic resolution design, but we have updated the structural design in three aspects:

First, we employ Interleaved-MRoPE. The original MRoPE divides the feature dimensions into blocks along the order of time (t), height (h), and width (w), causing all temporal information to be concentrated in high-frequency dimensions. In Qwen3-VL, we instead distribute t, h, and w in an interleaved manner, achieving full-frequency coverage across time, height, and width. This more robust positional encoding ensures comparable image understanding capability while significantly improving long video comprehension.

Second, we introduce DeepStack technology to fuse multi-level features from ViT, enhancing visual detail capture and text-image alignment accuracy. Following the core idea of DeepStack, we shift from the conventional multimodal large model (LMM) practice of injecting visual tokens into only a single layer, to injecting them across multiple layers of the large language model (LLM). This multi-layer injection approach enables finer-grained visual understanding. Furthermore, we optimize the visual feature tokenization strategy: visual features extracted from different ViT layers are tokenized and used as visual inputs. This design effectively preserves rich visual information ranging from low-level to high-level features. Experimental results demonstrate significant performance improvements across various visual understanding tasks.

Third, we upgrade the original video temporal modeling mechanism, T-RoPE, to a text-timestamp alignment mechanism. This mechanism adopts an interleaved input format of “timestamps-video frames”, enabling fine-grained alignment between frame-level temporal information and visual content. Additionally, the model natively supports two time output formats: “seconds” and “hours:minutes:seconds” (HMS). This improvement substantially enhances the model’s semantic perception and temporal localization accuracy for actions and events in videos, resulting in more robust performance and precise responses in complex temporal reasoning tasks—such as event localization, action boundary detection, and cross-modal temporal question answering.

Model Capabilities#

1. Visual Agent#

Qwen3-VL can not only understand images but also operate smartphones and computers like a human, automating many daily tasks. For example, it can open apps, tap buttons, or fill in forms—enabling intelligent interaction and automation.


2. Think with Images#

Qwen3-VL can carefully examine image details—just like humans—and use tools to perform complex reasoning. For instance, it can determine location by reading street signs or identify people from partial photos, completing fine-grained recognition and logical analysis tasks.


3. Code Programming#

By combining visual understanding with code generation, Qwen3-VL shows strong potential in frontend development. It can turn hand-drawn sketches into web pages or help debug UI issues, improving development efficiency.


4. Spatial Understanding#

Qwen3-VL can interpret spatial relationships from images and videos, recognizing directions, actions, and motion patterns. It can also plan future movements—laying the foundation for robotics, autonomous navigation, and embodied AI applications.


5. 2D/3D Grounding#

Qwen3-VL excels at locating objects in complex scenes with multiple items, accurately generating up to hundreds of detection boxes. The coordinate representation of 2D grounding changes from absolute coordinates to relative coordinates. It also supports 3D bounding box prediction, estimating the real-world position, size, and depth of objects.


6. Universal Recognition#

Rich visual knowledge is key to understanding the world. Qwen3-VL can accurately recognize celebrities, food, plants, animals, car brands, and anime characters—providing practical assistance in everyday life, education, and social interactions.


7. Creative Writing#

Qwen3-VL’s writing ability has improved significantly. It can generate vivid descriptions based on images or videos, supporting storytelling, copywriting, and video script creation.


8. STEM and Multi-Disciplinary Problem Solving#

Qwen3-VL shows stronger capabilities in solving problems across math, physics, chemistry, and other STEM fields. It understands questions, performs step-by-step reasoning, verifies results, and effectively solves academic and real-world scientific challenges.


9. Complex Instruction Following#

Qwen3-VL better understands complex instructions. Even when faced with multi-step, conditional, or highly structured requests, it can interpret them accurately and complete tasks as intended.


10. Complex Document Understanding & General Parsing#

Qwen3-VL has enhanced support for long and multi-page documents. It can clearly understand extremely wide web layouts or PDFs with dozens of scanned pages. In addition to supporting QwenVL HTML parsing, it now introduces a new QwenVL Markdown format that preserves text, formulas, tables, and image positions using fewer tokens—improving processing speed and accuracy.


11. Multilingual OCR & Visual Question Answering#

Qwen3-VL now supports OCR in 32 languages—up from 10—including Greek, Hebrew, Hindi, Romanian, and Thai. This improvement helps meet diverse regional needs and enables smoother multilingual conversations and document processing.


12. Multi-Image and Multi-Round Dialogue#

Qwen3-VL handles multiple images more effectively—comparing differences, finding connections, and maintaining context across turns. It performs well in extended conversations, remembering earlier discussions and building upon them meaningfully.


13. Video Understanding#

Qwen3-VL has stronger video comprehension, especially in pinpointing events over time and understanding long videos. It can describe content frame-by-frame and answer questions accurately—even for videos lasting over an hour and a half. It can even watch gameplay videos and generate corresponding game code.

API Usage#

To experience the most powerful model in the Qwen3-VL series, Qwen3-VL-235B-A22B, you can directly use our official API and obtain your DASHSCOPE_API_KEY here. Below is a usage example:}}

You can find more usage examples here.

Summary#

Qwen3-VL not only matches or even surpasses current top-tier closed-source models across multiple multimodal benchmarks, but more importantly, it genuinely integrates powerful text-only capabilities with visual understanding, eliminating any bias toward one modality. We have made substantial improvements in key areas including visual perception, spatial reasoning, long video comprehension, multilingual recognition, tool invocation, and code generation—enabling the model not just to "see", but to truly “understand, reason, and act". The Qwen3-VL-235B-A22B model we are open-sourcing today aims to serve as a solid stepping stone for the community’s exploration of large models. Moving forward, we will continue striving to enable multimodal large models to genuinely comprehend events and relationships in the real world, ultimately functioning as reliable assistants capable of proactively completing tasks and solving problems in both digital and physical environments.

原文来自 千问 Qwen Team,版权归原作者。

正文采集于 ,内容更新以原文为准。