Qwen3-235B-A22B (MoE) Architecture Parameters, Benchmarks & VRAM
235B total / 22B active MoE flagship suited for high-memory Mac Studios and multi-GPU servers.
🧬 Qwen3-235B-A22B (MoE) Architecture & Tensor Specifications
Verified directly against HuggingFace (Qwen/Qwen3-235B-A22B) config.json & Model Card
1. Backbone & Dimensions
qwen3_moe- Total / Active Params
- 235.1B / 22.2B
- Non-Embedding Params
- 234B
- Hidden Size (d_model)
- 4,096
- FFN Intermediate Size
- 12,288
- MoE Expert Dim
- 1,536
- MoE Experts
- 128 Routed · Top-8
- Activation / Norm
- SiLU (SwiGLU) · ε=1e-6
- Native Tensor Dtype
- BF16
2. Attention & Layer Layout
GQA- Num Hidden Layers
- 94 Layers
- Hidden Layer Layout
- 94-Layer Flagship MoE (128 Experts, 8 Activated)
- Attention Heads (Q / KV)
- 64 Q / 4 KV (16:1 GQA)
- Head Dimension
- 128
3. Context, RoPE & Tokenizer
Tokenizer- Native Context Length
- 32,768 tokens
- Max Extended Context
- 131,072 tokens
- RoPE Configuration
- RoPE (θ=1,000,000 · 原生 32K · YaRN 4x→128K)
- Vocab Size
- 151,936 (Untied)
- Special Token IDs
- BOS: 151643 · EOS: [151645,151643] · PAD: 151643
- Language Support
- 119 languages & dialects
📋 Qwen3-235B-A22B (MoE) Official API & System Card Specifications
Verified against official Alibaba Qwen developer documentation & pricing
1. Identity & Capabilities
API Spec- API Model ID
- qwen3-235b-a22b
- Release Date
- 2025-04-29
- Weight License
- Open-Weight (API Spec)
- Native Reasoning
- ✅ Supported
- Function Calling & JSON Schema
- ✅ 原生支持
- Batch API 异步批处理
- ✅ Supported (50% off)
2. Context, Output & Modalities
Limits- Max Context Window
- 128,000 tokens (128K)
- Max Output Tokens
- 16,000 tokens
- Input Modalities
- Text
- Output Modalities
- Text
- Direct China Access
- ✅ Direct CN + RMB
- Deployment Mode
- Open-Weight + Cloud API
3. Official Pricing & Cache
CNY / 1M- Input / 1M Tokens
- ¥ 2
- Cached Input / 1M
- ¥ 0.4 -80%
- Output / 1M Tokens
- ¥ 8
- Tiered Pricing Notes
- Alibaba Bailian: Hosted on Alibaba Cloud Bailian (CN Direct)
🏆 Official Benchmark Results
Sourced from official System Card / Model Card & technical reports
A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks.
An improved version of the MMLU benchmark featuring manually re-annotated questions to identify and correct errors in the original dataset. Provides more reliable evaluation metrics for language models by addressing dataset quality issues found in the original MMLU.
PhD-level Google-proof science reasoning questions
MATH-500 is a subset of the MATH dataset containing 500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions.
American Invitational Mathematics Examination 2024, consisting of 30 challenging mathematical reasoning problems from AIME I and AIME II competitions.
All 30 problems from the 2025 American Invitational Mathematics Examination (AIME I and AIME II), testing olympiad-level mathematical reasoning with integer answers from 000-999.
LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code.
Berkeley Function Calling Leaderboard v3 (BFCL-v3) is an advanced benchmark that evaluates large language models' function calling capabilities through multi-turn and multi-step interactions.
🎛️ Official Sampling & Reasoning Parameters
Recommended generation settings & API parameter constraints
💡 Do NOT use greedy decoding (temp=0); use 38,912 max tokens for competition math/code.
💡 Recommended for fast general chat; presence_penalty 0~2 reduces repetition.
📊 Qwen3-235B-A22B (MoE) Full Benchmark Scorecard
21 public benchmarks · ranks among 154 models · synced 2026-11-20
💻 Coding 3 items
LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code.
Aider is a comprehensive code editing benchmark based on 133 practice exercises from Exercism's Python repository, designed to evaluate AI models' ability to translate natural language coding requests into executable code that passes unit tests.
A rigorous code synthesis evaluation framework that augments existing datasets with extensive test cases generated by LLM and mutation-based strategies to better assess functional correctness of generated code, including HumanEval+ with 80x more test cases.
🤖 Agents & Tools 1 items
📐 Math 12 items
Grade School Math 8K, a dataset of 8.5K high-quality linguistically diverse grade school math word problems requiring multi-step reasoning and elementary arithmetic operations.
Big-Bench Hard (BBH) is a suite of 23 challenging tasks selected from BIG-Bench for which prior language model evaluations did not outperform the average human-rater.
Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains.
An improved version of the MMLU benchmark featuring manually re-annotated questions to identify and correct errors in the original dataset. Provides more reliable evaluation metrics for language models by addressing dataset quality issues found in the original MMLU.
Multilingual Massive Multitask Language Understanding dataset released by OpenAI, featuring professionally translated MMLU test questions across 14 languages including Arabic, Bengali, German, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Portuguese, Swahili…
American Invitational Mathematics Examination 2024, consisting of 30 challenging mathematical reasoning problems from AIME I and AIME II competitions.
MGSM (Multilingual Grade School Math) is a benchmark of grade-school math problems. Contains 250 grade-school math problems manually translated from the GSM8K dataset into ten typologically diverse languages: Spanish, French, German, Russian, Chinese, Japanese, Thai, Swahili…
All 30 problems from the 2025 American Invitational Mathematics Examination (AIME I and AIME II), testing olympiad-level mathematical reasoning with integer answers from 000-999.
LiveBench is a challenging, contamination-limited LLM benchmark that addresses test set contamination by releasing new questions monthly based on recently-released datasets, arXiv papers, news articles, and IMDb movie synopses.
MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions.
A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks.
SuperGPQA is a comprehensive benchmark that evaluates large language models across 285 graduate-level academic disciplines.
🧠 Reasoning & Science 3 items
Arena-Hard-Auto is an automatic evaluation benchmark for instruction-tuned LLMs consisting of 500 challenging real-world prompts curated by BenchBuilder. It includes open-ended software engineering problems, mathematical questions, and creative writing tasks.
A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
MBPP (Mostly Basic Python Problems) is a benchmark of 974 crowd-sourced Python programming problems designed to be solvable by entry-level programmers.
📚 Knowledge & General 2 items
INCLUDE is a multilingual language-understanding benchmark of 197,243 multiple-choice questions drawn from regional exams in 44 languages, built to measure regional and cultural knowledge rather than translated English content (Romanou et al., 2024).
MultiPL-E is a scalable and extensible system for translating unit test-driven code generation benchmarks to multiple programming languages.
Source: ZeroEval / llm-stats.com (✓ = third-party verified, others vendor-reported).
⚡ Qwen3-235B-A22B (MoE) Interactive VRAM & Token Speed Calculator
Physical 512 GB · Load 32.3%
MoE: reads only 13.46 GB active weights/step
Enable Q8/Q4 KV quantization to scale batch concurrency
OLLAMA_CONTEXT_LENGTH=16384 ollama run qwen3:235b-a22bMac Studio M5 Ultra 512G Fits Qwen3-235B-A22B (MoE) · Cost & TCO
Approx. $9,499 (~$19/GB VRAM) · 190W TDP
📊 Qwen3-235B-A22B (MoE) Quantization VRAM Breakdown (16K Context)
| Quantization | Bits/Weight | Weights | KV Cache (16K) | Total VRAM | Quality Note |
|---|---|---|---|---|---|
| FP16 / BF16 | 16 bpw | 470.2 GB | 3.15 GB | 483.21 GB | Lossless Full Precision |
| FP8 / Q8_0 | 8.5 bpw | 249.79 GB | 3.15 GB | 258.39 GB | 99.9% Near-Lossless |
| GGUF Q6_K | 6.56 bpw | 192.78 GB | 3.15 GB | 200.24 GB | Ultra High Quality |
| GGUF Q5_K_M | 5.69 bpw | 167.21 GB | 3.15 GB | 174.16 GB | High Balanced Quality |
| Q4_K_M / AWQ | 4.85 bpw | 142.53 GB | 3.15 GB | 148.98 GB | Sweet Spot (Recommended) |
| GGUF IQ4_XS | 4.25 bpw | 124.9 GB | 3.15 GB | 131 GB | Compact 4-bit |
| GGUF Q3_K_M | 3.91 bpw | 114.91 GB | 3.15 GB | 120.81 GB | Moderate Perplexity Loss |
| GGUF Q2_K | 3.15 bpw | 92.57 GB | 3.15 GB | 98.03 GB | Extreme Compression |
🖥️ GPU & Apple Silicon Compatibility for Qwen3-235B-A22B (MoE) (Q4_K_M)
Frequently Asked Questions
What are the exact architecture parameters of Qwen3-235B-A22B (MoE)?
Qwen3-235B-A22B (MoE) uses Qwen3MoeForCausalLM with 94 layers (94-Layer Flagship MoE (128 Experts, 8 Activated)), hidden_size=4096, 64 Q heads / 4 KV heads (head_dim=128), and vocab_size=151,936.
How much VRAM does Qwen3-235B-A22B (MoE) require?
At 4-bit quantization (Q4_K_M, 16K context), Qwen3-235B-A22B (MoE) requires 148.98 GB VRAM. Full FP16 requires 483.21 GB.