ai.canshu.com
中文
VRAM Calculator Advisor & TCO API Pricing Compare Models Leaderboard GPUs

Search models, GPUs, and tools

↑↓ Navigate · Enter Open · ★ Watchlist prioritized ai.canshu.com
Google DeepMind
Open-Weight Google DeepMind · MoE 26B / A4B · Text+Image+Video · BF16

Gemma-4-26B-A4B-IT (MoE) Architecture Parameters, Benchmarks & VRAM

Google's 26B total / 4B active multimodal MoE; fits easily inside 24GB VRAM with blazing token generation speed.

🧬 Gemma-4-26B-A4B-IT (MoE) Architecture & Tensor Specifications

Verified directly against HuggingFace (google/gemma-4-26B-A4B-it) config.json & Model Card

HF Class: Gemma4ForConditionalGeneration

1. Backbone & Dimensions

gemma4
Total / Active Params
26B / 4B
Hidden Size (d_model)
2,816
FFN Intermediate Size
2,112
MoE Expert Dim
704
MoE Experts
128 Routed · Top-8
Activation / Norm
GELU-Tanh (GeGLU) · ε=1e-6
Native Tensor Dtype
BF16

2. Attention & Layer Layout

HYBRID
Num Hidden Layers
30 Layers (5 Global KV)
Hidden Layer Layout
30 Hybrid MoE Layers: 25 Sliding (w=1024) + 5 Global Full Attention, 128 Experts
Attention Heads (Q / KV)
16 Q / 2 KV (8:1 GQA)
Head Dimension
512
Linear / KV Compression
Only 5 global layers scale with 256K context; 25 layers capped at 1024 window
Vision Encoder / MTP
ViT 27L

3. Context, RoPE & Tokenizer

Tokenizer
Native Context Length
262,144 tokens
RoPE Configuration
Dual RoPE (Global Proportional θ=1,000,000 partial=0.25 · Sliding θ=10,000)
Vocab Size
262,144 (Tied)
Special Token IDs
BOS: 2 · EOS: [1,106,50] · PAD: 0
Language Support
140+ languages pre-trained

📋 Gemma-4-26B-A4B-IT (MoE) Official API & System Card Specifications

Verified against official Google DeepMind developer documentation & pricing

API Model ID: google/gemma-4-26b-a4b-it

1. Identity & Capabilities

API Spec
API Model ID
google/gemma-4-26b-a4b-it
Release Date
2026-04-02
Weight License
Open-Weight (API Spec)
Native Reasoning
✅ Supported
Function Calling & JSON Schema
✅ 原生支持
Batch API 异步批处理
✅ Supported (50% off)

2. Context, Output & Modalities

Limits
Max Context Window
256,000 tokens (256K)
Max Output Tokens
236,000 tokens
Input Modalities
Text + Image + Video
Output Modalities
Text
Direct China Access
✅ Direct CN + RMB
Deployment Mode
Open-Weight + Cloud API

3. Official Pricing & Cache

USD / 1M
Input / 1M Tokens
$ 0.09
Cached Input / 1M
$ 0.05 -44%
Output / 1M Tokens
$ 0.3
Tiered Pricing Notes
OpenRouter: Unified multi-provider routing via OpenRouter

🏆 Official Benchmark Results

Sourced from official System Card / Model Card & technical reports

Arena Elo ~1442 AA Index 42
📚 Knowledge
MMLU-Pro 82.6 %

A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks.

📚 Knowledge
MMMLU 86.3 %

Multilingual Massive Multitask Language Understanding dataset released by OpenAI, featuring professionally translated MMLU test questions across 14 languages including Arabic, Bengali, German, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Portuguese, Swahili…

🧠 Reasoning
AIME 2026 (no tools) 88.3 %
🧠 Reasoning
GPQA Diamond 82.3 %

PhD-level Google-proof science reasoning questions

💻 Coding
LiveCodeBench v6 77.1 %

LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code.

🤖 Agentic
TAU2-Bench 68.2 %

Dual-control conversational tool-use benchmark

👁️ Multimodal
MMMU-Pro 73.8 %

A more robust multi-discipline multimodal understanding benchmark that enhances MMMU through a three-step process: filtering text-only answerable questions, augmenting candidate options, and introducing vision-only input settings.

👁️ Multimodal
MATH-Vision 82.4 %

🎛️ Official Sampling & Reasoning Parameters

Recommended generation settings & API parameter constraints

Official Default Multimodal & Thinking
temperature=1 top_p=0.95 top_k=64 max_tokens=32,768

💡 Supports interleaved multimodal input, built-in thinking mode, and native function calling.

📊 Gemma-4-26B-A4B-IT (MoE) Full Benchmark Scorecard

13 public benchmarks · ranks among 154 models · synced 2026-11-20

⚔️ Compare head-to-head

Source: ZeroEval / llm-stats.com (✓ = third-party verified, others vendor-reported).

⚡ Gemma-4-26B-A4B-IT (MoE) Interactive VRAM & Token Speed Calculator

Concurrency (Batch):
MoE 26B / A4B·30L (d=2816)·16Q/2KV·Ctx 256K·Text+Image+Video
Runnable Models →
GPUs
32 GB VRAM·1792 GB/s·Safe Usable 31.4 GB
Smooth Fit17.92 / 31.4 GB~420 tok/s
3. Weight Quantization
4. Per-Request Context16K (16,384 tok)
Verified: 2026-11-20·Roofline bandwidth/compute estimate (Batch=1)
Smooth Fit
17.92/ 31.4 GB usable

Physical 32 GB · Load 57.2%

Theoretical Decode Speed
~420tokens/s

MoE: reads only 2.42 GB active weights/step

Max Context at Batch=1
256K/ 256K

Enable Q8/Q4 KV quantization to scale batch concurrency

Weights: 15.76 GBKV Cache (1×16K): 0.55 GBOverhead: 1.62 GB
🚀 Blazing Fast (Instant CoT)~1.2s for a 500-token response
Reference Launch CLI:
OLLAMA_CONTEXT_LENGTH=16384 ollama run gemma4:26b-a4b

RTX 5090 Fits Gemma-4-26B-A4B-IT (MoE) · Cost & TCO

Approx. $2,199 (~$69/GB VRAM) · 575W TDP

📊 Gemma-4-26B-A4B-IT (MoE) Quantization VRAM Breakdown (16K Context)

Quantization Bits/Weight Weights KV Cache (16K) Total VRAM Quality Note
FP16 / BF16 16 bpw 52 GB 0.55 GB 54.89 GB Lossless Full Precision
FP8 / Q8_0 8.5 bpw 27.63 GB 0.55 GB 30.02 GB 99.9% Near-Lossless
GGUF Q6_K 6.56 bpw 21.32 GB 0.55 GB 23.59 GB Ultra High Quality
GGUF Q5_K_M 5.69 bpw 18.49 GB 0.55 GB 20.71 GB High Balanced Quality
Q4_K_M / AWQ 4.85 bpw 15.76 GB 0.55 GB 17.92 GB Sweet Spot (Recommended)
GGUF IQ4_XS 4.25 bpw 13.81 GB 0.55 GB 15.93 GB Compact 4-bit
GGUF Q3_K_M 3.91 bpw 12.71 GB 0.55 GB 14.81 GB Moderate Perplexity Loss
GGUF Q2_K 3.15 bpw 10.24 GB 0.55 GB 12.29 GB Extreme Compression

🖥️ GPU & Apple Silicon Compatibility for Gemma-4-26B-A4B-IT (MoE) (Q4_K_M)

Frequently Asked Questions

What are the exact architecture parameters of Gemma-4-26B-A4B-IT (MoE)?

Gemma-4-26B-A4B-IT (MoE) uses Gemma4ForConditionalGeneration with 30 layers (30 Hybrid MoE Layers: 25 Sliding (w=1024) + 5 Global Full Attention, 128 Experts), hidden_size=2816, 16 Q heads / 2 KV heads (head_dim=512), and vocab_size=262,144.

How much VRAM does Gemma-4-26B-A4B-IT (MoE) require?

At 4-bit quantization (Q4_K_M, 16K context), Gemma-4-26B-A4B-IT (MoE) requires 17.92 GB VRAM. Full FP16 requires 54.89 GB.