ai.canshu.com
中文
VRAM Calculator Advisor & TCO API Pricing Compare Models Leaderboard GPUs

Search models, GPUs, and tools

↑↓ Navigate · Enter Open · ★ Watchlist prioritized ai.canshu.com
Q4_K_M: Out of VRAM (OOM) 126.47 GB / 78.4 GB Usable

Step 3.7 Flash (201B MoE) on NVIDIA H100 SXM 80GB: VRAM Requirement & Speed Estimate

A single H100 80G (78.4 GB usable VRAM) cannot fully fit Step 3.7 Flash (201B MoE) at Q4_K_M (126.47 GB required). Consider multi-GPU tensor parallelism, a smaller parameter variant, or cloud GPU / API deployment.

Concurrency (Batch):
MoE 201B / A11B·45L (d=4096)·64Q/8KV·Ctx 256K·Text+Image+Video
Runnable Models →
GPUs
80 GB VRAM·3350 GB/s·Safe Usable 78.4 GB
Out of VRAM (OOM)126.47 / 78.4 GBOOM
3. Weight Quantization
4. Per-Request Context16K (16,384 tok)
Verified: 2026-11-20·Roofline bandwidth/compute estimate (Batch=1)
Out of VRAM (OOM)
126.47/ 78.4 GB usable

Physical 80 GB · Load 161.3%

Theoretical Decode Speed
—tokens/s

Exceeds usable VRAM by 48.07 GB

Max Context at Batch=1
0K/ 256K

Enable Q8/Q4 KV quantization to scale batch concurrency

Weights: 121.86 GBKV Cache (1×16K): 0.88 GBOverhead: 3.74 GB
⛔ Cannot Load
Reference Launch CLI:
OLLAMA_CONTEXT_LENGTH=16384 ollama run step-3-7-flash-200b

Insufficient Local VRAM? Ways to Run Step 3.7 Flash (201B MoE)

Rent an hourly high-VRAM cloud GPU pod, or check the 12-month Buy vs Cloud vs API break-even curve.

📊 All Quantization Levels for Step 3.7 Flash (201B MoE) on H100 80G

Quantization Total VRAM (16K) Verdict Est. Speed Max Context
FP16 / BF16 412.22 GB Out of VRAM (OOM) — —
FP8 / Q8_0 220.01 GB Out of VRAM (OOM) — —
GGUF Q6_K 170.29 GB Out of VRAM (OOM) — —
GGUF Q5_K_M 148 GB Out of VRAM (OOM) — —
Q4_K_M / AWQ 126.47 GB Out of VRAM (OOM) — —
GGUF IQ4_XS 111.09 GB Out of VRAM (OOM) — —
GGUF Q3_K_M 102.38 GB Partial RAM Offload (Slow) ~45.7 tok/s —
GGUF Q2_K 82.9 GB Partial RAM Offload (Slow) ~176 tok/s —

💡 Recommended Alternative GPUs & Macs for Step 3.7 Flash (201B MoE)

Frequently Asked Questions

Can NVIDIA H100 SXM 80GB run Step 3.7 Flash (201B MoE) locally?

A single H100 80G (78.4 GB usable VRAM) cannot fully fit Step 3.7 Flash (201B MoE) at Q4_K_M (126.47 GB required). Consider multi-GPU tensor parallelism, a smaller parameter variant, or cloud GPU / API deployment.

What is the best quantization for Step 3.7 Flash (201B MoE) on H100 80G?

Step 3.7 Flash (201B MoE) (201B parameters) exceeds single-device VRAM even at aggressive 2–3 bit quantization. Cloud GPUs or hosted APIs are recommended.

How much VRAM does Step 3.7 Flash (201B MoE) need in full FP16 precision?

In unquantized FP16/BF16 precision (16K context), Step 3.7 Flash (201B MoE) requires 412.22 GB VRAM (402 GB for weights alone). Because it uses a Mixture-of-Experts (MoE) architecture, all 201B parameters must sit in memory, while only 11B active parameters are read per token.