Q4_K_M: Out of VRAM (OOM) 196.03 GB / 94.1 GB Usable
Xiaomi MiMo-V2.6-Flash (311B MoE) on NVIDIA RTX PRO 6000 Blackwell 96GB: VRAM Requirement & Speed Estimate
A single RTX PRO 6000 96G (94.1 GB usable VRAM) cannot fully fit Xiaomi MiMo-V2.6-Flash (311B MoE) at Q4_K_M (196.03 GB required). Consider multi-GPU tensor parallelism, a smaller parameter variant, or cloud GPU / API deployment.
Concurrency (Batch):
MoE 311B / A12B·48L (d=4096)·64Q/4KV·Ctx 1M·Text+Image+Video
Out of VRAM (OOM)196.03 / 94.1 GBOOM· Q4_K_M