Q4_K_M: Partial RAM Offload (Slow) 17.92 GB / 15.7 GB Usable
Gemma-4-26B-A4B-IT (MoE) on NVIDIA GeForce RTX 4070 Ti Super 16GB: VRAM Requirement & Speed Estimate
A single RTX 4070 Ti Super (15.7 GB usable VRAM) cannot fully fit Gemma-4-26B-A4B-IT (MoE) at Q4_K_M (17.92 GB required). Consider multi-GPU tensor parallelism, a smaller parameter variant, or cloud GPU / API deployment.
Concurrency (Batch):
MoE 26B / A4B·30L (d=2816)·16Q/2KV·Ctx 256K·Text+Image+Video
Partial RAM Offload (Slow)17.92 / 15.7 GB~101.6 tok/s· Q4_K_M