LLM VRAM Calculator
Estimate how much GPU memory a large language model needs for inference (weights, KV cache and runtime overhead) and which GPUs can hold it alone or in 2, 4 or 8 cards.
Weights-
KV cache-
Overhead-
Total-
| GPU | Memory per GPU | GPUs needed |
|---|
Formulas
Weights (GB) = billions of parameters × bytes per parameter (4 in FP32, 2 in BF16/FP16, 1 in FP8/INT8, 0.5 in INT4)
KV cache (GB) = 2 × layers × KV dimension × context tokens × batch × 2 bytes / 10^9
Total = (weights + KV cache) × (1 + overhead %)
Presets assume a typical architecture for each size. Multi-GPU totals do not include the cost of splitting the model across cards.
Full calculator with live GPU prices on EverXCloud