Calculator
Will this LLM fit on this GPU?
Pick a model and a GPU. gpu-fit works out vLLM’s memory budget from the model’s real architecture, estimates how many requests it can hold at your context length and roughly how fast. Everything runs in your browser.
weightsoverheadKV cachereserved (8%)
Smallest estimated setup on each GPU
Same model, context and users. The fewest GPUs of each type (1, 2, 4 or 8) that should fit, by this estimate.
| GPU | Setup | Users at full context | KV tokens | tok/s, 1 user (est.) |
|---|
Prefer the terminal? pip install gpu-fit reads most Hugging Face models (grouped-query, hybrid, MLA, sliding-window; BF16, FP8, AWQ, GPTQ). Source and method: GitHub.