Calculator

Will this LLM fit on this GPU?

Pick a model and a GPU. gpu-fit works out vLLM’s memory budget from the model’s real architecture, estimates how many requests it can hold at your context length and roughly how fast. Everything runs in your browser.

Request slots (--max-num-seqs) follow “users at once”. Overhead is calibrated on measured vLLM runs (L40S, vLLM 0.31).

weightsoverheadKV cachereserved (8%)

        
      

Smallest estimated setup on each GPU

Same model, context and users. The fewest GPUs of each type (1, 2, 4 or 8) that should fit, by this estimate.

GPUSetupUsers at full contextKV tokenstok/s, 1 user (est.)

Prefer the terminal? pip install gpu-fit reads most Hugging Face models (grouped-query, hybrid, MLA, sliding-window; BF16, FP8, AWQ, GPTQ). Source and method: GitHub.