LLM VRAM Calculator
donaldJJ · Kit
Estimates how much GPU memory a Hugging Face model needs at any quantization and context length, split into weights, KV cache and overhead, and shows which GPU…
New
donaldJJ · Kit
Estimates how much GPU memory a Hugging Face model needs at any quantization and context length, split into weights, KV cache and overhead, and shows which GPU…