LOCAL INFERENCE / MEMORY PLANNING

Will your model fit?

A transparent VRAM estimate for quantized weights, KV cache, runtime overhead and breathing room. No benchmarks. No uploads.

Everything runs in your browser

Your setup

Hardware GPU memory Available VRAM (GiB)Use GiB (1 GiB = 1,073,741,824 bytes).

Model Checkpoint preset

Total parameters (B)Transformer layersKV headsUse KV heads, not attention heads (GQA).Head dimension

For MoE, enter total parameters—not active parameters. All experts occupy weight memory.

Runtime

Weight quantizationContext length (tokens)Batch / concurrent sequencesKV cache dtypeRuntime overhead (GiB)CUDA/runtime buffers, graph workspace, allocator slack.Planning headroom (%)

Block costs are lower bounds, not mixed Q_K_M files. For Q4_K_M / Q5_K_M use measured custom bpw. No universal 15% margin guarantees coverage.

Context includes generated tokens. INT8 KV excludes scales and padding.

Estimate

Block costs are lower bounds, not mixed Q_K_M files. Use measured custom bpw for mixed files.

——

Estimated required VRAM range—

——

Planning estimate, not a guarantee. Actual usage varies by runtime, kernels, allocator and GPU.

Planning band, not a confidence interval.

Where it goes

Quantized weights——

KV cache——

Runtime overheadYour planning allowance—

With headroom——

KV cache = layers × 2 (K+V) × KV heads × head dimension × context × batch × bytes per KV value.

Method, in plain English

Weights use total parameters and the effective bits-per-weight of the selected format, using single-block costs, not Q_K_M mixed recipes. Use measured effective bpw for mixed files. KV cache grows linearly with context and concurrent sequences. GQA uses the model's KV-head count.

MLA and other architectures whose KV layout is not represented here are not supported. This tool does not benchmark speed or predict quality.

Presets & sources

Llama 3.1 8B config — 32 × 8 × 128; 8.030261248B.
Qwen2.5-7B config — 28 × 4 × 128; 7.615616512B.
Mistral 7B v0.3 config — 32 × 8 × 128; 7.248023552B.
ggml block layout

Total parameter evidence: Llama · Qwen · Mistral

Model repositories are linked for verification. Total parameters are from public safetensors metadata checked 2026-09-15. Qwen/Mistral configs checked; gated Llama config was not re-fetched.

Privacy

Calculations stay in this page. No analytics or model uploads. Share query settings are sent to the static host when opened and may appear in logs. Do not put secrets in URLs.