LocalAI Cost — Buy the card or rent the tokens?
Model
Total params (B)
Active params (B)
KV per 1K tokens (GB)
Quant
FP8 (1.0)
Q8_0 (~1.06)
Q6_K (~0.75)
Q5_K_M (~0.69)
Q4_K_M (~0.6)
MXFP4 (~0.55)
KV cache
BF16 KV (1×)
FP8 KV (0.5×)
Hosted in (USD / M tok)
Hosted out (USD / M tok)
Cache discount (%)
Your usage
Usage hours / week
Electricity (USD/kWh)
Working context
128K
256K
512K
1M
Cache misses (%)
Response tokens / turn
Tool output tokens / turn
Measured t/s (override, blank = estimate)
t/s override applies to every card row. Working context is prefilled from the model preset — edit for your real value.
Card
VRAM, GB (used / total)
Bandwidth, TB/s
Verdict
t/s
First token, min
Agents
Fresh tokens / week (1 / agents)
Break-even, y (1 / agents)
Price
Accuracy