LOCAL AI · SPEED PLANNER v2.0

How fast would AI run on your own hardware?

Select your config to see how it will run. No jargon. The results panel on the right stays visible as you scroll.

01What will you use it for?

This matters more than you may realise. AI reads your request before it writes an answer, which includes prompt injections, as well as codebases or documents which can be large. Agentic harnesses like Hermes or Openclaw silently include in each prompt instruction files (AGENTS.md, system prompts, tool definitions, skills) into every single exchange - easily 10K–60K tokens before you've typed a word.

Framework / context injection — how much invisible overhead does your tool add?

8K tokens
500 tokens
32K tokens
02Choose a brain

Bigger models are smarter but heavier. Clever "MoE" models only use a slice of themselves per word, so they punch above their weight.

Model

Uses MLA memory compression (DeepSeek-style — much smaller memory footprint)

Compression ("quantization")

03Choose a machine

Two things decide speed: memory bandwidth (how fast it can read the model — controls writing speed) and compute (controls thinking time). Memory size decides what fits at all.

Device family

Memory
—
Bandwidth
—
per unit · writing speed
Compute
—
per unit · thinking speed
Power
—
under load
Blackwell-era FP4 hardware is already reflected in the device's bandwidth/compute — no extra toggle needed.
▶ Advanced knobs SAFE TO IGNORE — SENSIBLE DEFAULTS
Multi-token drafting (MTP)
The model sketches several words ahead, then checks its own work — roughly 2× faster writing with no quality loss. Supported on newer models.
High-performance server (vLLM)
Free, standard serving software that squeezes ~1.5× more speed out of NVIDIA chips. (No effect on Apple silicon.)
Machine preset: each device loads its typical real-world config — a DGX Spark defaults to MTP + vLLM on, other hardware to off. Switching device reloads the preset; you can tweak from there.
Compressed short-term memory (KV Q4)
Shrinks the model's running memory of your conversation to a quarter of the size — lets you fit much longer documents, at a tiny quality cost.
Prefix caching
Remembers the part of your prompt that never changes (like your agent's instructions) instead of re-reading it every single step. Huge time-saver for agents.
People (or agents) using it at once
One machine can serve several users at the same time. Each person gets slightly slower responses, and the model's memory eventually limits how many fit.
1
▶ How the math works FOR THE CURIOUS
WRITING SPEED (memory-bandwidth bound) words/sec = (bandwidth × split-efficiency ÷ active model size) × hardware-efficiency × compression-factor × drafting-boost THINKING SPEED (compute bound) tokens read/sec = (TFLOPS × split-efficiency × 1000 ÷ (2 × active params)) × hardware-efficiency × compression-factor ...degrading beyond 32K of context as attention gets expensive MEMORY model weights + conversation memory ≤ 85% of total RAM (if the weights don't fit, layers spill to the CPU — very slow) Calibrated against published real-world measurements; typical accuracy ±15%.
04See it in action

This uses the exact setup from above. Press run and watch how it would actually feel, in real time. The prefill (reading) and decode (writing) phases are shown separately so you can see where time goes.

—
LOCAL MODEL — LIVE SIMULATION
You:
READING YOUR REQUEST…
REAL TIME — IF IT'S SLOW, THAT'S THE POINT
Thinking time
—
before the first word appeared
Writing speed
—
Total
—
start to finish

Reference: what speeds feel like

SpeedExperience
05The money case

Cloud AI rents intelligence by the token. Local AI is a machine you buy once — and it's yours.

Private
Your prompts never leave the room. No logs, no training on your data, no breach that can expose them.
No rate limits
No sudden '5 hour window' service denials while in middle of doing great work!
Works offline
Plane, cabin, outage, apocalypse. If it has power, it works. No subscription can be revoked.

Country (electricity rate)

Heavy use (hours / day)

Using your Planner setup:

Your electricity · per 1M tokens
—
power draw only — hardware purchase not included
Cloud API · per 1M tokens
—
—
YOUR ELECTRICITY COST (monthly)—
CLOUD API COST (same usage)—

Roughly tokens processed per month at your usage. Local electricity cost only — the hardware is yours regardless. Even where cloud is cheaper on paper, local buys you privacy, no rate limits, and independence.

All cloud services (sorted by price)

Link copied — share it anywhere