Decode roofline

How fast can your model decode?

Estimate model fit, per-session speed, and useful aggregate throughput from the memory system up.

Configuration

Start with an example or enter exact values.

Live calculation

Hardware memory

Use sustained bandwidth when you have it. Catalog bandwidth produces a looser ceiling.

Common hardwareSingle-device production runs

Usable capacityGBMemory bandwidthGB/s

Runtime overheadGB

Model weights

Choose an exact model to load its architecture automatically, then set the precision of the weights you actually run.

Exact model profileHugging Face configSelect a known model so you do not have to enter its layers, KV heads, or recurrent state manually.Common model size4,544 production runsChanges parameter count only; architecture, cache, and quantization stay under your control.

Total parametersBWeight precisionbits

Serving target

The reservation controls memory fit; active context controls decode traffic.

Allocated contexttokens/sessionActive read contexttokens/session

Minimum speedtok/s/session

Decoding method

Model ordinary autoregressive decoding or account for a speculative draft model and its acceptance rate.

Advanced architecture and KV cache

64 attention layers · 8 KV heads · 16-bit cache

Global layersLocal layersSliding windowtokensKV headsHead dimensionKV storagebitsKV readbitsFixed state storedGB/sessionFixed state trafficGB/token

Speculative decoding controls are in the Decoding method section above.