LocalAI-functioncall-qwen2.5-7b-v0.5-Q4_K_M-GGUF — 156.5 tok/s on RTX 3090 · 48 GB ×2 | LocalMaxxing

Get started Leaderboard Decode calculator Models Organizations Reports Matrix runs Hardware Benchmarks Marketplace Rentals Pro API Docs

Language

Observed performance

Run snapshot

All runs matching the filters below.

Fastest run

Median speed

Memory ceiling

Slowest run

Measured runs

Community leaderboard

Speed test results

Compare real runs across hardware, engines, contexts, and quantizations.

Submit speed test

Observed performance

Run snapshot

All runs matching the filters below.

Fastest run

Median speed

Memory ceiling

Slowest run

Measured runs

#	Hardware	Engine	Depth	tok/s out	Prefill	tok/s total	TTFT ms	VRAM GB

LocalAI-functioncall-qwen2.5-7b-v0.5-Q4_K_M-GGUF — 156.5 tok/s on RTX 3090 · 48 GB ×2 | LocalMaxxing