Local LLM benchmark
Compare both SmolLM2 models with identical English and Portuguese prompts. Compare a strict WebGL2 run with a CPU baseline and check their numerical results.
Reproducible workload
135M: Q4 float256, Q4F16, INT8 and FP16. 360M: Q4 float256, Q4F16 and INT8. INT8 runs on CPU; other variants compare CPU and strict WebGL. One warmup and two measured runs per prompt, greedy decoding, at most 32 new tokens. Uncached variants download their weights; cached variants use IndexedDB. Each generation starts with a fresh KV cache.
WebGL2 uses dedicated SmolLM2 shaders and keeps its attention cache on the GPU. Each GPU case first checks both full benchmark prompts, decode steps, and reset against CPU results. Unsupported hardware or failed validation is reported as a failed case. INT8 currently uses the CPU because its WebGL output did not pass the numerical checks. The 360M INT8 model is a 363 MB download on its first run. Measured speed depends on your device. Keep this tab visible while testing.
Ready. Model loading and inference run in a worker.
| Model | Run | Actual backend | End-to-end tokens/s | Decode tokens/s | First token (ms) | CPU output |
|---|