Engineering
Browser AI needs a scheduler, not a wish
The hardest part of browser transcription is not running a model once on a developer laptop. The hard part is staying reliable across old integrated GPUs, memory-constrained tabs, inconsistent WebGPU support, and browsers that expose the same feature with very different behavior. Fixed pipelines break here. Adaptive orchestration is what turns browser AI from a demo into a product.
The problem with fixed browser pipelines
A hard-coded assumption like “GPU equals fast” fails quickly in the field. Some devices expose WebGPU but run out of memory with larger models. Some browsers advertise support yet behave poorly after repeated allocations. Some machines only become stable when the scheduler picks a smaller quantized model or abandons GPU entirely for a predictable WASM path.
Capability mismatch
The API exists, but the driver budget or adapter quality is too weak for the chosen model.
Runtime volatility
Long files expose problems that small smoke tests never reveal: drift, fragmentation, and repeated warmup cost.
Model overreach
A high-quality model on paper can be the wrong choice if it pushes the browser into instability or stalls.
Execution layers
| Layer | Strength | Failure mode | Scheduler response |
|---|---|---|---|
| WebGPU + ONNX Runtime Web | Best throughput on compatible devices | Adapter failure, memory pressure, unstable long runs | Reduce model size, lower batch profile, or fall back |
| ONNX Runtime WASM | Broad compatibility and predictable startup | Slower execution on long media | Prefer smaller quantized models and stable chunking |
| Whisper.cpp WASM | Useful portability fallback | Lower throughput on weak CPUs | Constrain expectations, preserve correctness, keep workflow alive |
Backend decision tree
incoming job
|
+-- detect browser + adapter + memory signals
|
+-- WebGPU available?
|
+-- no --> choose ONNX WASM or whisper.cpp WASM
|
+-- yes --> estimate model budget
|
+-- budget healthy --> try accelerated ONNX path
|
+-- budget weak --> load smaller quantized model
|
+-- unstable? --> fall back to WASM
The key detail is that the engine does not treat fallback as a rare exception. Fallback is part of the design. In heterogeneous environments, graceful degradation is the main path to reliability.
Scheduling logic in pseudo-code
hardware = probeDevice()
job = inspectMedia(input)
if hardware.webgpu and hardware.gpuBudget >= job.minimumGpuBudget:
candidateModels = rankModels(job.language, hardware, prefer="quality-within-budget")
for model in candidateModels:
result = tryOnnxWebGpu(model, job)
if result.ok:
return result
rememberFailure(model, result.reason)
candidateCpuModels = rankModels(job.language, hardware, prefer="stability")
for model in candidateCpuModels:
result = tryOnnxWasm(model, job)
if result.ok:
return result
return tryWhisperCppFallback(job)
Real implementations track more state than this sketch shows: previous backend failures, device-specific bad paths, warmup cost, thread count, and whether a user is processing one short clip or a folder of long recordings.
Why this matters for transcription quality
Quality is not just about the biggest model. A stalled job has zero usefulness. An adaptive engine can preserve effective quality by selecting the most capable path that still completes reliably. That often means using a smaller model that finishes in one pass instead of a larger model that crashes late in the run.
Related pages
WebGPU vs WASM AI
An honest comparison of acceleration, portability, and long-run stability.
Generalized vs adaptive inference
Why one-size-fits-all browser pipelines struggle on mixed hardware.
WebGPU explained
Browser-facing documentation on what GPU acceleration changes in practice.
Engineering article
A longer write-up on runtime orchestration and failure handling in the browser.