Evolve LLM

Why I built it

Running a language model locally usually means installing a runtime, drivers and a model manager, then learning which file to download. Browsers already ship a GPU programming interface. I wanted a single link that takes anyone from nothing to a private, local model in a minute, on hardware they already own.

Try it

Open the chat → Pick a model from the list: SmolLM2 135M is a 145 MB first test, and Qwen2.5 0.5B is the recommended start. The file downloads once from Hugging Face into the browser’s private storage and loads from there next time, even offline. You can paste any single-file GGUF on Hugging Face or open one from your disk.

WebGPU is needed: Chrome or Edge on Windows, macOS, ChromeOS and Android; Safari 26; Firefox on Windows. On Linux, Chrome needs its WebGPU flag. The page says what to do when WebGPU is missing.

An engine, not a wrapper

No machine-learning framework is involved. I wrote the GGUF parser, the decoders for 15 weight formats, the transformer forward pass as WGSL compute shaders, two tokenizer families and the chat-template renderer in Rust. The same crate runs natively through wgpu on Vulkan, Metal or Direct3D 12, and compiles to WebAssembly for the browser’s WebGPU.

• Weights stream to the GPU in 16 MB slices and are repacked on the way into word-aligned layouts, so the browser never holds the whole model in memory. Five- and six-bit weights keep their size, with their extra bits in a separate plane.
• Separate kernels handle one token at a time and whole prompts in tiles, with grouped-query attention, rotary embeddings, an f16 key-value cache and a streaming softmax.
• Each new turn computes only the part of the conversation that is not already cached, so follow-up questions start quickly.
• A module worker keeps the page responsive. Downloads resume after a dropped connection, and the only network destination is Hugging Face.

Checked against reference implementations

Speed means nothing if the numbers are wrong, so the engine is tested against independent code:

• Dequantization of every supported format matches gguf-py’s NumPy implementation.
• Tokenizers produce the same token ids as Hugging Face’s originals on 17 difficult strings across five models, and chat prompts match transformers’ template rendering exactly.
• Final-token logits for SmolLM2, Qwen2.5, Qwen3 and Llama 3.2 match transformers’ float32 PyTorch run on the same weights: same top token, top-20 logits within 0.2% of the range.
• In Chrome under the production security policy, greedy replies match the native Vulkan build word for word.

Measured speed

I measured on a laptop running its battery power profile. The profile held the RTX 3050 at its lowest clocks, so the integrated Intel GPU was the faster of the two. Chrome kept within about 15% of the native build.

• SmolLM2 135M: about 70 tokens per second in Chrome on the Intel GPU (82 natively).
• Qwen2.5 0.5B: about 35 tokens per second in Chrome (40 natively), and 415 prompt tokens per second natively.
• Llama 3.2 1B: about 22 tokens per second in Chrome (23 natively).
• On the capped RTX 3050, Chrome and native ran at the same speed, limited by the memory bandwidth the power profile allowed.

Limits

This is an early release. It runs Llama, Mistral, Qwen2, Qwen3 and SmolLM layouts; Gemma, Phi, mixture-of-experts models and images are not supported yet. Very large models are limited by the GPU memory a browser allows. Speed has not been tuned against llama.cpp or WebLLM, and models can be wrong, so check anything important.