WebGPU and On-Device AI in the Browser: What Already Works and Why It Doesn’t Replace Cloud APIs

Artificial intelligence models can be run directly in the browser, without sending data to a server and without paying for each request. This was made possible primarily by the expansion of WebGPU technology, which gives websites access to graphics card performance. However, headlines about “the end of dependence on cloud APIs” are getting ahead of reality: local AI in the browser works, but with clear limitations.

What Is WebGPU and Why Is It Important for AI

WebGPU is an interface that allows web applications to use the graphics card not only for 3D graphics but also for general-purpose computing. This is important for AI because models run on graphics chips. According to the web.dev blog, which is managed by Google, it is currently supported by Chrome, Edge, Firefox, and Safari. The ONNX Runtime and Transformers.js libraries are already using it to run models locally.

Browser support varies

• Chrome and Edge: starting with version 113 on Windows, macOS, and ChromeOS; on Android, starting with version 121 for devices running Android 12 or later with Qualcomm or ARM graphics.
• Firefox: Version 141 and later on Windows, and version 145 and later on Macs with ARM64 processors. According to web.dev, support for Linux, Android, and Macs with Intel processors is in the works.
• Safari: starting with version 26 on macOS Tahoe, iOS, iPadOS, and visionOS.

According to several blogs, overall coverage is around 80% of users, though I couldn’t find an exact figure in the primary source. Support for mobile phones and older devices is more fragmented.

How models run in the browser

The two best-known open-source tools are WebLLM and Transformers.js. According to the study’s authors, WebLLM offers an OpenAI-style interface and uses WebGPU and WebAssembly for computations. In the authors’ benchmarks, it achieved up to 80% of the performance of a native solution on the same device. This is a result reported by the tool’s authors, not an independent comparison.

Transformers.js from Hugging Face uses ONNX Runtime and, if WebGPU is unavailable, falls back to slower CPU-based computation. According to blog posts, the model is cached in the browser after the initial download.

Built-in AI in Chrome

Chrome also offers its own APIs where the Gemini Nano model does not need to be deployed separately. According to Chrome’s documentation, the Prompt API is in testing (an origin trial starting with Chrome 138), and the model is downloaded separately upon first use on a given page. The documentation also lists the system requirements: at least 22 GB of free disk space and more than 4 GB of graphics memory. The interfaces do not yet support Chrome for Android and iOS, and both input and output are available in English, Spanish, and Japanese.

What This Actually Offers

• Privacy: With local processing, data does not need to be sent to a server. The WebLLM developers cite this as one of the main advantages.
• Offline use: Once the model is downloaded, the app can run even without an internet connection (this is a general explanation).
• No per-request fees: However, the costs are shifted to the user’s device—specifically, its memory, battery, and processing power.

Why it doesn’t replace cloud APIs

Local models are significantly smaller than the largest cloud models. According to one blog post, models ranging in size from tens of millions to approximately eight billion parameters can be run in a browser, with a larger model potentially taking up several gigabytes of GPU memory. Furthermore:

• the model must be downloaded before first use, which takes time and consumes data,
• performance depends on the device, ranging from a powerful computer to a low-end phone,
• Chrome’s built-in interfaces are still experimental and tied to a specific browser,
• the quality of smaller models is lower for demanding tasks (this is my conclusion, not a figure from the source) than that of large cloud-based models.

When Is Local AI Worth It?

This is my recommendation, not information from the source. Running AI locally makes sense for simpler tasks, such as classification, text vector representations (embeddings), summarization, or a simple assistant, where privacy or zero per-request fees are important. For complex reasoning, long documents, or mobile devices, the cloud is still the more reliable option. Many applications will likely combine both approaches.

Conclusion

WebGPU has turned the browser into a usable environment for running smaller AI models. This doesn’t mean cloud APIs will disappear: local AI offers privacy and offline use, but at the cost of model size, download requirements, and hardware inconsistencies. So whenever you see a headline like this, it’s worth asking: How large is the model being run, on what device, and in which browser?