On-Device AI in the Browser: Private Shop Features

On-device AI moves language processing to where the data originates: into the customer's browser. This is made possible by WebGPU - the interface that gives web pages direct access to the graphics card, which reached Baseline status across all major browsers in January 2026 (web.dev/MDN). Around 83 % of all browser sessions now support WebGPU (caniuse). This lets small language models with 1 to 3 billion parameters run directly on the device - without search queries, cart data or chat histories ever reaching a foreign server. For online shops this opens up three concrete applications: privacy-friendly AI assistants, semantic on-site search and text snippets that also work offline and without ongoing API costs. This guide sorts out the technology, the limits and a realistic hybrid approach.

What WebGPU makes possible in 2026

WebGPU is the successor to WebGL and was specified as an open web standard by the W3C GPU for the Web Working Group (gpuweb) (W3C). The key difference: while WebGL was built for graphics rendering, WebGPU additionally provides compute shaders - exactly the building blocks needed for the matrix multiplications of a neural network. Chrome has shipped WebGPU by default since version 113 (May 2023), Edge also from 113, Safari from version 26 on macOS Tahoe, iOS and iPadOS, and Firefox since version 141 on Windows (MDN/caniuse). In January 2026 the interface reached Baseline status - the formal signal that a web feature is broadly and stably available (web.dev).

For shop operators this is more than a technical footnote. Every millisecond of response time affects revenue measurably: a Google-commissioned Deloitte study across 30 million user sessions found that just 0.1 seconds faster mobile load time raised retail conversion by 8.4 % and average order value by 9.2 % (Deloitte). On-device inference can deliver answers without a network round trip - the latency advantage is structural, not just an optimization.

How a language model runs in the browser

The process comes down to three steps. First, the model is downloaded once and stored in browser storage (Cache API or Origin Private File System), so later visits need no fresh download. Second, the weights are quantized - usually to 4 bits instead of 16 or 32, which drastically cuts file size and memory footprint at only minor quality loss. Third, WebGPU handles the actual computation on the graphics card while WebAssembly runs the management logic in the browser. Open-source engines such as WebLLM from the MLC AI project wrap this setup behind an OpenAI-compatible interface (WebLLM/MLC AI).

The WebLLM paper shows how fast this is: on a MacBook Pro with M3 Max, Phi-3.5-mini (3.8B) reaches around 71 tokens per second and a Llama-3.1-8B model about 41 tokens per second (WebLLM). Overall, browser execution retains roughly 80 % of the speed of a native install on identical hardware (WebLLM). Smaller 1-3B models, which suffice for clearly scoped shop tasks, run correspondingly smoother. The technical integration into an existing front end is, in practice, handled by our web development.

import { CreateMLCEngine } from "@mlc-ai/web-llm";

// The model is loaded once and kept in the browser cache
const engine = await CreateMLCEngine(
  "Llama-3.2-1B-Instruct-q4f16_1-MLC",
  {
    initProgressCallback: (p) => console.log(p.text) // download progress
  }
);

// OpenAI-compatible interface - all local, no network
const reply = await engine.chat.completions.create({
  messages: [
    { role: "system", content: "You are a product advisor for our shop." },
    { role: "user", content: "What size fits at 182 cm and 78 kg?" }
  ],
  temperature: 0.6
});

Three privacy-friendly shop features

Not every AI task needs a large cloud model. Recurring, clearly scoped shop functions in particular can be implemented on-device sensibly - with the added benefit that sensitive input never leaves the device.

Search in particular has leverage: Baymard research shows that 41 % of e-commerce sites do not fully support key search query types their customers use, and 70 % fail on synonyms for product types (Baymard). A semantic search that compares meanings rather than just character strings addresses exactly this gap. How to build it concretely is covered in our guide to AI-powered product search.

Privacy: why on-device is a real advantage

The central advantage is the same legally and technically: what does not leave the device does not have to be transmitted, stored or processed by a data processor. On-device inference therefore reduces the attack surface and the documentation burden around third-country transfers that regularly need clarifying with cloud AI. Users are more sensitive here than many shops assume: according to Bitkom, 79 % of internet users accept privacy policies without really understanding them, and 68 % feel poorly informed about how to protect their data (Bitkom).

The most data-minimizing dataset is the one that never leaves the device. On-device AI shifts the question of trust from transmission to non-transmission.

XICTRON development team

On-device complements existing cookieless strategies. Anyone already working on a first-party data strategy or looking to reduce the cookie banner adds another data-minimizing building block with local AI. What matters is honest communication - no absolute promises, but a clear statement of which processing happens locally and which server-side.

Being honest about the limits: model size, first download, devices

On-device AI is not a replacement for large cloud models but a complement for the right task profile. A 1-3B model understands instructions and phrases usable short answers, but hits limits with complex domain knowledge or long contexts. On top come three practical hurdles: the first download of the model, device dependence (weak smartphones and older GPUs) and higher initial latency until the model is loaded.

Chrome's built-in AI shows the requirements especially clearly: the Prompt API with the Gemini Nano model has been stable since Chrome 138 and runs entirely on-device via WebAssembly and WebGPU (Chrome for Developers). Chrome lists among its prerequisites 22 GB of free space on the profile volume, more than 4 GB of VRAM and at least 16 GB of RAM (Chrome for Developers). Not every device meets that - which is why feature detection and a fallback are mandatory.

Hybrid approach: on-device plus server fallback

The most robust route for production environments is rarely either-or, but a hybrid model following the principle of progressive enhancement. The shop first checks via feature detection whether WebGPU and a suitable GPU are present. If so, simple and privacy-sensitive tasks run locally; if the hardware is missing or a request needs a larger model, it escalates - with clear user information - to the server.

// Feature detection: is on-device inference possible?
async function pickInferenceMode() {
  if (!("gpu" in navigator)) return "server"; // no WebGPU
  const adapter = await navigator.gpu.requestAdapter();
  if (!adapter) return "server"; // no suitable GPU

  // WebGPU present -> run simple, private tasks locally
  return "on-device";
}

// Privacy-sensitive tasks preferably local, the rest optionally server-side
const mode = await pickInferenceMode();

• Local first: search, text suggestions and standard questions run on-device when the hardware allows.
• Server when needed: complex or knowledge-intensive requests go - transparently and ideally with consent - to a larger model.
• Graceful degradation: without WebGPU the shop stays fully functional; the AI feature does not fail but changes where it runs.
• Offline benefit: once loaded, local functions keep working without a connection - a natural fit for progressive web apps.

This logic belongs in a clean architecture, not in scattered inline scripts. Encapsulating it as a reusable building block, lazy-loading the engine and handling errors for older devices are classic tasks for our custom development.

Browser support and hardware in 2026

Availability is no longer a blocker in 2026, but it differs by environment. For the pure graphics and compute interface WebGPU, support is broad; the built-in AI models of individual browsers carry additional hardware requirements.

Two paths are available. WebLLM runs cross-browser wherever WebGPU is active and downloads the desired model per origin (WebLLM). Chrome's built-in AI, by contrast, ships the model with the browser but is for now limited to Chrome and sufficiently equipped devices (Chrome for Developers). Because the initial latency and memory footprint are noticeable, on-device AI belongs embedded in a performance strategy - the same way we generally recommend for Core Web Vitals: measure, budget, load only what the context needs.

Introducing on-device AI step by step

A pragmatic start begins small, measures the benefit and only then scales. The following order has proven itself for shop projects.

1. Pick a use case: start with a clearly scoped, privacy-sensitive function - such as on-site search or a product advisor - instead of a general-purpose assistant.
2. Check feasibility: analyze the target devices and browsers of your audience. Does the ratio of WebGPU coverage to device class fit the plan?
3. Define a hybrid architecture: feature detection, local execution and server fallback as a reusable building block - including transparent user communication.
4. Secure perceived performance: plan loading states and soft transitions so the initial latency does not disturb - here the View Transitions API helps with smooth changes.
5. Integrate cleanly: encapsulate the engine as a module and embed it in the shop platform - with Shopware in concert with the modern plugin and Vite architecture.
6. Measure and expand: observe usage, answer quality and drop-off, then add further functions. Our AI consulting and concept consulting support the design and prioritization.