Chrome Ships On-Device Gemini Nano, and Your Code Still Needs a Fallback

Chrome Ships On-Device Gemini Nano, and Your Code Still Needs a Fallback

The demo that took the number one spot on Hacker News was almost anticlimactic. No backend, no API key, no streaming proxy, no monthly invoice. One line, await LanguageModel.create(), and a chat side panel running entirely on the laptop it was rendered on. The model was already there. Chrome had downloaded Google's Gemini Nano during a routine background update, and the developer simply called it.

That is the Chrome Prompt API, and the demos hitting the front page every week have shifted from "look what's possible" to "I shipped this last weekend." The pitch is genuinely true this time: inference in the browser, zero marginal cost, user text that never leaves the machine. The catch is that the real engineering lives in what happens when the API is not there.

What actually shipped

Chrome's built-in AI is a family of APIs, and the Prompt API is the general-purpose surface. Extensions got it stable in Chrome 138 back in May 2026; Google says the web page surface reached stable in Chrome 148, two years after it first appeared behind the #prompt-api-for-gemini-nano flag and Origin Trial tokens. Edge inherits the whole thing automatically. Firefox and Safari offer nothing comparable.

One naming detail saves you an afternoon. The earliest previews exposed window.ai.createTextSession(), and Stack Overflow is still full of 2024 snippets using that shape. The 2026 surface is a LanguageModel global for pages and chrome.languageModel for extensions. If a copied snippet throws, the rename is almost certainly why.

The narrower task APIs shipped earlier: Summarizer, Translator, and Language Detector are stable since 138. Writer, Rewriter, and Proofreader are still listed as developer trial, and their Origin Trial windows have expired, so their surface can still change under you. The whole family is a WICG proposal hoping for cross-browser adoption, but no other vendor has committed to shipping it.

The three-call lifecycle

Every built-in AI API follows the same shape: check the API exists, ask about availability, create a session.

// 1. Does the API exist at all?
if (!('LanguageModel' in self)) return useCloudFallback();

// 2. Can it run on this device, with these inputs?
const status = await LanguageModel.availability({
  expectedInputs: [{ type: 'text', languages: ['en'] }],
  expectedOutputs: [{ type: 'text', languages: ['en'] }],
});
// 'available' | 'downloadable' | 'downloading' | 'unavailable'

// 3. Create a session (triggers the download if needed)
const session = await LanguageModel.create({
  systemPrompt: 'Reply in one sentence. No preamble.',
  monitor(m) {
    m.addEventListener('downloadprogress', (e) => {
      showProgress('model: ' + (e.loaded * 100).toFixed(0) + '%');
    });
  },
});
const answer = await session.prompt('Classify this review: ' + text);
session.destroy();

Two details matter more than they look. Pass the identical options object to availability() and create(), because availability is a question about a configuration, not about the API in general. A session expecting audio input reports unavailable on a CPU-only machine even when plain text works fine. And create() can block for minutes on a first call while the roughly 2 GB model downloads, so the monitor callback and its downloadprogress event are the honest way to render progress instead of a frozen spinner.

The part every demo skips

Chrome's own documentation says the on-device model fails open and your code should not. Fails open meaning: you get an unavailable status on a machine you expected to qualify, or a session created with different options than you asked for, rather than one clean error you can catch. Early analysis from Thinktecture estimated maybe 60% of real users qualify on hardware, before counting everyone who disabled the component updater.

async function getPrompter() {
  if (typeof LanguageModel !== 'undefined') {
    const status = await LanguageModel.availability().catch(() => 'unavailable');
    if (status === 'available') {
      const session = await LanguageModel.create({
        systemPrompt: 'You are a concise assistant.',
      });
      return (text) => session.promptStreaming(text);
    }
  }
  // Cloud fallback: same interface, different transport.
  return async function* (text) {
    const res = await fetch('/api/llm', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ prompt: text, stream: true }),
    });
    const reader = res.body.getReader();
    const decoder = new TextDecoder();
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      yield decoder.decode(value);
    }
  };
}

The router returns one streaming interface for both paths, so the feature code above it never branches. Write the fallback first. If your app breaks on Safari today, the on-device path will not save it next month either.

Sessions have a budget, and a destructor

Nano's practical window is roughly 4K input tokens and 1K output. session.countPromptTokens() prices a prompt before you send it, and session.tokensLeft tells you how much conversation remains. Quality degrades past a couple of thousand input tokens, so rotate long sessions with clone() before the budget runs thin. promptStreaming() accepts an AbortSignal if the user navigates away mid-answer. And call destroy() in a finally block, plus on beforeunload, because sessions hold GPU memory and RAM. A React component that leaks a session on every remount will cook a laptop that already has twenty tabs open.

What Nano is genuinely good at

The mental model that works: Nano replaces the regex and the hand-tuned classifier, not the frontier model. It is decent at sentiment and intent classification, summarizing content already on screen, translation, alt-text generation, and the quietly useful one, redacting or reshaping sensitive data before it goes to a cloud API. It is bad at multi-step reasoning, anything needing current world knowledge, long context, and code generation beyond trivia.

Once the model is warm, first-token latency runs around 50 to 300 milliseconds, against a few hundred milliseconds of network plus queue time for a cloud call. So the on-device path often feels snappier even when the cloud answer is better. For a classification chip in a settings panel, that is the entire trade.

The catch is the hardware, not the model

• 22 GB of free disk on the volume holding the Chrome profile. If space drops below 10 GB, Chrome deletes the model and re-downloads it later.
• More than 4 GB of VRAM, or 16 GB of RAM with four or more cores. Audio input requires the GPU.
• Windows 10/11, macOS 13+, Linux, or Chromebook Plus. Nothing on Android or iOS.
• The download starts when Chrome decides the device is idle on an unmetered connection. You cannot force it.
• The Origin Trial token binds to one origin, expires on a schedule, and signs you onto Google's generative AI prohibited-use policy, which is what drew Mozilla's public criticism of the whole effort.

Ship it, or wait

Adopt it now if you control the deployment target, meaning an extension, an internal tool, or a Chrome-only dashboard, or if the feature already has a cloud fallback and the on-device version is an optimization for privacy, offline use, cost, or latency. Wait if your audience skews Safari, if the task needs more context than Nano can hold, or if the only version you can imagine needs Writer or Rewriter, which are still developer trial.

The bet worth watching is smaller than it sounds: a browser now ships an inference runtime as a shared component, one download serving every origin on the machine. Whether that becomes a web standard or stays a Google exclusive depends on two other browser vendors who have not said much. The code that handles either outcome is the code you should be writing anyway. Ask politely, fall through, clean up after yourself.