Assistant

An in-page assistant grounded in your docs — suggested questions, custom instructions, retrieval sizing, provider adapters for OpenAI, Anthropic, Gemini, Grok, the Vercel AI Gateway, and any OpenAI-compatible endpoint, and the server output it needs.

Add an assistant that answers reader questions in an in-page chat panel, backed by a streaming server endpoint and the AI SDK. It’s opt-in, and static docs stay fully static until you turn it on:

ai: {
  assistant: {
    enabled: true,
  },
}

With nothing else written, answers stream through the Vercel AI Gateway from openai/gpt-5.5. Pick a different model or provider with an adapter.

Suggested questions

Seed the empty state with a few starter prompts. Each renders as a clickable suggestion — click one to send it — with an optional Lucide icon beside the label:

ai: {
  assistant: {
    enabled: true,
    suggestions: [
      { label: "What is Blume?", icon: "rocket" },
      { label: "How do I write a docs page?", icon: "file-text" },
      { label: "How do I configure the theme?", icon: "settings" },
    ],
  },
}

label is the question that gets asked; icon is optional. Leave suggestions unset (or empty) and the panel opens to a plain input.

Asking about code

With the assistant on, every code block gets an Ask button beside its copy button. It opens the assistant with the block attached, shown as a chip above the input, so the reader can ask what it does, why it fails, or how to adapt it. Sent without a question, it asks the assistant to explain the code. The code goes to the model as a fenced block after the question, up to 6,000 characters.

A custom chat UI built on useAssistant can do the same: listen for blume:open-assistant on window, whose detail.code holds the block’s language, source, and title.

Contact support

When the docs can’t answer, hand the reader to a person. Set support, and once there’s a conversation the panel shows a Contact support link under it:

ai: {
  assistant: {
    enabled: true,
    support: "mailto:help@example.com", // or "https://example.com/support", or "/support"
  },
},

A mailto: address starts an email with the conversation as its body, so the reader doesn’t have to explain again (a long conversation keeps its latest turns). A URL or a page on your site gets the conversation’s id as a thread query parameter. The assistant’s ask, ask_answer, and ask_error analytics events carry the same thread, so your support team can find the conversation in your analytics.

Custom instructions

Add your own system-prompt text with instructions — identity, language, tone, or anything else the assistant should keep in mind:

ai: {
  assistant: {
    enabled: true,
    instructions:
      "You are Bloomy, the Acme docs assistant. Answer in the language the question was asked in, and keep answers under three paragraphs.",
  },
}

Your text is appended to the built-in instructions rather than replacing them: the built-in part carries the grounding contract — answer only from the retrieved pages, cite them as Markdown links — that the chat panel’s citations depend on, so it stays intact whatever you add.

Grounding

The assistant is grounded in your docs. For each question it retrieves the most relevant pages — using the same lexical Orama index that powers on-page search — and injects them into the model’s system prompt, so answers come from your content instead of the model’s own knowledge. The assistant is told to answer only from the retrieved pages, to say when something isn’t covered, and to cite the pages it drew from.

The page the reader is currently on is added to the context first and used to scope retrieval to that page’s language — and, on a versioned site, to its docs version — so answers stay relevant to where they are in the docs. A question from outside the docs, such as a landing page, keeps to the language the URL’s locale prefix names (/ja), or to the default language. Retrieval runs at request time from a snapshot baked into the build, so it works regardless of your search provider — even with search: false — and needs no configuration.

Grounding is on for every adapter except Inkeep, which runs its own retrieval over the content you’ve indexed in its dashboard.

Searching and reading pages

The retrieved pages are a head start. The assistant can also look things up itself: it has a search_docs tool that searches your docs, and a read_page tool that reads a whole page, for when an excerpt stops short of the answer. It can search, read a page or two, search again, and then answer, all within one question. Tool calls run on your server against the same snapshot as the grounding, keep to the reader’s language and docs version, and never reach the reader; they only see the answer.

The tools are on by default for every adapter whose models support tool calling. They’re off for OpenAI-compatible endpoints, since a self-hosted model may not. Set tools to change either default:

ai: {
  assistant: {
    enabled: true,
    provider: openai({ baseUrl, model, apiKeyEnv }),
    tools: true, // this model supports tool calling
  },
}

Each question takes at most five model steps: up to four rounds of tool calls, then the answer. With tools: false, the assistant answers from the retrieved pages alone, as it did before tools existed.

Retrieval size

How much documentation a question carries is the biggest lever on how long the reader waits for the first word: the model reads every injected character before it emits a token. On a hosted frontier model that’s invisible, but on a self-hosted backend it dominates. retrieval sizes it:

ai: {
  assistant: {
    enabled: true,
    retrieval: {
      maxResults: 3, // fewer pages retrieved per question
      excerptChars: 1200, // shorter excerpt from each one
      contextBudget: 3000, // smaller total injection
    },
  },
}

The three aren’t interchangeable. contextBudget caps the whole injection, excerptChars decides how deep into a single long page its excerpt reaches — raise it when one page holds the whole answer and the excerpt cuts it off — and maxResults caps how many pages retrieval adds. The page the reader is viewing is injected on top of the retrieved ones, so an answer can cite up to one page more than maxResults.

The defaults suit a hosted model. Lower them when you’re serving from your own hardware and time-to-first-token matters more than recall; answers stay grounded either way, and the assistant is told to say when something isn’t covered rather than fill the gap.

On your own hardware, also size the model’s context window for the whole prompt, not only the excerpts: the conversation so far adds as many as 24,000 characters, and with tools on, one read_page result as many as 20,000, about 5,000 tokens on its own. Ollama, for one, defaults to a 4k-token window on smaller GPUs, and a prompt that outgrows it doesn’t fail: Ollama drops the start of the prompt, where the instructions and excerpts are, and the model answers without your docs.

External endpoint

Already have an API backend for AI? Point the panel at it and keep the docs build static:

ai: {
  assistant: {
    enabled: true,
    endpoint: "https://api.example.com/v1/docs/ask",
  },
}

Blume sends the same POST body as its built-in route:

{
  "messages": [{ "role": "user", "content": "How do I deploy?" }],
  "page": { "path": "/deployment" }
}

messages is the conversation so far, ending with the new question. The built-in route takes at most 40 messages and 24,000 characters of them as JSON, so once a conversation outgrows either, the panel sends its latest turns that fit and drops the oldest.

Return a successful response whose body is a plain UTF-8 text stream. If the endpoint is on another origin, allow the docs origin with CORS: accept OPTIONS and POST, permit the content-type request header, and return the CORS headers on both the preflight and streamed response. With endpoint set, Blume generates the chat UI but no server route, grounding snapshot, provider dependency, or provider-secret warning; your backend owns retrieval, authentication, rate limiting, model access, and citations. An adapter set alongside it is ignored.

Cross-origin callers

The generated endpoint answers the in-page assistant on its own origin. To call it from another site as well — a marketing page with an ask box, say — list that site’s origin in cors:

ai: {
  assistant: {
    enabled: true,
    cors: ["https://www.example.com"],
  },
}

The route then answers the browser’s OPTIONS preflight and names a listed origin on every response — the streamed answer and the error statuses alike, so the caller can tell a rejected body from a provider failure. Those responses expose Retry-After too, so after a rate limit’s 429 the caller can read how many seconds to wait. Origins that aren’t listed get no header and stay subject to the browser’s same-origin rule. Each entry is reduced to its origin, so https://www.example.com/docs/ and https://www.example.com mean the same thing. To let any page call the route, list "*" instead of origins.

The caller sends the same POST body the external endpoint contract describes and reads back the same text stream. Send it as JSON with a content-type: application/json header:

const response = await fetch("https://docs.example.com/api/ask", {
  body: JSON.stringify({
    messages: [{ role: "user", content: "How do I deploy?" }],
  }),
  headers: { "content-type": "application/json" },
  method: "POST",
});

The content type matters: Astro’s cross-site request check rejects a cross-origin POST that has no content type, or a form-like one such as text/plain, with a 403 before the route runs, and that response carries no CORS headers, so the browser reports it as a network error rather than a status. The preflight allows whatever request headers the caller asks for, so a fetch wrapper that adds its own headers needs no extra configuration.

cors only affects the generated route; with an external endpoint, CORS is that backend’s job, and setting both is a config error. The endpoint stays unauthenticated either way, so the rate limiting advice applies to cross-origin traffic too.

Server output required

Blume’s built-in assistant backend is a server route (POST /api/ask), so it can’t run on a static build. Name a host adapter from blume/deploy to switch to server output:

import { vercel } from "blume/deploy";

export default defineConfig({
  deployment: vercel(),
});

A static build with the assistant enabled and no external endpoint fails fast with a message telling you to set a host adapter. See Deployment for the adapters.

Adapters

provider picks the backend that answers. Its value is an adapter: a small function exported from blume/ai that takes that backend’s own options and returns a plain descriptor Blume writes into the generated route. Each adapter owns its model, the env var its key is read from, how it maps reasoning, and which provider SDK it needs — so there is no shared set of fields to reconcile across backends:

import { defineConfig } from "blume";
import { anthropic } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: anthropic({ model: "claude-sonnet-5" }),
    },
  },
});

openai(), anthropic(), gemini(), and grok() call the provider’s own API with your key — nothing sits in between. The SDKs are optional peer dependencies, so add the one your adapter needs to your project (npm install @ai-sdk/anthropic, say). If it’s missing, blume build stops before Vite runs, naming the package and the command that installs it, and blume doctor reports it too.

The descriptor an adapter returns is plain data — its kind, its options, the env vars it reads, and the SDK it needs — so the generated route (and the ejected one) inlines it as literals and imports the provider SDK by name. Nothing reads blume.config.ts at request time, and no secret is ever written into a route: adapters take the name of the env var holding the key, and the route reads the value through Astro’s getSecret(), so each deployment adapter supplies it its own way — environment variables on Node, Vercel, and Netlify, and the Worker’s bindings on Cloudflare.

OpenAI

Any OpenAI model, called directly through OpenAI’s own AI SDK provider:

import { defineConfig } from "blume";
import { openai } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: openai({ model: "gpt-5.5" }),
    },
  },
});

OpenAI-compatible endpoints

Any endpoint that speaks the OpenAI API — a self-hosted model, an internal gateway — works through openai() too: add its baseUrl, and usually apiKeyEnv for the env var holding its key (it defaults to OPENAI_API_KEY). name is the provider name the AI SDK reports, and the key the endpoint reads providerOptions under; it defaults to openai-compatible:

import { defineConfig } from "blume";
import { openai } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: openai({
        baseUrl: "https://my-gateway.example.com/v1",
        apiKeyEnv: "MY_GATEWAY_API_KEY",
        model: "gpt-4o",
        name: "my-gateway",
      }),
    },
  },
});

With a baseUrl, the route calls the endpoint’s Chat Completions API through @ai-sdk/openai-compatible, so install that package instead of @ai-sdk/openai: most compatible endpoints don’t serve OpenAI’s Responses API. The docs tools are off by default, since a self-hosted model may not support tool calling.

Anthropic

Any Claude model, called directly through Anthropic’s AI SDK provider:

import { defineConfig } from "blume";
import { anthropic } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: anthropic({ model: "claude-sonnet-5" }),
    },
  },
});

Gemini

Any Gemini model, called directly through Google’s AI SDK provider. The key is an API key from Google AI Studio:

import { defineConfig } from "blume";
import { gemini } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: gemini({ model: "gemini-3.5-flash" }),
    },
  },
});

Grok

Any xAI Grok model, called directly through xAI’s AI SDK provider:

import { defineConfig } from "blume";
import { grok } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: grok({ model: "grok-4.7" }),
    },
  },
});

Vercel AI Gateway

The default. model is a provider/model string, so you switch models by changing it (openai/gpt-5.5, anthropic/claude-sonnet-4-5, and so on) with no provider SDK to install. The gateway reads AI_GATEWAY_API_KEY from your environment and is wired up automatically when you deploy on Vercel, where it can also authenticate with the deployment’s OIDC token:

import { defineConfig } from "blume";
import { gateway } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: gateway({ model: "anthropic/claude-sonnet-4-5" }),
    },
  },
});

Leaving provider unset is the same as gateway({ model: "openai/gpt-5.5" }).

OpenRouter

Any model on OpenRouter, through its dedicated AI SDK provider:

import { defineConfig } from "blume";
import { openrouter } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: openrouter({
        model: "anthropic/claude-sonnet-4-5",
        reasoning: "none",
      }),
    },
  },
});

LLMGateway

Any model on LLMGateway, through its OpenAI-compatible endpoint:

import { defineConfig } from "blume";
import { llmgateway } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: llmgateway({ model: "openai/gpt-5.5" }),
    },
  },
});

baseUrl overrides the preset endpoint (https://api.llmgateway.io/v1) when you run LLMGateway yourself.

Inkeep

Inkeep answers from the content you’ve indexed in the Inkeep dashboard — it runs its own retrieval — so Blume leaves it ungrounded: no snapshot of this site’s pages is injected, and the retrieval size options don’t apply. It has no reasoning control either, so the adapter takes no reasoning:

import { defineConfig } from "blume";
import { inkeep } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: inkeep({ model: "inkeep-qa-expert" }),
    },
  },
});

baseUrl overrides the preset endpoint (https://api.inkeep.com/v1).

Options every adapter takes

apiKeyEnv points an adapter at a different env var than its default — gateway({ apiKeyEnv: "DOCS_GATEWAY_KEY" }) reads that variable instead of AI_GATEWAY_API_KEY, and the missing-secret warning at blume dev/build checks it too. Until the key is set, the deployed route answers 503 with a message naming the variable. The route reads a request body only up to 64 KB and answers anything larger with 413.

headers sends static request headers with every call — a caller-identifying header for a shared backend, say, so its own observability or rate limiting can tell your docs apart from other traffic:

provider: openai({
  baseUrl: "https://llm.internal.example.com/v1",
  apiKeyEnv: "INTERNAL_LLM_API_KEY",
  model: "gpt-4o",
  headers: { "X-Caller-Id": "docs" },
}),

The values are written into the generated route as-is, so keep secrets in apiKeyEnv rather than in headers. The API key’s Authorization header is applied first, so a custom header can’t displace it.

providerOptions passes anything else straight through to the AI SDK’s providerOptions, in the SDK’s own shape — keyed by provider, then by option — so a new model control never needs a Blume field of its own:

provider: gateway({
  model: "openai/gpt-5.5",
  providerOptions: { openai: { textVerbosity: "low" } },
}),

Blume maps only the options it names (model, reasoning, apiKeyEnv, headers) and forwards providerOptions verbatim, so it has to be JSON — it’s inlined into the route — and it has to use the key the underlying provider expects (openai for openai() or an OpenAI model behind the gateway, anthropic for anthropic(), google for gemini(), xai for grok(), openrouter on OpenRouter, and the adapter’s name on an OpenAI-compatible endpoint). Enabling the assistant also turns on React for the in-page island — see Customization.

Reasoning

Reasoning models think before they answer, and how much they do so by default varies by model. For grounded docs Q&A the retrieved excerpts carry the answer, so most of that thinking is latency the reader waits through. An adapter’s reasoning option sets how much the model reasons: "none", "minimal", "low", "medium", "high", or "xhigh":

provider: gateway({ model: "openai/gpt-5.5", reasoning: "none" }),

Each adapter sends the level as its backend’s own reasoning control, which is why it lives on the adapter rather than on assistant:

The model has to support the level you pick: OpenAI rejects a level a model doesn’t offer ("none" and "xhigh" exist only on some), so check the model’s documentation before setting one. Leave it unset to keep the model’s default. Like retrieval size, it trades thoroughness for time-to-first-token, and answers stay grounded either way.

Analytics

With an analytics provider configured, the assistant reports its usage through the same track() the page feedback widget uses, so questions land next to your pageviews:

path is the page the reader asked from (the served pathname, so it matches the feedback widget and your pageviews under a base), questionChars the question’s length, thread the conversation’s id (new when the reader clears it), ms the time from sending the question to the last chunk, and chars the answer’s length. status is the HTTP status: 0 when no response arrived at all (offline, DNS, CORS), and 200 when the response was fine but its stream broke mid-answer — how a provider or credential error surfaces, since the backend has already sent its headers — or delivered nothing. Clearing the conversation mid-answer reports neither outcome.

The question’s text never reaches a provider: it is free-form reader input (pasted keys, error logs, names) that would breach most providers’ terms and per-value size limits. It travels only on the blume:track DOM event, as question in detail.props, so a listener you write can forward it wherever you decide it belongs. A custom chat UI built on useAssistant from blume/hooks reports the same events. With no provider configured the built-in provider calls are no-ops, but the blume:track event still fires, so a custom integration listening for it receives them.

Rate limiting

The POST /api/ask endpoint is unauthenticated — it has to be, so the in-page assistant can call it. Blume validates each request — rejecting malformed bodies, capping it to 1–40 messages, and accepting only user/assistant roles so a caller can’t inject their own system prompt and repurpose the route as a general LLM proxy — to bound how much a single call can spend against your model. Rate limiting, on by default, then caps how many questions one reader can ask: past it, the panel tells them to try again in a few minutes.

Bot protection

Rate limiting counts by IP address, so a script that spreads its questions across many addresses slips past it. To stop scripts, add a bot check from blume/captcha: before each question, the panel gets a token from the check, and the route verifies it with the provider before the model runs. Most readers never see a challenge; the check only asks for a click when it’s unsure.

import { defineConfig } from "blume";
import { turnstile } from "blume/captcha";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      captcha: turnstile({ siteKey: "0x4AAAAAAA…" }),
    },
  },
});

The site key goes to the browser and is safe to commit; the secret key stays on the server. For Turnstile, create a widget in the Cloudflare dashboard and choose Managed or Invisible mode. Until the secret is set, the assistant answers that it isn’t configured, and blume build warns. The provider’s script loads with a reader’s first question, not with the page.

A failed check shows the reader a translated “we couldn’t check that you’re human” notice, and the route answers 403. Both providers publish test keys that always pass, for trying the check locally: Turnstile’s 1x00000000000000000000BB with the secret 1x0000000000000000000000000000000AA, and hCaptcha’s 10000000-ffff-ffff-ffff-000000000001 with 0x0000000000000000000000000000000000000000.

With an external endpoint, the panel still sends the token, as captcha in the request body, for your backend to verify.

The endpoint is advertised in the agent readability manifest alongside the rest of the site’s machine-readable surface.