--- title: Assistant description: An in-page assistant grounded in your docs — suggested questions, custom instructions, retrieval sizing, provider adapters from the Vercel AI Gateway to any OpenAI-compatible endpoint, and the server output it needs. --- Add an assistant that answers reader questions in an in-page chat panel, backed by a streaming server endpoint and the [AI SDK](https://ai-sdk.dev). It's opt-in, and static docs stay fully static until you turn it on: ```ts blume.config.ts lineNumbers ai: { assistant: { enabled: true, }, } ``` With nothing else written, answers stream through the [Vercel AI Gateway](#adapters) from `openai/gpt-5.5`. Pick a different model or provider with an [adapter](#adapters). ## Suggested questions Seed the empty state with a few starter prompts. Each renders as a clickable suggestion — click one to send it — with an optional [Lucide icon](/docs/content/components#icon) beside the label: ```ts blume.config.ts lineNumbers ai: { assistant: { enabled: true, suggestions: [ { label: "What is Blume?", icon: "rocket" }, { label: "How do I write a docs page?", icon: "file-text" }, { label: "How do I configure the theme?", icon: "settings" }, ], }, } ``` `label` is the question that gets asked; `icon` is optional. Leave `suggestions` unset (or empty) and the panel opens to a plain input. ## Custom instructions Add your own system-prompt text with `instructions` — identity, language, tone, or anything else the assistant should keep in mind: ```ts blume.config.ts lineNumbers ai: { assistant: { enabled: true, instructions: "You are Bloomy, the Acme docs assistant. Answer in the language the question was asked in, and keep answers under three paragraphs.", }, } ``` Your text is **appended to** the built-in instructions rather than replacing them: the built-in part carries the [grounding](#grounding) contract — answer only from the retrieved pages, cite them as Markdown links — that the chat panel's citations depend on, so it stays intact whatever you add. ## Grounding The assistant is **grounded in your docs**. For each question it retrieves the most relevant pages — using the same lexical [Orama](/docs/configuration/search) index that powers on-page search — and injects them into the model's system prompt, so answers come from your content instead of the model's own knowledge. The assistant is told to answer only from the retrieved pages, to say when something isn't covered, and to cite the pages it drew from. The page the reader is currently on is added to the context first and used to scope retrieval to that page's language, so answers stay relevant to where they are in the docs. Retrieval runs at request time from a snapshot baked into the build, so it works regardless of your [search](/docs/configuration/search) provider — even with `search: false` — and needs no configuration. Grounding is on for every adapter except **[Inkeep](#inkeep)**, which runs its own retrieval over the content you've indexed in its dashboard. ## Retrieval size How much documentation a question carries is the biggest lever on how long the reader waits for the first word: the model reads every injected character before it emits a token. On a hosted frontier model that's invisible, but on a self-hosted backend it dominates. `retrieval` sizes it: ```ts blume.config.ts lineNumbers ai: { assistant: { enabled: true, retrieval: { maxResults: 3, // fewer pages retrieved per question excerptChars: 1200, // shorter excerpt from each one contextBudget: 3000, // smaller total injection }, }, } ``` | Option | Default | Description | | --------------- | ------- | ----------------------------------------------- | | `maxResults` | `6` | Documents retrieved per question. | | `excerptChars` | `2000` | Characters kept from each retrieved page. | | `contextBudget` | `10000` | Total injected characters, across all excerpts. | The three aren't interchangeable. `contextBudget` caps the whole injection, `excerptChars` decides how deep into a single long page its excerpt reaches — raise it when one page holds the whole answer and the excerpt cuts it off — and `maxResults` caps how many pages retrieval adds. The page the reader is viewing is injected on top of the retrieved ones, so an answer can cite up to one page more than `maxResults`. The defaults suit a hosted model. Lower them when you're serving from your own hardware and time-to-first-token matters more than recall; answers stay grounded either way, and the assistant is told to say when something isn't covered rather than fill the gap. ## External endpoint Already have an API backend for AI? Point the panel at it and keep the docs build static: ```ts blume.config.ts lineNumbers ai: { assistant: { enabled: true, endpoint: "https://api.example.com/v1/docs/ask", }, } ``` Blume sends the same `POST` body as its built-in route: ```json { "messages": [{ "role": "user", "content": "How do I deploy?" }], "page": { "path": "/deployment" } } ``` Return a successful response whose body is a plain UTF-8 text stream. If the endpoint is on another origin, allow the docs origin with CORS: accept `OPTIONS` and `POST`, permit the `content-type` request header, and return the CORS headers on both the preflight and streamed response. With `endpoint` set, Blume generates the chat UI but no server route, grounding snapshot, provider dependency, or provider-secret warning; your backend owns retrieval, authentication, rate limiting, model access, and citations. An adapter set alongside it is ignored. ## Cross-origin callers The generated endpoint answers the in-page assistant on its own origin. To call it from another site as well — a marketing page with an ask box, say — list that site's origin in `cors`: ```ts blume.config.ts lineNumbers ai: { assistant: { enabled: true, cors: ["https://www.example.com"], }, } ``` The route then answers the browser's `OPTIONS` preflight and names a listed origin on every response — the streamed answer and the error statuses alike, so the caller can tell a rejected body from a provider failure. Origins that aren't listed get no header and stay subject to the browser's same-origin rule. Each entry is reduced to its origin, so `https://www.example.com/docs/` and `https://www.example.com` mean the same thing. To let any page call the route, list `"*"` instead of origins. The caller sends the same `POST` body the [external endpoint](#external-endpoint) contract describes and reads back the same text stream. Send it as JSON with a `content-type: application/json` header: ```ts const response = await fetch("https://docs.example.com/api/ask", { body: JSON.stringify({ messages: [{ role: "user", content: "How do I deploy?" }], }), headers: { "content-type": "application/json" }, method: "POST", }); ``` The content type matters: Astro's cross-site request check rejects a cross-origin `POST` that has no content type, or a form-like one such as `text/plain`, with a 403 before the route runs, and that response carries no CORS headers, so the browser reports it as a network error rather than a status. The preflight allows whatever request headers the caller asks for, so a fetch wrapper that adds its own headers needs no extra configuration. `cors` only affects the generated route; with an external `endpoint`, CORS is that backend's job, and setting both is a config error. The endpoint stays unauthenticated either way, so the [rate limiting](#rate-limiting) advice applies to cross-origin traffic too. ## Server output required Blume's built-in assistant backend is a server route (`POST /api/ask`), so it can't run on a static build. Name a host adapter from `blume/deploy` to switch to server output: ```ts blume.config.ts lineNumbers import { vercel } from "blume/deploy"; export default defineConfig({ deployment: vercel(), }); ``` A static build with the assistant enabled and no external `endpoint` fails fast with a message telling you to set a host adapter. See [Deployment](/docs/deployment) for the adapters. ## Adapters `provider` picks the backend that answers. Its value is an **adapter**: a small function exported from `blume/ai` that takes that backend's own options and returns a plain descriptor Blume writes into the generated route. Each adapter owns its model, the env var its key is read from, how it maps [reasoning](#reasoning), and which provider SDK it needs — so there is no shared set of fields to reconcile across backends: ```ts blume.config.ts lineNumbers import { defineConfig } from "blume"; import { openrouter } from "blume/ai"; export default defineConfig({ ai: { assistant: { enabled: true, provider: openrouter({ model: "anthropic/claude-sonnet-4-5" }), }, }, }); ``` | Adapter | Answers with | API key env var | SDK to install | | --- | --- | --- | --- | | [`gateway()`](#vercel-ai-gateway) (default) | a `provider/model` string via the Vercel AI Gateway | `AI_GATEWAY_API_KEY` | none — ships with Blume | | [`openrouter()`](#openrouter) | any [OpenRouter](https://openrouter.ai) model | `OPENROUTER_API_KEY` | `@openrouter/ai-sdk-provider` | | [`llmgateway()`](#llmgateway) | any [LLMGateway](https://llmgateway.io) model | `LLMGATEWAY_API_KEY` | `@ai-sdk/openai-compatible` | | [`inkeep()`](#inkeep) | an [Inkeep](https://inkeep.com) QA model | `INKEEP_API_KEY` | `@ai-sdk/openai-compatible` | | [`openaiCompatible()`](#openai-compatible-endpoints) | whatever your endpoint serves | the `apiKeyEnv` you name | `@ai-sdk/openai-compatible` | The SDKs are optional peer dependencies, so add the one your adapter needs to your project (`npm install @openrouter/ai-sdk-provider`, say). If it's missing, `blume build` stops before Vite runs, naming the package and the command that installs it, and [`blume doctor`](/docs/cli/doctor) reports it too. The descriptor an adapter returns is plain data — its kind, its options, the env vars it reads, and the SDK it needs — so the generated route (and the [ejected](/docs/configuration/customization#eject) one) inlines it as literals and imports the provider SDK by name. Nothing reads `blume.config.ts` at request time, and no secret is ever written into a route: adapters take the **name** of the env var holding the key, and the route reads the value through Astro's [`getSecret()`](https://docs.astro.build/en/guides/environment-variables/#retrieving-secrets-programmatically), so each deployment adapter supplies it its own way — environment variables on Node, Vercel, and Netlify, and the Worker's [bindings](https://docs.astro.build/en/guides/integrations-guide/cloudflare/#environment-variables-and-secrets) on Cloudflare. ### Vercel AI Gateway The default. `model` is a `provider/model` string, so you switch models by changing it (`openai/gpt-5.5`, `anthropic/claude-sonnet-4-5`, and so on) with no provider SDK to install. The gateway reads `AI_GATEWAY_API_KEY` from your environment and is wired up automatically when you deploy on Vercel, where it can also authenticate with the deployment's OIDC token: ```ts blume.config.ts lineNumbers import { defineConfig } from "blume"; import { gateway } from "blume/ai"; export default defineConfig({ ai: { assistant: { enabled: true, provider: gateway({ model: "anthropic/claude-sonnet-4-5" }), }, }, }); ``` Leaving `provider` unset is the same as `gateway({ model: "openai/gpt-5.5" })`. ### OpenRouter Any model on [OpenRouter](https://openrouter.ai), through its dedicated AI SDK provider: ```ts blume.config.ts lineNumbers import { defineConfig } from "blume"; import { openrouter } from "blume/ai"; export default defineConfig({ ai: { assistant: { enabled: true, provider: openrouter({ model: "anthropic/claude-sonnet-4-5", reasoning: "none", }), }, }, }); ``` ### LLMGateway Any model on [LLMGateway](https://llmgateway.io), through its OpenAI-compatible endpoint: ```ts blume.config.ts lineNumbers import { defineConfig } from "blume"; import { llmgateway } from "blume/ai"; export default defineConfig({ ai: { assistant: { enabled: true, provider: llmgateway({ model: "openai/gpt-5.5" }), }, }, }); ``` `baseUrl` overrides the preset endpoint (`https://api.llmgateway.io/v1`) when you run LLMGateway yourself. ### Inkeep [Inkeep](https://inkeep.com) answers from the content you've indexed in the Inkeep dashboard — it runs its own retrieval — so Blume leaves it **ungrounded**: no snapshot of this site's pages is injected, and the [retrieval size](#retrieval-size) options don't apply. It has no reasoning control either, so the adapter takes no `reasoning`: ```ts blume.config.ts lineNumbers import { defineConfig } from "blume"; import { inkeep } from "blume/ai"; export default defineConfig({ ai: { assistant: { enabled: true, provider: inkeep({ model: "inkeep-qa-expert" }), }, }, }); ``` `baseUrl` overrides the preset endpoint (`https://api.inkeep.com/v1`). ### OpenAI-compatible endpoints Any endpoint that speaks the OpenAI API works through `openaiCompatible()` — supply its `baseUrl`, the `model` it serves, and the env var holding its key. A generic endpoint has no preset for any of these, so all three are required; `name` is the provider name the AI SDK reports and defaults to `openai-compatible`: ```ts blume.config.ts lineNumbers import { defineConfig } from "blume"; import { openaiCompatible } from "blume/ai"; export default defineConfig({ ai: { assistant: { enabled: true, provider: openaiCompatible({ baseUrl: "https://my-gateway.example.com/v1", apiKeyEnv: "MY_GATEWAY_API_KEY", model: "gpt-4o", name: "my-gateway", }), }, }, }); ``` ### Options every adapter takes **`apiKeyEnv`** points an adapter at a different env var than its default — `gateway({ apiKeyEnv: "DOCS_GATEWAY_KEY" })` reads that variable instead of `AI_GATEWAY_API_KEY`, and the missing-secret warning at `blume dev`/`build` checks it too. Until the key is set, the deployed route answers `503` with a message naming the variable. The route reads a request body only up to 64 KB and answers anything larger with `413`. **`headers`** sends static request headers with every call — a caller-identifying header for a shared backend, say, so its own observability or rate limiting can tell your docs apart from other traffic: ```ts blume.config.ts lineNumbers provider: openaiCompatible({ baseUrl: "https://llm.internal.example.com/v1", apiKeyEnv: "INTERNAL_LLM_API_KEY", model: "gpt-4o", headers: { "X-Caller-Id": "docs" }, }), ``` The values are written into the generated route as-is, so keep secrets in `apiKeyEnv` rather than in `headers`. The API key's `Authorization` header is applied first, so a custom header can't displace it. **`providerOptions`** passes anything else straight through to the AI SDK's [`providerOptions`](https://ai-sdk.dev/docs/foundations/prompts#provider-options), in the SDK's own shape — keyed by provider, then by option — so a new model control never needs a Blume field of its own: ```ts blume.config.ts lineNumbers provider: gateway({ model: "openai/gpt-5.5", providerOptions: { openai: { textVerbosity: "low" } }, }), ``` Blume maps only the options it names (`model`, `reasoning`, `apiKeyEnv`, `headers`) and forwards `providerOptions` verbatim, so it has to be JSON — it's inlined into the route — and it has to use the key the underlying provider expects (`openai` for an OpenAI model behind the gateway, `openrouter` on OpenRouter). Enabling the assistant also turns on React for the in-page island — see [Customization](/docs/configuration/customization#interactive-islands). ## Reasoning Reasoning models think before they answer, and how much they do so by default varies by model. For grounded docs Q&A the retrieved excerpts carry the answer, so most of that thinking is latency the reader waits through. An adapter's `reasoning` option sets how much the model reasons: `"none"`, `"minimal"`, `"low"`, `"medium"`, `"high"`, or `"xhigh"`: ```ts blume.config.ts lineNumbers provider: gateway({ model: "openai/gpt-5.5", reasoning: "none" }), ``` Each adapter sends the level as its backend's own reasoning control, which is why it lives on the adapter rather than on `assistant`: | Adapter | What the level becomes | | --- | --- | | `gateway()` | The AI SDK's [`reasoning`](https://ai-sdk.dev/docs/ai-sdk-core/reasoning) call option, which the gateway maps to the model's own setting — OpenAI's `reasoning_effort`, for example. | | `openrouter()` | OpenRouter's `reasoning.effort`, set on the model. Its provider ignores the AI SDK's call option, so the level is placed where OpenRouter reads it. | | `llmgateway()` | `reasoning_effort` in the request, through the AI SDK's call option. | | `openaiCompatible()` | `reasoning_effort` in the request, so the endpoint has to accept that parameter. | | `inkeep()` | Not available. Inkeep runs its own QA pipeline with no reasoning control, so the adapter has no `reasoning` option and setting one is a config error. | The model has to support the level you pick: OpenAI rejects a level a model doesn't offer (`"none"` and `"xhigh"` exist only on some), so check the model's documentation before setting one. Leave it unset to keep the model's default. Like [retrieval size](#retrieval-size), it trades thoroughness for time-to-first-token, and answers stay grounded either way. ## Analytics With an [analytics provider](/docs/configuration/analytics) configured, the assistant reports its usage through the same `track()` the page feedback widget uses, so questions land next to your pageviews: | Event | When | Properties | | --- | --- | --- | | `ask` | A question is sent | `path`, `questionChars` | | `ask_answer` | The answer finishes streaming | `path`, `questionChars`, `ms`, `chars` | | `ask_error` | The request fails, breaks, or comes back empty | `path`, `questionChars`, `ms`, `status` | `path` is the page the reader asked from (the served pathname, so it matches the feedback widget and your pageviews under a `base`), `questionChars` the question's length, `ms` the time from sending the question to the last chunk, and `chars` the answer's length. `status` is the HTTP status: `0` when no response arrived at all (offline, DNS, CORS), and `200` when the response was fine but its stream broke mid-answer — how a provider or credential error surfaces, since the backend has already sent its headers — or delivered nothing. Clearing the conversation mid-answer reports neither outcome. The question's text never reaches a provider: it is free-form reader input (pasted keys, error logs, names) that would breach most providers' terms and per-value size limits. It travels only on the `blume:track` DOM event, as `question` in `detail.props`, so a listener you write can forward it wherever you decide it belongs. A custom chat UI built on `useAssistant` from `blume/hooks` reports the same events. With no provider configured the built-in provider calls are no-ops, but the `blume:track` event still fires, so a custom integration listening for it receives them. ## Rate limiting The `POST /api/ask` endpoint is **unauthenticated** — it has to be, so the in-page assistant can call it. Blume validates each request — rejecting malformed bodies, capping it to 1–40 messages, and accepting only `user`/`assistant` roles so a caller can't inject their own system prompt and repurpose the route as a general LLM proxy — to bound how much a single call can spend against your model, but it can't stop someone from calling the endpoint repeatedly. If cost abuse is a concern, put the route behind a rate limiter — your host's (e.g. Vercel's) edge rate limiting, a middleware, or your model provider's per-key spend limits. The endpoint is advertised in the [agent readability manifest](/docs/discoverability/agent-discovery#agent-readability) alongside the rest of the site's machine-readable surface.