Assistant

An in-page assistant grounded in your docs — suggested questions, custom instructions, retrieval sizing, provider adapters from the Vercel AI Gateway to any OpenAI-compatible endpoint, and the server output it needs.

Add an assistant that answers reader questions in an in-page chat panel, backed by a streaming server endpoint and the AI SDK. It’s opt-in, and static docs stay fully static until you turn it on:

ai: {
  assistant: {
    enabled: true,
  },
}

With nothing else written, answers stream through the Vercel AI Gateway from openai/gpt-5.5. Pick a different model or provider with an adapter.

Suggested questions

Seed the empty state with a few starter prompts. Each renders as a clickable suggestion — click one to send it — with an optional Lucide icon beside the label:

ai: {
  assistant: {
    enabled: true,
    suggestions: [
      { label: "What is Blume?", icon: "rocket" },
      { label: "How do I write a docs page?", icon: "file-text" },
      { label: "How do I configure the theme?", icon: "settings" },
    ],
  },
}

label is the question that gets asked; icon is optional. Leave suggestions unset (or empty) and the panel opens to a plain input.

Custom instructions

Add your own system-prompt text with instructions — identity, language, tone, or anything else the assistant should keep in mind:

ai: {
  assistant: {
    enabled: true,
    instructions:
      "You are Bloomy, the Acme docs assistant. Answer in the language the question was asked in, and keep answers under three paragraphs.",
  },
}

Your text is appended to the built-in instructions rather than replacing them: the built-in part carries the grounding contract — answer only from the retrieved pages, cite them as Markdown links — that the chat panel’s citations depend on, so it stays intact whatever you add.

Grounding

The assistant is grounded in your docs. For each question it retrieves the most relevant pages — using the same lexical Orama index that powers on-page search — and injects them into the model’s system prompt, so answers come from your content instead of the model’s own knowledge. The assistant is told to answer only from the retrieved pages, to say when something isn’t covered, and to cite the pages it drew from.

The page the reader is currently on is added to the context first and used to scope retrieval to that page’s language, so answers stay relevant to where they are in the docs. Retrieval runs at request time from a snapshot baked into the build, so it works regardless of your search provider — even with search: false — and needs no configuration.

Grounding is on for every adapter except Inkeep, which runs its own retrieval over the content you’ve indexed in its dashboard.

Retrieval size

How much documentation a question carries is the biggest lever on how long the reader waits for the first word: the model reads every injected character before it emits a token. On a hosted frontier model that’s invisible, but on a self-hosted backend it dominates. retrieval sizes it:

ai: {
  assistant: {
    enabled: true,
    retrieval: {
      maxResults: 3, // fewer pages retrieved per question
      excerptChars: 1200, // shorter excerpt from each one
      contextBudget: 3000, // smaller total injection
    },
  },
}

The three aren’t interchangeable. contextBudget caps the whole injection, excerptChars decides how deep into a single long page its excerpt reaches — raise it when one page holds the whole answer and the excerpt cuts it off — and maxResults caps how many pages retrieval adds. The page the reader is viewing is injected on top of the retrieved ones, so an answer can cite up to one page more than maxResults.

The defaults suit a hosted model. Lower them when you’re serving from your own hardware and time-to-first-token matters more than recall; answers stay grounded either way, and the assistant is told to say when something isn’t covered rather than fill the gap.

External endpoint

Already have an API backend for AI? Point the panel at it and keep the docs build static:

ai: {
  assistant: {
    enabled: true,
    endpoint: "https://api.example.com/v1/docs/ask",
  },
}

Blume sends the same POST body as its built-in route:

{
  "messages": [{ "role": "user", "content": "How do I deploy?" }],
  "page": { "path": "/deployment" }
}

Return a successful response whose body is a plain UTF-8 text stream. If the endpoint is on another origin, allow the docs origin with CORS: accept OPTIONS and POST, permit the content-type request header, and return the CORS headers on both the preflight and streamed response. With endpoint set, Blume generates the chat UI but no server route, grounding snapshot, provider dependency, or provider-secret warning; your backend owns retrieval, authentication, rate limiting, model access, and citations. An adapter set alongside it is ignored.

Cross-origin callers

The generated endpoint answers the in-page assistant on its own origin. To call it from another site as well — a marketing page with an ask box, say — list that site’s origin in cors:

ai: {
  assistant: {
    enabled: true,
    cors: ["https://www.example.com"],
  },
}

The route then answers the browser’s OPTIONS preflight and names a listed origin on every response — the streamed answer and the error statuses alike, so the caller can tell a rejected body from a provider failure. Origins that aren’t listed get no header and stay subject to the browser’s same-origin rule. Each entry is reduced to its origin, so https://www.example.com/docs/ and https://www.example.com mean the same thing. To let any page call the route, list "*" instead of origins.

The caller sends the same POST body the external endpoint contract describes and reads back the same text stream. Send it as JSON with a content-type: application/json header:

const response = await fetch("https://docs.example.com/api/ask", {
  body: JSON.stringify({
    messages: [{ role: "user", content: "How do I deploy?" }],
  }),
  headers: { "content-type": "application/json" },
  method: "POST",
});

The content type matters: Astro’s cross-site request check rejects a cross-origin POST that has no content type, or a form-like one such as text/plain, with a 403 before the route runs, and that response carries no CORS headers, so the browser reports it as a network error rather than a status. The preflight allows whatever request headers the caller asks for, so a fetch wrapper that adds its own headers needs no extra configuration.

cors only affects the generated route; with an external endpoint, CORS is that backend’s job, and setting both is a config error. The endpoint stays unauthenticated either way, so the rate limiting advice applies to cross-origin traffic too.

Server output required

Blume’s built-in assistant backend is a server route (POST /api/ask), so it can’t run on a static build. Name a host adapter from blume/deploy to switch to server output:

import { vercel } from "blume/deploy";

export default defineConfig({
  deployment: vercel(),
});

A static build with the assistant enabled and no external endpoint fails fast with a message telling you to set a host adapter. See Deployment for the adapters.

Adapters

provider picks the backend that answers. Its value is an adapter: a small function exported from blume/ai that takes that backend’s own options and returns a plain descriptor Blume writes into the generated route. Each adapter owns its model, the env var its key is read from, how it maps reasoning, and which provider SDK it needs — so there is no shared set of fields to reconcile across backends:

import { defineConfig } from "blume";
import { openrouter } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: openrouter({ model: "anthropic/claude-sonnet-4-5" }),
    },
  },
});

The SDKs are optional peer dependencies, so add the one your adapter needs to your project (npm install @openrouter/ai-sdk-provider, say). If it’s missing, blume build stops before Vite runs, naming the package and the command that installs it, and blume doctor reports it too.

The descriptor an adapter returns is plain data — its kind, its options, the env vars it reads, and the SDK it needs — so the generated route (and the ejected one) inlines it as literals and imports the provider SDK by name. Nothing reads blume.config.ts at request time, and no secret is ever written into a route: adapters take the name of the env var holding the key, and the route reads the value through Astro’s getSecret(), so each deployment adapter supplies it its own way — environment variables on Node, Vercel, and Netlify, and the Worker’s bindings on Cloudflare.

Vercel AI Gateway

The default. model is a provider/model string, so you switch models by changing it (openai/gpt-5.5, anthropic/claude-sonnet-4-5, and so on) with no provider SDK to install. The gateway reads AI_GATEWAY_API_KEY from your environment and is wired up automatically when you deploy on Vercel, where it can also authenticate with the deployment’s OIDC token:

import { defineConfig } from "blume";
import { gateway } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: gateway({ model: "anthropic/claude-sonnet-4-5" }),
    },
  },
});

Leaving provider unset is the same as gateway({ model: "openai/gpt-5.5" }).

OpenRouter

Any model on OpenRouter, through its dedicated AI SDK provider:

import { defineConfig } from "blume";
import { openrouter } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: openrouter({
        model: "anthropic/claude-sonnet-4-5",
        reasoning: "none",
      }),
    },
  },
});

LLMGateway

Any model on LLMGateway, through its OpenAI-compatible endpoint:

import { defineConfig } from "blume";
import { llmgateway } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: llmgateway({ model: "openai/gpt-5.5" }),
    },
  },
});

baseUrl overrides the preset endpoint (https://api.llmgateway.io/v1) when you run LLMGateway yourself.

Inkeep

Inkeep answers from the content you’ve indexed in the Inkeep dashboard — it runs its own retrieval — so Blume leaves it ungrounded: no snapshot of this site’s pages is injected, and the retrieval size options don’t apply. It has no reasoning control either, so the adapter takes no reasoning:

import { defineConfig } from "blume";
import { inkeep } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: inkeep({ model: "inkeep-qa-expert" }),
    },
  },
});

baseUrl overrides the preset endpoint (https://api.inkeep.com/v1).

OpenAI-compatible endpoints

Any endpoint that speaks the OpenAI API works through openaiCompatible() — supply its baseUrl, the model it serves, and the env var holding its key. A generic endpoint has no preset for any of these, so all three are required; name is the provider name the AI SDK reports and defaults to openai-compatible:

import { defineConfig } from "blume";
import { openaiCompatible } from "blume/ai";

export default defineConfig({
  ai: {
    assistant: {
      enabled: true,
      provider: openaiCompatible({
        baseUrl: "https://my-gateway.example.com/v1",
        apiKeyEnv: "MY_GATEWAY_API_KEY",
        model: "gpt-4o",
        name: "my-gateway",
      }),
    },
  },
});

Options every adapter takes

apiKeyEnv points an adapter at a different env var than its default — gateway({ apiKeyEnv: "DOCS_GATEWAY_KEY" }) reads that variable instead of AI_GATEWAY_API_KEY, and the missing-secret warning at blume dev/build checks it too. Until the key is set, the deployed route answers 503 with a message naming the variable. The route reads a request body only up to 64 KB and answers anything larger with 413.

headers sends static request headers with every call — a caller-identifying header for a shared backend, say, so its own observability or rate limiting can tell your docs apart from other traffic:

provider: openaiCompatible({
  baseUrl: "https://llm.internal.example.com/v1",
  apiKeyEnv: "INTERNAL_LLM_API_KEY",
  model: "gpt-4o",
  headers: { "X-Caller-Id": "docs" },
}),

The values are written into the generated route as-is, so keep secrets in apiKeyEnv rather than in headers. The API key’s Authorization header is applied first, so a custom header can’t displace it.

providerOptions passes anything else straight through to the AI SDK’s providerOptions, in the SDK’s own shape — keyed by provider, then by option — so a new model control never needs a Blume field of its own:

provider: gateway({
  model: "openai/gpt-5.5",
  providerOptions: { openai: { textVerbosity: "low" } },
}),

Blume maps only the options it names (model, reasoning, apiKeyEnv, headers) and forwards providerOptions verbatim, so it has to be JSON — it’s inlined into the route — and it has to use the key the underlying provider expects (openai for an OpenAI model behind the gateway, openrouter on OpenRouter). Enabling the assistant also turns on React for the in-page island — see Customization.

Reasoning

Reasoning models think before they answer, and how much they do so by default varies by model. For grounded docs Q&A the retrieved excerpts carry the answer, so most of that thinking is latency the reader waits through. An adapter’s reasoning option sets how much the model reasons: "none", "minimal", "low", "medium", "high", or "xhigh":

provider: gateway({ model: "openai/gpt-5.5", reasoning: "none" }),

Each adapter sends the level as its backend’s own reasoning control, which is why it lives on the adapter rather than on assistant:

The model has to support the level you pick: OpenAI rejects a level a model doesn’t offer ("none" and "xhigh" exist only on some), so check the model’s documentation before setting one. Leave it unset to keep the model’s default. Like retrieval size, it trades thoroughness for time-to-first-token, and answers stay grounded either way.

Analytics

With an analytics provider configured, the assistant reports its usage through the same track() the page feedback widget uses, so questions land next to your pageviews:

path is the page the reader asked from (the served pathname, so it matches the feedback widget and your pageviews under a base), questionChars the question’s length, ms the time from sending the question to the last chunk, and chars the answer’s length. status is the HTTP status: 0 when no response arrived at all (offline, DNS, CORS), and 200 when the response was fine but its stream broke mid-answer — how a provider or credential error surfaces, since the backend has already sent its headers — or delivered nothing. Clearing the conversation mid-answer reports neither outcome.

The question’s text never reaches a provider: it is free-form reader input (pasted keys, error logs, names) that would breach most providers’ terms and per-value size limits. It travels only on the blume:track DOM event, as question in detail.props, so a listener you write can forward it wherever you decide it belongs. A custom chat UI built on useAssistant from blume/hooks reports the same events. With no provider configured the built-in provider calls are no-ops, but the blume:track event still fires, so a custom integration listening for it receives them.

Rate limiting

The POST /api/ask endpoint is unauthenticated — it has to be, so the in-page assistant can call it. Blume validates each request — rejecting malformed bodies, capping it to 1–40 messages, and accepting only user/assistant roles so a caller can’t inject their own system prompt and repurpose the route as a general LLM proxy — to bound how much a single call can spend against your model, but it can’t stop someone from calling the endpoint repeatedly. If cost abuse is a concern, put the route behind a rate limiter — your host’s (e.g. Vercel’s) edge rate limiting, a middleware, or your model provider’s per-key spend limits.

The endpoint is advertised in the agent readability manifest alongside the rest of the site’s machine-readable surface.