---
title: Sitemap and robots
description: The sitemap.xml and robots.txt Blume writes for crawlers, and the Content-Signal line that tells AI crawlers how they may use your docs.
---

Search crawlers and AI crawlers both start from the same two files at your site root. Blume writes both on every build, and the `robots.txt` carries the content-usage signals that AI crawlers look for, so this is where the search side and the agent side of discoverability meet.

## Sitemap

Blume writes a `sitemap.xml` of every indexable page at build time. It needs an absolute [`deployment.site`](/docs/deployment) and lists every page except drafts, hidden, and [`noindex`](/docs/discoverability/metadata#per-page-overrides) pages. On a [versioned](/docs/content/versioning) site, archived pages whose canonical points at their live equivalent are left out too — the live page is the one to index. On by default:

```ts blume.config.ts lineNumbers
seo: {
  sitemap: true,
}
```

Ship your own `public/sitemap.xml` to take over — Blume never overwrites a file you place in `public/`.

## Robots

Blume writes a `robots.txt` that allows all crawlers, declares your [content signals](#content-signals), and adds a `Sitemap:` line pointing to the sitemap when one is available. On by default:

```ts blume.config.ts lineNumbers
seo: {
  robots: true,
}
```

```txt robots.txt
User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=yes
Allow: /

Sitemap: https://docs.example.com/sitemap.xml
```

### Content signals

The `Content-Signal` line — the emerging content-usage convention — declares how AI crawlers may reuse your docs. Blume emits it **on by default with every signal set to `yes`**, matching its stance that docs are open to humans and agents alike:

- `search` — traditional and AI search indexing
- `aiInput` — grounding / RAG at answer time
- `aiTrain` — model training

Restrict any signal by setting it to `false`; the ones you leave out stay `yes`:

```ts blume.config.ts lineNumbers
seo: {
  contentSignals: {
    aiTrain: false, // opt out of training, keep search + grounding
  },
}
```

```txt robots.txt
User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=no
Allow: /
```

Set `contentSignals: false` to drop the declaration entirely:

```ts blume.config.ts lineNumbers
seo: {
  contentSignals: false,
}
```

| Prop | Type | Default | Description |
| - | - | - | - |
| `seo.contentSignals?` | `boolean \| object` | - | Content-Signal declaration. true or omitted emits all signals as yes; false drops the line; an object sets signals individually. |
| `contentSignals.search?` | `boolean` | - | Allow use for search indexing (search). Default true. |
| `contentSignals.aiInput?` | `boolean` | - | Allow use for AI grounding / RAG at answer time (ai-input). Default true. |
| `contentSignals.aiTrain?` | `boolean` | - | Allow use for AI model training (ai-train). Default true. |

Content signals express a preference, not access control: they tell well-behaved crawlers how you'd like your content used, and it's on the crawler to honor them. The same policy is mirrored as `contentUsage` in the [agent readability manifest](/docs/discoverability/agent-discovery#agent-readability), so an agent that never reads `robots.txt` still sees it.

Ship your own `public/robots.txt` to take over.