Rate limiting

Limit how often one reader can call the assistant, the API playground proxy, and server-side search, on by default, with the count kept in memory, Upstash, Cloudflare, or Unkey.

The server routes a reader can call cost you something each time: the assistant spends model tokens, the API playground proxy relays requests to your API, and Mixedbread search runs a paid query. Blume limits how often one reader, identified by IP address, can call each of them. A reader over the limit gets 429 Too Many Requests with a Retry-After header, and the assistant tells them to try again in a few minutes.

Rate limiting is on by default, at 30 requests per reader per route every 10 minutes: well above what a person sends to the assistant or the playground, and low enough to stop a script. Search counts differently. The search dialog sends a query each time the reader pauses while typing, so one search can take several requests, and a reader who searches steadily for a few minutes can reach the limit. If your site uses Mixedbread search, raise requests, which applies to every route. Rate limiting only matters for a server build; a static site has no server routes to limit.

import { defineConfig } from "blume";
import { upstash } from "blume/ratelimit";

export default defineConfig({
  rateLimit: upstash({ requests: 30, window: 600 }), // window in seconds
});

Each route keeps its own count, so a busy playground session doesn’t use up the reader’s assistant questions.

Where the count lives

Memory

memory() needs nothing. On a server that runs as one process, like deployment: node(), the count is exact. Vercel, Netlify, and Cloudflare run many short-lived instances, and each keeps its own count, so there it stops a burst from one reader rather than enforcing the limit exactly. For an exact limit on those hosts, use one of the shared stores below.

import { memory } from "blume/ratelimit";

rateLimit: memory({ requests: 60, window: 600 }),

Upstash

upstash() keeps the count in Upstash Redis, which every instance shares. Create a database, then set its REST endpoint and token as UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN. Blume talks to Upstash’s REST API, so there’s nothing to install. Until both are set, the routes count in memory and blume build warns.

import { upstash } from "blume/ratelimit";

rateLimit: upstash(),

A database added from Vercel’s Marketplace sets its endpoint and token as KV_REST_API_URL and KV_REST_API_TOKEN instead, with any prefix you chose when you connected it. Name the variables with urlEnv and tokenEnv:

rateLimit: upstash({ urlEnv: "KV_REST_API_URL", tokenEnv: "KV_REST_API_TOKEN" }),

Cloudflare

cloudflare() counts with Workers rate limiting, for a site deployed with deployment: cloudflare(). Blume declares the binding in the Worker’s config when it builds, so there’s nothing to set up. Cloudflare counts per location and takes a window of 10 or 60 seconds, so its defaults are 10 requests per 60 seconds.

import { cloudflare } from "blume/deploy";
import { cloudflare as cloudflareRateLimit } from "blume/ratelimit";

export default defineConfig({
  deployment: cloudflare(),
  rateLimit: cloudflareRateLimit({ requests: 10, window: 60 }),
});

The binding’s namespace comes from the Worker’s name, so two sites on one account don’t share a count. Set namespaceId to choose it yourself.

Unkey

unkey() keeps the count in Unkey’s rate limiting, which every instance shares. Create a root key with the ratelimit.*.limit permission, plus ratelimit.*.create_namespace so Unkey can create the namespace on the first request, then set it as UNKEY_ROOT_KEY. Blume talks to Unkey’s REST API, so there’s nothing to install. Until the key is set, the routes count in memory and blume build warns.

import { unkey } from "blume/ratelimit";

rateLimit: unkey(),

The namespace is docs unless you name another with namespace. Name the variable that holds the key with rootKeyEnv:

rateLimit: unkey({ namespace: "acme-docs", rootKeyEnv: "DOCS_UNKEY_ROOT_KEY" }),

Unkey counts in each region and shares the counts between regions within moments, so a short burst split across regions can get a few more requests through than the limit. Every count fails with a key Unkey refuses, like one missing those permissions, and a request a shared store fails to count is let through, so if the server logs Rate limiting failed, check the key.

Behind a reverse proxy

Vercel, Netlify, and Cloudflare tell Blume each reader’s address. A node() server behind a reverse proxy, like nginx or Caddy, sees every request come from the proxy, so every reader shares one count and a busy site hits the limit for everyone. List the host the proxy answers for in allowedDomains:

import { node } from "blume/deploy";

deployment: node({
  site: "https://docs.example.com",
  allowedDomains: [{ hostname: "docs.example.com" }],
}),

This is Astro’s security.allowedDomains: for a request whose Host or X-Forwarded-Host header matches an entry, Astro takes the reader’s address from the first entry of X-Forwarded-For. Have the proxy pass the site’s host through and replace any X-Forwarded-For the reader sent with the address it sees, like nginx’s proxy_set_header X-Forwarded-For $remote_addr;. A proxy that appends to the header instead lets a reader pick their own address and get a fresh count with every request. Leave allowedDomains unset when readers reach the server directly.

Turning it off

Set rateLimit: false to let every request through, when your host’s firewall already limits these routes, say. A request whose address the host can’t tell is always let through, and so is one a shared store fails to count: the failure is logged, and readers never get locked out by a store outage.

For the assistant, pair rate limiting with a bot check to stop scripts that spread their requests across many addresses.