API Documentation

Introduction

AI Gateway API is a multi-provider REST gateway for LLM chat completions. Instead of hard-coding one vendor, you send every request to your own gateway and choose the AI provider per request. The gateway validates input, applies security and rate limits, attaches the correct server-side API key, and returns a normalized response.

It runs on two runtimes with the exact same code:

• Cloudflare Pages Functions (recommended) — serverless, global edge, free tier.
• Node.js 18+ — npm start for local development or self-hosting.

Quick start

1. Deploy the gateway (or run npm start locally) — see the deployment guide.
2. Set at least one provider key, e.g. OPENROUTER_API_KEY or POOLSIDE_API_KEY.
3. Send your first chat request:

curl -X POST https://YOUR-DOMAIN/api/v1/chat \
  -H "Content-Type: application/json" \
  -d '{ "message": "Hello!" }'

No-code option: paste this URL in any browser and the JSON answer appears directly — /ask?prompt=Hello gateway (full details in GET /api/v1/chat).

The base URL above is auto-detected — this same page shows your live domain when deployed (https://your-project.pages.dev) and http://localhost when running locally. No provider field? The gateway uses DEFAULT_PROVIDER (default: openrouter) and — if that provider errors — automatically fails over to the next configured one (see Auto-failover).

Authentication

The gateway has an optional shared key for protecting your API from third parties. It is independent of the provider keys.

GET /api/v1/health, the legacy GET /health and GET /api/v1/config always stay public so uptime monitors and the docs site work.

curl -X POST https://YOUR-DOMAIN/api/v1/chat \
  -H "Content-Type: application/json" \
  -H "X-API-Key: YOUR_GATEWAY_KEY" \
  -d '{ "message": "Hello!" }'

Wrong or missing keys get 401 UNAUTHORIZED. Key comparison is constant-time to prevent timing attacks.

Providers

A provider is any upstream AI service. Discovery endpoint: GET /api/v1/providers lists every registered provider, its default model and whether its key is configured. Built-ins:

Model overrides per provider: OPENROUTER_MODEL, OPENCODEZEN_MODEL, POOLSIDE_MODEL — or pass model in the request body. Adding more providers is documented on the Custom Provider page.

GET /api/v1/models — live catalog

GET /api/v1/models auto-fetches the model list from every provider — nothing is hardcoded. The gateway calls each provider's own /models endpoint in parallel and applies the per-provider free-only policy:

• OpenRouter → only :free models (e.g. z-ai/glm-5.2:free).
• OpenCode Zen → only -free models (e.g. deepseek-v4-flash-free).
• Every other provider (Poolside, custom, …) → the full catalog.

Results are cached for MODELS_CACHE_TTL_MS (default 6 h); ?refresh=1 re-fetches immediately and ?provider=<id> filters to one provider. A failing provider never breaks the endpoint — its error is reported inside providers[].error while the rest still return data. Browse it visually on the Models page.

curl https://YOUR-DOMAIN/api/v1/models

Automatic provider failover

On by default. If the chosen provider fails for any reason — invalid key, exhausted quota, rate limit, timeout, outage, malformed reply — the gateway transparently retries the next configured provider and keeps cycling the chain (rotating the starting provider each round) until a response is produced or every option is exhausted. Your client code does nothing; it just always gets an answer when at least one provider is healthy.

1. The preferred provider is tried first (the request provider field, else DEFAULT_PROVIDER).
2. On any error, the remaining configured providers are tried in registry order.
3. Each new round rotates the starting provider; up to FAILOVER_MAX_ROUNDS rounds (default 3).
4. Permanent errors (bad key, provider not configured) are not retried; transient ones (rate limit, timeout, outage) are.

{
  "success": true,
  "provider": "poolside",
  "reply": "Channels are typed conduits…",
  "model": "poolside/laguna-s-2.1",
  "usage": { "prompt_tokens": 14, "completion_tokens": 43, "total_tokens": 57 },
  "failover": {
    "enabled": true,
    "attempts": [
      { "provider": "openrouter", "status": "failed", "code": "UPSTREAM_RATE_LIMIT", "http": 429, "round": 1 }
    ],
    "rounds": 1,
    "servedOnRound": 1
  }
}

The winning provider is also reported in the X-AI-Gateway-Provider response header. Disable failover per request with "failover": false (useful for per-provider health probes), or globally with FAILOVER_ENABLED=false. When every provider fails you get 502 ALL_PROVIDERS_FAILED with the full attempt log.

curl -X POST https://YOUR-DOMAIN/api/v1/chat \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "poolside",
    "failover": false,
    "message": "What are channels in Go?"
  }'

GET /api/v1/chat — browser-clickable

A zero-setup entry point that shares the exact same failover core as the POST endpoint: open the URL in a browser (or integrate it anywhere a plain URL works) and the JSON answer appears directly — no headers, no body, no API key. The short alias /ask serves the identical handler: /ask?prompt=Say hello in Bangla

<span class="live-base">https://YOUR-DOMAIN</span>/api/v1/chat?prompt=What+is+an+API+gateway%3F
<span class="live-base">https://YOUR-DOMAIN</span>/ask?prompt=Write+a+haiku+about+clouds&raw=1

POST /api/v1/chat

The main endpoint. Accepts two input styles — exactly one is required.

Style 1 — simple message

Style 2 — multi-turn conversation

Shared optional fields

Examples

curl -X POST https://YOUR-DOMAIN/api/v1/chat \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "poolside",
    "model": "poolside/laguna-s-2.1",
    "system_prompt": "You are a concise senior Go developer.",
    "message": "What are channels in Go?",
    "temperature": 0.5,
    "max_tokens": 400
  }'

curl -X POST https://YOUR-DOMAIN/api/v1/chat \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "openrouter",
    "messages": [
      { "role": "system",    "content": "You are a helpful assistant." },
      { "role": "user",      "content": "Hi!" },
      { "role": "assistant", "content": "Hello! How can I help?" },
      { "role": "user",      "content": "Explain HTTP status codes briefly." }
    ]
  }'

Validation failures return 400 VALIDATION_ERROR with a details.field pointing at the problem.

Responses

Success — 200

{
  "success": true,
  "provider": "poolside",
  "reply": "Channels are typed conduits that let goroutines exchange values safely…",
  "model": "poolside/laguna-s-2.1",
  "usage": { "prompt_tokens": 14, "completion_tokens": 43, "total_tokens": 57 },
  "failover": { "enabled": true, "attempts": [], "rounds": 1, "servedOnRound": 1 }
}

• reply — the assistant text, extracted from the provider response.
• model — the model that actually answered (echoes upstream when available).
• usage — token counts normalized to the OpenAI shape; null when a provider omits them.

Error — any failure

{
  "success": false,
  "error": {
    "code": "UPSTREAM_RATE_LIMIT",
    "message": "OpenRouter rate limit reached (free tiers throttle temporarily). Retry shortly.",
    "details": { "retryAfterSeconds": 5 }
  }
}

Error codes

Every failure uses the same envelope. These are all the codes the gateway can emit:

Rate limits

Responses include X-RateLimit-Limit and X-RateLimit-Remaining; 429s include Retry-After.

Runtime config & base URL auto-detection

GET /api/v1/config (public, no auth) returns the live server configuration. The docs site calls it from every page, which is how the version badge, base URL and failover status stay in sync with your environment:

{
  "success": true,
  "appName": "AI Gateway API",
  "version": "1.0.0",
  "apiBaseUrl": "https://your-project.pages.dev",
  "defaultProvider": "openrouter",
  "failover": { "enabled": true, "maxRounds": 3, "retryDelayMs": 800 },
  "providers": [
    { "id": "openrouter", "label": "OpenRouter", "configured": true, "defaultModel": "z-ai/glm-5.2:free" },
    { "id": "poolside", "label": "Poolside AI", "configured": true, "defaultModel": "poolside/laguna-s-2.1" },
    { "id": "custom", "label": "Custom (OpenAI-compatible)", "configured": false, "defaultModel": null }
  ]
}

Base URL auto-detection: apiBaseUrl is derived from the incoming request itself, so the same deployment reports the correct URL on localhost, on *.pages.dev and on any custom domain. In the browser, pages use their own origin by default (override at build time with API_BASE_URL if you host the API on a separate domain).

Admin panel (GitHub-backed, full CRUD)

/admin/ is an owner-only panel that manages this site's editable configuration through your GitHub repository. Sign in with a repository name + branch + fine-grained GitHub token (permission needed: Contents: Read and write on that single repository) and the panel:

• Loads everything from the repo — config/site.json (branding, announcement banner, chat-bot settings) and config/providers.json (display order, taglines, visibility).
• Commits every change straight back to the repo via the GitHub API — no local files, no manual git.
• Cloudflare Pages then rebuilds and redeploys automatically, so admin changes go live in 1–2 minutes.
• Shows recent commits touching config/ so you can audit every change.

Security: the token stays in your browser tab (session storage) and is sent only to api.github.com. Provider API keys are never editable in the panel — those remain Cloudflare environment secrets.

Configuration

All settings are environment variables — on Cloudflare you set them in the dashboard; locally put them in .env (Node) or .dev.vars (wrangler). Change a value once and every endpoint, page and badge reflects it after redeploy.

CORS

Browser calls are allowed from any origin by default (Access-Control-Allow-Origin: *), with preflight OPTIONS handled automatically. To restrict to your own sites, set ALLOWED_ORIGINS to a comma-separated list:

ALLOWED_ORIGINS=https://mysite.com,https://app.mysite.com

Deployment

The project is built for Cloudflare Pages: build command npm run build, build output out, root directory empty. The API itself runs as Pages Functions from the functions/ directory — no extra config needed.

Step-by-step instructions (GitHub push → Pages connect → env vars) live on the Integration Guide → Deployment section.