Custom Provider Guide

1. How providers work

A provider is any upstream AI service the gateway can chat with. Every provider lives in src/providers/ and exposes the same tiny contract, so the rest of the gateway (validation, errors, rate limits, response shape) works identically no matter who answers.

your app ──▶ POST /api/v1/chat { "provider": "poolside", ... }
                    │
                    ▼
            AI Gateway (src/index.js)
                    │  validate → rate-limit → pick provider (src/providers/registry.js)
                    ▼
   ┌─────────────────┼──────────────────┐
   ▼                 ▼                  ▼
openrouter.js    poolside.js        custom.js
OpenRouter API   Poolside API       ANY OpenAI-compatible URL (env-driven)

2. Three ways to add a provider

3. Method 1 — Zero code (the custom provider)

The gateway ships with a universal adapter, src/providers/custom.js. It reads its target from environment variables, so pointing it at a new vendor requires no code and no redeploy of code — only env vars:

Then call it with "provider": "custom":

curl -X POST https://YOUR-DOMAIN/api/v1/chat \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "custom",
    "message": "Hello from my own provider!"
  }'

Known-good base URLs

4. Method 2 — Add a dedicated provider file (recommended)

This is exactly how Poolside AI was added to this project. A provider that speaks the OpenAI protocol needs one tiny file using the factory, then a 2-line registration.

Step 1 — create src/providers/poolside.js

import { createOpenAICompatible } from './openai-compatible.js';

export const poolsideProvider = createOpenAICompatible({
  id: 'poolside',                                   // unique, lowercase
  label: 'Poolside AI',                             // shown in /providers
  description: 'Poolside Laguna code-intelligence models…',
  defaultBaseUrl: 'https://inference.poolside.ai/v1',
  envKey: 'POOLSIDE_API_KEY',                       // env var holding the key
  modelEnvKey: 'POOLSIDE_MODEL',                    // optional model override
  defaultModel: 'poolside/laguna-s-2.1',
  models: ['poolside/laguna-s-2.1'],
  supportsMaxTokens: true,
});

Step 2 — register it in src/providers/registry.js

import { openrouterProvider } from './openrouter.js';
import { poolsideProvider } from './poolside.js';    // ← 1. import
import { customProvider } from './custom.js';

const BUILT_IN_PROVIDERS = [
  openrouterProvider,
  poolsideProvider,                                  // ← 2. add to the list
  customProvider,
];

Step 3 — set the API key as an env var

Locally: add POOLSIDE_API_KEY=your-key to .env (Node) or .dev.vars (wrangler). On Cloudflare: see section 7. Never hard-code the key in the file itself.

Step 4 — restart / redeploy

Local: stop (Ctrl+C) and re-run npm start. Cloudflare: push to GitHub and Pages redeploys automatically.

Step 5 — call it

curl https://YOUR-DOMAIN/api/v1/chat \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{ "provider": "poolside",
        "model": "poolside/laguna-s-2.1",
        "messages": [{ "role": "user", "content": "What are channels in Go?" }] }'

5. Method 3 — the template (full control)

If the provider does not speak the OpenAI protocol (different request shape, different auth, custom response format), copy src/providers/_template.js. It is a fully annotated skeleton of the provider contract — you implement isConfigured(env) and chat(payload, ctx) yourself, throw typed ApiErrors, and register the result exactly like Step 2 above.

export const myProvider = {
  id: 'myprovider',
  label: 'My Provider',
  description: '…',
  defaultModel: 'my-model-v1',
  models: ['my-model-v1'],
  requiredEnvKeys: ['MYPROVIDER_API_KEY'],

  isConfigured(env) {
    return Boolean(env.MYPROVIDER_API_KEY);
  },

  async chat(payload, { env, config }) {
    // build YOUR request shape, fetch with utils/http.js postJson(),
    // map failures with upstreamError() from core/errors.js,
    // then return { reply, model, usage } — the gateway does the rest.
  },
};

6. Provider contract reference

Every option accepted by createOpenAICompatible() — the factory used by OpenRouter, Poolside and custom:

The factory handles everything else: auth header, timeout, error mapping (401→UPSTREAM_AUTH_ERROR, 429→UPSTREAM_RATE_LIMIT…), usage normalization and the success envelope.

7. Setting provider keys on Cloudflare Pages

1. Open the Cloudflare dashboard → Workers & Pages → click your Pages project.
2. Go to Settings → Variables and Secrets.
3. Fastest: click Import .env and upload a prepared env file (all variables at once) — or click + Add, choose type Secret, and enter the name (e.g. POOLSIDE_API_KEY) and the key value.
4. Add it to both environments: Production and Preview (tabs at the top of the section).
5. Redeploy — Deployments → latest deployment → Retry deployment, or just push any commit. New variables only take effect on a fresh deployment.

8. Verify it works

Two quick checks after deploying:

curl https://YOUR-DOMAIN/api/v1/providers
# → find your provider in the list; "configured": true is required

curl -X POST https://YOUR-DOMAIN/api/v1/chat \
  -H "Content-Type: application/json" \
  -d '{ "provider": "poolside", "failover": false, "message": "ping" }'

Prefer clicking over curl? The API Status page has a per-provider Test button that runs this check from the browser.

9. FAQ & troubleshooting

Can I add more than one custom provider?

Yes — Method 1's custom slot holds one vendor, but Method 2 lets you add unlimited providers (one file each). Many teams keep custom as their staging playground and promote serious vendors to dedicated files.

How do I know if a vendor is OpenAI-compatible?

If this curl returns JSON with choices[0].message.content, it is compatible:

curl https://VENDOR-URL/v1/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d '{ "model": "their-model", "messages": [{ "role": "user", "content": "hi" }] }'

Troubleshooting table