Add BYO-LLM to your product

In this tutorial, we are going to add BYO-LLM to an existing application.

Your backend will keep using the OpenAI or Anthropic SDK. The difference is that each request goes through AI Gateway with the ID of the customer making it. AI Gateway then sends the request to the provider account that customer connected.

╔══════════════════════╗      ┌────────────────┐      ╔════════════════════════╗
║ Your cloud           ║      │ Alien          │      ║ Customer provider      ║
║                      ║      │                │      ║                        ║
║ Backend              ║─────▶│ AI Gateway     │─────▶║ Model provider account ║
║ + customer ID        ║◀─────│                │◀─────║ connected during setup ║
╚══════════════════════╝      └────────────────┘      ╚════════════════════════╝

For example, the same request for byo/claude-opus-5 can use:

Amazon Bedrock in one customer's AWS account;
Vertex AI in another customer's Google Cloud project;
Azure AI Foundry, Anthropic, or Databricks for another customer.

Your backend does not need AWS, Google Cloud, Azure, or provider credentials. It keeps one Alien API key and one model integration.

Start with the request your product already makes

Suppose your backend currently calls OpenAI like this:

src/ai.ts

import OpenAI from "openai"

const ai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })

const response = await ai.chat.completions.create({
  model: "gpt-5.6-sol",
  messages: [{ role: "user", content: "Summarize this document" }],
})

Adding AI Gateway changes three values:

src/ai.ts

const ai = new OpenAI({
  baseURL: "https://ai.alien.dev/v1", 
  apiKey: process.env.ALIEN_AI_API_KEY,
  defaultHeaders: {
    "X-Alien-External-ID": customer.id, 
  },
})

const response = await ai.chat.completions.create({
  model: "byo/gpt-5.6-sol", 
  messages: [{ role: "user", content: "Summarize this document" }],
})

The Alien API key selects your project. customer.id selects the customer's connection. byo/gpt-5.6-sol selects the model.

Resolve customer.id from the authenticated account in your backend. Do not accept an arbitrary customer ID from browser input.

Choose the models your product supports

Open Infrastructure → Models in the Alien dashboard. Select the models and client APIs your application uses.

You can do the same from the CLI:

alien projects capabilities enable ai \
  --model byo/claude-opus-5

Mark a model as required only when your product cannot work without it. A required model limits customer setup to providers that can serve it.

The dashboard shows which providers can serve each model. That matters because a model publisher and the service running the model are not always the same thing: Claude may come from Bedrock, Vertex AI, Azure AI Foundry, Anthropic, or Databricks.

Create the server API key

alien api-keys create \
  --for ai-gateway \
  --description production-backend

The secret is shown once. Store it in your backend as ALIEN_AI_API_KEY. Never put it in browser code.

Let the customer connect a provider

For a test, create a setup link for a customer your application already knows as org_123:

alien onboard "Acme" \
  --external-id org_123 \
  --setup-items models

Open the returned link. If Acme chooses AWS, they connect the AWS account and region where Bedrock is available. If they choose Google Cloud, they connect a project where Vertex AI is enabled. Direct Anthropic, OpenAI, and Databricks connections use the corresponding provider account instead.

In a real product, create this link from your backend when the customer opens your BYO-LLM settings. See Customer setup for the TypeScript SDK and REST API.

See what this customer can use

Model availability belongs to the customer connection. Ask AI Gateway what org_123 can use now:

curl "https://ai.alien.dev/v1/models" \
  -H "Authorization: Bearer $ALIEN_AI_API_KEY" \
  -H "X-Alien-External-ID: org_123"

The returned IDs use the same byo/ namespace your application sends in model requests.

Send the first request

curl "https://ai.alien.dev/v1/chat/completions" \
  -H "Authorization: Bearer $ALIEN_AI_API_KEY" \
  -H "X-Alien-External-ID: org_123" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "byo/claude-opus-5",
    "messages": [{"role": "user", "content": "Say hello in five words."}]
  }'

If org_123 connected AWS, this request uses Claude through Bedrock in that AWS account. The same application request uses Vertex AI when another customer connects Google Cloud.

AI Gateway also supports OpenAI Responses and Anthropic Messages. You can keep the client API your application already uses, as long as the selected model and customer provider support it.

Handle unavailable models

A connected account does not guarantee that every model is ready. The model may need one-time activation, may not exist in the selected region, or may not have usable quota.

Check /v1/models after setup and before showing a model as available. If a request fails, inspect diagnostics without exposing prompts or responses:

alien logs --source ai-gateway \
  --status provider-error \
  --model byo/claude-opus-5 \
  --since 1h

What you added

Your product now has one model integration and a different provider connection for each customer. Customers control where their model requests run and which provider account governs that usage. Your backend never receives their provider credentials.

Continue with AI Gateway requests and routing, or add the customer setup flow to your product.

Add BYO-LLM to your product

On this page