Add AI to a Worker

In this example, we are going to build an AI endpoint inside a customer's cloud. GET /models lists the models available through that cloud account, and GET /ask sends a question to one of them.

The model call happens inside the Worker: Amazon Bedrock on AWS, Vertex AI on Google Cloud, or Azure AI Foundry on Azure. This lets an application process customer data with the model service in the same cloud account without putting a provider key in its source code or sending the request through your hosted backend.

╔════════════════════════════════╗        ╔══════════════════════════════════╗
║ Your product                   ║        ║ Customer environment             ║
║                                ║        ║                                  ║
║ Send an HTTP request           ║───────▶║  ┌──────────┐    ┌────────────┐  ║
║                                ║◀───────║  │ Worker   │───▶│ AI models  │  ║
║ Receive the model response     ║        ║  │ your code│    │ available  │  ║
╚════════════════════════════════╝        ║  └──────────┘    └────────────┘  ║
                                          ╚══════════════════════════════════╝

We will first describe the Worker and AI resource in alien.ts. Then we will implement both HTTP routes in src/index.ts and try them locally.

Describe the Worker and model access

alien.ts

const assistant = new alien.AI("assistant").build()

const api = new alien.Worker("api")
  .code({ type: "source", src: "./", toolchain: { type: "typescript" } })
  .publicEndpoint("api")
  .link(assistant)
  .permissions("execution")
  .build()

export default new alien.Stack("ai-quickstart")
  .add(assistant, "live")
  .add(api, "live")
  .permissions({ profiles: { execution: { "*": ["ai/invoke"] } } })
  .build()

Linking assistant makes it available inside the Worker. The permission profile allows the Worker to invoke it.

Show the models this deployment can use

src/index.ts

app.get("/models", async c => {
  const models = await ai("assistant").getAvailableModels()
  return c.json({ models })
})

Use the returned IDs in a model picker or choose one in your application.

Send a question

src/index.ts

app.get("/ask", async c => {
  const question = c.req.query("q")
  if (!question) return c.json({ error: "pass ?q=..." }, 400)

  const assistant = ai("assistant")
  const models = await assistant.getAvailableModels()
  const model = c.req.query("model") ?? models[0]?.id
  if (!model) return c.json({ error: "no models available" }, 500)

  const completion = await assistant.chat.completions.create({
    model,
    messages: [{ role: "user", content: question }],
  })

  return c.json({ answer: completion.choices[0]?.message?.content ?? "" })
})

Run it locally

alien init ai-quickstart-ts
cd ai-quickstart-ts
OPENAI_API_KEY=sk-... alien dev

Local development uses OPENAI_API_KEY. A cloud deployment uses the AI configuration available there.

curl http://localhost:<port>/models
curl 'http://localhost:<port>/ask?q=Reply+with+one+word:+pong'

Put it in a customer's cloud

1 · Release

alien release

Publishes a version. Nothing is deployed for a customer yet.

2 · Invite

alien onboard acme-corp

Creates a deployment link for that customer.

3 · Deploy

The customer opens the link and deploys into their environment.

The Worker then calls the AI models available in that customer deployment, using its cloud identity rather than a provider key in your source.

What you built

You built an AI endpoint whose compute, data handling, and model request can stay in the customer's cloud account. The same application code discovers and invokes the models available there without hard-coding provider credentials into the Worker.

Complete source: ai-quickstart-ts.

To route requests from your hosted backend to a provider connected by each customer, follow Use a customer's model provider.