Implement Soba

Soba is the LLM layer for an application whose users bring their own compute. Requests go to Soba instead of to a model provider; Soba routes each one to the machine or plan belonging to the user it names, meters it, and bills for it.

See it working first#

The parts of an integration, placed where they go in one small app: the three every app needs, plus the navbar button an app adds when its AI only works once a user connects. Switch the state to watch each part react: nothing connected, a run refused, a machine attached, a machine asleep, an allowance spent.

What your user sees#

Every Connect in your app hands off to the Soba app, on your user's Mac, which lends their AI to your app. You place none of it, but it is the half of the journey you cannot see from your code, so here it is step by step — the real app's screens, beside what your app is doing meanwhile.

How an integration is shaped#

Every app, whatever it is — a chat, one AI button, a background job with a page — gets the same two halves:

1. The server: every LLM call goes through Soba as the signed-in user, a session route lets the browser act for that user, and Soba's answer (including a refusal) reaches the browser intact.
2. The browser: one <SobaProvider>, and a surface for each moment somebody meets their compute — the settings page, every control that starts a run, the place a run's result appears and, when the app's AI only works on the user's own compute, the navbar.

Each surface has a component and a hook. Mount the component; build from the hook only when the person asks for their own markup, and then show everything the component would (see "Component or hook"). The integration is finished when every surface in the required list exists, not when the first request succeeds.

The server#

1. Point every LLM call site at Soba#

baseURL   https://api.soba.so/v1
apiKey    process.env.SOBA_KEY

An OpenAI client already speaks this, so on most projects the base URL is the only line that changes. Prompts, tools, streaming and error handling stay exactly as they are. Find every call site — a shared client, a route that builds its own, a raw fetch to the provider's REST API — and move all of them.

Pass the signed-in user's id as the user field on every request.

Resolve that id on the SERVER, from the session, never from the request body. It is what decides whose machine a run may reach, so a value the browser can choose is a value any user can use to spend any other user's compute. This is the one line in the integration that must not be guessed at: if the project's auth is not obvious, read it until it is, or stop and ask. A wrong id here does not throw — it silently pairs runs to the wrong person.

Derive it from something that outlives the database. A primary key minted at sign-up (an autoincrement, a random UUID) is not: reset the database and the same person arrives as somebody new, stranded from the machine they paired. An email is not either, if the app lets people change it. sobaUserId(stable) from @soba-so/sdk/identity turns a stable value — a username, an auth provider's subject — into the id; use one helper for it and call it everywhere the user is named. See https://soba.so/docs/identity.

2. Where the key comes from#

If .env.local already sets SOBA_KEY, use it. Otherwise sign the terminal in, from the project root:

npx -y @soba-so/cli login --no-link

It opens the person's browser once, they approve, and the terminal keeps a token of its own; no key passes through you. Tell the person to approve before you run it; it waits up to fifteen minutes, so give the command a long timeout or run it in the background. Add --headless where no browser can open, and pass the link it prints on to the person.

Then find out which app this project is. npx -y @soba-so/cli apps lists theirs by workspace. Ask the person which one, or whether to make a new one, and do not guess from the names; then link it:

npx -y @soba-so/cli link --workspace "<workspace>" --app "<app>"
npx -y @soba-so/cli link --workspace "<workspace>" --new "<name>"   # a new app

That writes SOBA_KEY (a development key, for this machine), SOBA_PROD_KEY (production) and SOBA_APP_ID into .env.local, adding that file to .gitignore if git would commit it. With nobody at the terminal the CLI never waits on a question: a choice it needs ends the command with the flag to pass. Every command is on https://soba.so/docs/reference/cli.

.env.local is right for Next.js, Vite and most Node servers. Where the code that holds the key reads a different file, pass that file with --env-file to link instead, from the project root, so the key lands where the server will look:

Find the file by reading how that server loads its environment, not by its framework's reputation. The CLI adds whatever file it writes to .gitignore if git would commit it.

On a host that keeps secrets itself — wrangler secret put SOBA_KEY, Vercel's or Railway's environment — production takes SOBA_PROD_KEY's value under the name SOBA_KEY. That is the person's to paste; tell them the command.

A development key reaches only the user's own AI — their machine, their subscription, their API key. In production, SOBA_KEY is set to the value of SOBA_PROD_KEY in the host's environment; say so to the person rather than doing it.

If you cannot run commands but have a soba_create_key tool, call it: the secret is returned once and nothing can show it again, so write it into .env.local in the same step. Failing both, ask the person to run the command above themselves. Do not leave SOBA_KEY empty and do not invent a value; both fail at the first request, and the first fails as an auth error rather than as a missing variable.

Put SOBA_KEY in .env.local (or the server's own file, above), and check that file is gitignored before writing it. Never in .env, never in .env.example, never in anything tracked, and never echoed back to the terminal.

3. The session route#

POST /api/soba/session, a route in the app that proxies POST /v1/end_users/session with the secret key and the same server-resolved user id, and returns the body as it came ({ session, expires_in }). It is what every surface in the browser runs on, and the browser never sees SOBA_KEY. Signed out, it answers 401.

4. Pass Soba's answer through#

A refused run is not an outage: no_compute (409) means nothing the run could use is reachable for this user, payment_required (402) means the plan's allowance is spent. The browser surfaces below turn each into the way out — connect a machine, or upgrade — but only if the refusal reaches them with its code. So in every route that calls Soba, return Soba's status and error body as they came, instead of letting the client library's exception become a 500 or a string:

} catch (err) {
  if (err instanceof OpenAI.APIError) {
    return Response.json({ error: err.error }, { status: err.status });
  }
  throw err;
}

A route that streams has to do this before the first byte is written: create the completion (or start the run) first, and only then return the stream. In the browser, keep the parsed body (setError(await res.json())) rather than its text; that object is what the surfaces read. Branch on the code, never on the message.

5. Which API#

chat/completions is stateless and the caller owns the loop, so it reaches models. Reaching a user's own Claude or ChatGPT subscription means letting Soba own the loop instead:

import { Soba } from "@soba-so/sdk";
import { sobaUserId } from "@soba-so/sdk/identity";

const soba = new Soba({ apiKey: process.env.SOBA_KEY });

const run = await soba.run({
  user: sobaUserId(session.user.username),
  model: "auto",
  prompt: "Summarise everything I saved this week",
  tools: [listNotes, readNote],   // your code, executed in your app
});

Tools passed to a run execute in the app's own process, so they can read the app's own data. The model asks for data by calling them; the data does not travel in the prompt.

The browser: one provider, a surface for each moment#

Everything below reads from <SobaProvider sessionEndpoint="/api/soba/session">. Mount it where a surface is, with sessionCacheKey set to the signed-in user's id so reloads, navigations, tabs and other providers on the page share one session. Never in a root layout that also renders the login page or marketing pages: there is nobody there to mint a session for. A navbar's provider goes in the layout that draws the signed-in navbar. All the components raise the same pairing modal — the provider mounts exactly one, so an app with every surface cannot put two pairings on screen at once.

Required in every app: the first three. The navbar follows from how the app's AI is paid for (below); the composer, billing and gates follow from what the app has — a model choice to offer, plans to sell, features to limit — and the plans themselves are the person's to decide, not yours.

What each surface is for, and the mistakes it exists to prevent:

• Navbar, when the AI only works on their compute. Add <ConnectButton /> to the signed-in navbar when nothing can run until a user connects — a bring-your-own plan, a route that only allows their machine or their subscription, an app on a development key. There, connecting is a prerequisite rather than an offer, so the door belongs on every page: it says "Connect compute" until something is attached, then names the machine and its state, and once connected it is a pill that opens a popover with the machines and what they are running. An app whose runs fall back to compute it pays for does not need it; settings, the nudge and the run limit already meet the person where connecting matters. Wherever it is used, it lives in the header and only there — a popover pill is wrong inside a page.
• Settings. A settings page is where somebody checks on a machine they already attached, not where they decide to attach one. <ComputeSettings /> draws it as rows of that page in its own type: each machine, which agent the app's runs go to there, and whether it is awake, asleep or serving, with "Connect compute" or "Connect another computer". Nothing else from the package goes on that page — <ConnectButton /> or <ComputeStatus /> beside it draws the same machine twice. (<ConnectCompute /> is its older name.) If there is no settings page, build one, and do not point the connect return URL at a page that does not render it.
• Above each run-starting control. Not only a chat composer: every "Tidy", "Summarise", "Draft", "Generate", every form that kicks off a job — find each call site from step 1 and the control that triggers it. <ConnectNudge /> stands there before anything is pressed and speaks only when it can help: nothing connected and nothing else can serve, a paired machine gone quiet, or an allowance running down that connecting would lift. It draws nothing otherwise.
• The result's place. When a run fails, <RunLimit error={error} /> renders where the answer was going to be: for no_compute the way to connect (or wake the machine), for payment_required the plan to upgrade to. It goes quiet by itself once a machine is online, so a refusal from before connecting is never shown as the answer to connecting. <RunError error={error} /> beside it says every other failure in a sentence; the two never both speak.

One run-starting control, written both ways#

The same button in any app, as components:

"use client";
import { ConnectNudge, RunError, RunLimit } from "@soba-so/react";

function TidyButton({ noteId }) {
  const [error, setError] = useState(null);

  async function tidy() {
    setError(null);
    const res = await fetch("/api/tidy", { method: "POST", body: JSON.stringify({ noteId }) });
    if (!res.ok) return setError(await res.json());   // Soba's body, per step 4
    // …read the result as the app already does
  }

  return (
    <>
      <ConnectNudge />
      <button onClick={tidy}>Tidy</button>
      <RunLimit error={error} />
      <RunError error={error} />
    </>
  );
}

And built from the hooks, in the app's own components (Your… below stands for whatever the app already draws banners, buttons and cards with), saying everything the components would:

import { runErrorMessage, useConnectNudge, useRunLimit } from "@soba-so/react";

function TidyButton({ noteId }) {
  const [error, setError] = useState(null);
  const nudge = useConnectNudge();       // before: is anything in the way?
  const limit = useRunLimit(error);      // after: the way out of a refusal
  const message = runErrorMessage(error); // after: every other failure

  return (
    <>
      {nudge.visible && (
        <YourBanner title={nudge.title} detail={nudge.detail} steps={nudge.steps}
                    onAction={nudge.open} onDismiss={nudge.dismissible ? nudge.dismiss : undefined} />
      )}
      <YourButton onClick={tidy}>Tidy</YourButton>
      {limit ? (
        <YourCard title={limit.title} body={limit.body}
                  onConnect={limit.connect}
                  onUpgrade={limit.upgradeTo ? limit.upgrade : undefined} />
      ) : message ? (
        <p className="error">{message}</p>
      ) : null}
    </>
  );
}

Both halves are inside a provider. In a chat, the nudge stands above the composer and the limit and error render in the message that failed; see "The chat stays theirs".

Component or hook#

Both halves ship, and the component is the answer. Mount it even when the app already has its own composer, its own billing cards, its own design system and Tailwind everywhere. Do not look at the repo's styling and decide to rebuild one of these from its hook.

The reason is that the components are not just markup. The vendor configures them in the Soba dashboard: the model picker's layout (by provider, one list, custom only), which of Automatic, Fast and Best are offered, search, the agents' marks and the line under each choice, the connect popup's look and theme. The component reads all of that from the provider and draws it. A hand-built copy reads none of it, so every setting the vendor changes in the dashboard silently does nothing in their app, and nobody finds out until they compare the two.

What that looks like in practice, from a real integration: an app with its own Tailwind composer took useModel() and rendered it as a native <select> — no marks, no descriptions, no submenu per agent, a flat list where the dashboard said "by provider". It worked, and it was wrong.

The components fit a foreign screen on purpose: className on each one for placement and size, theme on <SobaProvider> (or the dashboard) for light and dark, <ComputeSettings /> inherits the page's type, and <ModelPicker /> is a pill beside a send button, not a card. If one still clashes, say so to the person and leave the component in.

Reach for a hook only when the person asks for their own markup, or for a decision no component draws (useMachineRuns(), useOpenSobaConnect(), gating a server action on useFeature()). Every component is one rendering of an exported hook and derives nothing of its own, and a surface built from a hook must still show everything the component shows. Each line in the last column is something a hand-built copy has dropped:

<SobaBilling /> and <ConnectModal /> are compositions of the above and have no rule of their own. <ConnectPill /> is the half of <ConnectButton /> that reports — the mark, the state and the machine, with no popover under it and no flow behind it. It is for an app whose person asked for their own panel; otherwise the header gets <ConnectButton />, panel included. Pressing the pill is yours to wire, and it still opens Soba's flow through useSoba().connect.open(); everything it says is connectSays. <SobaProvider> is needed either way: the hooks read from it, so a hook with no provider above it is the one mistake this split invites.

A gate draws the button; it does not secure the action. On the server, ask before you create: soba.users.check(user, "workspaces", { use: 1 }) answers { allowed, reason } and counts the use in the same call, so two requests for the last one cannot both get it. Record use you did not ask about with soba.users.track(user, "workspaces", -1) when one is deleted. Prices, allowances and the connected reward come from the plan objects, so changing them later is not a deploy. Pass showConnect to <SobaPlans /> only on a page with no current-plan card: a bring-your-own plan would otherwise state a requirement with no way to meet it.

Three belong to no component of their own. useOpenSobaConnect() brings the Soba app forward on this person's machine and hands back the one honest thing a page can say about having tried — a browser cannot tell whether a soba:// link was answered, and a page that claims it worked is guessing. useMachineRuns() is the "Running now" list out of <ConnectButton />'s panel with no panel around it: what this person's machines are running FOR THIS APP, and stop(runId) to end one as the person whose machine it is on; the app that started that run sees it finish with reason: "stopped_by_user". useRedeemCode() is a promo code field's behaviour: redeem(code) gives the user what the app's reward hands out and re-reads their plan, so the allowance on screen moves without a reload; <RedeemCode /> is that field already drawn.

useSoba() is the raw provider state. Reach for a named hook first and use this one for what no hook covers — run(), connect.open(), plan.refresh(). A page that reads compute.connected or plan.remaining straight off it is re-deriving a decision that is already exported, and the two copies are how our own apps drifted.

What not to write yourself#

These are not style preferences. Each one is a judgement this package already makes, and each has been got wrong in production — in our own code, which is why they are exported at all. If you find yourself writing one of them, delete it and import the answer:

• A list of model names. The names come off the person's own machine, so a list in your code refuses the model they can see on their own screen. useModel() reads it from what they have connected.
• A model id on a run without its agent. Soba applies a person's own agent preference only to a run whose model is blank or a size, so a named model travelling alone goes to whichever agent is found first. Spread useModel().run whole — it is { model, runtime? } — and have your route forward both fields.
• "Nothing connected yet." That is a claim about somebody's account and it is only true once the account has been read. Said while the first request is in flight, or after it failed, it tells a person with a paired laptop that it has vanished. useCompute().view is the four-way answer.
• A match on an error message, or an error line of your own for a refusal. useRunLimit and runErrorMessage read the CODE. Messages are prose we rewrite, matching on them breaks on a copy edit, and Soba's own refusal text names internal ids no user should see.
• Words for a cost class. tierLabel(tier) is the one glossary for the five economics this product is about. Four hand-written copies in this repo disagreed with each other, and three reported somebody's own API key as frontier supply the app had paid for.
• Where the pick is remembered, or how a stale one is handled. A model whose machine is gone falls back to Automatic and stays picked. useModel() does that and tells you with unavailable.

Read https://soba.so/docs/components for each hook's full shape.

The chat stays theirs#

There is no chat component, on purpose. An app built on Soba already has its own composer and its own messages, and they are the product. Keep every line of that markup. What Soba adds goes INTO it, and these are the things a chat on Soba has to handle — check each one against the code, not against memory:

In the composer

• <ModelPicker /> in the composer's footer, beside the send button. The component, per the section above, not a copy.
• Spread useModel().run onto the request, and have the route forward model and runtime both.
• Send becomes Stop while a run is going. Stop aborts the request's signal, and a run the person stopped is not an error to show. Neither is reason: "stopped_by_user", which is the machine's owner stopping it from their side.
• <ConnectNudge /> above the composer. It draws nothing until it has something to say.
• Images, only if the composer already takes them: forward each message's images and the runtimes that can see them will.

In the messages

• delta text is appended as it arrives. thinking is the running total and REPLACES what is shown — appending it repeats every sentence. Fold it away by default.
• status events are what the agent is doing: show message, and toolInput beside it when there is one. state: "interrupted" means the person's machine went offline mid-run and "resumed" that it came back; say so in the answer, because the run goes on either way.
• A failed run renders <RunLimit /> where the answer was going to be, and <RunError /> beside it for everything else. Branch on the reason code, never the message. Offer Retry only when retryable is not false.
• Copy on every answer, on hover, copying the text as written (markdown, not rendered HTML). Retry on the last one.
• done carries usage and tier. If the app shows what answered, word the tier with tierLabel().

Across turns

• done and error carry a session. Send it back as resume on the next run, and have the route forward it, so a follow-up continues the agent's conversation instead of starting a new one with the whole transcript pasted in.

An app with no chat at all gets one built in its own components and styles, the same way it would build any other screen — with everything above in it.

Proving it#

soba/echo is answered by Soba itself — no machine, no metered supply, nothing billed — and still proves the key, the base URL and the user field:

curl https://api.soba.so/v1/chat/completions \
  -H "Authorization: Bearer $SOBA_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "model": "soba/echo", "user": "usr_123",
        "messages": [{ "role": "user", "content": "ping" }] }'

A curl is evidence the credential works. It is NOT evidence the app is integrated — the dashboard's integrate step ticks on a request arriving from the real call site, and it cannot tell a curl from a deployment. Do not send one to make a checklist go green. Before calling the integration done, walk the required surfaces as a signed-in user with nothing connected: settings says "Connect compute", the nudge stands above each run-starting control when connecting would help, pressing one shows the limit card where the result would have been (and the navbar says "Connect compute", where there is one) — then connect, and each of them goes quiet.

A new project rather than an existing one#

There is no separate procedure for this and no question to ask the person: look at the directory. If there is no app yet, run npx create-soba@latest and let it write one — it ships a correct /api/soba/session, which is the security boundary of the whole integration, and a plausible hand-written one is the failure mode that matters. Then the server half is already done and what remains is the key.

What is not yours to do#

Pairing is a person installing the Soba app on their own laptop and approving it. Pricing, allowances and rewards are decisions about somebody's business. Do neither on their behalf; say what is left and stop.

Every part, live#

Each component in every state that changes what it draws, with the line of code that places it and the hook to build your own version from. The same parts have a page of their own at soba.so/components, with more room for each.