Core concepts

BatchRouter routes batch-AI workloads across many providers. To use it well, it helps to know the handful of nouns that show up everywhere in the API: a batch is the unit of work, made of items; a quote prices it before you commit; routing picks a lane on a provider; and receipts explain what happened. This page defines each term and shows how they relate. Each entry links to the deeper guide.

For the full request and response field list, see the interactive API reference or the raw OpenAPI spec.

The lifecycle at a glance

A job moves through these objects in order:

1. Quote — price a representative sample, free. Returns a quote_id (qlock_…).
2. Batch — submit the quote plus your items. Returns a batch id (bat_…), status pending.
3. Routing — BatchRouter scores eligible lanes and dispatches to the winner(s).
4. Results — completed output, plus a billing receipt and per-lane rejection receipts.
5. Delivery — results are polled, pushed to a webhook, or written to your bucket.

The terms below are the pieces of that flow.

Batch

A batch is one durable job: a set of model requests you submit once and track with a single id (bat_…). You create it with POST /v1/batches, then poll GET /v1/batches/{batchId} until it reaches a terminal state. Statuses progress pending → queued → routing → dispatched → processing → completing → completed; the terminal failure states are failed, cancelled, and expired.

A batch carries the routing knobs (routing mode, SLA tier, privacy tier), holds the credit reservation, and is the anchor for results, receipts, and delivery.

Submit a batch Create, poll, and retrieve a batch end to end.

Item

An item is a single model request inside a batch. Each item has:

• customer_item_id — your own id for the row, echoed back so you can match outputs to inputs.
• operation — the task type: responses (default), embeddings, or vision.
• model — the model slug to run (see the model catalog).
• input — the prompt payload (e.g. messages for responses).

A canonical item looks like:

{
  "customer_item_id": "item-1",
  "operation": "responses",
  "model": "gpt-4o-mini",
  "input": {
    "messages": [
      { "role": "user", "content": "Summarize: BatchRouter routes batch-AI workloads across providers." }
    ]
  }
}

Note

gpt-4o-mini is illustrative. Fetch the real available models from GET /v1/catalog/models.

Submit a batch See how items are shaped and batched.

Quote

A quote is a free, up-front price estimate. You send a representative sample of items to POST /v1/quotes/model (a “direct route” — you choose the model) or POST /v1/quotes/workflow (a curated outcome contract). The response contains:

• quote_id — a locked quote handle matching ^qlock_…. Pass it as quote_id to POST /v1/batches to accept the estimate and reserve credits.
• pricing_estimate — the estimated customer price for the work.

Quotes are versioned snapshots: they capture the provider offer prices at quote time and expire quickly if not submitted. Creating a quote never costs credits — you are charged only when you create a batch, and settled against actual token usage.

Quotes and pricing Get a price before you commit credits.

Lane

A lane is one routable path to run your work: a specific provider plus a specific model offering, with its own price, capacity, region, SLA window, and privacy posture. BatchRouter evaluates every eligible lane, applies hard gates (capability, capacity, price ceiling, privacy, region, health), scores the survivors, and dispatches to the best one — or splits across several lanes when that meets capacity or deadline. The lanes considered (and why each was or wasn’t picked) are recorded as rejection receipts.

Routing explained How lanes are gated, scored, and selected.

Routing mode

The routing mode tells the router what to optimize for. Set it per batch:

• cheapest (default) — lowest eligible price wins.
• sla_aware — favor lanes that reliably meet the deadline.
• public_only — only public cloud providers (OpenAI, Anthropic, …).
• edge_only — only edge / BatchProviderApi nodes.
• hybrid — mix public and edge lanes.
• privacy_constrained — restrict to lanes meeting your privacy requirements.

Routing explained When to override the default mode.

SLA tier

The SLA tier is the completion-window class you’re buying: standard (24h), flex, or priority. It constrains which lanes are eligible — a lane whose provider window can’t satisfy the tier is gated out before scoring.

Routing explained How the SLA tier interacts with lane selection.

Privacy tier

The privacy tier sets data-handling requirements: standard, confidential (routes only to providers with a data-retention opt-out), or restricted (routes only to private BatchProviderApi nodes). A lane that can’t meet the requested tier is rejected with a privacy_tier_mismatch code.

Routing explained Privacy-constrained routing in practice.

Rejection receipt

A rejection receipt is the auditable record explaining why a lane was not selected. Each carries a normalized rejection_code — for example capacity_full, stale_heartbeat, context_window_exceeded, privacy_tier_mismatch, or region_unavailable — plus a customer-safe rejection_reason. These make routing explainable: you can see exactly which lanes were considered and the gate each one failed.

Receipts Reading routing and rejection receipts.

Billing receipt

A billing receipt is the settled cost record for a completed batch, available at GET /v1/batches/{batchId}/billing-receipt. It breaks down the quoted versus final price, the model or workflow used, the lane summary, input/output usage, and the disclosed batchrouter_fee (the platform’s control-plane margin). It’s the source of truth for what you were charged.

Receipts What a billing receipt contains.

Provider

A provider is a batch-capable backend BatchRouter routes to — a public cloud (OpenAI, Anthropic, OpenRouter, and others) or an edge/BatchProviderApi node. Providers publish versioned offerings (models, prices, capacity, region, SLA window). You don’t contract with providers directly; you buy a quote from BatchRouter, which owns routing, pricing, and settlement. Browse the public directory at GET /v1/providers.

Provider marketplace How the provider marketplace works.

Model catalog

The model catalog is the public list of models you can route to, served by GET /v1/catalog/models (and a single entry at GET /v1/catalog/models/{slug}). Always read the catalog for live, available model slugs rather than hardcoding names — availability and pricing change as providers update their offerings.

Browse the catalog Models, providers, and fee schedule.

Workflow product

A workflow product is a BatchRouter-curated outcome contract — you buy a result (e.g. “classify into this taxonomy”, “extract these fields”) instead of choosing a model. BatchRouter picks the provider/model lane that satisfies the contract and may change that mix between versions. List them at GET /v1/catalog/workflow-products; quote one with POST /v1/quotes/workflow.

Workflow products Outcome contracts vs. direct routes.

Delivery target

A delivery target is where results land. By default you poll GET /v1/batches/{batchId}/results or fetch a signed …/artifact-url. You can also have BatchRouter push results: to a webhook (per-batch {url, secret} or an org default via PUT /v1/auth/account/delivery-webhook; signed with X-BatchRouter-Signature, HMAC-SHA256), or to your own S3-compatible bucket registered via POST /v1/delivery-targets. How long results are retained is governed by GET|PUT /v1/retention-settings.

Delivery and webhooks Poll, webhook, or push to your bucket.

Credits

Credits are your prepaid balance — BatchRouter charges against them. Creating a quote is free; creating a batch reserves credits, and completion settles the final amount (releasing any unused reservation, refunding failed work per policy). Credits are purchased in the dashboard at batchrouter.com/app/billing — there is no public billing-checkout endpoint. You can configure spending limits, alerts, and auto top-up under /v1/billing/*, and review charges with GET /v1/usage.

Credits and billing Buying credits, limits, and auto top-up.

How they relate

Tip

In one sentence: you quote a sample of items, accept the quote_id to create a batch, BatchRouter routes it onto a provider lane under your chosen routing mode / SLA tier / privacy tier, then settles credits and emits billing + rejection receipts while the result reaches your delivery target.

Next steps

Quickstart Make your first batch in a few minutes.

Submit a batch The full create → poll → results flow.

Routing explained Lanes, modes, tiers, and scoring in depth.

API reference Every endpoint, interactively.