Core concepts

BatchRouter routes batch-AI workloads across many providers. To use it well, it helps to know the handful of nouns that show up everywhere in the API: a batch is the unit of work, made of items; a quote prices it before you commit; routing picks a lane on a provider; and receipts explain what happened. This page defines each term and shows how they relate. Each entry links to the deeper guide.

For the full request and response field list, see the interactive API reference or the raw OpenAPI spec.

The lifecycle at a glanceCopy Anchor Link

A job moves through these objects in order:

1. Quote — price a representative sample, free. Returns a quote_id (qlock_…).
2. Batch — submit the quote plus your items. Returns a batch id (batch_…), state priced.
3. Routing — BatchRouter scores eligible lanes and dispatches to the winner(s).
4. Results — completed output, plus a billing receipt and per-lane rejection receipts.
5. Delivery — results are polled, pushed to a webhook, or written to your bucket.

The terms below are the pieces of that flow.

BatchCopy Anchor Link

A batch is one durable job: a set of model requests you submit once and track with a single id (batch_…). You create it with POST /v1/batches, then poll GET /v1/batches/{batchId} until it reaches a terminal state. The state field progresses received → validated → priced → queued → dispatching → running → finalizing → completed; the other terminal states are failed, canceled, and expired.

A batch carries the routing knobs (routing mode, privacy tier), holds the credit reservation, and is the anchor for results, receipts, and delivery.

ItemCopy Anchor Link

An item is a single model request inside a batch. Each item has:

• customer_item_id — your own id for the row, echoed back so you can match outputs to inputs.
• operation — the task type: responses (default), embeddings, or vision.
• model — the model id to run, such as gpt-5.4-mini (see the model catalog).
• input — the prompt payload (e.g. messages for responses).

A canonical item looks like:

{
  "customer_item_id": "item-1",
  "operation": "responses",
  "model": "gpt-5.4-mini",
  "input": {
    "messages": [
      { "role": "user", "content": "Summarize: BatchRouter routes batch-AI workloads across providers." }
    ]
  }
}

QuoteCopy Anchor Link

A quote is a free, up-front price estimate. You send a representative sample of items to POST /v1/quotes/model (a "direct route" — you choose the model) or POST /v1/quotes/workflow (a curated outcome contract). The response contains:

• quote_id — a locked quote handle matching ^qlock_…. Pass it as quote_id to POST /v1/batches to accept the estimate and reserve credits.
• pricing_estimate — the estimated customer price for the work.

Quotes are versioned snapshots: they capture the provider offer prices at quote time and expire quickly if not submitted. Creating a quote never costs credits — you are charged only when you create a batch, and settled against actual token usage.

LaneCopy Anchor Link

A lane is one routable path to run your work: a specific provider plus a specific model offering, with its own price, capacity, region, SLA window, and privacy posture. BatchRouter evaluates every eligible lane, applies hard gates (capability, capacity, price ceiling, privacy, region, health), scores the survivors, and dispatches to the best one — or splits across several lanes when that meets capacity or deadline. The lanes considered (and why each was or wasn't picked) are recorded as rejection receipts.

Routing modeCopy Anchor Link

The routing mode tells the router what to optimize for. Set it per batch:

• cheapest (default) — lowest eligible price wins.
• sla_aware — favor lanes that reliably meet the deadline.
• public_only — only public cloud providers (OpenAI, Anthropic, …).
• edge_only — only edge / BatchProviderApi nodes.
• hybrid — mix public and edge lanes.
• privacy_constrained — restrict to lanes meeting your privacy requirements.

SLA tierCopy Anchor Link

Every batch has one SLA: a 24-hour completion window. The sla_tier field accepts standard, flex, and priority for compatibility, and all three are treated as standard. The window constrains which lanes are eligible — a lane whose provider window can't satisfy it is gated out before scoring.

Privacy tierCopy Anchor Link

The privacy tier selects providers by the privacy tiers they declare: standard, confidential (routes only to providers that declare the confidential tier; it does not by itself require zero data retention — the router adds that check only when routing_mode is privacy_constrained), or restricted (providers that declare the restricted tier; the declaration is the provider's own, and no built-in provider makes it). A lane whose provider does not declare the requested tier is rejected with a privacy_tier_mismatch code.

Rejection receiptCopy Anchor Link

A rejection receipt is the auditable record explaining why a lane was not selected. Each carries a normalized rejection_code — for example capacity_full, stale_heartbeat, context_window_exceeded, privacy_tier_mismatch, or region_unavailable — plus a customer-safe rejection_reason. These make routing explainable: you can see exactly which lanes were considered and the gate each one failed.

Billing receiptCopy Anchor Link

A billing receipt is the settled cost record for a completed batch, available at GET /v1/batches/{batchId}/billing-receipt. It breaks down the quoted versus final price, the model or workflow used, the lane summary, input/output usage, and the disclosed batchrouter_fee (the platform's control-plane margin). It's the source of truth for what you were charged.

ProviderCopy Anchor Link

A provider is a batch-capable backend BatchRouter routes to — a public cloud (OpenAI, Anthropic, OpenRouter, and others) or an edge/BatchProviderApi node. Providers publish versioned offerings (models, prices, capacity, region, SLA window). You don't contract with providers directly; you buy a quote from BatchRouter, which owns routing, pricing, and settlement. Browse the public directory at GET /v1/providers.

Model catalogCopy Anchor Link

The model catalog is the public list of models you can route to, served by GET /v1/catalog/models (and a single entry at GET /v1/catalog/models/{slug}). Always read the catalog for live, available model ids rather than hardcoding names — availability and pricing change as providers update their offerings.

Workflow productCopy Anchor Link

A workflow product is a BatchRouter-curated outcome contract — you buy a result (e.g. "classify into this taxonomy", "extract these fields") instead of choosing a model. BatchRouter picks the provider/model lane that satisfies the contract and may change that mix between versions. List them at GET /v1/catalog/workflow-products; quote one with POST /v1/quotes/workflow.

Delivery targetCopy Anchor Link

A delivery target is where results land. By default you poll GET /v1/batches/{batchId}/results or fetch a signed …/artifact-url. You can also have BatchRouter push results: to a webhook (per-batch {url, secret} or an org default via PUT /v1/auth/account/delivery-webhook; signed with X-BatchRouter-Signature, HMAC-SHA256), or to your own S3-compatible bucket registered via POST /v1/delivery-targets. How long results are retained is governed by GET|PUT /v1/retention-settings.

CreditsCopy Anchor Link

Credits are your prepaid balance — BatchRouter charges against them. Creating a quote is free; creating a batch reserves credits, and completion settles the final amount (releasing any unused reservation, refunding failed work per policy). Credits are purchased in the dashboard at batchrouter.com/app/billing — there is no public billing-checkout endpoint. You can configure spending limits, alerts, and auto top-up under /v1/billing/*, and review charges with GET /v1/usage.

How they relateCopy Anchor Link

Next stepsCopy Anchor Link