SLA-aware lanes & routing

BatchRouter doesn’t just forward your batch to one provider — it evaluates every eligible provider/model lane, prices them, and picks placement based on the controls you set. This page explains those controls (sla_tier, routing_mode, privacy_tier), how multi-lane routing works, and how to read the rejection receipts that explain why a lane wasn’t used.

All three controls are set per quote and per batch. Set them when you request a quote so the estimate reflects your constraints, then pass the same values (plus the quote_id) to POST /v1/batches.

What a lane is

A lane is one candidate execution path — a specific provider running a specific model that could fulfill some or all of your items. When you request a quote, BatchRouter expands your items into every eligible lane, prices each one, and returns them as quote_lanes. The selected lane(s) carry selected: true; the rest come back with a rejection receipt explaining why they were skipped.

Because different items in one batch can declare different models, a single customer batch can be split across multiple internal lanes — each handling the items it’s best suited for. You still get one batch.id, one status to poll, and one merged result set; the per-lane execution is BatchRouter’s concern. The completed batch’s billing receipt lists the provider_lanes that actually ran and the rejected_lanes that didn’t.

Note

Multi-lane routing optimizes for cost, SLA, and live capacity simultaneously. If one provider is at capacity for part of your batch, eligible items can still flow to another lane instead of failing the whole batch.

SLA tiers (sla_tier)

sla_tier sets the deadline BatchRouter commits to. Default is standard.

Tier	Deadline	Use it when
standard	24h SLA (default)	The normal balance of speed and price.
flex	Up to 48h, lower price	You’re not in a hurry and want the cheapest placement.
priority	Expedited	You need results back sooner than standard.

{ "sla_tier": "flex" }

The deadline is surfaced on the batch you poll. Tier choice also affects which lanes are eligible — a lane that can’t meet priority within its capacity window is rejected for that batch.

Routing modes (routing_mode)

routing_mode controls which lanes BatchRouter is allowed to consider and how it ranks them. Default is cheapest.

Mode	What it does
cheapest	Lowest total price across all eligible lanes (default).
sla_aware	Balances price against the lane’s ability to comfortably hit your sla_tier.
public_only	Routes only to public provider lanes (OpenAI, Anthropic, and other public APIs).
edge_only	Routes only to edge / BatchProviderApi provider lanes.
hybrid	Considers both public and edge lanes and picks the best fit.
privacy_constrained	Restricts to lanes that satisfy your privacy requirements (pair with privacy_tier).

{ "routing_mode": "sla_aware" }

Tip

Start with cheapest and let the quote show you the lane spread. Switch to sla_aware if a cheap lane is cutting your deadline close, or to public_only / edge_only when you have a hard requirement about where compute runs.

Privacy tiers (privacy_tier)

privacy_tier constrains placement by data-handling guarantee. Default is standard. Each lane carries a data-privacy proof (retention behavior, output handling, declared privacy tiers) that BatchRouter checks against this value.

Tier	Routing constraint
standard	No additional data-handling constraint (default).
confidential	Routes only to providers with data-retention opt-out.
restricted	Routes only to private BatchProviderApi nodes.

{ "routing_mode": "privacy_constrained", "privacy_tier": "confidential" }

If no eligible lane can satisfy the requested tier, the lanes that fail it come back rejected with privacy_tier_mismatch — and if none qualify, quote creation fails rather than silently routing you somewhere that doesn’t meet the constraint.

Rejection receipts

Every non-selected lane in a quote (and every rejected lane in a completed batch’s receipt) carries a normalized rejection receipt: a stable rejection_code, a customer-safe rejection_reason, and a rejection_receipt object with the failed eligibility checks. This makes routing auditable — you can always see why a cheaper or different lane wasn’t used.

Common rejection codes:

Code	Meaning
capacity_full	The lane had no live routable capacity for your SLA/volume.
stale_heartbeat	The provider’s capacity heartbeat was stale, so it wasn’t treated as live supply.
context_window_exceeded	An item exceeds the model’s context window on that lane.
privacy_tier_mismatch	The lane can’t satisfy your requested privacy_tier.
region_unavailable	The lane isn’t available in your allowed_regions.
missing_web_search (and similar missing_<tool>)	The lane’s provider doesn’t declare a hosted tool you required via required_tools.

Inspect rejected lanes straight from the quote response:

cURL

curl -s https://api.batchrouter.com/v1/quotes/model \
  -H "Authorization: Bearer $BATCHROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "models": ["gpt-4o-mini"],
    "routing_mode": "sla_aware",
    "items": [
      {"customer_item_id":"item-1","operation":"responses","model":"gpt-4o-mini","input":{"messages":[{"role":"user","content":"Summarize: BatchRouter routes batch-AI workloads across providers."}]}}
    ]
  }' | jq '.quote_lanes[] | {provider, model, selected, rejection_code, rejection_reason}'

Node

const res = await fetch("https://api.batchrouter.com/v1/quotes/model", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.BATCHROUTER_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    models: ["gpt-4o-mini"],
    routing_mode: "sla_aware",
    items: [
      {
        customer_item_id: "item-1",
        operation: "responses",
        model: "gpt-4o-mini",
        input: { messages: [{ role: "user", content: "Summarize: BatchRouter routes batch-AI workloads across providers." }] },
      },
    ],
  }),
});
const quote = await res.json();
for (const lane of quote.quote_lanes) {
  console.log(lane.provider, lane.model, lane.selected, lane.rejection_code ?? "(selected)");
}

Python

import os, requests

res = requests.post(
    "https://api.batchrouter.com/v1/quotes/model",
    headers={"Authorization": f"Bearer {os.environ['BATCHROUTER_API_KEY']}"},
    json={
        "models": ["gpt-4o-mini"],
        "routing_mode": "sla_aware",
        "items": [
            {
                "customer_item_id": "item-1",
                "operation": "responses",
                "model": "gpt-4o-mini",
                "input": {"messages": [{"role": "user", "content": "Summarize: BatchRouter routes batch-AI workloads across providers."}]},
            }
        ],
    },
)
for lane in res.json()["quote_lanes"]:
    print(lane["provider"], lane["model"], lane["selected"], lane.get("rejection_code") or "(selected)")

Caution

gpt-4o-mini is illustrative. List the models and the providers/regions/tools each one actually supports via GET /v1/catalog/models before pinning a model or region.

Putting it together

A single batch can combine all three controls. For example, “run this confidentially, only on private nodes, and don’t sweat the deadline”:

{
  "quote_id": "qlock_...",
  "sla_tier": "flex",
  "routing_mode": "privacy_constrained",
  "privacy_tier": "restricted"
}

Set the same values on the quote first so the price you’re quoted reflects the constrained lane set — then carry them onto the batch.

Next steps

Quotes & pricing Get a free quote, read the lane spread, and lock a price before you submit.

Models & catalog List models with per-provider pricing, capacity, regions, retention, and supported tools.

Submit a batch Create a batch with your chosen SLA tier, routing mode, and privacy tier.

API reference Full schema for quote lanes, rejection receipts, and batch parameters.