Idempotency & retries

Network calls fail. A request times out, a connection drops, or your process restarts mid-submit — and you’re left not knowing whether the batch was created. BatchRouter makes that safe: every POST /v1/batches carries an Idempotency-Key, so you can retry the same submission as many times as you need without ever creating a duplicate batch.

This guide covers the idempotency key, a safe client-retry pattern with exponential backoff, the difference between retrying a request and retrying failed items, and the automatic retries BatchRouter performs against providers on your behalf.

The Idempotency-Key header

POST /v1/batches requires an Idempotency-Key request header. It is a client-generated string of 8–128 characters that uniquely identifies one logical submission.

• The first request with a given key creates the batch and returns 202 with the batch summary (id bat_…).
• Any replay with the same key returns the same 202 response — the original batch — without creating a new one.
• Re-sending the same key with a different request body is a conflict: the API returns 409 rather than silently creating or mutating a batch.

cURL

curl -X POST https://api.batchrouter.com/v1/batches \
  -H "Authorization: Bearer $BATCHROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: order-4821-2026-06-18" \
  -d '{
    "sla_tier": "standard",
    "routing_mode": "cheapest",
    "items": [
      {
        "customer_item_id": "item-1",
        "operation": "responses",
        "model": "gpt-4o-mini",
        "input": {
          "messages": [
            { "role": "user", "content": "Summarize: BatchRouter routes batch-AI workloads across providers." }
          ]
        }
      }
    ]
  }'

Node

const res = await fetch("https://api.batchrouter.com/v1/batches", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.BATCHROUTER_API_KEY}`,
    "Content-Type": "application/json",
    "Idempotency-Key": "order-4821-2026-06-18",
  },
  body: JSON.stringify({
    sla_tier: "standard",
    routing_mode: "cheapest",
    items: [
      {
        customer_item_id: "item-1",
        operation: "responses",
        model: "gpt-4o-mini",
        input: {
          messages: [
            { role: "user", content: "Summarize: BatchRouter routes batch-AI workloads across providers." },
          ],
        },
      },
    ],
  }),
});

const { batch } = await res.json();
console.log(batch.id); // bat_…

Python

import os, requests

res = requests.post(
    "https://api.batchrouter.com/v1/batches",
    headers={
        "Authorization": f"Bearer {os.environ['BATCHROUTER_API_KEY']}",
        "Content-Type": "application/json",
        "Idempotency-Key": "order-4821-2026-06-18",
    },
    json={
        "sla_tier": "standard",
        "routing_mode": "cheapest",
        "items": [
            {
                "customer_item_id": "item-1",
                "operation": "responses",
                "model": "gpt-4o-mini",
                "input": {
                    "messages": [
                        {"role": "user", "content": "Summarize: BatchRouter routes batch-AI workloads across providers."}
                    ]
                },
            }
        ],
    },
)

batch = res.json()["batch"]
print(batch["id"])  # bat_…

Choosing a stable idempotency key

Derive the key from something that already uniquely identifies the work in your system — an order id, a job row’s primary key, a content hash of the items — not from a fresh random value generated at call time. The whole point is that a retry of the same logical submission reuses the same key, so a key that’s regenerated on every attempt defeats the protection. A good key is stable across retries, unique across distinct submissions, and within the 8–128 character limit. UUIDs work well when you persist one per logical job before the first attempt.

Safe client retries with exponential backoff

Because the key makes POST /v1/batches safe to repeat, the correct response to a network failure or a 5xx is simply to retry the exact same request — same key, same body. Use exponential backoff with jitter so a transient blip doesn’t turn into a thundering retry storm.

What to retry on:

• Network errors (timeout, connection reset) — outcome unknown; retry the identical request.
• 429 Too Many Requests — back off, then retry. Honor a Retry-After header if present.
• 5xx — server-side and transient; retry.

What not to retry:

• 400 / 422 — the request is malformed; fix the body, don’t retry as-is.
• 401 / 403 — fix authentication or permissions first.
• 409 — the key was reused with a different payload; reuse the original payload, or pick a new key for genuinely new work.

Node

async function createBatchWithRetry(body, idempotencyKey, maxAttempts = 5) {
  let delay = 500; // ms
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    const res = await fetch("https://api.batchrouter.com/v1/batches", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.BATCHROUTER_API_KEY}`,
        "Content-Type": "application/json",
        "Idempotency-Key": idempotencyKey, // same key every attempt
      },
      body: JSON.stringify(body),
    });

    if (res.ok) return res.json();

    // Don't retry client errors except rate limiting.
    if (res.status < 500 && res.status !== 429) {
      throw new Error(`Batch create failed: ${res.status} ${await res.text()}`);
    }

    if (attempt === maxAttempts) throw new Error("Exhausted retries");

    const jitter = Math.random() * 250;
    await new Promise((r) => setTimeout(r, delay + jitter));
    delay *= 2; // exponential backoff
  }
}

Python

import os, time, random, requests

def create_batch_with_retry(body, idempotency_key, max_attempts=5):
    delay = 0.5  # seconds
    for attempt in range(1, max_attempts + 1):
        res = requests.post(
            "https://api.batchrouter.com/v1/batches",
            headers={
                "Authorization": f"Bearer {os.environ['BATCHROUTER_API_KEY']}",
                "Content-Type": "application/json",
                "Idempotency-Key": idempotency_key,  # same key every attempt
            },
            json=body,
        )

        if res.ok:
            return res.json()

        # Don't retry client errors except rate limiting.
        if res.status_code < 500 and res.status_code != 429:
            res.raise_for_status()

        if attempt == max_attempts:
            raise RuntimeError("Exhausted retries")

        time.sleep(delay + random.uniform(0, 0.25))
        delay *= 2  # exponential backoff

cURL

# curl's built-in retry handles transient 429/5xx and connection errors
# with exponential backoff. Keep the Idempotency-Key constant across retries.
curl --retry 5 --retry-delay 1 --retry-all-errors \
  -X POST https://api.batchrouter.com/v1/batches \
  -H "Authorization: Bearer $BATCHROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: order-4821-2026-06-18" \
  -d @batch.json

Note

The same safe-retry guidance applies to read endpoints like GET /v1/batches/{id} when you poll for status — they’re idempotent by nature, so back off and repeat on transient failures. For the full set of error code values and their meaning, see the errors reference. Every error response uses the envelope { "error": { "code", "message", "details"? } }.

Retrying a request vs. retrying failed items

These are two different operations — keep them straight:

	What it retries	When to use
Idempotency-Key replay	Your POST /v1/batches call	The submission’s outcome is unknown (timeout, 5xx). Safe, never duplicates.
POST /v1/batches/{id}/retry beta	The work of a finished-but-failed batch	The batch reached terminal failed or expired and you want another dispatch attempt.

Replaying the idempotency key is about delivering your request reliably. POST /v1/batches/{id}/retry is about re-running a batch that already ran and didn’t succeed — it re-queues a batch that is in failed or expired status for another dispatch attempt. It does not apply to a batch that completed or is still in flight (you’ll get a 409).

cURL

curl -X POST https://api.batchrouter.com/v1/batches/bat_123/retry \
  -H "Authorization: Bearer $BATCHROUTER_API_KEY"

Node

const res = await fetch("https://api.batchrouter.com/v1/batches/bat_123/retry", {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.BATCHROUTER_API_KEY}` },
});
const batch = await res.json();

Python

res = requests.post(
    "https://api.batchrouter.com/v1/batches/bat_123/retry",
    headers={"Authorization": f"Bearer {os.environ['BATCHROUTER_API_KEY']}"},
)
batch = res.json()

Caution

POST /v1/batches/{id}/retry is currently in beta and its behaviour may change. Check the interactive API reference for the latest contract before depending on it in production.

Re-delivering results

If a batch completed but delivery to your customer-owned bucket failed, that’s a separate concern from re-running the work. Use POST /v1/batches/{id}/redeliver to re-enqueue delivery — it’s guarded against double-delivery, returning 409 if the batch is already delivered or delivering. See delivery targets for setup. For webhook delivery, results are also always retrievable via GET /v1/batches/{id}/results, so a missed webhook never means lost data.

Automatic provider-side retries

You don’t have to handle transient provider failures yourself. Once a batch is dispatched, BatchRouter manages the lifecycle on your behalf:

1. Transient provider errors are retried automatically. If a provider returns a temporary error (rate limit, timeout, a transient 5xx), BatchRouter retries the affected work internally — your batch simply stays in its in-flight status (routing → dispatched → processing) while this happens.

2. SLA-aware routing may re-route. Depending on your routing_mode and sla_tier, work that can’t be completed on one lane can be routed to another eligible provider rather than failing outright.

3. Only durable failures surface to you. A batch reaches terminal failed only after the automatic retries are exhausted. Per-item failures are reported in the results — inspect them with GET /v1/batches/{id}/results or GET /v1/batches/{id}/items.

Note

Because BatchRouter already retries transient provider issues for you, your client-side retry logic only needs to cover getting your request to BatchRouter reliably (the idempotency-key pattern above) and polling for status. You do not need to implement provider-level backoff yourself.

Next steps

Submit a batch The full quote → create → poll → results flow.

Errors reference Every error code, what it means, and whether to retry.

API reference Interactive reference for every endpoint and field.