Idempotency & retries

Network calls fail. A request times out, a connection drops, or your process restarts mid-submit — and you're left not knowing whether the batch was created. BatchRouter makes that safe: every POST /v1/batches carries an Idempotency-Key, so you can retry the same submission as many times as you need without ever creating a duplicate batch.

This guide covers the idempotency key, a safe client-retry pattern with exponential backoff, the difference between retrying a request and retrying failed items, and the automatic retries BatchRouter performs against providers on your behalf.

The Idempotency-Key headerCopy Anchor Link

POST /v1/batches requires an Idempotency-Key request header. It is a client-generated string of 8–128 characters that uniquely identifies one logical submission.

• The first request with a given key creates the batch and returns 202 with the batch summary (id batch_…).
• Any replay with the same key returns the same 202 response — the original batch — without creating a new one.
• Re-sending the same key with a different request body is a conflict: the API returns 409 rather than silently creating or mutating a batch.

curl -X POST https://api.batchrouter.com/v1/batches \
  -H "Authorization: Bearer $BATCHROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: order-4821-2026-06-18" \
  -d '{
    "sla_tier": "standard",
    "routing_mode": "cheapest",
    "items": [
      {
        "customer_item_id": "item-1",
        "operation": "responses",
        "model": "gpt-5.4-mini",
        "input": {
          "messages": [
            { "role": "user", "content": "Summarize: BatchRouter routes batch-AI workloads across providers." }
          ]
        }
      }
    ]
  }'

Safe client retries with exponential backoffCopy Anchor Link

Because the key makes POST /v1/batches safe to repeat, the correct response to a network failure or a 5xx is simply to retry the exact same request — same key, same body. Use exponential backoff with jitter so a transient blip doesn't turn into a thundering retry storm.

What to retry on:

• Network errors (timeout, connection reset) — outcome unknown; retry the identical request.
• 429 Too Many Requests — back off, then retry. Honor a Retry-After header if present.
• 5xx — server-side and transient; retry.

What not to retry:

• 400 / 422 — the request is malformed; fix the body, don't retry as-is.
• 401 / 403 — fix authentication or permissions first.
• 409 — the key was reused with a different payload; reuse the original payload, or pick a new key for genuinely new work.

async function createBatchWithRetry(body, idempotencyKey, maxAttempts = 5) {
  let delay = 500; // ms
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    const res = await fetch("https://api.batchrouter.com/v1/batches", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.BATCHROUTER_API_KEY}`,
        "Content-Type": "application/json",
        "Idempotency-Key": idempotencyKey, // same key every attempt
      },
      body: JSON.stringify(body),
    });

    if (res.ok) return res.json();

    // Don't retry client errors except rate limiting.
    if (res.status < 500 && res.status !== 429) {
      throw new Error(`Batch create failed: ${res.status} ${await res.text()}`);
    }

    if (attempt === maxAttempts) throw new Error("Exhausted retries");

    const jitter = Math.random() * 250;
    await new Promise((r) => setTimeout(r, delay + jitter));
    delay *= 2; // exponential backoff
  }
}

Retrying a request vs. retrying failed itemsCopy Anchor Link

These are two different operations — keep them straight:

Replaying the idempotency key is about delivering your request reliably. POST /v1/batches/{id}/retry is about re-running the items of a batch that already ran and didn't succeed — it creates a new batch from the failed and canceled items of a batch that is in a terminal state. It does not apply to a batch that is still in flight, or one with no failed or canceled items (you'll get a 409).

curl -X POST https://api.batchrouter.com/v1/batches/batch_123/retry \
  -H "Authorization: Bearer $BATCHROUTER_API_KEY"

Re-delivering resultsCopy Anchor Link

If a batch completed but delivery to your customer-owned bucket failed, that's a separate concern from re-running the work. Use POST /v1/batches/{id}/redeliver to re-enqueue delivery — it's guarded against double-delivery, returning 409 if the batch is already delivered or delivering. See delivery targets for setup. For webhook delivery, results are also always retrievable via GET /v1/batches/{id}/results, so a missed webhook never means lost data.

Automatic provider-side retriesCopy Anchor Link

You don't have to handle transient provider failures yourself. Once a batch is dispatched, BatchRouter manages the lifecycle on your behalf:

Next stepsCopy Anchor Link