BatchRouter Docs

JSONL input format

Define BatchRouter batch items in JSONL — per-line item objects, responses/embeddings/vision input shapes, mixed-model batches, and file uploads.

A BatchRouter batch is a list of items, where each item is one JSON object: a model call you want run. You can send items inline in the request body or upload them as a JSONL file (one item per line) and reference it by input_file_id. This page describes the item shape so your JSONL validates on the first try.

The item object

Every item — inline or one JSONL line — has the same four fields:

FieldRequiredDescription
customer_item_idyesYour stable ID for this item. Echoed back on every result so you can join outputs to inputs. Must be unique within the batch.
operationyesresponses, embeddings, or vision. Determines the shape of input.
modelyesA model ID. List real values with GET /v1/catalog/models.
inputyesThe operation-specific payload (see below).

The canonical item, shown with the catalog model gpt-5.4-mini (run GET /v1/catalog/models for the models available now):

{"customer_item_id":"item-1","operation":"responses","model":"gpt-5.4-mini","input":{"messages":[{"role":"user","content":"Summarize: BatchRouter routes batch-AI workloads across providers."}]}}

As JSONL, each item is one line — no array, no trailing commas, no blank lines:

{"customer_item_id":"item-1","operation":"responses","model":"gpt-5.4-mini","input":{"messages":[{"role":"user","content":"Summarize: BatchRouter routes batch-AI workloads across providers."}]}}
{"customer_item_id":"item-2","operation":"responses","model":"gpt-5.4-mini","input":{"messages":[{"role":"user","content":"List three benefits of batch inference."}]}}
{"customer_item_id":"item-3","operation":"embeddings","model":"text-embedding-3-small","input":{"input":"BatchRouter routes batch-AI workloads across providers."}}

If validation fails, the quote and batch endpoints return 400 with error.details.preflight describing the problem (for example a malformed JSONL line, an unsupported file type, or a context window exceeded). Fix the input and resubmit — no credits are spent on a rejected batch.

Input shapes by operation

The input object differs per operation.

responses

A chat-style request. Provide a messages array of { role, content }:

{
  "customer_item_id": "item-1",
  "operation": "responses",
  "model": "gpt-5.4-mini",
  "input": {
    "messages": [
      { "role": "user", "content": "Summarize: BatchRouter routes batch-AI workloads across providers." }
    ]
  }
}

embeddings

Provide the text to embed under input:

{
  "customer_item_id": "item-3",
  "operation": "embeddings",
  "model": "text-embedding-3-small",
  "input": { "input": "BatchRouter routes batch-AI workloads across providers." }
}

vision

A messages array where a user message's content is a list of content blocks. Use a text block for the prompt and an input_image block that references an uploaded file by file_id:

{
  "customer_item_id": "item-4",
  "operation": "vision",
  "model": "gpt-5.4-mini",
  "input": {
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "Describe what is in this image." },
          { "type": "input_image", "file_id": "file_abc123" }
        ]
      }
    ]
  }
}

input_image blocks reference a file_id returned by POST /v1/files (see below). Document inputs use an input_file block of the same shape: { "type": "input_file", "file_id": "file_…" }. Use GET /v1/catalog/models to confirm a model accepts file uploads before you build vision items.

Mixed-model batches

Items in one batch can target different models. BatchRouter splits them into model-specific internal lanes, routes and prices each lane independently, and reports everything under a single customer batch id. You poll, retrieve results, and bill against that one batch.

{"customer_item_id":"item-1","operation":"responses","model":"gpt-5.4-mini","input":{"messages":[{"role":"user","content":"Summarize: BatchRouter routes batch-AI workloads across providers."}]}}
{"customer_item_id":"item-2","operation":"responses","model":"claude-haiku-4-5","input":{"messages":[{"role":"user","content":"Summarize: BatchRouter routes batch-AI workloads across providers."}]}}
{"customer_item_id":"item-3","operation":"embeddings","model":"text-embedding-3-small","input":{"input":"BatchRouter routes batch-AI workloads across providers."}}

To let BatchRouter pick the cheapest eligible model instead of pinning one per item, omit per-item model selection and pass models[] at quote time. See Quote a batch.

Uploading a JSONL or binary file

Inline items are simplest for small batches. Upload a file when your input is large (a big JSONL of items) or binary (an image or document referenced from an item). Send the raw file body to POST /v1/files with the size and type in headers.

HeaderRequiredValue
Content-LengthyesExact body size in bytes (enforced before storage).
Content-TypeyesThe actual MIME type — text/plain (or application/json) for a JSONL of items, image/png, application/pdf, etc. for binary.
X-BatchRouter-FilenamenoOriginal filename. URL-encode spaces or non-ASCII characters.
X-BatchRouter-PurposenoDefaults to model_input.

Supported binary categories include image/*, audio/*, video/*, text/*, PDF, Word, PowerPoint, Excel, RTF, JSON, and XML. The response returns a file_id.

# Upload a JSONL file of items
curl -X POST https://api.batchrouter.com/v1/files \
  -H "Authorization: Bearer $BATCHROUTER_API_KEY" \
  -H "Content-Type: text/plain" \
  -H "Content-Length: $(wc -c < items.jsonl)" \
  -H "X-BatchRouter-Filename: items.jsonl" \
  -H "X-BatchRouter-Purpose: model_input" \
  --data-binary @items.jsonl
# => { "file_id": "file_abc123", ... }

Reference the file in a batch

For a JSONL of items, create the batch with input_file_id instead of inline items. Mixed-model JSONL is supported, and you still pass your quote_id:

{
  "input_file_id": "file_abc123",
  "quote_id": "qlock_..."
}

For a binary input (an image or document), keep your items inline (or in another JSONL) and reference the uploaded file by its file_id inside a content block, as in the vision example above:

{ "type": "input_image", "file_id": "file_abc123" }

Provide either inline items or input_file_id on POST /v1/batches — not both.

Next steps

Quote a batch

Price your items for free and pick a routing mode before you spend credits.

Submit a batch

Create the batch with inline items or input_file_id and an idempotency key.

Model catalog

List real models, supported operations, file support, and per-provider pricing.

API reference

Full schemas for /v1/files, /v1/batches, and the item object.

How routing works

Previous Page

Files & large batches

Next Page

On this page

The item objectInput shapes by operationresponsesembeddingsvisionMixed-model batchesUploading a JSONL or binary fileReference the file in a batchNext steps