JSONL input format

A BatchRouter batch is a list of items, where each item is one JSON object: a model call you want run. You can send items inline in the request body or upload them as a JSONL file (one item per line) and reference it by input_file_id. This page describes the item shape so your JSONL validates on the first try.

The item object

Every item — inline or one JSONL line — has the same four fields:

Field	Required	Description
customer_item_id	yes	Your stable ID for this item. Echoed back on every result so you can join outputs to inputs. Must be unique within the batch.
operation	yes	responses, embeddings, or vision. Determines the shape of input.
model	yes	A model ID. List real values with GET /v1/catalog/models.
input	yes	The operation-specific payload (see below).

The canonical item, illustrated with gpt-4o-mini (run GET /v1/catalog/models for real, available models):

{"customer_item_id":"item-1","operation":"responses","model":"gpt-4o-mini","input":{"messages":[{"role":"user","content":"Summarize: BatchRouter routes batch-AI workloads across providers."}]}}

As JSONL, each item is one line — no array, no trailing commas, no blank lines:

{"customer_item_id":"item-1","operation":"responses","model":"gpt-4o-mini","input":{"messages":[{"role":"user","content":"Summarize: BatchRouter routes batch-AI workloads across providers."}]}}
{"customer_item_id":"item-2","operation":"responses","model":"gpt-4o-mini","input":{"messages":[{"role":"user","content":"List three benefits of batch inference."}]}}
{"customer_item_id":"item-3","operation":"embeddings","model":"text-embedding-3-small","input":{"input":"BatchRouter routes batch-AI workloads across providers."}}

Caution

If validation fails, the quote and batch endpoints return 400 with error.details.preflight describing the problem (for example a malformed JSONL line, an unsupported file type, or a context window exceeded). Fix the input and resubmit — no credits are spent on a rejected batch.

Input shapes by operation

The input object differs per operation.

responses

A chat-style request. Provide a messages array of { role, content }:

{
  "customer_item_id": "item-1",
  "operation": "responses",
  "model": "gpt-4o-mini",
  "input": {
    "messages": [
      { "role": "user", "content": "Summarize: BatchRouter routes batch-AI workloads across providers." }
    ]
  }
}

embeddings

Provide the text to embed under input:

{
  "customer_item_id": "item-3",
  "operation": "embeddings",
  "model": "text-embedding-3-small",
  "input": { "input": "BatchRouter routes batch-AI workloads across providers." }
}

vision

A messages array where a user message’s content is a list of content blocks. Use a text block for the prompt and an input_image block that references an uploaded file by file_id:

{
  "customer_item_id": "item-4",
  "operation": "vision",
  "model": "gpt-4o-mini",
  "input": {
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "Describe what is in this image." },
          { "type": "input_image", "file_id": "file_abc123" }
        ]
      }
    ]
  }
}

Note

input_image blocks reference a file_id returned by POST /v1/files (see below). Document inputs use an input_file block of the same shape: { "type": "input_file", "file_id": "file_…" }. Use GET /v1/catalog/models to confirm a model accepts file uploads before you build vision items.

Mixed-model batches

Items in one batch can target different models. BatchRouter splits them into model-specific internal lanes, routes and prices each lane independently, and reports everything under a single customer batch id. You poll, retrieve results, and bill against that one batch.

{"customer_item_id":"item-1","operation":"responses","model":"gpt-4o-mini","input":{"messages":[{"role":"user","content":"Summarize: BatchRouter routes batch-AI workloads across providers."}]}}
{"customer_item_id":"item-2","operation":"responses","model":"claude-haiku","input":{"messages":[{"role":"user","content":"Summarize: BatchRouter routes batch-AI workloads across providers."}]}}
{"customer_item_id":"item-3","operation":"embeddings","model":"text-embedding-3-small","input":{"input":"BatchRouter routes batch-AI workloads across providers."}}

Tip

To let BatchRouter pick the cheapest eligible model instead of pinning one per item, omit per-item model selection and pass models[] at quote time. See Quote a batch.

Uploading a JSONL or binary file

Inline items are simplest for small batches. Upload a file when your input is large (a big JSONL of items) or binary (an image or document referenced from an item). Send the raw file body to POST /v1/files with the size and type in headers.

Header	Required	Value
Content-Length	yes	Exact body size in bytes (enforced before storage).
Content-Type	yes	The actual MIME type — text/plain (or application/json) for a JSONL of items, image/png, application/pdf, etc. for binary.
X-BatchRouter-Filename	no	Original filename. URL-encode spaces or non-ASCII characters.
X-BatchRouter-Purpose	no	Defaults to model_input.

Supported binary categories include image/*, audio/*, video/*, text/*, PDF, Word, PowerPoint, Excel, RTF, JSON, and XML. The response returns a file_id.

cURL

# Upload a JSONL file of items
curl -X POST https://api.batchrouter.com/v1/files \
  -H "Authorization: Bearer $BATCHROUTER_API_KEY" \
  -H "Content-Type: text/plain" \
  -H "Content-Length: $(wc -c < items.jsonl)" \
  -H "X-BatchRouter-Filename: items.jsonl" \
  -H "X-BatchRouter-Purpose: model_input" \
  --data-binary @items.jsonl
# => { "file_id": "file_abc123", ... }

Node

import { readFileSync } from 'node:fs';

const body = readFileSync('items.jsonl');

const res = await fetch('https://api.batchrouter.com/v1/files', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.BATCHROUTER_API_KEY}`,
    'Content-Type': 'text/plain',
    'Content-Length': String(body.byteLength),
    'X-BatchRouter-Filename': 'items.jsonl',
    'X-BatchRouter-Purpose': 'model_input',
  },
  body,
});

const { file_id } = await res.json();

Python

import os
import requests

with open("items.jsonl", "rb") as f:
    body = f.read()

res = requests.post(
    "https://api.batchrouter.com/v1/files",
    headers={
        "Authorization": f"Bearer {os.environ['BATCHROUTER_API_KEY']}",
        "Content-Type": "text/plain",
        "Content-Length": str(len(body)),
        "X-BatchRouter-Filename": "items.jsonl",
        "X-BatchRouter-Purpose": "model_input",
    },
    data=body,
)

file_id = res.json()["file_id"]

Reference the file in a batch

For a JSONL of items, create the batch with input_file_id instead of inline items. Mixed-model JSONL is supported, and you still pass your quote_id:

{
  "input_file_id": "file_abc123",
  "quote_id": "qlock_..."
}

For a binary input (an image or document), keep your items inline (or in another JSONL) and reference the uploaded file by its file_id inside a content block, as in the vision example above:

{ "type": "input_image", "file_id": "file_abc123" }

Caution

Provide either inline items or input_file_id on POST /v1/batches — not both.

Next steps

Quote a batch Price your items for free and pick a routing mode before you spend credits.

Submit a batch Create the batch with inline items or input_file_id and an idempotency key.

Model catalog List real models, supported operations, file support, and per-provider pricing.

API reference Full schemas for /v1/files, /v1/batches, and the item object.