BatchRouter Docs

Embeddings & multimodal inputs

Build batch items for the responses, embeddings, and vision operations — message inputs, embedding inputs, and image content blocks from uploaded files.

Every BatchRouter batch item declares an operation that tells the router what kind of work it is and what input shape to expect. There are three: responses (chat-style text generation, the default), embeddings (vector generation), and vision (multimodal prompts that mix text and images). This guide covers the input shape for each, with JSONL you can drop into a batch.

The operation is matched against each model's supported operations — see GET /v1/catalog/models for which models accept responses, embeddings, or vision. If you omit operation, BatchRouter treats the item as responses.

These are the native BatchRouter item shapes. If you already have OpenAI-Batch JSONL, you can submit it unchanged through the OpenAI-compatible surface instead — see Files & large batches.

responses — chat-style generation

Use responses for text generation: summarization, extraction, classification, rewriting, Q&A. The input is a messages array of {role, content} objects, the same shape you already use with chat-completions-style APIs.

{
  "customer_item_id": "item-1",
  "operation": "responses",
  "model": "gpt-5.4-mini",
  "input": {
    "messages": [
      { "role": "user", "content": "Summarize: BatchRouter routes batch-AI workloads across providers." }
    ]
  }
}

customer_item_id is your own per-item identifier — it is echoed back on every result so you can join outputs to inputs. model is the model id from GET /v1/catalog/models (gpt-5.4-mini here). operation may be omitted for responses since it is the default.

embeddings — vector generation

Use embeddings to turn text into vectors for search, retrieval, clustering, or deduplication. The input is an input field that is either a single string or an array of strings (one vector per string).

{
  "customer_item_id": "doc-1",
  "operation": "embeddings",
  "model": "text-embedding-3-small",
  "input": { "input": "BatchRouter routes batch-AI workloads across providers." }
}

Pass an array when you want several vectors back from one item — it is cheaper and produces fewer rows than one item per string. Pick an embedding model whose operations list includes embeddings (check GET /v1/catalog/models/{slug}).

vision — multimodal (text + images)

Use vision for prompts that combine text with one or more images. The shape is the same messages array as responses, but a message's content becomes an array of content blocks instead of a plain string. Mix text blocks with image blocks in the order you want the model to see them.

An image block references a file you uploaded first via POST /v1/files, by its returned file_id:

{ "type": "input_image", "file_id": "file_..." }

Upload the image, then reference it

  1. Upload the file. POST /v1/files stores the image and returns a file_id. The upload defaults to purpose model_input; send the raw bytes with a Content-Length header (and X-BatchRouter-Purpose: model_input if you want to be explicit).

    curl -X POST https://api.batchrouter.com/v1/files \
  -H "Authorization: Bearer $BATCHROUTER_API_KEY" \
  -H "Content-Type: image/png" \
  -H "X-BatchRouter-Purpose: model_input" \
  --data-binary @receipt.png

  2. Reference the file_id in an input_image block inside your vision item (below).

  3. Submit the batch as usual via POST /v1/batches. See Submit a batch.

Upload images before you submit the batch — input_image blocks resolve file_id at submission time, so the file must already exist. Use a model whose input_modalities include image (see GET /v1/catalog/models/{slug}).

Vision JSONL example

{
  "customer_item_id": "receipt-1",
  "operation": "vision",
  "model": "gpt-5.4-mini",
  "input": {
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "input_image", "file_id": "file_abc123" },
          { "type": "text", "text": "Extract the merchant, date, and total from this receipt as JSON." }
        ]
      }
    ]
  }
}

To pass a non-image document (a PDF, for example) to a model that supports it, use a file block instead: { "type": "input_file", "file_id": "file_...", "filename": "report.pdf" }. Upload it the same way with POST /v1/files. For exact per-model support and any size limits, see the API reference.

Mixing operations in one batch

A single batch can contain items with different operations and different models — BatchRouter splits them into model-specific internal lanes under your one customer batch. For example, you can embed a corpus and run a vision extraction in the same submission. Each item carries its own operation and model.

Quote before you submit

Quoting is free and works for every operation. Send a representative subset of items (you do not need to send them all) to POST /v1/quotes/model with the matching operation to see the price before creating the batch.

curl -X POST https://api.batchrouter.com/v1/quotes/model \
  -H "Authorization: Bearer $BATCHROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "operation": "embeddings",
    "model": "text-embedding-3-small",
    "items": [{ "customer_item_id": "doc-1", "operation": "embeddings", "input": { "input": "Sample text." } }]
  }'

Next steps

JSONL format

The full item line schema — fields, validation, and one item per line.

Files & large batches

Upload large inputs and reference them by file id instead of inlining items.

Submit a batch

Quote, create, poll, and fetch results end to end.

API reference

Interactive reference for every endpoint, field, and response shape.

Manage & monitor batches

Previous Page

Idempotency & retries

Next Page

On this page

responses — chat-style generationembeddings — vector generationvision — multimodal (text + images)Upload the image, then reference itVision JSONL exampleMixing operations in one batchQuote before you submitNext steps