Embeddings & multimodal inputs

Every BatchRouter batch item declares an operation that tells the router what kind of work it is and what input shape to expect. There are three: responses (chat-style text generation, the default), embeddings (vector generation), and vision (multimodal prompts that mix text and images). This guide covers the input shape for each, with JSONL you can drop into a batch.

The operation is matched against each model’s supported operations — see GET /v1/catalog/models for which models accept responses, embeddings, or vision. If you omit operation, BatchRouter treats the item as responses.

Note

These are the native BatchRouter item shapes. If you already have OpenAI-Batch JSONL, you can submit it unchanged through the OpenAI-compatible surface instead — see Files & large batches.

responses — chat-style generation

Use responses for text generation: summarization, extraction, classification, rewriting, Q&A. The input is a messages array of {role, content} objects, the same shape you already use with chat-completions-style APIs.

{
  "customer_item_id": "item-1",
  "operation": "responses",
  "model": "gpt-4o-mini",
  "input": {
    "messages": [
      { "role": "user", "content": "Summarize: BatchRouter routes batch-AI workloads across providers." }
    ]
  }
}

customer_item_id is your own per-item identifier — it is echoed back on every result so you can join outputs to inputs. model is a slug from GET /v1/catalog/models (gpt-4o-mini here is illustrative). operation may be omitted for responses since it is the default.

embeddings — vector generation

Use embeddings to turn text into vectors for search, retrieval, clustering, or deduplication. The input is an input field that is either a single string or an array of strings (one vector per string).

single string

{
  "customer_item_id": "doc-1",
  "operation": "embeddings",
  "model": "text-embedding-3-small",
  "input": { "input": "BatchRouter routes batch-AI workloads across providers." }
}

array of strings

{
  "customer_item_id": "doc-2",
  "operation": "embeddings",
  "model": "text-embedding-3-small",
  "input": { "input": ["First chunk of text.", "Second chunk of text.", "Third chunk."] }
}

Tip

Pass an array when you want several vectors back from one item — it is cheaper and produces fewer rows than one item per string. Pick an embedding model whose operations list includes embeddings (check GET /v1/catalog/models/{slug}).

vision — multimodal (text + images)

Use vision for prompts that combine text with one or more images. The shape is the same messages array as responses, but a message’s content becomes an array of content blocks instead of a plain string. Mix text blocks with image blocks in the order you want the model to see them.

An image block references a file you uploaded first via POST /v1/files, by its returned file_id:

{ "type": "input_image", "file_id": "file_..." }

Upload the image, then reference it

1. Upload the file. POST /v1/files stores the image and returns a file_id. The upload defaults to purpose model_input; send the raw bytes with a Content-Length header (and X-BatchRouter-Purpose: model_input if you want to be explicit).

   cURL

   curl -X POST https://api.batchrouter.com/v1/files \
     -H "Authorization: Bearer $BATCHROUTER_API_KEY" \
     -H "Content-Type: image/png" \
     -H "X-BatchRouter-Purpose: model_input" \
     --data-binary @receipt.png

   Node

   import { readFileSync } from 'node:fs';
   
   const body = readFileSync('receipt.png');
   const res = await fetch('https://api.batchrouter.com/v1/files', {
     method: 'POST',
     headers: {
       Authorization: `Bearer ${process.env.BATCHROUTER_API_KEY}`,
       'Content-Type': 'image/png',
       'X-BatchRouter-Purpose': 'model_input',
       'Content-Length': String(body.length),
     },
     body,
   });
   const { file_id } = await res.json();

   Python

   import os, requests
   
   body = open("receipt.png", "rb").read()
   res = requests.post(
       "https://api.batchrouter.com/v1/files",
       headers={
           "Authorization": f"Bearer {os.environ['BATCHROUTER_API_KEY']}",
           "Content-Type": "image/png",
           "X-BatchRouter-Purpose": "model_input",
       },
       data=body,
   )
   file_id = res.json()["file_id"]

2. Reference the file_id in an input_image block inside your vision item (below).

3. Submit the batch as usual via POST /v1/batches. See Submit a batch.

Caution

Upload images before you submit the batch — input_image blocks resolve file_id at submission time, so the file must already exist. Use a model whose input_modalities include image (see GET /v1/catalog/models/{slug}).

Vision JSONL example

{
  "customer_item_id": "receipt-1",
  "operation": "vision",
  "model": "gpt-4o-mini",
  "input": {
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "input_image", "file_id": "file_abc123" },
          { "type": "text", "text": "Extract the merchant, date, and total from this receipt as JSON." }
        ]
      }
    ]
  }
}

Note

To pass a non-image document (a PDF, for example) to a model that supports it, use a file block instead: { "type": "input_file", "file_id": "file_...", "filename": "report.pdf" }. Upload it the same way with POST /v1/files. For exact per-model support and any size limits, see the API reference.

Mixing operations in one batch

A single batch can contain items with different operations and different models — BatchRouter splits them into model-specific internal lanes under your one customer batch. For example, you can embed a corpus and run a vision extraction in the same submission. Each item carries its own operation and model.

Quote before you submit

Quoting is free and works for every operation. Send a representative subset of items (you do not need to send them all) to POST /v1/quotes/model with the matching operation to see the price before creating the batch.

cURL

curl -X POST https://api.batchrouter.com/v1/quotes/model \
  -H "Authorization: Bearer $BATCHROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "operation": "embeddings",
    "model": "text-embedding-3-small",
    "items": [{ "customer_item_id": "doc-1", "operation": "embeddings", "input": { "input": "Sample text." } }]
  }'

Node

const res = await fetch('https://api.batchrouter.com/v1/quotes/model', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.BATCHROUTER_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    operation: 'embeddings',
    model: 'text-embedding-3-small',
    items: [
      { customer_item_id: 'doc-1', operation: 'embeddings', input: { input: 'Sample text.' } },
    ],
  }),
});
const quote = await res.json();

Python

import os, requests

res = requests.post(
    "https://api.batchrouter.com/v1/quotes/model",
    headers={
        "Authorization": f"Bearer {os.environ['BATCHROUTER_API_KEY']}",
        "Content-Type": "application/json",
    },
    json={
        "operation": "embeddings",
        "model": "text-embedding-3-small",
        "items": [
            {"customer_item_id": "doc-1", "operation": "embeddings", "input": {"input": "Sample text."}}
        ],
    },
)
quote = res.json()

Next steps

JSONL format The full item line schema — fields, validation, and one item per line.

Files & large batches Upload large inputs and reference them by file id instead of inlining items.

Submit a batch Quote, create, poll, and fetch results end to end.

API reference Interactive reference for every endpoint, field, and response shape.