Batch tier pricing: state the trade-off

Which calls can wait

A batch tier sells the same model at a lower rate in exchange for results that arrive later, within a stated window. The buyer's question is which of their calls can wait that long, and they can answer it only if the page states the window, the limits on a batch and both batch rates. Without the window the discount cannot be weighed against anything.

We hold a batch rate on 121 listings. The rest may offer one in a console or a sales conversation, but a buyer comparing published pages counts only what is printed.

A batch rate that nobody can find is a discount that only existing customers use. Put it on the public pricing page, on the same row as the standard rate, so a buyer comparing sellers sees it while they are still choosing.

Work that suits a batch

Backfilling embeddings for an archive. Tagging last month's support tickets. Running an evaluation set against a new prompt. Translating a product catalogue overnight. None of these has a person waiting on the far end, and all of them are large enough that the rate matters more than the hour.

The opposite list is short and obvious: anything a user is watching, and anything a later step depends on within the same session. Buyers sort their own traffic between the two. The page only has to give them the facts to do it.

The middle ground is where the window decides. A report a manager reads at nine can be built overnight. A moderation queue a person clears within the hour probably cannot. The same buyer will put some of their work on each side once they know the window.

How a buyer prices the wait

The buyer has two costs to compare: the standard rate with an answer now, and the batch rate with an answer later plus the engineering to submit files, poll for results and handle failures. For a one-off job the engineering can outweigh the saving. For a recurring job it is paid once.

They also price the risk. If a batch misses its window, does the morning report go out empty, or does a fallback run at the standard rate? A buyer who has to build that fallback will count its cost against your discount. A page that says how often windows are missed, measured and dated, shrinks that cost; a page that says nothing leaves the buyer to assume the worst.

None of this needs a figure from us. It needs the window, the failure behaviour and both rates, written where the buyer reads the price.

The window is the price of the discount

State the completion window as a promise or a target, and say which. For a nightly job, the question is whether results can arrive after the morning run starts, and what happens to requests that are not finished in time: are they returned, retried, cancelled, or billed anyway.

If most batches finish well inside the window, it is tempting to say so. Say it only if it is measured and dated, because a buyer who plans on the typical case and gets the promised case will not trust the next sentence on the page.

Say what the window starts from: submission, validation of the file, or the moment the batch begins to run. Someone who submits at midnight and reads results at six needs to know which clock you are using.

Limits that only apply to batches

Batches often have their own limits: requests per file, file size, total tokens queued at once, and how many batches can run together. These are separate from the per-minute limits on synchronous calls and can be missing from the pricing page. If batch traffic counts against the synchronous allowance, say so, or a team will find out when its live product slows down during a backfill.

A buyer with a large backfill will hit the queued-token limit before anything else. Put it on the same page as the batch rate, by tier, so they can work out how many submissions the job will take before choosing between your batch tier and a cheaper synchronous seller.

State the file format too, and whether a malformed line fails the whole batch or only itself. Otherwise the first run finds out, and a failed overnight batch costs a day.

How the batch rate is compared

Give the batch input and output rates as separate figures per million tokens, beside the standard rates on the same row. A batch rate written only as a discount from the standard one makes the buyer do arithmetic, and a table of token prices holds rates, not discounts.

Say whether cached input pricing also applies inside a batch. Some buyers run large batches over the same long prefix, and the answer can change which provider is cheaper for them. A cost estimate from published rates will only include the combination if the page states it.

Name the models that have a batch tier. A batch tier that covers some models and not others, without a list, will be assumed to cover the one the buyer wanted until it does not.

If the batch rate differs by model, give it on each model row. A single batch discount stated once for the whole catalogue is read as applying everywhere, including to the models it does not cover.