Skip to content

Reservation and settlement: how limits hold under concurrent calls

Before dispatch, KazenAI reserves maximum exposure:

  • estimated input cost, plus
  • enforceable maximum output at the recorded pricing version

The reservation is not the final bill. It prevents concurrent work from oversubscribing a shared budget.

Maximum output is the request’s max_tokens (max_output_tokens on the Responses API). A request without one is sent with the model’s default output cap, and that is what is held: for gpt-4o-mini, 16,384 tokens, about $0.011, for a call that may cost $0.00002. A limit with less left than the hold refuses the call. Set a max_tokens that fits your replies to hold close to the real cost; from 1.2.6 an over-limit refusal says what the call could cost and what was left.

For a stream, KazenAI advances that reservation to in_flight at the provider dispatch boundary. This distinction matters: an expired reservation that was never dispatched may be released, while provider-started work with unknown usage must remain pending for reconciliation.

When authoritative provider usage arrives:

  • settle actual cost
  • release unused reserved exposure
  • emit final accounting evidence

If usage is missing or the outcome is unknown—for example, an active stream is cancelled before final provider usage arrives—settlement remains pending. KazenAI records outcome_unknown; it is never silently recorded as exact zero or treated as a call that never reached the provider.

The exception is a definitive refusal: when the provider rejects the request with an HTTP 4xx (bad request, invalid key, unknown model, rate limited — not 408, 409 or 499), nothing was generated. That reservation settles at exactly $0 and its hold is released straight away, so a typo in a model name does not tie up budget.

The optional stream_cutoff_usd guard estimates only output observable at the client. It cannot see hidden reasoning or provider-side work after a disconnect, so that estimate is never substituted for authoritative settlement.

A limit in shadow mode reserves and settles like any other, so real spend is always recorded. It also keeps an enforced-world ledger that counts only calls enforcement would have admitted, and it never blocks.

Each retry is a new attempt_id with an idempotent reservation key. Prior attempts must not be double-settled.

When local or shared budget authority is unavailable:

  • fail-closed (the default) refuses provider dispatch (BudgetUnavailable)
  • fail-open is an organization setting an admin chooses in Agent FinOps: the call runs with only the SDK’s local caps, and it is not charged to the organization’s limits

See Configuration and Errors.