Find retry loops that inflate LLM spend

Find expensive retry loops by grouping actual model attempts under one user task and recording why each retry happened. A lower per-call price will not resolve repeated attempts caused by an invalid request or an unbounded outer loop.

Look for the same failure fingerprint

Repeated schema failures with the same invalid arguments are different from a temporary connection failure. Record a sanitized failure category and stable operation signature, not an unrestricted payload dump. Check whether retrying changes any condition that could make the next attempt succeed.

Account for nested retries

An application may retry a model client that already retries internally, while an agent loop repeats the entire tool operation. Inspect the effective attempt count across these layers. Set one understandable budget and decide which layer owns retry decisions.

Reduce amplification safely

1. Count actual provider attempts, distinguishing them from repeated telemetry deliveries.
2. Stop retrying permanent validation and authorization failures without a changed input or permission state.
3. Use bounded delays and a total task deadline for transient failures.
4. Before retrying an operation with side effects, determine whether a previous attempt completed and whether an appropriate idempotency mechanism exists.

Interpret the ledger example

The fixture finds three attempts with the same permanent-invalid-input fingerprint and sums their synthetic cost. It is a diagnostic accounting check, not an automatic retry policy for all provider errors. Keep unknown outcomes and late usage records visible rather than assuming a timed-out call cost nothing.

Run the worked example

Save the synthetic Node.js example and run node find-llm-retry-loops.mjs with Node.js 22 or newer. The assertions check the illustrated records and calculations; they do not test a deployed provider, SDK, or access control. No credentials or network connection are needed.

// Synthetic teaching fixture. No credentials, network calls, or production verification.
import assert from 'node:assert/strict';

const attempts = [
  { task: 't1', fingerprint: 'invalid-input:missing-account', costUnits: 2 },
  { task: 't1', fingerprint: 'invalid-input:missing-account', costUnits: 2 },
  { task: 't1', fingerprint: 'invalid-input:missing-account', costUnits: 2 },
];
const cost = attempts.reduce((sum, a) => sum + a.costUnits, 0);
assert.equal(new Set(attempts.map(a => a.fingerprint)).size, 1);
assert.equal(cost, 6);
assert.equal(attempts.slice(1).reduce((sum, a) => sum + a.costUnits, 0), 4);
console.log('Three identical permanent failures cost 6 synthetic units; investigate retry amplification.');

References and further reading

• RFC 9110: idempotent HTTP methods
• DriftRail integration documentation

Maintained by DriftRail. AI assisted the October 2026 content and template revision. Examples are illustrative; they are not measured product benchmarks. Suggest a correction.