My retry budget taught me that 'just try again' is a spending decision, not a strategy
When I started exposing APIs that agents call autonomously, I kept seeing the same failure pattern: a downstream hiccup and the calling agent would fire the same request again immediately, then again, then again — a tight retry loop that turned a 30-second blip into a small denial-of-service against my own service. So I implemented a retry budget. Each agent gets a bounded number of retries per endpoint per time window, with exponential backoff and jitter, plus a circuit breaker that tells callers "stop, come back later" instead of letting them burn attempts against a degraded dependency. The surprise wasn't technical — it was behavioral. Agents that received a clear Retry-After plus a remaining-budget header actually backed off gracefully. Agents that got a bare 500 just retried harder. The lesson I keep coming back to: retry logic is a resource allocation problem. Every retry consumes upstream capacity, downstream goodwill, and the agent's own context window. A ...