InventoryAllocation

Implementing Exponential Backoff in Python for OTA Rate Limits

When a channel push returns 429 Too Many Requests, the naive fix — sleep a fixed second and retry — is exactly the reflex that keeps a property throttled all morning: every sync worker retries on the same tick, re-trips the limit the instant the window opens, and the last rate an OTA actually accepted goes stale. This page shows how to compute exponential backoff with full jitter in Python correctly, so that a throttled push waits a mathematically predictable, de-synchronised interval instead. It is the backoff mechanics referenced by handling OTA API rate limits, where the same curve powers the token-bucket governor’s 429/Retry-After retry policy across the wider API Sync & Data Ingestion Workflows pipeline.

Prerequisites & environment

The backoff computation itself is pure standard library, but the production wrapper leans on a small, pinned stack. tenacity in particular changed its wait-composition API across major versions, so pin it:

Outbound payloads should already be validated and carry rate-plan identifiers resolved against the rate plan taxonomy; backoff only decides when to re-send a well-formed mutation, never whether it is correct.

Full-jitter exponential backoff delay curve A line chart of the computed backoff delay against retry attempt zero to five. The solid violet line is the capped exponential ceiling base times two to the power of the attempt — one, two, four, eight, sixteen, and thirty-two clamped to thirty seconds. The shaded region beneath the line is the full-jitter draw, a uniform sample over the interval zero to the computed delay. A dashed pink horizontal line marks the thirty-second max_delay ceiling, which the raw curve would breach at attempt five where thirty-two is clamped to thirty. 0 10 20 30 delay (seconds) retry attempt (n) max_delay = 30s ceiling 1s 2s 4s 8s 16s 30s 0 1 2 3 4 5 2⁵=32 clamped computed cap · base·2ⁿ full-jitter draw · uniform [0, delay]

Step-by-step implementation

We build backoff in four small pieces: the delay-curve function, the retryable-status classifier, a hand-rolled async retry loop that shows the mechanics explicitly, and finally the idiomatic tenacity decorator you would actually ship.

Step 1 — Compute the jittered delay curve

The delay for attempt n is base_delay * multiplier ** n, clamped to a ceiling, then randomised. Using full jitter — a uniform draw over [0, computed_delay] rather than a fixed fraction added on top — spreads retries across the widest interval and is the single most effective defence against a synchronised retry storm.

python
import random

def compute_backoff(
    attempt: int,
    base_delay: float = 1.0,
    multiplier: float = 2.0,
    max_delay: float = 30.0,
) -> float:
    """Full-jitter exponential backoff: a uniform draw over [0, capped_delay]."""
    capped = min(base_delay * (multiplier ** attempt), max_delay)
    return random.uniform(0.0, capped)

Clamping with min(...) before drawing the jitter matters: if you cap after jittering, the ceiling stops bounding the curve once the raw exponential blows past max_delay, and a late attempt can still sleep for an unbounded fraction of a runaway value.

Step 2 — Classify which statuses may be retried

Backoff must never fire on a deterministic client error. Retrying a 400 malformed payload or a 422 invalid rate wastes the channel’s request budget and delays the alert that a genuine schema drift needs. Only throttling and transient upstream faults are eligible.

python
RETRYABLE_STATUSES = frozenset({408, 429, 500, 502, 503, 504})

def is_retryable(status_code: int) -> bool:
    # 408 Request Timeout is retryable; 4xx below it (400/401/403/404/409/422) is not.
    return status_code in RETRYABLE_STATUSES

408 is deliberately in the retryable set while its 4xx neighbours are not: a request timeout is an upstream/network condition that a later attempt can clear, whereas a 409 conflict or 422 validation failure will fail identically on every retry.

Step 3 — Drive an async retry loop that honours Retry-After

This loop pushes one rate mutation for a property_id / room_type_code / rate_plan_code triple to an OTA and retries on a retryable status. When the channel returns a Retry-After header it tells you exactly when the window resets, so that value must win over the locally computed curve.

python
import asyncio
import httpx
import structlog

log = structlog.get_logger("rate_parity.backoff")

async def push_rate_with_backoff(
    client: httpx.AsyncClient,
    payload: dict,
    max_retries: int = 5,
) -> httpx.Response:
    ota = payload["ota"]  # e.g. "booking_com", "expedia", "agoda"
    for attempt in range(max_retries + 1):
        resp = await client.post(f"/{ota}/rates/sync", json=payload)
        if resp.status_code < 400:
            log.info("rate_committed", ota=ota,
                     rate_plan_code=payload["rate_plan_code"], attempt=attempt)
            return resp
        if not is_retryable(resp.status_code) or attempt == max_retries:
            return resp  # terminal: caller dead-letters or alerts

        retry_after = resp.headers.get("Retry-After")
        delay = float(retry_after) if retry_after else compute_backoff(attempt)
        log.warning("rate_throttled", ota=ota, status=resp.status_code,
                    attempt=attempt, delay_s=round(delay, 2),
                    honoured_retry_after=bool(retry_after))
        await asyncio.sleep(delay)  # non-blocking: never time.sleep in async code
    return resp

asyncio.sleep — not time.sleep — is load-bearing here: a blocking sleep inside a coroutine would freeze the entire event loop, stalling every other channel’s concurrent pushes for the full backoff interval instead of just pausing this one.

Retry-loop control flow honouring Retry-After A flowchart of the async retry loop from Step 3. A rate push is posted to the OTA. If the status is under four hundred the response is committed and returned. Otherwise, if the status is not retryable the request terminates and is dead-lettered; if it is retryable but the attempt equals max_retries it also terminates. If it is retryable and attempts remain, the loop checks for a Retry-After header: when present the delay is float(Retry-After), otherwise the delay is compute_backoff(attempt). The worker then awaits asyncio.sleep(delay) and loops back to re-post with the attempt counter incremented. yes · 2xx/3xx no no yes yes no yes no retry · attempt += 1 POST rate push /{ota}/rates/sync status < 400 ? is_retryable ? attempt == max ? Retry-After present ? delay = float(Retry-After) delay = compute_backoff(attempt) await asyncio.sleep(delay) return resp rate_committed return resp dead-letter / alert

Step 4 — Ship the idiomatic tenacity decorator

In production you rarely hand-roll the loop. Declaring the policy once at module level with tenacity builds the retry machinery a single time and keeps the retry contract readable. Defining the decorator inside the calling function would rebuild that state on every invocation and defeat attempt counting.

python
from tenacity import (
    retry, stop_after_attempt, wait_exponential_jitter,
    retry_if_exception, before_sleep_log,
)

class Throttled(Exception):
    def __init__(self, status_code: int):
        self.status_code = status_code
        super().__init__(f"throttled: {status_code}")

@retry(
    retry=retry_if_exception(lambda e: isinstance(e, Throttled)),
    wait=wait_exponential_jitter(initial=1, max=30, jitter=2),  # same curve as Step 1
    stop=stop_after_attempt(5),
    before_sleep=before_sleep_log(log, log_level=30),  # WARNING
    reraise=True,
)
async def push_rate(client: httpx.AsyncClient, payload: dict) -> httpx.Response:
    resp = await client.post(f"/{payload['ota']}/rates/sync", json=payload)
    if is_retryable(resp.status_code):
        raise Throttled(resp.status_code)
    resp.raise_for_status()  # deterministic 4xx surface as HTTPStatusError, not a retry
    return resp

Raising a dedicated Throttled exception rather than retrying on any exception keeps the policy narrow: wait_exponential_jitter fires only for the statuses you chose in Step 2, while a raise_for_status() on a 400/422 escapes the retry loop immediately as an HTTPStatusError.

Gotchas & production notes

Verification snippet

Backoff is easy to break silently, so assert the three properties that actually matter: the curve stays under its ceiling, jitter never exceeds the capped value, and classification is deterministic.

python
def test_backoff_curve_is_bounded_and_jittered():
    # Full jitter: every sample sits within [0, capped_delay] and never exceeds the cap.
    for attempt in range(0, 8):
        capped = min(1.0 * 2.0 ** attempt, 30.0)
        samples = [compute_backoff(attempt, max_delay=30.0) for _ in range(500)]
        assert all(0.0 <= s <= capped for s in samples)
        assert max(samples) <= 30.0                 # ceiling holds at every attempt
    # Later attempts must, on average, wait longer than early ones.
    early = sum(compute_backoff(1) for _ in range(500)) / 500
    late = sum(compute_backoff(4) for _ in range(500)) / 500
    assert late > early

def test_status_classification_is_deterministic():
    assert is_retryable(429) and is_retryable(503) and is_retryable(408)
    assert not is_retryable(400) and not is_retryable(422) and not is_retryable(401)

The ceiling assertion is the one that guards production: a curve that silently loses its cap does not fail a smoke test, it just turns a brief throttle into a multi-minute stall that shows up only as stale rates on the channel.

← Back to Handling OTA API Rate Limits