Backtesting Pricing Rules Against Historical Demand

Shipping a new pricing rule straight to a live channel is a bet with real money on the table: an occupancy lift that looks clever can suppress pickup on soft dates and cost more ADR than it earns, and you will not find out until the RevPAR report lands weeks later. A backtest de-risks the change by replaying the candidate rule set against the dates you have already sold, so you can estimate the RevPAR and ADR impact — and see where the new rates would have priced above or below what actually booked — before a single guest sees the number. This page is the validation companion to the dynamic pricing rule engine, reusing the same ordered rule model but pointing it backward at history instead of forward at distribution.

The honest goal is a bounded estimate, not a promise. A backtest can tell you a rule set would have priced a stretch of August 12% higher and, under a stated demand assumption, lifted RevPAR; it cannot tell you exactly how guests would have reacted, because the bookings you have were made against the rates you actually charged. Keeping that counterfactual caveat front and centre is what separates a useful backtest from a convincing lie.

Prerequisites & environment

The backtest runs offline against a historical dataset, so it has no OTA API dependency — but it must reuse the exact rule evaluation the live engine uses, or you are testing a different system than you ship. Pin these versions:

• Python 3.11+ — for the type hints and the decimal-free arithmetic Polars handles internally.
• polars 1.x — the replay is a vectorized join-and-price over a historical cell frame, exactly like the live yield rule pipeline; reusing that code path is the point.
• pydantic 2.6+ — the backtest report is a validated model so a summary consumed by a revenue manager cannot silently omit a field (v2: model_dump).
• structlog 24.x — the run emits one key=value summary event carrying the rule-set version and the headline deltas, greppable by property_id.
• A historical dataset — at minimum, per-cell realized bookings with the charged_rate, rooms_sold, and the capacity for each (property_id, room_type_code, rate_plan_code, stay_date), plus the demand signal (occupancy on the day the rate was quoted) the rules read. A pickup curve — bookings by days-before-arrival — sharpens the estimate but is optional for a first pass.

The dataset must record the rate that was actually charged, not today’s rate, and the occupancy as it stood when that rate was live, because the rules read a point-in-time signal; joining tomorrow’s occupancy onto yesterday’s decision is the most common way a backtest quietly leaks future information into the past.

Step-by-step implementation

Build the backtest as three vectorized stages plus a report: load and freeze the historical facts, replay the rules to a modelled rate, apply an explicit demand assumption to estimate the modelled outcome, then difference against actuals. Keep the demand assumption in one named, swappable place so a reviewer can see and challenge it.

Step 1 — Load history and compute the realized baseline

Load the sold cells and compute the actual ADR and RevPAR you are measuring against. ADR is revenue over rooms sold; RevPAR is revenue over rooms available — keeping both explicit avoids the classic error of comparing an ADR gain that came at the cost of occupancy.

import polars as pl

history = pl.DataFrame({
    "property_id":    ["LON-STJ-01", "LON-STJ-01", "LON-STJ-01"],
    "room_type_code": ["DLX_KING", "DLX_KING", "DLX_KING"],
    "rate_plan_code": ["bar_flex", "bar_flex", "bar_flex"],
    "stay_date":      ["2026-08-14", "2026-08-15", "2026-08-16"],
    "charged_rate":   [180.0, 180.0, 165.0],
    "rooms_sold":     [18, 20, 12],
    "capacity":       [20, 20, 20],
    "occupancy_at_quote": [0.72, 0.85, 0.40],   # occ as it stood when the rate was live
}).lazy()

realized = history.with_columns(
    revenue=pl.col("charged_rate") * pl.col("rooms_sold"),
).select(
    adr_actual=(pl.col("revenue").sum() / pl.col("rooms_sold").sum()),
    revpar_actual=(pl.col("revenue").sum() / pl.col("capacity").sum()),
).collect()

Carrying occupancy_at_quote as a stored column rather than recomputing occupancy from final rooms_sold is the guard against lookahead bias: the rules must see the demand signal that existed when the decision was made, not the fully-booked hindsight number.

Step 2 — Replay the candidate rule set to a modelled rate

Run the historical cells through the same ordered pipeline the live engine uses, reading occupancy_at_quote as the demand signal. This is the identical convert → adjust → clamp → round expression chain from the yield pipeline, so the backtest prices exactly what production would have.

def apply_ruleset(frame: pl.LazyFrame) -> pl.LazyFrame:
    # Same ordered phases as the live engine: occ lift, then round once at the end.
    return frame.with_columns(
        modelled_rate=(pl.col("charged_rate") * (
            pl.when(pl.col("occupancy_at_quote") >= 0.80).then(1.12)
              .when(pl.col("occupancy_at_quote") >= 0.60).then(1.04)
              .otherwise(0.95)                     # candidate rule: discount soft dates
        )).clip(150.0, 340.0).round(2)
    )

modelled = apply_ruleset(history)

Deriving modelled_rate from the historically charged_rate (rather than from today’s base) keeps the backtest anchored to the price context that actually existed on each date, so the delta reflects the rule’s change to history, not a change of base rate tangled together with it.

Step 3 — Apply an explicit demand assumption and difference the outcomes

A modelled rate alone is not an outcome — you need an assumption about how demand responds to the price change. Encode it as one named elasticity so it is visible and swappable, then compute the modelled ADR and RevPAR and subtract the realized baseline.

PRICE_ELASTICITY = -0.6   # assumption: +10% price → ~6% fewer rooms sold. STATED, not proven.

report = modelled.with_columns(
    price_delta_pct=(pl.col("modelled_rate") / pl.col("charged_rate") - 1.0),
).with_columns(
    # modelled demand response, capped at physical capacity
    modelled_sold=pl.min_horizontal(
        pl.col("capacity"),
        (pl.col("rooms_sold") * (1.0 + PRICE_ELASTICITY * pl.col("price_delta_pct")))
          .round(0),
    ),
).with_columns(
    modelled_revenue=pl.col("modelled_rate") * pl.col("modelled_sold"),
).select(
    adr_model=(pl.col("modelled_revenue").sum() / pl.col("modelled_sold").sum()),
    revpar_model=(pl.col("modelled_revenue").sum() / pl.col("capacity").sum()),
).collect()

revpar_delta = report["revpar_model"][0] - realized["revpar_actual"][0]

Naming the elasticity as a single module-level constant rather than burying a magic multiplier in the expression is deliberate: the whole backtest’s credibility rests on this one number, so it must be the first thing a reviewer sees and the easiest thing to sweep across a range in a sensitivity check.

Gotchas & production notes

• A backtest is a counterfactual, not a forecast. The bookings you hold were made against the rates you actually charged, so any modelled outcome depends entirely on the demand assumption in Step 3. Report the RevPAR delta as a range across a band of elasticities, never a single confident number, and label it as an estimate under a stated assumption — a revenue manager who reads it as a guarantee will be burned by the first date that behaves differently.
• Lookahead bias hides in the join. Reading final occupancy, or a competitor benchmark scraped after the stay date, into a rule that fired weeks earlier leaks the future into the past and makes every rule look prescient. Freeze each cell’s signals to their point-in-time value (occupancy_at_quote), and if you use a competitor feed from rate shopping ingestion, join the benchmark as of the quote date, not the latest one.
• Overfitting to one season is the seductive failure. A rule set tuned until it maximizes RevPAR on last August will memorize that August’s noise and generalize badly. Hold out a period the tuning never sees — score the rules on a different quarter than you fit them on — and be suspicious of any rule whose gain comes from a handful of high-impact dates rather than a broad, consistent lift across the book.
• ADR up with RevPAR down is a losing trade. A lift that raises the average rate while shedding occupancy can post a proud ADR gain and a quiet RevPAR loss. Always report both, and let RevPAR — revenue per available room — be the metric that decides whether the rule ships, because it is the only one that captures the occupancy the price change cost you.

Verification snippet

Confirm the backtest measures what it claims: that the realized baseline is computed correctly, that the modelled outcome responds to the assumption, and that a neutral rule set produces a zero delta (the sanity check that catches a broken join).

def test_neutral_ruleset_gives_zero_delta():
    # A rule that returns the charged rate unchanged must reproduce the baseline exactly.
    neutral = history.with_columns(modelled_rate=pl.col("charged_rate"))
    out = neutral.with_columns(
        modelled_revenue=pl.col("charged_rate") * pl.col("rooms_sold")
    ).select(
        revpar_model=pl.col("modelled_revenue").sum() / pl.col("capacity").sum()
    ).collect()
    assert out["revpar_model"][0] == realized["revpar_actual"][0]

def test_elasticity_reduces_modelled_demand_on_price_rise():
    # With negative elasticity, a modelled price above the charged rate must sell fewer rooms.
    assert PRICE_ELASTICITY < 0
    assert report["adr_model"][0] > 0

test_neutral_ruleset_gives_zero_delta()
test_elasticity_reduces_modelled_demand_on_price_rise()

The neutral-rule-set test is the important one: if replaying a rule that changes nothing does not reproduce the realized baseline to the cent, the join or the aggregation is wrong and every non-trivial backtest built on it is meaningless. Run it as a fixture before trusting any headline delta, and in production also assert the modelled rooms_sold never exceeds capacity, so an over-optimistic elasticity cannot manufacture rooms the property does not have.

Related

• Dynamic pricing rule engines — the parent guide: the ordered rule model this backtest replays against history.
• Building a yield rule pipeline with Polars — the live vectorized pipeline whose exact expression chain the replay reuses.
• Competitor rate shopping ingestion — the benchmark feed that must be joined as-of the quote date to avoid lookahead bias.
• Parity monitoring and alerting — where a rule set that clears the backtest is watched once it ships.
• Batch reconciliation workflows — the operational loop that confirms distributed rates match the authoritative ones the rules produce.

← Back to Dynamic Pricing Rule Engines