Normalizing Competitor Rates from Gross vs Net to a Comparable Basis

A scraped competitor rate is almost never on the same footing as your own: one source quotes a tax-and-fee-inclusive gross rate for two guests, another quotes a net room-only rate before a city tax and a resort fee, and a third quotes both in a different currency. Compare them raw and the benchmark is fiction. This page specifies the arithmetic that turns those mismatched quotes into one comparable number, as the reconciliation step of the competitor rate shopping ingestion workflow that feeds the pricing engine. Every transformation here uses Decimal, because a benchmark that is later diffed against your rate to the cent cannot survive floating-point drift.

The goal is a single, defensible rule: given a raw observation and what the source tells you about its basis, produce the room-only, tax-normalized, single-currency, per-occupancy figure your own rate is expressed in — and refuse to guess when the source does not say enough to reconcile safely.

Prerequisites & environment

The reconciliation runs inside the normalizer, after collection and before the staleness and outlier guards. Pin these so the math is identical across every worker:

Python 3.11+ — for the decimal module and modern type hints.
pydantic 2.6+ — the fee and tax basis for each source is expressed as a validated config model (v2 API: field_validator, model_dump).
structlog 24.x — every reconciliation that cannot be completed is logged as a key=value event keyed by source and competitor_property_id.
A pinned FX snapshot — one exchange-rate table captured at run start and passed through every observation, never a live per-row lookup. The reasons this must be pinned are covered under gotchas below.
A per-source basis registry — for each source, whether its quote is gross or net, which taxes and fees are already folded in, and the meal-plan value to strip. This is configuration, not code, and adding a source is a reviewed change.

The rule this page enforces is one clause of the property-wide data schema standardization contract; the canonical CompetitorRate shape the normalized value lands in is defined on the parent guide.

Figure 1: a gross tax-inclusive quote and a net tax-exclusive quote in two currencies both resolve to one room-only, single-currency, per-occupancy figure by applying currency, tax/fee, board, and occupancy adjustments in a fixed order.

Step-by-step implementation

The reconciliation is four operations applied in a fixed order — currency, then tax/fee basis, then board, then occupancy. Order matters: a percentage tax must be applied to a converted amount, and a per-person board credit must be subtracted before you divide by occupancy.

Step 1 — Model each source’s basis as validated config

Encode what every source’s quote already includes so the normalizer never has to guess. A source is gross or net, includes a known set of taxes and fees, and quotes a meal plan — all declared, all reviewed.

python

from decimal import Decimal
from pydantic import BaseModel, Field, field_validator

class SourceBasis(BaseModel):
    source: str
    is_gross: bool                         # True: taxes+fees already folded in
    tax_pct: Decimal = Field(ge=0, le=1)   # e.g. Decimal("0.08") for 8% city tax
    fixed_fee: Decimal = Field(ge=0)       # per-stay resort/city fee in source currency
    meal_plan: str                         # source's board token: ro, bb, hb...

    @field_validator("tax_pct", "fixed_fee", mode="before")
    @classmethod
    def to_decimal(cls, v) -> Decimal:
        return Decimal(str(v))             # coerce via str so 0.08 never enters as a float

BASIS = {
    "booking_com": SourceBasis(source="booking_com", is_gross=True,
                               tax_pct=Decimal("0.08"), fixed_fee=Decimal("15"),
                               meal_plan="bb"),
    "vendor_x":    SourceBasis(source="vendor_x", is_gross=False,
                               tax_pct=Decimal("0.08"), fixed_fee=Decimal("15"),
                               meal_plan="ro"),
}

Coercing every rate-bearing field through Decimal(str(v)) rather than Decimal(v) is the subtle guard: passing a raw float like 0.08 into Decimal preserves its binary-rounding error, while stringifying first captures the exact decimal the config author wrote.

Step 2 — Pin currency against the run snapshot

Convert to the property’s base currency first, using one FX table captured at the start of the shop run. Never call a live FX service per observation — two cells priced seconds apart would otherwise use different rates and manufacture drift.

python

def to_base_currency(amount: Decimal, currency: str, base: str,
                     fx_snapshot: dict[str, Decimal]) -> Decimal:
    if currency == base:
        return amount
    rate = fx_snapshot.get(currency)
    if rate is None:                       # a missing rate is a hard stop, not a guess
        raise ValueError(f"no pinned FX rate for {currency} in this run's snapshot")
    return amount * rate                   # rate expressed as units of base per 1 unit source

Raising on a missing FX rate rather than defaulting to 1.0 is what stops a currency the snapshot never loaded from silently passing through unconverted — a THB amount treated as EUR would look like a 40x undercut and could trip the outlier guard or, worse, slip through as a plausible number.

Step 3 — Reconcile to a common gross-or-net basis

Bring every quote to the same tax/fee footing. Here the target is a net-of-tax, net-of-fee room figure. A gross quote has its embedded tax and fee removed; a net quote is already there. Applying the tax adjustment to the converted amount is why this step follows currency conversion.

python

def to_net_of_tax_and_fees(amount: Decimal, basis: SourceBasis,
                           fee_in_base: Decimal) -> Decimal:
    if basis.is_gross:
        # gross = (net + fee) * (1 + tax_pct); invert to recover net room amount
        net_plus_fee = amount / (Decimal("1") + basis.tax_pct)
        return net_plus_fee - fee_in_base
    # net quote: tax and fee are quoted separately, so the amount is already net room
    return amount

Inverting the gross formula with division rather than subtracting a naive amount * tax_pct matters because the tax is levied on the taxable base, not on the gross total — subtracting eight percent of the gross would leave a residual cent or two that accumulates into a visible basis error across a full comp set.

Step 4 — Strip board and normalize per occupancy

Finally remove the meal-plan value to reach a room-only basis, then express the figure per the occupancy you price against. The board credit is per person, so it must be subtracted before any per-guest division.

python

from decimal import ROUND_HALF_EVEN

MEAL_CREDIT = {"ro": Decimal("0"), "bb": Decimal("18"), "hb": Decimal("42")}

def to_room_only_per_occupancy(net_room: Decimal, basis: SourceBasis,
                               quoted_occupancy: int) -> Decimal:
    board = MEAL_CREDIT.get(basis.meal_plan, Decimal("0")) * quoted_occupancy
    room_only = net_room - board
    # quantize once, at the end, so intermediate steps keep full precision
    return room_only.quantize(Decimal("0.01"), rounding=ROUND_HALF_EVEN)

Quantizing only at the very end — never after each intermediate operation — preserves precision through the currency and tax math and rounds exactly once, so the result is deterministic regardless of how many transformations preceded it. Banker’s rounding (ROUND_HALF_EVEN) matches the rounding mode the pricing engine uses, so the benchmark and your own rate round the same way.

Gotchas & production notes

A source that flips gross to net overnight is the classic silent undercut. If booking_com starts quoting net rates and the basis registry still marks it gross, the normalizer strips a tax that was never there and the benchmark drops by the tax percentage across every cell — enough to make the pricing engine chase a competitor that never moved. Validate the basis periodically against a known reference room, and alert on a step-change in the whole source’s normalized level, not just per-cell outliers.
Pin FX for the entire run, and stamp which snapshot was used. A live per-observation FX lookup means the first shop and the last shop of a batch price against different rates, so identical competitor rooms normalize to different benchmarks purely by fetch order. Capture one snapshot at run start, pass it through, and record its timestamp on each benchmark so a disputed number can be reproduced exactly.
Fees are per-stay, taxes are per-night-percentage — do not conflate them. A €15 resort fee is a fixed subtraction; an 8% city tax scales with the rate. Folding the fee into the tax percentage (or vice versa) produces a benchmark that is correct at one price point and wrong everywhere else, and the error grows with the nightly rate.
Refuse to reconcile when the source does not declare its basis. If an observation arrives without enough information to know whether it is gross or net, quarantine it — never assume gross. A wrong assumption here does not fail loudly; it produces a plausible-looking number that quietly biases the whole benchmark.

Verification snippet

Confirm the two reconciliation directions — collapsing a gross quote and passing a net quote — land on the same comparable figure before trusting the normalizer in front of the pricing engine.

python

from decimal import Decimal

def test_gross_and_net_reconcile_to_same_basis():
    fx = {"USD": Decimal("0.92")}
    # Gross USD quote: $210, breakfast, 8% tax, $15 fee, 2 guests
    gross = to_base_currency(Decimal("210"), "USD", "EUR", fx)          # 193.20 EUR
    fee_eur = Decimal("15") * fx["USD"]                                 # 13.80 EUR
    gross_net = to_net_of_tax_and_fees(gross, BASIS["booking_com"], fee_eur)
    gross_final = to_room_only_per_occupancy(gross_net, BASIS["booking_com"], 2)

    # Net EUR quote for the equivalent room, room-only, quoted net of tax and fee
    net_final = to_room_only_per_occupancy(Decimal("165.06"), BASIS["vendor_x"], 2)

    # Both should describe the same room-only, per-2-guest figure within a cent
    assert abs(gross_final - net_final) <= Decimal("0.01")

def test_missing_fx_rate_raises():
    import pytest
    with pytest.raises(ValueError, match="no pinned FX rate"):
        to_base_currency(Decimal("7200"), "THB", "EUR", {"USD": Decimal("0.92")})

test_gross_and_net_reconcile_to_same_basis()

The first test is the load-bearing one: it proves a gross tax-inclusive quote and a net tax-exclusive quote for the same underlying room converge, which is the entire point of normalization. The second proves the FX guard fails closed rather than letting an unconverted amount through. In production, also assert that a source whose declared basis is toggled produces a benchmark shift of exactly the tax percentage, so a mis-set basis flag is caught by a regression test rather than in a revenue review.

Competitor rate shopping ingestion — the parent workflow: collection, the canonical rate contract, and the guards this reconciliation feeds.
Scheduling competitor rate shops with async httpx — the polite collector that produces the raw observations reconciled here.
Dynamic pricing rule engines — the consumer that diffs this normalized benchmark against your own rate to the cent.
Data schema standardization — the property-wide contract this reconciliation rule is one clause of.
Parity monitoring and alerting — where a mis-normalized benchmark would surface as a phantom parity deviation.

← Back to Competitor Rate Shopping Ingestion