Designing Immutable Audit-Log Schemas for Rate and Inventory Changes
An audit log is only worth keeping if you can prove nobody edited it, and that proof cannot rest on “our storage is locked down” — an auditor wants tamper-evidence baked into the records themselves. This page specifies the schema for an append-only audit record whose integrity is self-verifying: a SHA-256 hash chain that makes any retroactive edit mathematically detectable, sitting one level below the compliance and audit logging workflow that owns the end-to-end trail. The reader here is the Python engineer designing the record shape that will hold years of rate and inventory history.
The core idea is borrowed from block-linked ledgers without any of the distributed-consensus machinery: each record carries the hash of the one before it, so the records form a chain where altering entry n changes its hash and therefore breaks the prev_hash of entry n+1, and every entry after it. A verifier that walks the chain from a known genesis detects the first break and reports exactly where it is.
Prerequisites & environment
The schema and verifier below run wherever audit records are written and wherever they are later read for an integrity check. Pin these so the hash input is byte-identical on both sides:
- Python 3.11+ — for modern type hints and the
hashlibandhmacmodules used throughout. pydantic2.6+ — the record contract is a v2 model (field_validator,model_dump), which both validates a record and provides the canonical dict the hash is computed over.structlog24.x — verification failures are emitted as key=value events keyed bysequence, so a broken chain is greppable rather than buried in a formatted string.- A monotonic sequence source — a per-property counter (a database sequence or a Redis
INCR) that never reuses or reorders a number, since the chain’s ordering guarantee rests on it. - A canonical JSON serializer —
json.dumps(..., sort_keys=True, separators=(",", ":")), so the digest of a record does not depend on dict insertion order or incidental whitespace.
The one hard requirement is determinism: if the bytes hashed at write time differ in any way from the bytes hashed at verify time, every record reads as tampered. Money and timestamps are the usual culprits, so both are normalized to strings in the record before hashing.
record_hash feeds the next entry's prev_hash, so editing sequence 1's value breaks the link to sequence 2 and the verifier reports the break at sequence 1.Step-by-step implementation
Step 1 — Model the record with actor, action, and before/after
The record’s semantic core is who did what to which value. Capture the actor, the action type, and the before/after pair, plus the fields that let a verifier reproduce the hash: a monotonic sequence and the prev_hash it links to.
from datetime import datetime
from enum import Enum
from pydantic import BaseModel, Field, field_validator
class Action(str, Enum):
RATE_PUBLISHED = "rate.published"
INVENTORY_ADJUSTED = "inventory.adjusted"
STOP_SELL_TOGGLED = "stop_sell.toggled"
class AuditRecord(BaseModel):
sequence: int = Field(ge=0) # monotonic, per property_id
occurred_at: datetime # timezone-aware UTC
property_id: str = Field(pattern=r"^[A-Z]{3}-[A-Z0-9]{3,}-\d{2}$")
actor: str = Field(pattern=r"^(user|worker|system):[a-z0-9_\-]+$")
action: Action
subject: str # e.g. "DLX_KING/BAR" — what changed
before: str | None # prior value as a string, None on create
after: str # new value as a string
prev_hash: str = Field(min_length=8) # record_hash of sequence - 1, or "GENESIS"
@field_validator("occurred_at")
@classmethod
def utc_only(cls, v: datetime) -> datetime:
if v.tzinfo is None or v.utcoffset() is None:
raise ValueError("occurred_at must be timezone-aware UTC")
return v
Storing before and after as strings rather than native numbers is the deliberate choice that keeps the hash deterministic: a Decimal("180.00") and a float 180.0 render differently, so pinning the on-record form to a string means the digest cannot drift between a write in one service and a verify in another.
Step 2 — Compute the record hash over a canonical serialization
The hash covers the record’s content plus its sequence and prev_hash, serialized canonically so key order and whitespace never affect the digest. Exclude the record_hash field itself — it is the output, not an input.
import hashlib
import json
def compute_record_hash(record: AuditRecord) -> str:
body = record.model_dump(mode="json", exclude={"prev_hash"})
payload = json.dumps(body, sort_keys=True, separators=(",", ":"))
# prev_hash is folded in explicitly so the chain link is part of the digest
return hashlib.sha256(f"{record.prev_hash}|{payload}".encode()).hexdigest()
Using model_dump(mode="json") gives a JSON-safe dict (datetimes and enums already stringified) that matches exactly what gets stored, so the verifier can rebuild the identical byte string from the persisted record without a second, divergent serialization path.
Step 3 — Append a record by linking it to the chain head
Appending means reading the current head’s hash, allocating the next sequence number, computing this record’s hash, and writing it. The sequence and the prev_hash must come from the same head read to avoid a torn append under concurrency.
def append_record(new: AuditRecord, chain_head: AuditRecord | None) -> dict:
prev_hash = "GENESIS" if chain_head is None else compute_record_hash(chain_head)
expected_seq = 0 if chain_head is None else chain_head.sequence + 1
if new.sequence != expected_seq:
raise ValueError(f"sequence gap: expected {expected_seq}, got {new.sequence}")
linked = new.model_copy(update={"prev_hash": prev_hash})
return {**linked.model_dump(mode="json"),
"record_hash": compute_record_hash(linked)}
Asserting new.sequence == chain_head.sequence + 1 before writing turns a concurrent double-append into a loud failure instead of a silent fork in the chain, so two workers racing to extend the same property’s log cannot both succeed with the same sequence number.
Step 4 — Verify the chain and pinpoint the first break
The verifier walks records in sequence order, recomputing each hash from the previous record’s hash. It returns the sequence of the first record whose stored hash or prev-link does not match — the tamper location.
def verify_chain(records: list[dict]) -> int | None:
"""Return the sequence of the first broken record, or None if the chain is intact."""
prev_hash = "GENESIS"
for raw in sorted(records, key=lambda r: r["sequence"]):
stored_hash = raw["record_hash"]
rebuilt = AuditRecord(**{k: v for k, v in raw.items()
if k != "record_hash"})
if rebuilt.prev_hash != prev_hash:
return rebuilt.sequence # link to predecessor is broken
if compute_record_hash(rebuilt) != stored_hash:
return rebuilt.sequence # record body was edited
prev_hash = stored_hash
return None
Checking prev_hash continuity and recomputing the body hash catches two distinct attacks: editing a record’s content (body hash changes) and splicing a record out of the sequence (the link to the predecessor no longer matches), neither of which a single check alone would detect.
Gotchas & production notes
- A genesis anchor you can trust. The first record links to a fixed
"GENESIS"string, but nothing stops an attacker from truncating the log to a new, self-consistent genesis. Defend against wholesale truncation by periodically writing the current head hash to an independent, separately-controlled store (or the WORM object-lock sink) so a shortened chain is detectable against an external checkpoint. - Sequence allocation must be atomic per property. If two sync workers both read head sequence 41 and each write 42, one silently overwrites the other or forks the chain. Allocate the sequence from a single atomic source — a database sequence or a Redis
INCRkeyed byproperty_id— not from an application-side counter. - Never renumber to “fix” a gap. A crash between allocating a sequence and writing leaves a hole. The instinct to renumber later records to close it rewrites the very history the chain protects and breaks every downstream hash. Record the gap as a known operational incident and keep the sequence monotonic.
- Hash-chaining is tamper-evidence, not access control. The chain proves an edit happened; it does not prevent one. Pair it with immutable storage so an attacker cannot simply rewrite the whole chain and its external checkpoint together — the two mechanisms are complementary, and the schema alone is not sufficient.
Verification snippet
Confirm the two behaviors the whole design rests on: an intact chain verifies clean, and any edit is caught at the exact record where it happened.
from datetime import datetime, timezone
def _rec(seq: int, after: str, prev: str) -> AuditRecord:
return AuditRecord(sequence=seq, occurred_at=datetime.now(timezone.utc),
property_id="LON-STJ-01", actor="worker:pricing",
action="rate.published", subject="DLX_KING/BAR",
before=None, after=after, prev_hash=prev)
def test_intact_chain_verifies():
r0 = {**_rec(0, "180.00", "GENESIS").model_dump(mode="json"),
"record_hash": compute_record_hash(_rec(0, "180.00", "GENESIS"))}
h0 = r0["record_hash"]
r1 = {**_rec(1, "185.00", h0).model_dump(mode="json"),
"record_hash": compute_record_hash(_rec(1, "185.00", h0))}
assert verify_chain([r0, r1]) is None
def test_edit_is_caught_at_its_sequence():
r0 = {**_rec(0, "180.00", "GENESIS").model_dump(mode="json"),
"record_hash": compute_record_hash(_rec(0, "180.00", "GENESIS"))}
h0 = r0["record_hash"]
r1 = {**_rec(1, "185.00", h0).model_dump(mode="json"),
"record_hash": compute_record_hash(_rec(1, "185.00", h0))}
r1["after"] = "95.00" # tamper: edit the stored value in place
assert verify_chain([r0, r1]) == 1 # break detected exactly at sequence 1
test_intact_chain_verifies()
test_edit_is_caught_at_its_sequence()
The second test is the one that matters: it mutates a stored record’s after field without recomputing its hash — exactly what an in-place database edit would do — and asserts the verifier returns the tampered sequence rather than a vague failure. Run this verifier on a schedule and alert on any non-None result, so the chain’s promise is continuously exercised rather than trusted in the abstract.
Related
- Compliance & Audit Logging for Sync Pipelines — the parent workflow: the end-to-end immutable trail this record schema feeds.
- GDPR retention policies for reservation data — erasing guest fields on a lawful-basis timeline while keeping the chained financial record.
- Sanitizing PCI cardholder data in sync logs — stripping card data from an event before it is chained and sealed.
- Security & authentication boundaries — the scoped credentials that stop a writer from rewriting the chain.
- Data schema standardization — the canonical field shapes the before/after values are drawn from.