What Should a Data Change Log Contain? Re-Extractions, Backfills and Removals in a Chinese Case Law Corpus

“How often do you refresh?” is the question that gets asked in the technical call, and on its own it is close to useless. It describes the ingestion job: when new records arrive, and nothing at all about what happened to the records you already hold. In a corpus of Chinese court judgments, that second population is where the expensive surprises live.

The corpus you licensed in March is not a subset of the corpus you hold in September. Records were added, which everyone plans for. Records were also rewritten without the underlying case changing, arrived in bulk carrying publication dates a decade old, had their identity keys reassigned, and left the source entirely. A supplier who reports only the first of those is reporting a growth number and calling it change tracking.

The coverage verification protocol lists a change log as the fifth of five artefacts to request in writing before signature, and gives it one line: re-extractions, backfills and removals, each with a date. This page is that line expanded — what belongs in the log, what it cannot decide for you, and how to consume it without re-embedding your index every week.

The corpus is not append-only, and the source says so

Start above the vendor layer; the mutability starts there. The Supreme People's Court provisions on publishing judgment documents on the internet, in force since 1 October 2016, explicitly contemplate documents leaving and returning. Article 16 provides that a document published online which does not match the original, or which has been subject to improper processing, is to be withdrawn and republished after correction; a document found to fall within the categories that should not have been published is likewise withdrawn. Article 15 requires the published text to match the original apart from the processing the rules themselves require.

Two consequences follow, and both precede any question about a supplier's pipeline quality.

The same docket number can lawfully return different text at two points in time. Not because someone's parser improved — because the authoritative document was replaced upstream. A design that treats a case number as a pointer to immutable text is wrong at the source layer, before a vendor is involved.

A correction does not always look like a change. The same rules direct that where a court corrects a clerical error, the correcting ruling is itself published as a document. So a correction can arrive as a new record standing in a relationship to an old one, rather than as an edit to the old one. A change feed built purely on content diffs will faithfully record an addition and completely miss that it supersedes something you are still serving.

That second case is the shape of the whole problem: the changes that hurt are relational, and content diffing is blind to them.

Five classes of change, and why they cost different amounts

Before specifying a log, name what it must carry; lumping these together produces feeds that are both noisy and uninformative.

The right-hand column is the argument for the whole exercise: one class is cheap to absorb, two corrupt analysis quietly, one corrupts your vector store, and one is a compliance exposure that produces no error message.

The one question that reveals whether any of this is instrumented

You do not need to audit a supplier's architecture to find out whether they track change. Ask this instead: how many records changed content last month without the underlying case changing?

A supplier who maintains change events produces it in minutes. One who does not will answer with a description of the pipeline, an assurance about quality, or a cadence — all answers to a different question. And if they do not keep the number, you cannot recover it either: computing it yourself means retaining the full previous state of every record and diffing it, which is exactly the cost a change log exists to spare you.

The minimum viable change event

One event per change, one line per event, append-only. The field list below is the argument, not the syntax; no standard has been ratified for this.

{"event_id":      "ce_01J9F7...",              // unique, stable, never reused
 "event_type":    "add|revise|backfill|remove|relink",
 "record_id":     "sv_88f0c1...",              // supplier identity key
 "case_number":   "(2021)...民初12345号",       // public docket number
 "observed_at":   "2026-08-19T04:11:07Z",      // when WE saw it
 "effective_at":  "2026-08-11",                // when it happened upstream; null if unknown
 "fields_changed": ["full_text", "court_name"],
 "hash_before":   "sha256:1c9d...",            // over a defined canonical serialisation
 "hash_after":    "sha256:7a2e...",
 "reason":        "upstream_replacement",      // parser_v7 | dedup | out_of_scope | ...
 "batch_id":      "b_2026-08-19_parser_v7",    // groups one engineering action
 "supersedes":    null,                        // record_id, for correction chains
 "removal_class": null}                        // source_withdrawal | scope | duplicate

Two timestamps, never one. observed_at is a fact about the supplier's pipeline; effective_at is a fact about the world. Collapsing them into a single “date” destroys the only distinction that matters when you are working out whether a delay is the court's or the vendor's. effective_at must be nullable and honestly null, because for many changes the upstream timing genuinely is not observable — a supplier who populates it for every event is inferring, and you want to know that.

The changed-field list is what makes the feed affordable. A change to a court name is a metadata update; a change to the judgment text is a re-embedding, a different order of expense that belongs in a queue rather than a synchronous handler. Without fields_changed, every event is priced at the most expensive tier it might be, and a feed that forces a re-embed on every metadata correction gets switched off within a quarter. That is how buyers end up with stale indexes despite having a feed.

Hashes make the event verifiable rather than asserted. hash_before lets a consumer check that the state the supplier believes you held is the state you actually hold; divergence is a sync defect you want to find on an ordinary Tuesday rather than during an audit. It only works if both sides hash over a defined canonical serialisation — field order, whitespace, null handling and encoding fixed in writing — otherwise both compute honest hashes that never agree.

The batch identifier separates the world from the pipeline. Four hundred thousand revision events sharing one batch_id and the reason parser_v7 are one engineering decision, not four hundred thousand legal developments. Without it, a routine re-extraction reads as a catastrophic month, and a consumer with any alerting will either page someone or learn to ignore the feed.

supersedes carries the relationship content diffing cannot see — the corrected ruling published as its own document, the merged duplicate, the record re-keyed under a new identity. It determines whether your appeal chains survive.

Removals must be positive records, and they must outlive the record

An absence is not a signal. If a record simply stops appearing, you cannot distinguish removal from a missed sync, a re-keyed identifier, or a query that differed subtly from last month's. A feed carrying only additions and revisions is, silently, a feed of additions — discovered the first time someone asks why a document that no longer exists upstream is still being cited in the product's output.

Two properties are non-negotiable. The removal must be an explicit event with its own identifier and timestamps. And it must be retained after the record itself is gone — the one class of event where retention has to survive the thing it describes, because the question it answers, why do we no longer hold this and on whose instruction, is asked months later, and a tombstone garbage-collected along with its record cannot answer it.

removal_class is not bookkeeping. Withdrawal at source is a fact about the world and may carry obligations that run through your whole stack. Falling outside licensed scope is a fact about your arrangement; the record still exists and may be licensable on other terms. A duplicate merged into another record has not gone anywhere at all, and treating it as a deletion will silently reduce your coverage while looking like compliance. Three causes, three different correct responses, one word of difference in the payload.

What you owe after a removal is a contract question — and one layer cannot comply

The log tells you a record left. It cannot tell you what to do about the copies you already made, and after a judgment has been in a production system for a year there are more of those than the deletion clause imagines: the primary store, the search index, the embedding store, cached snippets and summaries, query logs and traces that quote it, the evaluation sets your team built, and — if you trained — model weights.

The honest part, which suppliers and buyers both tend to skate over: you cannot un-train a model on a document. A clause requiring deletion from “all copies and derivative works” is therefore either unmeetable or is being read by both parties as not meaning what it says, and the second is worse, because it is the version that surfaces in a dispute.

The workable form separates the layers explicitly. Deletion at the record, index, embedding and cache layers within a stated window. Suppression at retrieval and display, so a withdrawn record is neither returned nor rendered regardless of what remains in storage. Pruning of evaluation artefacts, which is easy if those artefacts store pointers rather than payloads. And a written acknowledgement of what is irreversible in trained parameters, expressed as a control on output rather than a promise about weights. That belongs in the agreement rather than the feed spec; the term-sheet piece covers where it sits among the clauses that cost money.

Consuming the feed without rebuilding your index every week

A well-specified log is still unusable if it is delivered badly. Four requirements, all cheap to agree before signature and awkward afterwards.

• A cursor, not a date range. “Give me everything since Tuesday” is ambiguous across time zones, breaks on events observed out of order, and silently drops anything that arrived late. An opaque cursor over a total order of (observed_at, event_id) lets a consumer that crashed halfway resume exactly where it stopped.
• Idempotency by event_id. Replaying yesterday must be a no-op. Every consumer replays eventually — after an outage, a migration, a bug — and a feed that cannot be replayed safely forces a full resync each time.
• Priced tiers before action. Map fields_changed onto what each change actually costs you: metadata updates applied inline, text changes queued for batched re-embedding at off-peak, removals executed immediately because they are the only class with a deadline attached. The retrieval pipeline piece covers where those stages sit.
• Periodic reconciliation anyway. No change feed is complete, including a good one, and the classes hardest to detect are the ones nobody instrumented. Quarterly, take a stratum you care about — one court level, one era, one case type — and compare counts and hashes against what you hold. The log reduces how often you must reconcile in full; it does not remove the need. A supplier who says the feed is provably complete has never diffed it against their own store.

Five questions to ask before you sign

1. How many records changed content last month without the underlying case changing?
2. Is a removal a positive event in the feed, is it classified by cause, and how long is it retained after the record is gone?
3. Does an event state which fields changed, or only that something did?
4. Do you distinguish when you observed a change from when it took effect upstream, and what do you do when the second is unknown?
5. When a correction arrives as a new document rather than an edit, does anything in the feed link it to what it corrects?

The answers matter less than their form. A supplier who returns tables and field definitions is showing you instrumentation that exists — the strongest signal available while you are outside a contract. A supplier who returns prose is telling you the events are not being kept, and by extension that whatever you measure during a pilot is a snapshot you will have no way to re-verify once the pilot ends.

What this page does not settle

We sell this data, so declare the interest plainly: every requirement above is an obligation on a supplier, and recommending that you demand it is against our own convenience. That is deliberate.

This page does not claim that any particular supplier, ourselves included, currently publishes a feed in the shape specified above. It is a specification argued from what breaks without it, not a product description. Nor does it settle the legal position: the publication rules cited govern how courts publish and withdraw, and say nothing about what a private licence obliges a buyer to do downstream. Finally, the design is opinionated about cost: a team holding a corpus small enough to re-embed in an afternoon can ignore field-level pricing and simply reprocess everything. The argument is for the case where that is not an option, which is what a corpus at this scale puts you in.