Accepting a Chinese Case Law Delivery: Ten Checks to Run Before You Sign the Acceptance Note

There is a short window between the moment a bulk delivery lands and the moment somebody signs the acceptance note. Inside that window, a defect is the supplier's problem by default. Outside it, the same defect is a support ticket, a workaround in your loader, and eventually a line in your own documentation explaining why a field cannot be trusted.

Most buyers spend their diligence effort before signature and almost none at delivery. That is backwards. Pre-signature work is done against a trial key on a curated slice; the drop is the whole corpus, and it is the first time anyone can measure the parts nobody demoed. The coverage verification protocol is what you run before you sign. This page is what you run after the files arrive and before you accept them.

Every check below is one we have run against our own index of 160,291,678 PRC judgment records. Every one of them found something. The findings are linked in place, with the dates they were measured, so you can see what a real result looks like rather than a checklist item.

Write the acceptance clause before the data moves

An acceptance protocol with no contractual hook is a report nobody has to act on. Four things need to exist in the agreement before the first byte is transferred, and all four are cheap to agree in advance and expensive to argue afterwards.

The unit, defined once. Whatever number appears in the commercial terms — records, documents, case numbers, cases — should be defined in the same paragraph it appears in. This is not pedantry, as the next section shows.

A manifest that travels with the delivery. Per batch: record count, date range, field list, and a checksum. Without it, every check on this page has to be run against a single undifferentiated pile, which is exactly the shape that hides defects.

An inspection window with a number of days in it. Ten to fifteen business days is workable for a bulk drop; anything shorter and the checks that need a parse over the full corpus cannot finish. The window should start when the data is readable, not when it is sent.

A definition of material defect and a remedy. Re-delivery is the usual remedy, and it should be the default; price adjustment is the fallback when a defect is inherent to the source rather than to the supplier's processing. That distinction matters more in this market than in most, because a great deal of what looks like supplier error in PRC court data is a property of how the documents were published. Say which is which in advance, and specify who keeps the artefacts produced during the inspection — the evaluation-artefact question is easier to settle before anybody has produced anything valuable.

The ten checks at a glance

Check 10 belongs first in execution order and last on the page, because it is the one that invalidates the other nine and the one nobody writes into a contract.

1. Reconcile the count to the unit you actually bought

Our corpus can be described in three ways, all of them true on the same day. It holds 160,291,678 records. Those resolve to 127,191,468 distinct case numbers. And they represent at most 130,700,854 distinct cases. The extremes are about 26 per cent apart. Nobody has to be lying for a slide that says "130 million cases" and an invoice that says "160 million records" to describe the same delivery.

The gap is structural rather than accidental. One dispute produces a first-instance judgment, often an appellate judgment, sometimes a retrial decision, and frequently several enforcement rulings. It is not evenly distributed: redundancy varies by case type, by court, and by year, which is why the year-by-year census reports both records and approximate distinct case numbers per year rather than one series.

The same problem appears inside a single extract. In one 500-row sample we counted 1,026 case-number occurrences resolving to 359 distinct case numbers — a row is not a case, and a loader that assumes it is will double-count some matters and silently merge others. That measurement sits here.

What to do at acceptance: compute all three numbers yourself, put them in the acceptance note side by side, and state which one the commercial terms refer to. If your internal reporting later quotes a different one, the reconciliation is already written down.

2. Reconcile per batch, never in total

This is the check that makes the other nine work. A corpus assembled from separate ingest runs does not degrade as a gradient; it partitions. Defects sit inside batch boundaries, which means a corpus-wide average is a weighted mixture of slices that are individually fine and slices that are individually unusable.

Our index has fifteen ingest batches. The empty-body rate across them runs from 100 per cent in one batch to about 23 per cent in two others, 3 per cent in another, and none at all in the 200 records sampled from a fifth. A single corpus-wide figure would have described none of them. The batch-provenance piece walks through how far this reaches: metadata patterns that look like litigation trends turn out to be schedules.

What to do at acceptance: require the batch manifest as a delivery artefact, then run checks 3 through 9 per batch and report the worst cell, not the mean. If the supplier cannot produce a batch breakdown, that is itself a finding: it means they are not measuring the thing that is most likely to be wrong.

3. Count the empty documents before you measure any field

The most expensive defect in this data type is invisible to completeness checking, because the field is present and well-formed. It is just empty.

Across a corpus-wide random sample of 1,500 records, 8.3 per cent had an empty body — roughly 13 million records — and 2.9 per cent had no title. Worse, one batch of 4,414,931 records contains no text at all: a case number, a court, a date, and nothing else. Queries inside that batch for 法院, 判决, 本院 and 原告 each return exactly zero, which no genuine batch of Chinese judgments can do. The full audit is here.

What to do at acceptance: run the four-term probe per batch. It takes minutes and it is unfakeable — a batch of real PRC judgments cannot return zero for all four. Then compute the empty-body and empty-title rate per batch and write both into the note. A delivery priced on record count where eight per cent of the records have no document in them is a conversation to have inside the inspection window.

4. Look for the storage cap, not the average length

Average document length tells you almost nothing. The distribution tells you everything, and what you are looking for is a spike at a round number.

When we first measured our index, in September 2026, 2.5 per cent of records had a body of exactly 6,000 characters. That was not a document; it was a cap applied at storage time, and it was backfilled later that month (re-measured on 24 September 2026: 0.003 per cent). It mattered far more than the percentage suggests, because a PRC judgment puts its operative part at the end. A truncated document reads as complete, parses as complete, and is missing precisely the passage anybody would want to extract.

The cost is measurable. When we computed dispositions for one category, 8,401 cases had to be excluded because the body was cut at 6,000 characters — alongside 4,358 with an empty or near-empty body. Those two lines belong in every denominator that gets published, which is why our own category work prints them in the table rather than in a footnote.

What to do at acceptance: histogram body length per batch. Any spike at a round figure is a cap. Then ask two questions in writing: what the cap is, and whether the untruncated text exists upstream and can be re-delivered.

5. Do not derive document type from the opening line

A PRC judgment and a PRC ruling are different instruments with different legal weight, and a pipeline that mixes them produces analysis that cannot be defended.

The tempting shortcut is to read the document's own first line, which usually names its type. In one of our batches that shortcut is wrong at scale: 19.3 million rulings open with text calling themselves judgments, measured on 18 September 2026. The title field is right; the prose is not. The audit that found it found it while looking for something else entirely, which is the usual way.

What to do at acceptance: ask which field in the delivery is authoritative for document type, then cross-tabulate that field against the text header per batch and report the disagreement rate. If the disagreement rate is high in any batch, your loader needs to know which field wins before it ingests anything.

6. Ask which date each date is

Every delivery of court data has a date field. Very few state which event it records. The candidates — filing, hearing, signature, publication, ingest — can differ by months, and a downstream time series built on the wrong one measures the supplier's schedule rather than the courts' activity.

In three of our fifteen batches — 19,833,947 records, 12.4 per cent of the corpus — the stored date is the date the document was published, not the date the judgment was signed. We found it by parsing the signature date printed at the foot of the document, on 150 records drawn at random from each batch, and comparing years. In a separate category study, one batch had 51.9 per cent of records signed a calendar year before their stored date. Both results are in the date-semantics audit and the shareholder-inspection piece.

What to do at acceptance: run the same test. Parse the signature date out of the document text for a sample per batch, compare it to the stored date, and report the share where the year differs. This is one of the few checks that uses the document to audit its own metadata, which makes it hard for any processing error to survive.

7. Test the identity key on the repeats, not on a sample of singletons

A de-duplication rule has to be agreed, not assumed, because the wrong one deletes documents you paid for.

The instinct is to collapse records sharing a case number and keep the longest. Whether that is right depends on the year. Within repeated case numbers in our index, the share of groups whose titles are identical ran 50.1 per cent on 18 May 2016, 43.0 per cent on 12 June 2019, 98.1 per cent on 27 September 2021 and 0.0 per cent on 11 April 2023. On the 2021 date the copies really are copies. On the 2023 date they are different documents that happen to share a number, and collapsing them destroys data. The survivorship analysis sets out why the rule cannot be uniform.

Case numbers themselves need normalisation before any of this works, and the normalisation is not trivial: the province code inside the case number changed meaning at the 2016 reform, and court names carry their own entity-resolution seams, including invisible bidirectional marks sitting inside the field.

What to do at acceptance: ask for the identity key specification in writing — which fields form the key, and exactly how case numbers are normalised. Then test it on groups of repeats per batch, not on a random sample of the corpus, because a random sample is mostly singletons and will tell you the key is fine.

8. Cross every category field with era before you trust it

Category and cause-of-action fields are the ones product teams reach for first, and they are among the least stable across a multi-decade corpus.

For PRC shareholder inspection suits, the cause label reaches 16,323 of 25,115 distinct case numbers — 65.0 per cent overall — but only 12.5 per cent of 2020 records, because two ingest batches carry no cause value at all. An overall figure of sixty-five per cent would have hidden a year that is effectively unlabelled. Separately, taxonomy drift means a case-type filter behaves partly as a date filter, and labels can arrive in the data two years after the vocabulary existed.

Procedural fields are worse. Our corpus carries 405 distinct procedure values across four encodings, which is a crosswalk you cannot finish so much as one you version and re-check. Field-level variation across the corpus generally is mapped in the metadata census.

What to do at acceptance: require a field-completeness matrix crossing each field with year band and batch, and check it against your own counts. Accept the matrix as a delivery artefact, not as a slide — it is the document your engineers will reference for years.

9. If you bought retrieval, accept the retrieval — not just the files

When the delivery includes an API or a hosted index, a file-level acceptance misses an entire failure class: the data is present, correct and complete, and the query returns nothing.

We found this in our own index and published it rather than fixing it quietly. Because the index was built with one Chinese analyzer and queried with another, an exact-phrase query returned zero results for nine of our forty largest causes of action and five of seven company names we tested — terms with millions of genuine matches behind them. The analyzer-asymmetry piece has the reproduction. A related trap sits in script handling: "we support both traditional and simplified" is two separate questions, one about storage and one about query expansion, and a vendor can honestly answer yes to one while failing the other.

What to do at acceptance: take twenty terms you already know exist in the corpus — ideally terms you verified during the pre-signature trial — and run them as exact phrases, as loose matches, and in both scripts. A zero on any of them is an acceptance finding even though nothing is missing from the data.

10. Audit the sampler before you believe checks 1 through 9

Almost everything above is measured on a sample, which makes the sampler the single point of failure for the whole protocol.

Ours failed. A seeded random scoring function applied to an internal sequence number turned out to be shard-local, so a nominally random 1,500-record sample drew from two of fifteen ingest batches and reported zero hits for a defect that was present in 19.3 million records. Nothing in the result looked wrong; the sample size was respectable, the seed was recorded, and the answer was still worthless. We re-ran the affected figures with a stratified sampler and published what changed and what had been right for the wrong reason.

What to do at acceptance: stratify by batch explicitly rather than trusting a random function to do it, record the seed and the sample size next to every number, and sanity-check that the batch composition of your sample matches the batch composition of the corpus. If it does not, the sampler is broken, and every figure you have produced this week needs re-running before it goes in the note.

What acceptance cannot prove

Four limits, stated plainly, because an acceptance note that overclaims is worse than none.

1. A clean drop says nothing about the next one. Corpora are re-extracted, backfilled and corrected between deliveries, and those movements are usually silent. The protection is contractual rather than analytical: a change-log obligation covering re-extractions, backfills and removals, with dates. What that log should contain is a short specification, and a supplier who already keeps one is showing you their instrumentation.
2. Quality and permission are separable. A delivery can pass every check here and still be unusable for the thing you intend, because granted uses, derivative artefacts and what survives termination are decided in the agreement. Embeddings and model weights after termination is the clause most often discovered late.
3. Per-batch cells get thin fast. Splitting a few thousand sampled records across fifteen batches and several year bands leaves very few per cell. Treat cell-level results as directional, and resist letting a per-cell percentage get quoted internally as though it were measured on the whole corpus.
4. These are our measurements, on our index, on the dates stated. They are not a claim about any other supplier's corpus, and they are not a census of published PRC judgments as a whole — nobody can state the size of that universe, so nobody can state a defensible share of it. What they are is evidence that each of these ten checks finds something real, including when the supplier being audited is us.

What the acceptance note should actually contain

Short, tabular, and signed by someone who ran the checks rather than someone who received the summary:

• The three counts — records, distinct case numbers, distinct cases — with the contractual unit named
• The batch manifest as received, with your own recount per batch beside it
• Per batch: empty-body rate, empty-title rate, truncation-spike share, document-type disagreement rate, date-semantics year-mismatch share
• The identity key specification as supplied, and your repeat-group test result against it
• The field-completeness matrix as supplied, and the cells where your count disagrees
• For a retrieval delivery: the twenty-term query result, including any zeros
• The sampler used, the seed, the sample size, and the batch composition of the sample
• A list of findings classified as material or immaterial, against the definition agreed before delivery

If the deal is still ahead of you rather than behind you, the upstream pieces are license versus scrape on build-or-buy, pilot scoping and exit criteria on what the trial has to prove, pricing structures and term-sheet clauses on what the numbers should look like, and bulk, API and sync delivery models on the shape of the drop itself. Downstream, preparing a licensed corpus for a training run and building the retrieval pipeline assume the checks on this page have already been run.