The Two Fields Nobody Audits: A Date That Is Not the Decision Date, and 4.4 Million Records With No Text

Due diligence on a case law corpus concentrates on the fields that are obviously fragile. Buyers ask about the cause-of-action label, the court name, the case type, the de-duplication rule — and they are right to, because we have written about how each of those breaks. Almost nobody asks about the two fields that look unimpeachable: the date, and the text itself. A date is a date. A judgment either is in the database or it is not.

We audited ours on 16 September 2026 and found that neither holds. In three of fifteen ingest batches — 19,833,947 records, 12.4 per cent of the corpus — the stored date is the date the document was published, not the date the judgment was signed. And one batch of 4,414,931 records contains no text at all: a case number, a court, a date, and an empty document. Both defects are invisible to every completeness check we have published, because in both cases the field is present and well-formed. It is just answering a different question than the one you asked.

The first defect made a page on this site wrong. Our year-by-year census states that the year field is the date attached to the decision. For most of the corpus that is true. For the row it labels 2024 it is not: every one of those 6,984,435 records comes from a single batch, and only 53 per cent of them were signed in 2024. A correction is now on that page. This is the audit behind it.

What was measured, and how to re-run it

The index holds 160,291,678 records across fifteen ingest batches, each tagged with a dataset value. 8,369,010 records (5.22 per cent) carry no year and are excluded from every year figure here. Three things were measured.

One. Whether the year field is anything other than the year of the date field. We drew 200 records at random from each of fourteen batches; 2,760 of the 2,800 had both fields populated, and in all 2,760 the year was exactly the year of the date. The year field carries no independent information. Whatever is wrong with date is wrong with year.

Two. Whether the stored date is the date on the judgment. PRC judgments print their signature date at the foot of the document, above the bench. We drew 150 records at random from each batch, parsed that date out of the text, and compared years. Where the batch had no text, the test could not run.

Three. Whether the document is there at all: a single corpus-wide random sample of 1,500 records, plus exact whole-batch counts.

GET /judgments/_search
{ "size": 150, "_source": ["year", "date", "body"],
  "query": { "function_score": {
    "query": { "term": { "dataset": "2024" } },
    "random_score": { "seed": 20260916, "field": "_seq_no" } } } }

Every sample on this page uses seed 20260916, so the same rows come back. Set track_total_hits explicitly on any count you run; left at its default the engine stops at ten thousand and reports that as the answer.

Ten batches agree. Three do not.

The result is not a gradient. It is a partition.

Ten batches, 136,035,800 records, land between 95.2 and 100 per cent. Three batches land between 52.3 and 65.8. There is nothing in between. In a separate corpus-wide sample of 1,500 records, 1,234 had a parseable signature date and 1,132 of those matched the stored year: 8.3 per cent of the records whose signature date we could read sit in the wrong year, and 101 of the 102 disagreements ran the same direction — the stored date is later than the signature, usually by one year.

The direction is the whole argument. Parser noise is symmetric; a publication lag is not. Below are three records drawn from a fresh random sample of each suspect batch, taken from among those where the two dates disagree. Their case numbers corroborate the signature date without relying on our parser: a PRC case number carries the year the case was docketed, and a case docketed in 2023 was not decided in 2022.

What it does to the histogram we published

A defect confined to three batches would be a footnote if those batches were spread evenly across the time axis. They are not. Batches are era-shaped — a point we made about the cause-of-action field and which turns out to be true of the date field too. The three suspect batches supply 31.3 per cent of the 2021 row, 31.9 per cent of 2022, 23.4 per cent of 2023 and 100 per cent of 2024.

So we re-bucketed them. For each batch we took the empirical distribution of signature years within each stored year and applied it to that batch's exact record counts. The three largest cells hold 395, 267 and 245 sampled records; the smallest, 2021_x stored as 2023, holds four, and contributes 77,442 records. Treat the restated column as an estimate with a real error bar, not a measurement.

About 7.8 million records are in the wrong row. The 2024 row loses 47 per cent of its volume. The 2021 row gains 2.6 million, almost entirely from 2021_x records stored as 2022 — 98.5 per cent of that cell was signed in 2021.

The headline of the census survives, smaller. The fall from 2022 to 2023 was published at 67.0 per cent on raw records. Restated by signature date it is 59.3 per cent. The share of the corpus in the last three years falls from 10.7 to 9.4 per cent. That page also reported a separate correction on a different axis — 54.5 per cent once duplicate case numbers are collapsed — and the two corrections are independent of each other. We have not combined them, and we would not quote a combined figure without measuring it.

One caution about our own restatement: we redistributed only the three suspect batches. The other ten still carry roughly three to five per cent of records dated late, about 3.8 million in total, spread thin. The restated column therefore still leans slightly late. It is closer to the truth than the published column, not equal to it.

The cheap version of the test: look at the day of the month

Parsing signature dates out of Chinese judgment text is an afternoon of work. There is a version of the test that costs one query and no parsing, and it flags all three batches.

Signature dates fall on working days, roughly evenly across a month; about 16 per cent of them should land on the 27th or later. Publication dates cluster, because documents are released in lots.

About 600 records sampled per batch. The month-of-year test is the one people reach for first, and it is the weaker of the two: it catches 2021_x and 2022_x loudly and misses 2024 entirely. The day-of-month test catches all three. If you run one diagnostic on a supplier's date field before you run anything else, run that one.

The other field nobody audits: the text

Our metadata census measured six categorical fields and did not measure the document. So we measured it.

Batch 2019_2 — 4,414,931 records, 2.75 per cent of the corpus — has no text whatsoever. This is not a sample. A full-text query for 法院 (court) across that batch returns exactly 0 records. So does 判决 (judgment), 本院 (this court) and 原告 (plaintiff). All 4,414,931 carry a case number; 200 of 200 sampled have a zero-length body and a zero-length title. They are real case references with the document missing.

Across a corpus-wide random sample of 1,500 records, 8.3 per cent had an empty body — roughly 13 million records — and 2.9 per cent had no title. Batch by batch the empty rate is again a partition rather than a gradient: 100 per cent in 2019_2, about 23 per cent in 2019_1 and 2021, 3 per cent in ws_new, and none at all in the 200 records sampled from ws.

When this audit was run, a separate 2.5 per cent of records had a body of exactly 6,000 characters — a storage cap, not a document; the cap was backfilled in September 2026, and re-measured on 24 September 2026 it affected 0.003 per cent of the corpus. That mattered most for anything parsed from the end of a judgment, which is where the operative part sits; we measured what it costs when computing an outcome rate and it removed the operative part of more than one judgment in eight.

None of this shows up in a null check. The field exists, it is a string, it is the right type. It is empty.

Why these are batch-shaped, and what that means when you buy

Every defect we have published on this corpus has the same shape. The case-type filter, the cause label, the court name, the date semantics, the presence of text: the unit of failure is the ingest batch, not the record. Batches were acquired at different times from different upstream shapes, and each carries its own conventions in whole.

Two consequences follow, and they are the reason this page exists rather than a patch note.

First, a random sample of a corpus will not find these defects, but it will average them away. Draw a thousand records from the whole index and about eight per cent of the readable ones will carry a wrong year while about eight per cent have no text at all — two unrelated defects that both read as ordinary noise. Stratify by batch and the same data says three batches are systematically wrong and eleven are fine. That is a different finding and it implies a different fix.

Second, defects correlate with the time axis, because batches do. So the error is never where the arithmetic says it should be. An eight per cent date error spread evenly would be a rounding note. Concentrated as it actually is, it removes 47 per cent of one year's records.

If you are evaluating any supplier of Chinese case law, including us, ask which ingest batches make up the corpus, get a per-batch record count, and run every quality metric per batch. A supplier who cannot tell you how the corpus was assembled cannot tell you where it is wrong, and the aggregate number they give you will be a weighted average of populations that do not belong in the same average. Our coverage verification protocol sets out the rest of the checks.

Five checks worth an afternoon

1. Is the date field the decision date? Sample 150 records per batch, parse the signature date from the foot of the text, compare years. If a batch lands below 90 per cent agreement, it is storing something else.
2. The one-query version. Histogram the day of the month. Above roughly 20 per cent falling on the 27th or later means the dates were assigned in lots.
3. Is the document there? Run a full-text query for a term that appears in every judgment — 法院 works — and compare the hit count to the record count, per batch. A batch that returns zero is not a coverage gap you will find any other way.
4. Is the document whole? Count records whose body length is exactly the same number. If a specific length recurs, that is a cap, and everything near the end of the document is gone.
5. Do it per batch. Every check above, grouped by ingest batch. The corpus-level answer will hide all of it.

What this audit does not tell you

• We tested no other supplier. Every defect described here is our own, and nothing on this page is a comparison.
• Signature dates were parsed by rule from the last date in the tail of each document. 17.7 per cent of the corpus-wide sample had no parseable signature date, and those records are excluded rather than assumed correct. Records with no text are excluded by definition, which means 2019_2’s 4.4 million contribute nothing to the date finding.
• The restated year table is an estimate built from per-cell samples, one of which has four observations. It is not a re-measurement of 160 million records.
• We did not establish why three batches carry publication dates. The pattern is consistent with a bulk release date applied at ingest, but we did not reach the upstream source to confirm it.
• We have fixed nothing. No backfill, no second date field, no re-ingest. The corpus today is exactly the corpus described here, which is also why the numbers reproduce.
• Nothing here says whether our coverage is good. It says what the coverage numbers mean. Those are different questions, and the second one has to be settled first.