Your Case-Type Filter Is a Date Filter

Yesterday we tried to write a crosswalk for the procedure field of our own PRC judgment corpus and concluded that it cannot be finished: 405 values in four incompatible encodings, a fifth of the records covered by values that resolve to nothing. We also said the neighbouring field, caseType, was the easy one — it agreed with the case number on 96 to 99 per cent of records, so its problem was “only” vocabulary, and vocabulary can be mapped.

That was true and it was not the point. We wrote the crosswalk today. It fits on one screen, which is the good news. The bad news is what fell out while writing it: which vocabulary a record uses is decided almost entirely by the year of the judgment. The field is not a mixture, it is a stratigraphy. And that means a case-type filter, the single most common filter anyone applies to case law, quietly doubles as a date filter. You ask for civil cases and you get a decade.

Thirty-two values, and they sort themselves

The field holds 32 distinct values. Thirty of them fall cleanly into three vocabularies plus an empty string, and — this is the difference from the procedure field — no string appears in two of them. The vocabularies are mutually exclusive as text, so every record announces which word-list it is speaking simply by the value it carries. You do not need provenance to disambiguate. That is the entire reason this crosswalk is writable at all.

An independent aggregation over every matching record put the same four shares within 0.6 percentage points of the sample, so the sample is doing its job. The third row is the one to sit with: close to a quarter of the corpus stores, in the field called case type, a value that is not a type of case. It is the type of document — judgment, ruling, mediation agreement, notice, decision. Those are orthogonal axes. A ruling can be civil or criminal; a criminal case produces both judgments and rulings.

The last row is two records: the registry class names for civil and enforcement cases, each with its first character replaced by a Unicode replacement character. One record apiece. They are worthless as data and useful as evidence — they prove a lossy transcoding step exists somewhere upstream, on a path that also carries 38 million intact copies of the same two strings. A further 7,002 records carry no caseType key at all; they belong to a small benchmark slice that shares the index.

The stratigraphy

Here is the finding that reorganised the rest of the day. For each judgment year, the share of that year’s own records carrying each vocabulary:

Each row’s denominator is that year’s own record count, not the corpus. Read down the columns and the field stops looking like a merge accident and starts looking like a chronology: document types until roughly 2017, short header classes across 2018 and 2019, registry classes from 2020, and then — the part nobody would guess — a reversion to the short vocabulary in 2025. The taxonomy did not improve monotonically. It regressed.

The 2024 row hides a clean edge. Within the single ingest batch covering that year, the registry vocabulary’s last dated record is 31 October 2024 and it appears on nothing after that date. Of the records dated in 2024 that carry the short vocabulary, 41 are dated before 1 November and 490,404 fall in November and December. This is not a gradient. It is a switch thrown on a date, inside one batch, on records the pipeline treated as homogeneous.

We chased two wrong explanations before that one. The province mix of the two slices differs sharply — one province is 23.4 per cent short-vocabulary, another 0.6 per cent — which looked like a per-province feed until we noticed both vocabularies co-occur in 33 of 35 provinces. The court mix differs too, but 120 courts in a single province appear on both sides. Only the date separates them without exception. The province and court asymmetries are downstream of it: courts publish at different rates, so a cut-off date reshuffles the geography. It is a good reminder that an ingest artefact can impersonate a geographic pattern.

What that does to a filter

The consequence is not subtle and it does not announce itself. In our corpus the registry term for civil cases is carried by 38,775,178 records and the short term for civil by 26,038,604. A query for the first returns almost nothing before 2020; a query for the second returns almost nothing between 2020 and October 2024. Neither returns anything from 2008 to 2017, where 39,314,528 records answer the case-type question with a document type instead.

So the request “civil judgments, 2008 to 2025” has no expression as a case-type filter. Any single term is an era. The union of both civil terms is still missing an era. The measurement, on the same 10,000-record random sample: 40.32 per cent of records carry one of the two civil labels, while 57.01 per cent resolve to the civil family when the case number on the same record is read instead. The gap of nearly 17 points is not error; it is the part of the corpus that spells its answer in a vocabulary the filter did not ask for.

If you evaluate a corpus by running category filters and comparing counts to public statistics, this defect will read as a coverage gap. It is not. The judgments are there, indexed and retrievable by every other means. Only the label is era-specific — which is why a coverage test has to be run per year, never as a single lifetime count.

The crosswalk, and its four bad cells

Now the table we came to write. Each registry class was tested against a reference the record carries internally: the case-family code inside its own case number. Composition figures come from random samples of up to 1,500 records per class (three seeds, de-duplicated), or from the entire class where it holds fewer.

Six registry classes have no short-vocabulary counterpart at all, covering 1,009,605 records; a seventh, “other case”, has a counterpart in name only. Translating registry to short is therefore lossy in a knowable, countable way: those records have to go somewhere, and the only somewhere is a residual bucket. Translating short to registry is worse, because the information to split them was never stored. That asymmetry is the crosswalk’s real content. Everything else is a rename.

The name collision is the expensive cell

One pair of classes shares a word in both vocabularies, and it is the pair a careless mapping would join first. On a random sample of 4,954 records, the short value glossed “compensation” is 68.61 per cent ordinary civil damages claims — civil first-instance and civil appeal codes — with 12.35 per cent administrative-compensation, 5.29 per cent enforcement and only 4.00 per cent carrying a state-compensation code. On a random sample of 4,701 records, the registry class whose name also contains that word is 47.88 per cent administrative state-compensation dockets, 33.08 per cent enforcement — judicial-relief decisions issued under enforcement case numbers — and 10.25 per cent state-compensation codes.

They are not the same class. They are barely overlapping classes that share a morpheme. Joining them on the name moves millions of ordinary tort and contract damages records into a state-liability bucket, and the join will look successful because both sides are populated.

Two residual buckets, neither residual, and not the same

Both vocabularies have an “other”. Neither behaves like one. The short vocabulary’s other is carried by 5,888,436 records; on a 4,998-record random sample it is 58.20 per cent ordinary civil, 18.99 per cent enforcement and 3.34 per cent criminal, with 16.05 per cent carrying no case number to test. The registry vocabulary’s other case is carried by 36,713 records and is 77.2 per cent enforcement. One is a missing-value marker wearing a category’s clothes; the other is a dumping ground for enforcement dockets that did not fit the registry’s own enforcement class. Mapping one “other” to the other “other” would be the most natural row in the crosswalk and one of the most wrong.

The classes do not nest

Even where names line up, populations do not. The registry vocabulary holds preservation review and jurisdiction as top-level classes, so its civil class excludes them. The short vocabulary has no such classes, so its civil value absorbs them: on a 4,999-record sample, 2.28 per cent of short-civil records carry preservation codes and a further fraction carry jurisdiction codes. The registry class is not innocent either — 1.58 per cent of registry-civil records carry preservation codes despite a dedicated class existing for exactly those. A count of civil cases is therefore not comparable across the 2019/2020 boundary even after the vocabularies are mapped, because the boundary of the concept moved as well as its name.

The fallback, and where it is thinnest

The remedy is the one we reached yesterday for procedure: derive from the case number, which carries a case-family code directly. Reading it gave a family on 95.29 per cent of the 10,000-record random sample. The honest part is the distribution of the failures.

Roughly 3,000 records per era, three seeds, de-duplicated; the disagreement column is computed only over records where both routes fire. The fallback is weakest exactly where the stored field is weakest. In the oldest era the stored value names a case family on one record in ten, and the case number rescues only three-quarters of the rest, leaving a quarter of that era with no usable case family from either route. There is no free lunch hiding in the case number; there is a good lunch for 2014 onward and a thin one before.

Some of that thinness is our parser rather than the data. Inspecting the failures showed two repeating pre-2016 shapes: a commercial-division code that the 2016 reform later folded into civil, and codes interrupted by a parenthetical division marker, as in a number that writes the civil-first-instance code with a bracketed sub-division inserted mid-code. Two rules — strip parenthetical inserts inside the code, treat the commercial and IP codes as civil family — moved 2008–2013 coverage from 72.42 to 98.20 per cent, a gain of 25.78 points, and whole-corpus coverage from 95.29 to 97.41. Of 773 records gained in the oldest era, 770 were civil. The parser is an asset you maintain, not a problem you solve; two rules bought 26 points in one era and under two corpus-wide, and the next two rules will buy less.

What we would hand a buyer

1. Never filter case type without a date range, and never compare counts across one. If a category filter must span eras, it has to be a union of per-era terms, and the union has to be written down.
2. Derive the case family from the case number; use the stored value as a tiebreak. Where both fire they disagree on 0.21 to 4.26 per cent depending on era, and the disagreement rate is itself a per-era quality signal worth publishing.
3. Never join two classes on a shared name. Show the measured composition of both first. The one pair in our field that shares a name overlaps on roughly four per cent of one side.
4. Treat every “other” as missing until a random sample proves otherwise, and publish what the sample found. Two “other” buckets in the same field were 58 per cent civil and 77 per cent enforcement respectively.
5. Publish the vocabulary-by-year table itself. It is four aggregations and it is the single artefact that makes era bias visible before a contract rather than after a model is trained.

What we are not claiming

We measured no other provider’s data, so nothing here compares vendors. The case number is a strong reference but not an oracle: it records the programme the issuing court assigned, and a court that registers a case wrongly is copied faithfully by our validator. Composition figures are sample estimates with the sample size stated on each; classes near a boundary will move a little under other seeds. Nothing has been repaired — the vocabularies are not normalised, the two mojibake records are not deleted, and the “other” buckets are not reclassified, because changing stored values silently breaks every downstream join written against the current ones. Any of it is fixable; all of it needs a change log and notice first. And the 2025 reversion to the short vocabulary is described here, not explained: we know the date it happened and not the reason.