The Litigation Trend That Was an Ingest Schedule

Yesterday we found that one Chinese intermediate court sits in our own facets under two different names, and that which name a record carries is decided by which ingest generation loaded it. We closed with an obvious question and left it open: if the batch decides the court’s name, what else does the batch decide?

Today we asked the index directly, batch by batch, field by field, with exact counts rather than a sample. The answer is worse than a naming quirk. For the fields buyers actually filter on, completeness is close to binary at the batch level — a batch either populates a field on nearly every record or on none of them — and because batches are era-shaped, that binary turns into a curve when you plot anything by year. The curve looks like a story about Chinese litigation. It is a picture of our loading history.

The field is never missing. It is present and empty.

Start with the least dramatic finding, because it is the one that defeats the usual test. Our records carry a cause-of-action field. Across fourteen of our fifteen ingest batches, that field is absent on 0.0 per cent of records — it is there, on every single record. A schema conformance check therefore returns full compliance, and a vendor could truthfully say the field is present corpus-wide.

Now ask whether it has anything in it.

Read the cause-of-action column as a distribution and it has no middle. Twelve batches sit between 84 and 100 per cent. Two sit at exactly zero, and they are the largest and the third largest batches we hold. There is nothing in between. The same shape appears in the province column, where two legacy batches carry the field at 0.0 per cent because the upstream schema for that generation simply had no province concept, and in the procedure column.

This is what a merged corpus looks like from the inside. Completeness is not a quality level that the corpus has; it is a mixing ratio between batches that each have their own, near-absolute answer. Any single number you are quoted — ours or anyone’s — is that mixing ratio evaluated over whatever population the person quoting it happened to measure.

Batches are era-shaped, so completeness becomes a function of year

If batches were scattered randomly across judgment dates, the mixing ratio would be constant and the averages would be honest. They are not. Each batch was loaded from a source that covered a period, so the batch mix changes as you move along the time axis, and the completeness of every field moves with it.

Note what this is not. It is not an ageing curve: 2013 scores higher than 2018 on every column, and the years 2000 to 2012 sit between 78 and 98 per cent on cause of action. The damage is in the middle of the corpus, not at the old end. Nor is it monotonic in recency — province coverage is 95.3 per cent in 2022 and 73.9 per cent in 2024, because a recent batch happens to be thinner on that one field.

The mechanism is visible if you break a single year down by the batches that supply it. For 2018, one batch supplies 75.0 per cent of the year and populates cause of action on 0.0 per cent of its records; the batches supplying the next 12.9 and 7.4 per cent populate it on 83.3 and 93.6 per cent. For 2020, one batch supplies 88.7 per cent of the year at 0.0 per cent cause coverage, while the batch supplying the next 8.9 per cent sits at 93.7 per cent. Same year, same courts, same kind of document. The only thing that differs is which load it arrived in.

What this does to a buyer’s first chart

The first thing almost every evaluator does with a corpus is count something by year. So we did that, the way a buyer would: filter on a common cause of action — labour and employment disputes — and normalise per 10,000 documents in that year, so that the changing size of each year cancels out. Then we built a control over the same documents that does not read the cause field at all: a phrase search in the judgment text.

The filtered series falls 84 per cent from 2015 to 2020 and then rises nine-fold to 2023. Peak to trough it moves by a factor of nine. The control moves by a factor of 1.8 across the same years, and its dip is both shallower and differently shaped. An analyst who saw only the first column would write a paragraph about a collapse in labour litigation and a post-2021 surge, and every word of it would be about our loading schedule.

Be careful about what the control proves. A phrase in a judgment’s text is not a reliable indicator that the case is of that type — documents mention dispute categories they are not about. The control is not a better estimate of litigation volume, and we are not offering it as one. Its only job is to be a measurement of the same documents that does not depend on the field under suspicion. When two such measurements disagree by a factor of five in their dynamics, the disagreement is information, and here it lines up exactly with the batch mix in the table above.

The same judgment, more than once, in more than one vocabulary

There is a second mechanism underneath the first, and it is the one that makes the first hard to repair. To measure it without sampling error we picked four calendar dates and enumerated every document on each of them, one by one, then grouped by case number. Enumerated counts matched the index total on all four days.

Two of these numbers deserve emphasis. The first is the spread: 0.6 per cent on one day and 16.3 per cent on another. Redundancy in a merged corpus is not a constant you can subtract; like the completeness rates above, it is a function of which batches overlap on the slice you are looking at. Quote it as a single corpus figure and you will be wrong on most slices in both directions. The second is that these counts are conservative, because we grouped on the exact case-number string; normalising bracket forms and whitespace raised the 2016 figure from 9.8 to 10.2 per cent and the 2023 figure from 0.6 to 0.9 per cent. Repetition also happens within a single batch: documents sharing both a batch label and a case number account for 1.89, 3.57, 1.26 and 0.50 per cent of the documents carrying a case number on those four days, so the cross-batch merge is not the only source.

Now the part that matters more than the row count. The copies do not agree. Taking every case number that appeared more than once on 18 May 2016:

Almost none of these are contradictions about the case. They are different vocabularies for the same fact. In the case-type column, one batch stores the document type in that position — judgment, ruling — while another stores the case type: civil, criminal, enforcement. In the procedure column, one batch stores numeric codes and another stores labels, so the same first-instance case reads as 1 in one copy and as the words for first instance in the other. In the province column the disagreement is an administrative suffix present or absent. In the court column it is yesterday’s finding, now visible inside a single case rather than across a facet: the province-prefixed official style in one copy and the common short style in the other.

A representative group, one case number, four copies across two batches: the older batch carries a specific cause of action, the province name with its suffix, a numeric procedure code and the document type in the case-type position, and names the court without its province prefix. Three copies from the newer batch carry a blank cause of action, the province name without its suffix, a worded procedure, the case type in the case-type position, and the court name with its province prefix — and one of those three has the province and procedure blank as well, while its two siblings do not.

So de-duplication is not free. If you drop repeats by keeping an arbitrary copy, on 27 September 2021 94.0 per cent of repeated case numbers had one copy with a cause of action and one without, and the expected outcome is a blank cause on 47.0 per cent of them. On 18 May 2016 those figures are 74.4 and 38.4 per cent. Keep the wrong copy and you have not removed a duplicate row; you have destroyed a field. Keep the right copy and you need a rule that knows which vocabulary you are standardising on — that rule is the actual deliverable, and it is not in any record.

The 7,000 documents no filter can reach

One last item, small, exact and complete. One of our fifteen batches holds 7,000 documents whose cause, court, province, procedure, case type and year fields are all absent — not empty, absent, the only place in the corpus where that is true. They carry a title, a case number, a date and full text. They are real judgments: in a 300-document check, 271 named a court or tribunal within the first eighty characters of their text, just not in the court field.

We checked whether they are new material by taking the upstream record identifier of each and looking for it under every other batch label. All 7,000 are re-labelled copies of records that already sit in another batch — not a sample, all of them — and the twins carry the metadata these copies lack: court on 7,000, case type on 7,000, year on 7,000, procedure on 6,998, cause on 6,796, province on 5,996.

The consequence is precise. These documents are reachable by full-text search and unreachable by every structured filter. They inflate any raw document count. They can never appear in a count by court, by province, by year or by cause, so they will never make a facet look wrong. A buyer running coverage tests through filters would never see them; a buyer running retrieval evaluations would hit them and score them as unlabelled noise.

Reproduce any of this on us in one request

Every claim about a specific case above is checkable from outside, without a key and without asking us. Our public search endpoint accepts an exact case number and returns every copy we hold, each stamped with the batch it came from:

GET https://www.wenshucha.com/wscx2/search?caseNo=<case number>

Returns one record per copy, each with: dataset, court,
cause, province, procedure, case_type, doc_id

Take any case number from any judgment you already have, ask for it, and count the rows. If more than one comes back, put the fields side by side and see whether they agree. That is the entire test, it takes one request, and it works the same way on any vendor who will let you look up a case number and see a provenance stamp. If they will not tell you which batch a record came from, you cannot run it — and that is itself the answer to a different question.

Four checks worth minutes of your time

1. Ask for completeness by batch and by year, never as one number. A corpus-wide average is a mixing ratio. Ours would tell you the cause field is roughly half populated, and there is no query for which that is the right expectation: within a batch it is 0 per cent or above 84 per cent, and within a year it ranges from 10.7 to 97.1 per cent.
2. Build every count twice, once through the filter and once through a control that does not read that field. Compare shapes, not levels. Divergence in dynamics is the signature of a metadata artefact, and it is cheap to detect long before you can explain it.
3. Ask whether a judgment can appear twice, what the de-duplication key is, and which copy wins. Then ask for the disagreement rate between copies on the fields you care about. A vendor who has never computed it has not de-duplicated; they have deleted rows.
4. Ask whether an empty field is stored as empty or as absent. Schema validation cannot tell the difference and will report full compliance either way. This one question separates “the field exists” from “the field is usable”.

What we did not do

We fixed nothing today. Normalising vocabularies across batches, and choosing a surviving copy for each duplicate, both change values that anyone joining on the current data already depends on; that belongs in a change log with notice, not in a quiet edit, and it needs a decision about which vocabulary wins rather than a script. We did not measure any other vendor’s corpus and make no comparison. The four enumerated dates were chosen to span eras, not sampled, so their duplicate rates are four worked examples and not an annual estimate. The text control is a control, not a truth: it does not measure how much labour litigation occurred, and we have not claimed it does. And every rate here is a rate within the slice named beside it.

We publish this for the same reason as yesterday’s finding. Every corpus assembled from more than one source generation carries these seams, almost none of them are visible in a single record, and the tests that buyers habitually run — schema validation, spot-checking documents, eyeballing a facet — are all structurally blind to them. The chart is the giveaway. If a legal data time series has a cliff in it, check the loading history before you write the analysis.