Every Column Reads Complete. That Is the Filter Talking.

Yesterday we published a coverage audit that ended on a single lesson: a completeness figure is only as honest as its denominator. Today we applied the same discipline to a different object — not the full-text corpus, but a small, heavily structured dataset of our own, the kind of file that gets emailed to a buyer during evaluation because it looks tidy.

It is very tidy. Twelve of its fourteen columns read 100 per cent populated, or within two points of it. That reading is worthless, and the reason it is worthless is the most transferable thing in this article: in a derived dataset, column completeness is a property of the row filter, not of the extraction. A judgment whose amount could not be parsed never became a row. The columns read complete because the incomplete rows were never shipped.

Below that surface, three independent tests found defects that no completeness metric can see: an outcome label with no losing value, in which 865 rows record an amount the court expressly refused to award; three numeric fields that are individually present and mutually inconsistent on roughly one row in eight; and a delivered evidence span so short that for about half the rows the headline number cannot be re-derived from the file at all.

One correction belongs at the top

This same dataset was described in our own pre-sales material as holding 1.1 million records. On 29 July 2026 we measured it: 80,000. The description was wrong by roughly fourteen times, sat in a machine-readable field for months, and was caught by a coverage note in the same API response contradicting the number beside it — not by a customer.

What the completeness table actually says

Four derived columns at exactly 100.00 per cent is not a quality result; it is a definition. Producing all four was the condition of entry, so their accuracy is unconstrained by the completeness table. Only monthly_salary — the one derived field the pipeline was allowed to leave blank — carries information, and it says that on 9,436 rows the salary could not be read out of a judgment from which an award amount could be read.

There is a second casualty. Yesterday, in the unfiltered corpus, we measured that blank fields travel together: blank case numbers were 305 times denser in the slice whose procedure field was also blank. That test is impossible here. The rows that would have shown a correlation between two failed extractions were deleted by the filter, so every pairwise test returned a degenerate zero. A selected dataset cannot tell you how its own extraction fails.

The joint denominator is the only one a buyer uses

No analytic question uses one column. The questions buyers bring to a labor dataset — what multiple of salary courts award, how it scales with tenure, how that differs by termination reason and province — need several columns at once, and the joint is pinned by the weakest member.

So the honest headline for this file is not 80,000. For any question involving salary it is 70,564, and for the panel question a pricing model would ask it is 70,123. The distance between 100.00 per cent and 87.65 per cent is 9,877 rows, invisible in a per-column completeness report.

An outcome label with no losing value

The plaintiff_won column has exactly two values across 80,000 rows: win (41,835 rows, 52.29 per cent) and partial (38,165 rows, 47.71 per cent). There is no third value. The schema has no way to represent nothing was awarded, and compensation_judged has no zero and no null — its minimum is 100 yuan.

A schema with no null hypothesis will invent a positive one. To find out whether it had, we screened the delivered text for refusal language — the operative formulas a Chinese court uses to decline a payment (无需支付 / 无须支付 / 不予支付, “need not pay”, “shall not pay”) — and then asked a narrower question: is the amount the court refused the same number the row records as awarded?

We then read 20 of the 865 by hand, drawn at random with a fixed seed. Eighteen were unambiguous polarity errors. In a 2017 Beijing appellate judgment ((2017)京01民终5364号) the operative paragraph holds that the company need not pay 23,000 yuan in severance; the row records 23,000 yuan as the compensation awarded and labels the case a win. In a 2017 Wuhan appellate judgment ((2017)鄂01民终4179号) the court holds the employer need not pay 11,572.34 yuan; the row records 11,572.3. In a 2020 Hangzhou judgment ((2020)浙0106民初1091号) the employer need not pay 53,123 yuan; the row records 53,123.

The two false positives are instructive about the screen rather than the data. One was a double negative — the court dismissed the employer’s claim that it need not pay, and then ordered the payment, so the positive amount is correct. The other matched a refusal that belonged to an arbitration-acceptance ruling, not to the award. At 18 of 20, the screen’s precision is about 90 per cent (95 per cent interval roughly 70 to 97 per cent), which puts roughly 780 rows — about 1 per cent of the file — in the category of an amount the court expressly declined to order, stored as an award.

That is a floor, not an estimate of the total. The screen can only see refusals that happen to fall inside a 260-character excerpt, and only when the refused figure is written next to them. The population of rows where the court refused and the text of the refusal was truncated away is not measurable from this file at all.

Whose win is it?

The field is named for the plaintiff, which in PRC labor litigation is a coin flip. Of the 67,640 rows whose excerpt names a plaintiff — 84.55 per cent of the file — 49.09 per cent name an organisation and 50.91 per cent an individual. Employers file about half of these cases, usually to challenge an arbitration award, and when the employer is the plaintiff, a plaintiff win means the worker was paid nothing.

Read the label as written — the plaintiff prevailed — and this file says employers win 62.71 per cent of the cases they bring. Read it as the pipeline probably intended — money moved to the worker — and the rate should be near-independent of who filed. It is neither: an 18.60-point asymmetry, of which the polarity defect accounts for about 1.18 points. We do not know what the remaining 17 points are, and that is the finding. An outcome label that cannot be interpreted without re-reading the judgment is not an outcome label; it is a second research task, shipped as a column.

Three numbers that are all present and do not agree

Chinese statutory severance is anchored to a multiple of average monthly wage per year of service, with damages for unlawful termination at twice that base. So compensation_judged divided by (monthly_salary × work_years) has an expected neighbourhood. Over the 70,564 rows where all three are present:

The median of 1.50× is where statutory arithmetic would put it, which says nothing about the tails. 61.74 per cent of the rows fall in a plausible 0.5×–3.0× band; 4,956 rows sit above 12× and 1,641 below 0.1×. The band is an indicator, not ground truth — an award can legitimately include unpaid wages, double-wage penalties or unused leave — so an out-of-band ratio is a question, not a verdict. But reading the extremes answers the question in several cases:

• The top row records 29,173,300 yuan against a 1,630 yuan monthly salary over 1.2 years of service. No head of damages reaches that ratio; the amount belongs to something else in the document.
• A row at 836× is, on the face of the delivered text, a wrongful-death award — a funeral subsidy and a lump-sum death compensation — sitting inside a table of labor-dispute severance.
• At the other end sits the clearest single row in the file. It records 101 yuan of compensation and a 23,000 yuan monthly salary. Its delivered text is one refusal sentence: the employer need not pay severance of 101.029 yuan, a wage differential of 23,000 yuan, or a winter heating allowance of 335 yuan. Both of the row’s numbers were lifted out of the same refused list, and one of them was not a salary.

The lesson is not that the extractor is bad. It is that a cross-field consistency test finds defects per-field validation cannot, because each of the three numbers is individually well-formed, non-null, positive and in range.

The audit trail is truncated before it reaches the number

Each row ships a body_excerpt. Its median length, 95th percentile and maximum are all 260 characters — a hard cap. 93.10 per cent of excerpts end mid-sentence. 67.45 per cent contain both the findings section and the operative section; 0.16 per cent contain no operative section at all. And then the number that matters most to anyone planning to spot-check:

For roughly half the rows, and four fifths of the salary figures, the delivered artifact contains no evidence for its own headline value. A buyer can still verify against the full judgment — that is what the full-text corpus is for — but not from the file they were sent, which is the file that gets loaded into a notebook during evaluation. Ask for the evidence span, not the excerpt, and ask for its character budget in writing.

Half the file is at a sampling cap

Nine provinces contain exactly 4,500 rows each: Jiangsu, Shandong, Hubei, Guangdong, Hunan, Shanghai, Henan, Zhejiang and Sichuan. That is 40,500 rows, 50.6 per cent of the file, pinned at a per-province extraction ceiling rather than at any real case volume. Cross-province comparison on this file measures the cap, not China.

The province column holds 39 distinct values for a country with 34 province-level divisions. Two are not provinces at all (the Supreme People’s Court, 9 rows; a Xinjiang production-corps branch court, 4 rows). Six are parser residue with one row each — a province name with a stray digit, underscore, backtick or semicolon welded to the front. Small in volume, fatal in a GROUP BY.

The time axis is worse, and it produces a publishable-looking trend:

The 2018 rows stop in July and the single 2019 row comes from one court in one province; 2023 stops in March; 2024 onward is empty; 509 rows carry no date at all. Plot litigation volume from this file and you will report a 95 per cent collapse in 2018, a near-total disappearance in 2019 and a twentyfold rebound in 2020. None of that happened. It is an ingest window with a hole in it.

The primary key remembers which pipeline built it

doc_id is unique across all 80,000 rows, in three shapes: 73,564 32-character hexadecimal identifiers, 6,046 carrying a ws_ prefix from a second ingest path, and 390 that are literal content-management URL paths of the form rmfy/ms/201406/t20140620_1601098.htm. The key is stable and joinable, but its shape encodes provenance, so any length or format assumption written against it will silently drop one of the three cohorts.

case_no is not a key. 79,908 rows carry one, covering 79,223 distinct values: 685 rows share a case number with another row. Bracket conventions are mixed across rows — 78,290 rows full-width, 1,514 half-width, 104 with no bracket at all — though the negative result belongs here too: normalising bracket width collapses zero additional duplicates in this file, so the mixed convention is a join hazard rather than a live collision.

The natural key is worse than the nominal one. Grouping on (title, court, date) finds 452 duplicate groups covering 1,015 rows, one of them 24 rows deep. In 134 of those 452 groups the derived numbers disagree — the same judgment, two rows, two different awards. De-duplication here is not a GROUP BY; it is an adjudication, and whoever writes the survivorship rule decides the answer.

What to ask, if someone sends you a tidy file

1. Which columns are derived, and what was the condition of entry? A derived column at 100.00 per cent is reporting the row filter. Ask for the count of source judgments rejected by it — that is the real completeness figure, and it is usually not in the file.
2. Give me the joint coverage for my question, not the marginals. Twelve columns at 100 per cent and one at 88.2 per cent means the answerable population for anything involving that field is 88.2 per cent, and less once you add a date and a geography.
3. Does the outcome vocabulary contain a losing value? If it does not, the pipeline had nowhere to put a refusal, and refusals will be stored as wins with a number attached. Screen for the court’s refusal formulas against the amount column before you trust an outcome field.
4. Whose outcome is it? In a jurisdiction where the employer files half the cases, a field named for the plaintiff does not tell you who received money. Ask for party roles as data, not as a label inferred from them.
5. Run one cross-field consistency test before any per-field validation. Three individually valid numbers that disagree with each other will pass every null check and every range check you write.
6. Ask how many characters of evidence ship with each row, and measure what share of the delivered values appear in the delivered text. If it is half, your spot-check plan needs the full documents and a licence that covers them.

What we are not claiming

We measured no other provider’s data, so nothing here is comparative. We fixed nothing: no value was rewritten, no row removed, no label corrected, because changing shipped values breaks every join already written against them and belongs in a change log with notice rather than in an audit. The hand adjudication of 20 rows is our own read of our own data, not independent replication, and the 90 per cent precision it implies carries a wide interval at that sample size. The statutory band is an indicator, not ground truth, so the 38.26 per cent outside it are questions rather than confirmed errors. The refusal screen sees only what survives a 260-character truncation, which makes 865 a floor. And the plaintiff-role figures come from that same truncated text, so they describe the 84.55 per cent of rows naming a plaintiff, not the file.

One finding has no comfortable reading. This file is the tidiest artifact we ship, and the one buyers see first, because it loads in a second and every column looks full. Tidiness was the property being optimised and completeness metrics rewarded it. The defects above survived because nothing in the delivery format could report them.