“Has Full Text” Is Three Numbers: Measuring Body Coverage in a Chinese Case Law Dataset

Every vendor of Chinese court judgment data will tell you what share of its records have full text. It is usually the second number on the data sheet, after the record count, and it is usually one number. It should be three.

We measured all three on our own index of 160,291,678 PRC judgment records on 26 September 2026. The field-presence check that most engineers reach for first says 100 per cent. The share of records whose body is not empty is 91.4 per cent. The share whose body actually contains a judgment is about 84 per cent. All three are true statements about the same corpus on the same day. Only the third is the number a buyer is paying for, and the gap between the second and the third is invisible to every check that looks at whether a field is populated rather than at what it holds.

This page is the measurement protocol we now use on ourselves, written so that you can run it on any delivery, including ours. We had been quoting “about 91 per cent with text” on our own pages until 24 September; that figure was correct as a count of non-empty bodies and wrong as a description of what those bodies contained. The corrections are on those pages. This is the audit behind them.

Three numbers for one question

The sample behind the second and third rows is fifteen strata, one per ingest batch, 600 records each (9,000 in total), drawn with a seeded random score (seed 20260926) and weighted back to batch size. The 7.17 per cent is not a sample estimate; it is an exact count, explained below. The sample and the exact count agree: in the sample, records with a body of at least 30 characters come to 84.57 per cent.

Why the presence check says 100 per cent

A field that contains an empty string has a value. It is not missing. In a search index, the exists query matches any document that has a value for the field, and an empty string qualifies; in a relational store, body IS NOT NULL behaves the same way. Neither check can tell you whether there is anything in the field, only that the field was written.

The clearest demonstration in our index is one ingest batch of 4,414,931 records. Every one of them passes the presence check:

POST /judgments/_count
{"query":{"bool":{"filter":[
  {"term":{"dataset":"2019_2"}},
  {"exists":{"field":"body"}}]}}}

→ 4,414,931   (of 4,414,931)

A 600-record seeded sample from the same batch found 600 empty bodies. A full-batch search for the four words that appear in almost every PRC judgment — 法院 (court), 本院 (this court), 判决 (judgment), 原告 (plaintiff) — returns 94, 90, 44 and 42 records respectively. The batch is, for text purposes, empty. It still carries case numbers, court names, titles and dates, which is why it looks healthy in any schema-level report. We described this batch in an earlier corpus audit; what that audit did not say, and should have, is that the obvious query for detecting it reports the opposite.

Run the same presence check across the whole corpus and it also returns 160,291,678 of 160,291,678. A vendor who reports field completeness from a schema profiler, a data catalogue, or a “null rate” column will report the body field as complete. That report will not be a lie. It will simply be an answer to a question nobody should be asking.

The records that have text but no judgment

The larger and less obvious gap is between non-empty and judgment-bearing. It exists because of how Chinese courts publish.

The Supreme People's Court's 2016 provisions on publishing judgments on the internet list the kinds of judgment that are not to be published: those involving state secrets, juvenile crime, cases closed by mediation or confirming a people's mediation agreement, divorce and child custody or guardianship, and a residual category of cases a court considers unsuitable for publication. The same provisions say that for a judgment withheld on those grounds, the court should still publish the case number, the court, the date and the reason it is not being published.

So the public source contains a large population of records that are, correctly, a case number plus a sentence explaining why there is no document. Any pipeline that collects the source faithfully will collect those records. Many will store the reason in the same field that normally holds the judgment text, because on the source page it sits where the text would sit. In our index, six such strings account for every one of these records:

The six strings map onto the five grounds in the provisions, with the mediation ground split in two. They are between 7 and 19 characters long. Each passes every non-empty check, most pass a naive “has Chinese text” check, and one of them contains the word 法院 — the same word we used above to prove a batch was empty. A keyword-based text detector will count the residual-category notice as a judgment.

They are also concentrated. 11,464,704 of them — about 99.7 per cent — sit in a single ingest batch of 40,705,653 records, where they make up 28.2 per cent of the batch. Every other batch carries between six and roughly 13,000. That concentration is why a corpus-wide sample of a few hundred records can miss the pattern entirely or dismiss it as noise, and why the per-batch reporting argued for in our acceptance protocol matters here as much as anywhere.

A second trap inside the first: counting them returns zero

Once you know the strings, the natural next step is to count them with an exact-phrase query. On our index, that query with the default analyzer returns zero for five of the six strings. Only the shortest, 以调解方式结案的, comes back with its true count. The other five return their true counts only when the query is forced to use the same Chinese analyzer the field was indexed with (ik_max_word in our case). This is the same analyzer asymmetry that breaks exact-phrase retrieval for nine of our forty largest causes of action, turning up again in an audit query.

The consequence is specific and unpleasant: an auditor who suspects placeholder bodies, writes the obvious query and gets zeros will conclude the problem does not exist. A zero from a phrase query over Chinese text is a claim about the query before it is a claim about the data. Always run a string you know is present as a control.

Short is not the same as defective

The tempting shortcut is a length threshold: treat anything under, say, 200 characters as junk. On our sample, bodies of 30–199 characters make up 1.88 per cent of the corpus, and when we read them they are overwhelmingly real documents — enforcement case-closure notices (结案通知书), notices of preservation results, short withdrawal rulings, the occasional brief mediation statement. They are short because the document is short. A length filter at 200 characters would delete about three million genuine court documents along with the notices.

The placeholders, by contrast, are all under 30 characters. In the sample, 162 records had a non-empty body shorter than 30 characters; 161 were one of the six strings above and one was a scraper artefact, 文书加载中，请稍后。。。 (“document loading, please wait”), which is a captured loading screen rather than a document. The right filter is an exact match against a known list, plus a short-length review queue for anything the list does not catch. It is not a length threshold.

A measurement protocol you can run on any delivery

This works on a search index, on JSONL files or on a columnar table. It assumes you have the batch identifier for every record; if the vendor has not supplied one, ask for it before you measure anything else.

1. Stratify by batch. Draw a seeded random sample of at least 500 records per ingest batch. Record the seed and the sample size. Weight results back to batch size; never pool the sample and report one rate.
2. Measure length, not presence. For each sampled record compute the length of the trimmed body. Bucket into empty, under 30 characters, 30–199, and 200 or more.
3. Read the short bucket. Tabulate the distinct values of every body under 30 characters. On PRC data you should expect the non-publication reasons above; anything else is a pipeline artefact and belongs on the defect list.
4. Count the placeholders exactly. Once you know the strings, count them over the whole corpus by exact match — and if you are querying an index, force the analyzer and run a known-present control string alongside.
5. Report three numbers per batch, with the date: present, non-empty, judgment-bearing. The last is non-empty minus placeholders minus artefacts.

On a search index, the length measurement is one request per batch:

POST /judgments/_search
{"size": 600, "_source": ["title"],
 "script_fields": {"blen": {"script": {"source":
   "params._source.body == null ? -1 : params._source.body.trim().length()"}}},
 "query": {"function_score": {
   "query": {"term": {"dataset": "<batch>"}},
   "random_score": {"seed": 20260926, "field": "_seq_no"},
   "boost_mode": "replace"}}}

On delivered files, the whole protocol is a few lines of SQL. In DuckDB, for example:

SELECT batch,
       count(*)                                          AS records,
       avg((length(trim(coalesce(body,''))) = 0)::INT)   AS empty_rate,
       avg((trim(body) IN (
             '以调解方式结案的',
             '人民法院认为不宜在互联网公布的其他情形',
             '离婚诉讼或者涉及未成年子女抚养、监护的',
             '确认人民调解协议效力的',
             '未成年人犯罪的',
             '涉及国家秘密的'))::INT)                    AS notice_rate
FROM read_parquet('delivery/*.parquet')
GROUP BY batch
ORDER BY notice_rate DESC;

If you are exact-matching on delivered files, check first whether the vendor normalised whitespace or punctuation in the body field. A full-width comma rendered as a half-width one will make an exact match fail silently, which is the file-based equivalent of the analyzer trap.

What to do with the answer

The non-publication records are not worthless. Each carries a case number, a court, a date and a reason, and the reason is itself data: the 5.6 million mediation notices in our index are a direct record of cases closed by mediation, which is a research quantity in its own right and one that no judgment-text corpus can give you. If you are building a docket-level view — how many cases a court handled, how many ended in mediation, how many divorce matters were filed — you want these records.

What they are not is text. For anything that consumes the body — a training run, a retrieval index, a holding extractor, a citation graph — they are noise at best. At worst they are actively harmful: a model trained on 11 million copies of six sentences will learn those sentences very well, and a retrieval system that indexes them will return a one-line notice as the top hit for any query about divorce or mediation.

Commercially, the definition belongs in the contract. If the price or the acceptance criteria refer to “full text” or “documents with text”, say which of the three numbers that means, and say that it is measured per batch. Our note on pricing structures and term-sheet clauses covers where such definitions usually sit. Because the notices are a property of the public source rather than a processing fault, the honest treatment is usually a documented carve-out or a price that reflects the judgment-bearing count, not a demand for re-delivery: no vendor can deliver a judgment the court chose not to publish.

For a training corpus specifically, the filtering step belongs early in corpus preparation, before deduplication — otherwise a near-duplicate detector will collapse 11 million notices into six clusters and report an alarming duplication rate that has nothing to do with judgments.

What this page does not claim

• The figures describe our own index on the dates stated. They are not a measurement of any other vendor's corpus, and they are not a census of all judgments published in China — nobody can state the size of that population, so nobody can state a share of it.
• The 11,493,790 figure is a count of records whose body matches one of six phrases. In principle a genuine judgment could quote one of those phrases and be counted; the independent length sample, in which every short match was the whole body, suggests this is rare, but we have not verified all eleven million individually.
• “About 84 per cent” is derived (non-empty rate from the sample, minus the exact placeholder count), not a direct census. The direct sample estimate is 84.57 per cent. We quote the rounder figure because the two methods agree to within half a point.
• The 30–199 character bucket was characterised by reading a sample, not by classifying every record in it.
• These rates can change. Our 6,000-character storage cap, which affected 2.5 per cent of records when we first measured it, was backfilled in September 2026 and now affects 0.003 per cent. Any figure on this page should be re-run against the delivery you actually receive, which is the point of the protocol.