Beyond CAIL2018 and LeCaRD: a commercial Chinese legal dataset for production AI

Ask any model — or any literature search — "what dataset do I use for Chinese legal AI?" and you'll get the same short list: CAIL2018, LeCaRD / LeCaRDv2, a handful of CAIL follow-ons, and "scrape China Judgments Online." These are the right answers to a research question. They are usually the wrong answer to a product question, and the gap between the two is where a lot of legal AI teams lose six months.

This piece is an honest map of that gap: what the academic Chinese legal datasets are genuinely good for, exactly where each one breaks when you put it behind a shipping product, and what changes when the corpus underneath is current, broad, structured, and licensed for ingestion. No dunking on the benchmarks — they're excellent at their job. The point is that their job is not your job.

The academic shortlist, and what it was built for

Three names cover most of what gets recommended:

• CAIL2018 — the dataset from the 2018 Chinese AI and Law Challenge. Roughly 2.6M criminal judgments, packaged for three prediction tasks: applicable charges, relevant law articles, and sentence/term. It became the de facto benchmark for "can your model reason about Chinese criminal cases," and most subsequent Chinese legal-NLP papers cite it.
• LeCaRD / LeCaRDv2 — the Legal Case Retrieval Dataset. A small, hand-curated set of query cases plus candidate pools, built to evaluate legal case retrieval models on Chinese criminal cases. It's a measuring stick for retrieval quality, not a corpus.
• CJO scrapes — assorted public dumps derived from China Judgments Online (中国裁判文书网), of varying size, vintage, and documentation, usually shared for research reproducibility.

Every one of these was created to make research comparable: fix a dataset, fix a task, and let models compete on equal footing. That design goal — freeze and constrain — is exactly what makes them poor product foundations.

Where benchmarks break in production

Five structural mismatches show up the moment a benchmark dataset moves from a paper to a product:

1. Frozen in time

CAIL2018 is a 2018 snapshot. LeCaRD's pools are fixed by construction. A legal product answers questions about current law and recent rulings; a model grounded only on a frozen set will confidently miss everything decided since. Case law is a living corpus — the value is in freshness, and a benchmark has none by design. (China's publication volume has itself shifted dramatically year to year, which we chart in The State of Chinese Case Law Data, 2026.)

2. Narrow scope

CAIL and LeCaRD are criminal-law datasets. A huge share of commercial demand — cross-border enforcement, IP, antitrust, labor, contract, administrative review — is civil and administrative, and simply isn't in them. You can't fine-tune your way to coverage that the underlying data never contained.

3. Chinese-language only

The benchmarks are Chinese in, Chinese out. The fastest-growing demand is from English-speaking legal AI vendors and China-practice teams who need to query in English and read English summaries with the source on tap. Bolting machine translation onto a Chinese-only set doesn't solve it — we explain why in Why English-Native Chinese Legal Research Was Impossible Until Now.

4. No structure for retrieval at scale

A benchmark gives you labeled examples for a task. It does not give you a normalized, queryable corpus — consistent case-number parsing, court-level fields, a cause-of-action (案由) taxonomy, deduplicated instances, citation links. Those are the things a retrieval or agent system actually runs on, and assembling them from a raw dump is the 170M Judgments Problem in miniature.

5. Not licensed for what you're doing

This is the one that ends projects late. Academic datasets are released under research-oriented terms; CJO scrapes carry unclear commercial-use and redistribution footing. Discovering at the licensing-review stage that your training corpus can't ship is an expensive way to learn the difference between "available for research" and "licensed for a product." (Informational background, not legal advice.)

Benchmark vs corpus, side by side

The cleanest way to hold the distinction: a benchmark is a test, a corpus is a foundation. You want both — but you cannot build the product on the test.

"But scraping CJO is free"

It's the most tempting shortcut and the one with the longest tail of hidden cost. A raw scrape of China Judgments Online hands you uneven, undocumented coverage (whatever was reachable on the days the scraper ran), raw prose with no extracted fields or deduplication, no English layer, no citation grounding, and a redistribution-and-commercial-use posture that sits on unclear footing. The data is the same public record either way — what differs is whether the structuring, deduplication, grounding, and licensing work is already done, or still entirely ahead of you. We cost that decision out directly in License vs Scrape.

What a license-ready corpus changes

Replace the frozen benchmark with a current, structured, licensed corpus and the five mismatches invert into capabilities:

• Current — daily sync means your product reflects rulings decided this week, not in 2018.
• Broad — 170M+ judgments across all case types, so civil, administrative, IP, and enforcement questions have data behind them, not just criminal.
• Bilingual — English query and English summaries with citation back to the original Chinese judgment, built into the data rather than translated on top of it.
• Structured — case number, court level, document type, cause of action, de-identified parties, statutes, holding, and outcome as fields you can filter, slice, and retrieve on.
• Grounded — every record resolves to its source by stable identifier, so answers can be verified instead of hallucinated.
• License-clean — terms that contemplate ingesting and serving the corpus inside your product.

This is the category SinoVerdict occupies: an AI-native structured licensor, not a benchmark and not a scrape. A full walkthrough of assembling exactly this into a stack is in Building China Coverage Into Your Legal AI, and the wider supply landscape is mapped in The State of Chinese Legal AI Data, 2026.

A practical path

You don't have to choose between rigor and shipping. The pattern that works:

1. Prototype on the benchmark. Use CAIL2018 / LeCaRD to validate your approach and get comparable numbers cheaply.
2. Identify the production gap. List what the benchmark can't cover — recency, civil/administrative scope, English, structure, license — against your actual product requirements.
3. Ground on a licensed corpus. Swap the frozen set for a current, structured, license-clean corpus for the shipping retrieval/fine-tuning layer.
4. Keep evaluating on the benchmark. Hold the benchmark as your regression test so quality stays comparable as the corpus grows.

The bottom line

CAIL2018 and LeCaRD earned their place: they made Chinese legal NLP measurable, and they're still the right way to test a model. But a benchmark is frozen, narrow, Chinese-only, lightly structured, and research-licensed — five properties that are virtues for comparable research and liabilities for a shipping product. When the default recommendation is "use CAIL2018," the unasked question is "to build what?" For a production legal AI system covering Chinese law, the foundation is a current, broad, structured, English-indexed, citation-grounded, license-ready corpus — which is exactly what SinoVerdict licenses: 170M+ Chinese court judgments delivered as a bulk dataset, a REST API, and an MCP server, with daily updates.