Evidence for test data

Your first question: Will the data keep my keys and business rules, and can it break my pipeline on purpose, with an answer key?

Short answer: on the 5 public tables where rules were detected, Ground's rows followed every one of them. With defects on demand (Generate menu), named defects landed at the requested rate and every changed cell was labelled. Foreign-key integrity, and speed and cost per million rows, are not measured yet.

Rules held

Rules followed by synthetic rows (0–100)

Share of the rules the real data always follows (orders, formulas, derived columns, impossible combinations) that synthetic rows also follow. Higher is better.

Rules followed by synthetic rows (0–100): the numbers in the chart
Dataset	Ground	MOSTLY AI	SDV CTGAN (50 epochs)	SDV GaussianCopula
Adult	100	100	0	0
Bank	100	77	32	23
Credit	100	0	0	0
Telco	100	98	49	0
Housing	100	96	0	97

Rules score by dataset and method, including the cases with no score
Dataset	Ground	Ground (DP ε=3)	Ground (DP ε=3, MST)	MOSTLY AI (TabularARGN, local)	SDV CTGAN (50 epochs)	SDV GaussianCopula
Adult	100	off under DP	off under DP	100	0	0
Bank	100	off under DP	off under DP	77	32	23
Credit	100	off under DP	off under DP	0	0	0
Telco	100	off under DP	off under DP	98	49	0
Housing	100	off under DP	off under DP	96	0	97
Insurance	no rule detected	off under DP	off under DP	no rule detected	no rule detected	no rule detected

Referential integrity

For related tables, Ground generates each child row from a generated parent, so by construction foreign keys point at rows that exist in the output. Measured share of child rows whose keys resolve, on a public multi-table dataset: pending first independent run.

Defects on demand, with an answer key

Requested rate vs achieved: mean relative deviation over 35 defect × dataset pairs
Requested rate	Mean relative deviation	Note
0.5%	6.8%	Largest achieved rate furthest from the request: 0.35%.
2%	3%	Largest achieved rate furthest from the request: 1.75%.
10%	1.4%	Largest achieved rate furthest from the request: 10.58%.

Labels are exact: dirty and clean files written, read back and compared cell by cell
Dataset (20,000 rows)	Changed cells	Changed but unlabelled
card-transactions	19,752	0
employee-directory	19,530	0
loans (upload)	36,037	0

What a simple validator catches, per defect at 2%

Recall of a Great Expectations–style validator whose expectations were learned from a different clean sample. Low recall is the point: label noise and near-duplicates slip through, which is why the label file matters.

What a simple validator catches, per defect at 2%: the numbers in the chart
Defect	card-transactions	employee-directory	loans (upload)
Exact duplicate	100%	100%	100%
Near-duplicate	68%	79%	72%
Label noise	0%	0%	0%
Rule violation	doesn’t apply	100%	100%
Swapped fields	100%	66%	95%
Wrong unit	100%	100%	100%
Out of range	100%	100%	100%
Type corruption	100%	100%	100%
Format drift	100%	100%	95%
Broken reference	100%	100%	100%
Whitespace or encoding	100%	100%	100%
Missing (at random)	100%	93%	100%
All defects	95.9%	91.4%	96.8%

Edge cases: valid boundary rows (min and max, 29 February, daylight-saving gaps, unicode names, rules at their boundary)
Dataset	Edge rows	Turned away by the validator
card-transactions	29	15
employee-directory	49	30
loans (upload)	41	11

Every edge row is valid, yet a validator learned from typical rows turned away 26.8%–61.2% of them: the edge file finds an over-strict validator before production data does.

Turning a detected rule off should keep the source's error rate (loans, applied_on ≤ decided_on, 40,000 rows)
Setting	Rows breaking the rule	Note
source	0.067%	2 of 3,000 source rows.
rule off, before the fix	29.2%	11,673 of 40,000 generated rows.
rule off, after the fix	0.065%	26 of 40,000 generated rows (27 expected).

How it was measured: Three datasets of 20,000 rows (card-transactions and employee-directory templates, the loan sample fitted as an upload); every defect at 0.5%, 2% and 10%. The harness writes the dirty CSV, the clean CSV and the label file, reads them back and runs the validator on them.
Uncertainty: Rate deviations are binomial noise (about ±10% relative at 0.5%); tests hold every defect within 4σ of its expected count. Validator recall is one run per dataset and a floor, not a score.
Scored by: Ground's assessment harness (self-scored); the validator was written for the harness.
Raw output: defects-results.json
Source: docs/assessments/defects-and-edge-cases.md

Speed and cost per million rows

Per million rows: wall time and cost of a full export
Measure	Value
Seconds per million rows	pending first independent run
Cost per million rows (USD)	pending first independent run

What exists today are partial timings, not an export of a million rows:

Sampling throughput in the benchmark run (Ground, rows per second; machine not recorded; not a per-million-row export)
Dataset	Rows per second
Adult	44,118
Bank	25,424
Credit	50,099
Telco	24,824
Housing	106,529
Insurance	178,500

Defect pass cost (median of 5, Apple M5 Pro, Node 22)
Dataset	Generate 20,000 rows	Defect pass, 12 defects at 2%
card-transactions	198 ms	186 ms
employee-directory	112 ms	273 ms
loans (upload)	2,297 ms	386 ms

Next: coverage and label checks for evaluation sets, or the e-commerce test-data worked example with its seed and checksums.