How to evaluate synthetic data quality

1. Compare distributions and relationships

Ground measures numeric and date column shapes using the Kolmogorov–Smirnov statistic, and categorical shapes using total variation distance. Missing-value rates are checked as well. Pairwise metrics compare associations between columns in real and synthetic data.

Inspect individual fields and important segments, not just the average. A dataset can match individual columns while missing a dependency that matters to your model. Pairwise similarity can also miss higher-order relationships, so add checks for domain rules and the queries you intend to run.

2. Read detection AUC in context

The report trains a classifier to distinguish real rows from synthetic rows. An AUC near 0.5 means that classifier has difficulty separating them; higher values indicate detectable differences. It is a diagnostic for this test, not proof that all reviewers or models will find the datasets indistinguishable.

Ground's overall score combines column shapes (35%), column pairs (35%) and detection (30%). Task usefulness and privacy checks are reported separately. A strong overall score can therefore coexist with weak performance for a particular target or an unacceptable privacy result.

3. Test usefulness on unseen real rows

Train-on-synthetic, test-on-real evaluation compares models trained on synthetic and real data against a real holdout. Ground reports classification AUC or regression R². Look at the gap between the two models and whether that gap is acceptable for your task.

Repeat important evaluations with your own model and the split your application needs. For analysis workflows, compare the actual queries and aggregates you expect users to run. Similar distributions alone do not guarantee similar decisions.

4. Review privacy independently of realism

Ground reports distance to the closest record and membership-inference, attribute-inference and singling-out risk. These tests help identify whether outputs may expose information about source rows. Default copy guards and category suppression reduce specific risks; they are not a formal privacy guarantee.

Optional differential privacy uses a separate fitting mode with an explicit budget. Interpret both empirical tests and configured privacy controls in the context of the planned release. No report score alone certifies anonymity, compliance or suitability for a sensitive use.

Make the comparison reproducible

Keep the source split, model version, generation settings and exported artifact with your evaluation. Ground keeps analysis versions and provides a shareable data card. Check what a card exposes before publishing it.

Ground's public benchmark page reports results on public datasets, with the same splits across the compared synthesizers and caveats about the run. Treat it as evidence about those experiments. Your dataset, task and privacy settings may produce different results.

What is a good synthetic data quality score?

There is no universal threshold. Set acceptance criteria for your intended task, compare against a baseline and review important fields, segments, usefulness and privacy separately. Ground's overall score summarizes fidelity and detection, not all release criteria.

Does detection AUC of 0.5 prove that synthetic data is private?

No. It means the tested classifier struggles to distinguish real and synthetic rows. Privacy requires separate analysis, including attacks and consideration of the synthesis settings and release context.