Benchmarks
Ground on public datasets, scored by the same quality and privacy report every user gets. Each dataset is shuffled with a fixed seed and split 80/20: every method trains on the same 80% (at most 30,000 rows) and is scored against the 20% it never saw. Other synthesizers are run on the same splits and scored the same way. The results are measured by Ground; read the metric definitions and the limitations below before interpreting them.
New to these metrics? Read how to evaluate synthetic data quality for definitions, interpretation and limitations.
For a decision-oriented explanation of this run, read the Ground, SDV and MOSTLY AI benchmark walkthrough.
Detection: AUC of a classifier telling real from synthetic rows (0.5 = indistinguishable). Utility: a model trained on synthetic data and tested on real data, vs trained on real (AUC, or R² for regression). Rules: how consistently synthetic rows follow rules the real data always follows (100 = always). MIA: membership-inference AUC (0.5 = an attacker learns nothing). AIR: attribute-inference risk (0 = none).
adult
UCI Adult census income (48,842 rows, 15 columns): income above $50K · classification target income
| Method | Overall | Shapes | Pairs | Detection | Utility (syn / real) | Rules | MIA | AIR | Fit |
|---|---|---|---|---|---|---|---|---|---|
| Ground | 94 | 99 | 97 | 0.57 | 0.897 / 0.904 AUC | 100 | 0.52 | 0.05 | 100.9s |
| Ground (DP ε=3) | 77 | 97 | 92 | 0.82 | 0.874 / 0.904 AUC | – | 0.51 | 0.01 | 2.3s |
| Ground (DP ε=3, MST) | 75 | 97 | 90 | 0.83 | 0.884 / 0.904 AUC | – | 0.50 | 0.00 | 0.4s |
| MOSTLY AI (TabularARGN, local) | 95 | 98 | 97 | 0.55 | 0.897 / 0.904 AUC | 100 | 0.52 | 0.00 | 235.4s |
| SDV CTGAN (50 epochs) | 67 | 86 | 94 | 0.94 | 0.865 / 0.904 AUC | 0 | 0.50 | 0.00 | 168.9s |
| SDV GaussianCopula | 65 | 90 | 89 | 0.96 | 0.867 / 0.904 AUC | 0 | 0.50 | 0.00 | 6.9s |
bank
UCI Bank marketing (41,188 rows, 21 columns, semicolon-separated): subscribed to a deposit · classification target y
| Method | Overall | Shapes | Pairs | Detection | Utility (syn / real) | Rules | MIA | AIR | Fit |
|---|---|---|---|---|---|---|---|---|---|
| Ground | 90 | 96 | 96 | 0.61 | 0.913 / 0.919 AUC | 100 | 0.51 | 0.02 | 110.8s |
| Ground (DP ε=3) | 64 | 96 | 87 | 1.00 | 0.869 / 0.917 AUC | – | 0.50 | 0.02 | 4.1s |
| Ground (DP ε=3, MST) | 65 | 95 | 86 | 0.98 | 0.880 / 0.919 AUC | – | 0.49 | 0.04 | 0.6s |
| MOSTLY AI (TabularARGN, local) | 65 | 91 | 96 | 1.00 | 0.896 / 0.919 AUC | 77 | 0.51 | 0.07 | 231.9s |
| SDV CTGAN (50 epochs) | 60 | 81 | 89 | 1.00 | 0.722 / 0.919 AUC | 32 | 0.50 | 0.02 | 190.6s |
| SDV GaussianCopula | 59 | 81 | 87 | 1.00 | 0.882 / 0.919 AUC | 23 | 0.49 | 0.04 | 9.9s |
credit
Credit scoring (4,446 rows, 26 columns, incl. 12 binned copies of numeric columns): good or bad loan · classification target Status
| Method | Overall | Shapes | Pairs | Detection | Utility (syn / real) | Rules | MIA | AIR | Fit |
|---|---|---|---|---|---|---|---|---|---|
| Ground | 91 | 98 | 97 | 0.61 | 0.827 / 0.833 AUC | 100 | 0.52 | 0.06 | 32.5s |
| Ground (DP ε=3) | 63 | 92 | 88 | 0.99 | 0.755 / 0.835 AUC | – | 0.52 | 0.00 | 0.7s |
| Ground (DP ε=3, MST) | 66 | 92 | 92 | 0.97 | 0.776 / 0.835 AUC | – | 0.55 | 0.00 | 0.1s |
| MOSTLY AI (TabularARGN, local) | 79 | 94 | 96 | 0.80 | 0.823 / 0.833 AUC | 0 | 0.54 | 0.09 | 153.5s |
| SDV CTGAN (50 epochs) | 61 | 87 | 87 | 1.00 | 0.456 / 0.833 AUC | 0 | 0.53 | 0.00 | 20.7s |
| SDV GaussianCopula | 63 | 89 | 91 | 1.00 | 0.801 / 0.833 AUC | 0 | 0.52 | 0.00 | 2.5s |
telco
IBM Telco customer churn (7,043 rows, 21 columns, IDs and blank totals) · classification target Churn
| Method | Overall | Shapes | Pairs | Detection | Utility (syn / real) | Rules | MIA | AIR | Fit |
|---|---|---|---|---|---|---|---|---|---|
| Ground | 92 | 99 | 98 | 0.62 | 0.829 / 0.833 AUC | 100 | 0.52 | 0.07 | 34.3s |
| Ground (DP ε=3) | 64 | 97 | 82 | 0.98 | 0.804 / 0.833 AUC | – | 0.52 | 0.05 | 0.5s |
| Ground (DP ε=3, MST) | 69 | 97 | 87 | 0.92 | 0.810 / 0.833 AUC | – | 0.52 | 0.00 | 0.1s |
| MOSTLY AI (TabularARGN, local) | 90 | 98 | 96 | 0.63 | 0.821 / 0.833 AUC | 98 | 0.53 | 0.00 | 139.6s |
| SDV CTGAN (50 epochs) | 64 | 90 | 86 | 0.96 | 0.610 / 0.833 AUC | 49 | 0.52 | 0.07 | 1350.5s |
| SDV GaussianCopula | 65 | 98 | 82 | 0.96 | 0.810 / 0.833 AUC | 0 | 0.52 | 0.00 | 2.5s |
housing
California housing (20,640 rows, 10 columns): median house value · regression target median_house_value
| Method | Overall | Shapes | Pairs | Detection | Utility (syn / real) | Rules | MIA | AIR | Fit |
|---|---|---|---|---|---|---|---|---|---|
| Ground | 95 | 99 | 98 | 0.57 | 0.628 / 0.688 R² | 100 | 0.51 | 0.08 | 53.9s |
| Ground (DP ε=3) | 75 | 96 | 95 | 0.87 | 0.533 / 0.688 R² | – | 0.51 | 0.00 | 0.7s |
| Ground (DP ε=3, MST) | 76 | 96 | 89 | 0.80 | 0.397 / 0.688 R² | – | 0.50 | 0.08 | 0.1s |
| MOSTLY AI (TabularARGN, local) | 93 | 98 | 96 | 0.59 | 0.607 / 0.688 R² | 96 | 0.51 | 0.12 | 145.7s |
| SDV CTGAN (50 epochs) | 53 | 75 | 76 | 1.00 | 0.058 / 0.688 R² | 0 | 0.53 | 0.00 | 73.4s |
| SDV GaussianCopula | 60 | 79 | 90 | 1.00 | 0.436 / 0.688 R² | 97 | 0.49 | 0.00 | 2.4s |
insurance
Medical insurance (1,338 rows, 7 columns): annual charges · regression target charges
| Method | Overall | Shapes | Pairs | Detection | Utility (syn / real) | Rules | MIA | AIR | Fit |
|---|---|---|---|---|---|---|---|---|---|
| Ground | 98 | 98 | 97 | 0.51 | 0.813 / 0.871 R² | – | 0.49 | 0.00 | 12.3s |
| Ground (DP ε=3) | 80 | 94 | 92 | 0.75 | 0.643 / 0.871 R² | – | 0.55 | 0.06 | 0s |
| Ground (DP ε=3, MST) | 82 | 95 | 93 | 0.73 | 0.726 / 0.871 R² | – | 0.50 | 0.00 | 0s |
| MOSTLY AI (TabularARGN, local) | 77 | 98 | 90 | 0.82 | 0.019 / 0.871 R² | – | 0.48 | 0.00 | 47s |
| SDV CTGAN (50 epochs) | 68 | 86 | 90 | 0.90 | 0.088 / 0.871 R² | – | 0.49 | 0.00 | 7.4s |
| SDV GaussianCopula | 82 | 96 | 94 | 0.75 | 0.507 / 0.871 R² | – | 0.52 | 0.06 | 0.5s |
Caveats
- The report is Ground's own. Overall is 35% column shapes, 35% column pairs and 30% detection; it doesn't include Rules, which checks the rules Ground detects and enforces, so Rules favors Ground by design.
- SDV CTGAN was trained for 50 epochs (its default is 300) to keep the run within a few hours on one CPU; more epochs would likely improve it. SDV GaussianCopula used its defaults. Both ran on SDV 2 with metadata detected from the training split.
- Differential privacy at ε = 3 keeps column shapes and the label mix, but a classifier can still tell DP rows from real ones (detection AUC 0.75–1.0): the noise breaks fine-grained structure, and rules can't be learned from the data without spending budget.
- Single runs with fixed seeds; differences of a point or two in Overall are within noise.
- MOSTLY AI ran locally from its open-source SDK (TabularARGN) with a 5-minute training cap per dataset; Synthcity ran with its plugin defaults. Gretel isn't included: it trains in Gretel's cloud, which would mean uploading the data to a third party.
- Where MOSTLY AI scores low we checked the rows, not just the numbers: on bank, 35% of its euribor3m values are off the 308-value grid the real rate sits on, which a classifier spots; on insurance it lost the smoker effect on charges (smokers average $14.2k vs $31.4k real), after 47 seconds of training, well under its 5-minute cap.
Reproducibility
Ground maintains a reproduction harness and pinned synthesizer environments in its private repository. This published snapshot is not yet independently reproducible by public readers. Download the recorded results and read the benchmark walkthrough for the evidence and missing provenance.
Run 2026-10-20. Want this on your data? Upload a table and the same report is generated for it.