Evidence for privacy and compliance teams

Short answer, from one benchmark run on six public tables: the strongest of four membership attacks reached an AUC of 0.50–0.55 against Ground's rows (0.5 is a coin flip; Ground flags anything above 0.6 for review), synthetic rows were closer to a training row than to an unseen one 47.2%–52.1% of the time (about 50% means no memorisation), and no synthetic row copied a rare real record. These are measured attacks, not a guarantee. A formal guarantee needs differential privacy, which costs fidelity (below).

Membership attacks

The report computes TPR at 1% FPR for every attack on every dataset (it flags anything above 5%), but the published benchmark run recorded only the AUC, so the low-false-positive number is pending first independent run. AUC averages over all thresholds and can hide an attack that is very sure about a few people; TPR at low FPR is the number to ask for.

Distance to closest record, copies and inference

How it was measured

Each table is split 80/20 with seed 7; the synthesizer sees only the 80%. Attacks try to tell those training rows from the unseen 20% using the synthetic rows. DCR share: synthetic rows closer to a training row than to a holdout row (about 50% is ideal, above 60% flags review). Attribute inference and singling out are Anonymeter-style excess risks of training over holdout rows (above 0.1 flags review).

Uncertainty

One run per table with fixed seeds; no confidence intervals. Attack AUCs within a few hundredths of 0.5 are indistinguishable from chance at these sizes.

Scored by

Ground's own report (self-scored). Independent attack implementations are pending first independent run.

Raw output

benchmark-results.json · evidence.json

Source

apps/web/lib/benchmark-results.json; docs/benchmarks.md

Canary records

The canary audit plants made-up records, refits the same kind of model, and checks whether the output gives them away. It reports the attack AUC on the canaries and an empirical lower bound on ε (Clopper–Pearson 95%). On the public tables: attack AUC pending first independent run, ε lower bound pending first independent run. The audit is part of the report Ground computes for an upload, so your own dataset's report has its own result.

Differential privacy at each ε

Other values of ε: pending first independent run. At ε = 3 column shapes and the label mix survive, but a classifier can still tell the private rows from real ones (detection AUC near 1 on several tables): the noise breaks fine-grained structure, and rules are off because they can't be learned without spending budget. The schema, including every class of the label, is treated as public and isn't covered by ε.

What happens to your upload

• Files go from your browser straight to encrypted storage through short-lived signed links; nothing passes through the web app.
• Names, emails, phones, IDs, card numbers and addresses are detected and replaced with fake values, never modeled.
• In the default mode every source row is fingerprinted, and a generated row that would reproduce a rare real record is redrawn. (Private mode never reads real rows while generating, so it needs no copy guard.)
• Raw uploads, and the copies of their rows made for training, are deleted after 30 days by default or on request.
• When GPU engines are on for a dataset, the modeled columns (never identifiers or personal data) go to Ground's own Modal workspace through short-lived links, and Modal keeps nothing once a job ends. Free text goes there only if you turn on private fine-tuning for it, and the canary audit sends a second copy of the training rows. Those rows can still be personal data in combination (an age, a postcode and a diagnosis).
• For AI-written text columns, Claude sees only short, scrubbed examples, never your files. Uploads are never used to train AI models.

Next: methods and reproducibility, or read how to evaluate synthetic data quality and privacy.