Sample report. The brand and every number on this page are made up to show the format. Real reports go up only with the brand's written OK, after the test has finished.
Example Brand: Meta ads holdout test on Amazon
Meta ads were paused in 15 US states for 56 days (Jan 12, 2026 to Mar 8, 2026) and kept on in the other 36. With ads on, Amazon orders were 18% higher than the paused states predict (95% range 5% to 32%).
Result
- Lift in orders
- +18% (5% to 32%)
- Extra orders per day, ads on nationally
- 12 (3.5 to 20.5)
- Extra sales per day
- $480 ($140 to $820)
- Incremental ROAS
- 1.20 (0.35 to 2.05)
Ranges are 95% intervals; p = 0.008. The whole range is above zero, so the ads caused extra orders. Computed Mar 10, 2026 from orders through Mar 8, 2026.
Design, locked before the test started
- Registered
- Jan 5, 2026
- Test
- Jan 12, 2026 to Mar 8, 2026 (56 days)
- Baseline from
- Oct 6, 2025
- Holdout share
- 30% of orders
- Smallest lift the test could detect
- 15% (80% power, 5% significance)
- Meta spend before the test
- $400 a day
Ads paused in: Arizona, Colorado, Connecticut, Iowa, Kansas, Kentucky, Maryland, Minnesota, Nevada, Oklahoma, Oregon, South Carolina, Tennessee, Utah, Wisconsin.
Method: ratio-geo-holdout-v1, a difference-in-differences on orders by ship-to state. See how lift is calculated.
Check it yourself
The design was written down and fingerprinted before the test began. Its SHA-256 hash is 18800b7f2e6776ea1731773c9ad6e085be440d690c550b4b721365eb35be7398.
To check it, hash the design below as JSON with its keys sorted and no spaces. If the hash matches, nothing in the design changed after it was registered. Results are added to the record, never edited.
The registered design (JSON)
{"alpha":0.05,"brandId":"example-brand","createdAt":"2026-01-05T17:00:00.000Z","dailyMetaSpend":400,"dataSource":"amazon-all-orders-report","expectedMinimumDetectableLift":0.15,"experimentId":"00000000-0000-4000-8000-000000000001","groups":{"adsOff":["AZ","CO","CT","IA","KS","KY","MD","MN","NV","OK","OR","SC","TN","UT","WI"],"adsOn":["AK","AL","AR","CA","DC","DE","FL","GA","HI","ID","IL","IN","LA","MA","ME","MI","MO","MS","MT","NC","ND","NE","NH","NJ","NM","NY","OH","PA","RI","SD","TX","VA","VT","WA","WV","WY"]},"holdoutShare":0.3,"method":"ratio-geo-holdout-v1","metric":"orders","notes":"Sample report with made-up numbers.","phi":1.8,"power":0.8,"preStart":"2025-10-06","schemaVersion":1,"seed":1,"testDays":56,"testStart":"2026-01-12","timeZone":"America/Los_Angeles"}What this report leaves out
Profit margin and break-even stay private to the brand. Amazon Attribution figures aren't shown next to Meta results while Amazon's approval is pending. Every number here comes from this brand's own orders and spend; nothing is pooled with other brands.