Reproducible tests
Lab evidence matrix
This page is not a product dashboard — it documents how we plan to prove claims on the public site. The lab injects known faults on sim.punchop.com; scenario definitions live inpunchop-lab. The investigator never uses that answer key during a real case.
Until the Enterprise lab run finishes, the table below is expected visibility from the scenario catalog, not measured lag or live BigQuery results. After the run, we publish observed cells and dates here (same method as guides marked verification: lab).
Method
- Emitter fleet sends with
X-Punchop-Scenarioand known fault fixtures. - Receiver + Enterprise export + Reports API capture events.
coverage-matrixtool compares expected vs observed visibility.
Expected coverage (catalog — not yet measured on tenant)
| Scenario | Fault | BigQuery export | Reports API | Receiver |
|---|---|---|---|---|
| transport_greylist | greylist_451 | partial | — | yes |
| gmail_5_7_26 | gmail_auth_policy | yes | yes | yes |
| async_dsn_bounce | accepted_not_delivered | yes | — | yes |
| outlook_5_7_515 | outlook_high_volume | yes | — | yes |
Full JSON: regenerate with go run ./cmd/coverage-matrix/ after each lab run.