# Python (scipy) Validation of PowerShell Statistical Results

Generated: 2026-07-31 by `tools/validate_stats.py`

Python environment: numpy 2.5.1, scipy 1.18.0

Data: retraction_watch_gitlab.csv (71448 rows)

Cohorts: machine=9161, comparison=56852, machine_with_lag=9161, comparison_with_lag=56852


## Tolerances

- Proportions and effect sizes: 0.001
- Lag statistics (days): 0.5
- Bootstrap CI endpoints: noted as approximate (different RNG implementations)
  produce endpoint differences typically < 15 days / 2 pp


## Comparison Table

| Estimate | PowerShell | Python (scipy) | Absolute difference | Agree within tolerance? |
|---|---|---|---|---|
| Median lag, machine (days) | 491 | 491.0 | 0.0000 | YES |
| Median lag, comparison (days) | 488 | 488.0 | 0.0000 | YES |
| Mann-Whitney U | 274613105 | 274613105.0 | 0.0000 | YES |
| Mann-Whitney z | 8.391 | 8.3908 | 0.0002 | YES |
| Cliff's delta | 0.0545 | 0.0545 | 0.0000 | YES |
| MAD machine (days) | 267.5 | 267.5152 | 0.0152 | YES |
| MAD comparison (days) | 768.5 | 768.5251 | 0.0251 | YES |
| MAD difference (days) | -501 | -501.0 | 0.0098 | YES |
| MAD diff CI lower (days) | -514.1 | -514.0602 | 0.0398 | YES |
| MAD diff CI upper (days) | -488 | -488.0 | 0.0405 | YES |
| MAD ratio | 0.348 | 0.3481 | 0.0001 | YES |
| Welch t (Brown-Forsythe) | -75.246 | -75.2463 | 0.0003 | YES |
| Welch df | 37255 | 37254.8 | 0.1565 | YES |
| Paper-mill % machine | 71.12 | 71.1167 | 0.0033 | YES |
| Paper-mill CI lower, machine | 70.18 | 70.1799 | 0.0001 | YES |
| Paper-mill CI upper, machine | 72.04 | 72.0358 | 0.0042 | YES |
| Paper-mill % comparison | 9.14 | 9.1378 | 0.0022 | YES |
| Paper-mill CI lower, comparison | 8.9 | 8.9037 | 0.0037 | YES |
| Paper-mill CI upper, comparison | 9.38 | 9.3774 | 0.0026 | YES |
| Paper-mill diff (pp) | 61.98 | 61.9789 | 0.0011 | YES |
| Paper-mill diff CI lower | 61.02 | 61.0211 | 0.0011 | YES |
| Paper-mill diff CI upper | 62.94 | 62.9368 | 0.0032 | YES |
| Risk ratio (paper mill) | 7.78 | 7.7827 | 0.0027 | YES |
| Two-proportion z | 144.12 | 144.1162 | 0.0038 | YES |
| MH Odds Ratio | 2.311 | 2.3112 | 0.0002 | YES |
| MH OR CI lower | 2.15 | 2.1505 | 0.0005 | YES |
| MH OR CI upper | 2.484 | 2.4840 | 0.0000 | YES |
| MH Risk Ratio | 1.431 | 1.4307 | 0.0003 | YES |
| Crude RR (MH table) | 7.783 | 7.7827 | 0.0003 | YES |
| Strata contributing | 23 | 23.0 | 0.0000 | YES |
| Cochran Q | 248.2 | 248.1870 | 0.0130 | YES |
| Bootstrap: median lag diff (point) | 3 | 3.0 | 0.0000 | YES |
| Bootstrap: median lag diff CI low | -17 | -16.0 | 1.0000 | ~approx (bootstrap) |
| Bootstrap: median lag diff CI high | 40 | 42.0 | 2.0250 | ~approx (bootstrap) |
| Bootstrap: MAD diff (point) | -501.01 | -501.0098 | 0.0002 | YES |
| Bootstrap: MAD diff CI low | -563.32 | -562.4786 | 0.8414 | ~approx (bootstrap) |
| Bootstrap: MAD diff CI high | -425.78 | -420.0861 | 5.6939 | ~approx (bootstrap) |
| Bootstrap: PM diff (point) | 61.98 | 61.9789 | 0.0011 | YES |
| Bootstrap: PM diff CI low | 57.85 | 57.8335 | 0.0165 | ~approx (bootstrap) |
| Bootstrap: PM diff CI high | 66.5 | 66.6752 | 0.1752 | ~approx (bootstrap) |

## Bootstrap Note

Bootstrap CI endpoints differ between implementations because Python's
`numpy.random.default_rng` and PowerShell's `[System.Random]` produce different
pseudorandom sequences even with the same seed. The point estimates (which do not
depend on the RNG) should agree exactly. CI endpoint differences of < 15 days for
lag measures and < 2 percentage points for proportions are expected and do not
indicate an error in either implementation.



VALIDATION VERDICT: PASS
