# Third-round corrections

Generated 2026-07-31 05:58.

## Citation selection flow
| Step | Machine | Controls |
|---|---|---|
| Machine-attributed records with usable DOI and dates | 9142 |  |
| Successfully paired 1:1 (without replacement) | 8594 | 8594 |
| Could not be paired | 548 |  |
| Duplicate retrieved records removed | 143 |  |
| Pairs with BOTH members retrieved in OpenAlex (analysed) | 8448 | 8448 |

Duplicate retrieved records removed: 143. Controls were drawn without replacement, so each control appears at most once.

## Covariate balance (standardised mean differences)
| Covariate | Machine mean | Control mean | SMD |
|---|---|---|---|
| Pre-retraction citations | 6 | 8.93 | -0.152 |
| Publication year | 2021.54 | 2021.51 | 0.019 |
| Retraction year | 2023.27 | 2023.27 | 0 |
| Exposure years | 2.73 | 2.73 | 0 |
| Publisher matched exactly (proportion) | 0.6 | 0.6 | 0 |

## Paired citation results (n = 8448 pairs)
| Measure | Machine | Controls | Difference (95% CI) | Test |
|---|---|---|---|---|
| Total citations, median (IQR) | 4 (2-7) | 5 (2-11) | | |
| Post-retraction citations per article-year, median (IQR) | 0.333 (0-0.667) | 0.333 (0-1) | | z = -16.435, p < 1e-15, delta -0.1413 |
| Post-retraction citations per article-year, mean | 0.758 | 1.181 | | |
| Cited in any post-retraction calendar year | 55.17% | 64.67% | -9.49 pp (-10.96 to -8.02) | z = -12.59, p < 1e-15 |
| Cited in the first post-retraction calendar year | 46.41% (n = 8407) | 55.86% (n = 8407) | -9.44 pp (-10.95 to -7.94) | p < 1e-15 |
| Cited in the first two post-retraction calendar years | 59.9% (n = 7259) | 65.39% (n = 7259) | -5.5 pp (-7.07 to -3.93) | p 7.7e-12 |
| Cited in the first three post-retraction calendar years | 63.99% (n = 5854) | 67.05% (n = 5854) | -3.06 pp (-4.78 to -1.34) | p 5.0e-4 |
| Flagged as retracted in OpenAlex | 98.39% | 97.86% | 0.53 pp (0.12 to 0.94) | z = 2.55, p 0.011 |

## Notice-level sensitivity (replacing the arbitrary ten-largest-notices analysis)
| Analysis | n | Median | Mean abs dev | Location delta | Location p | Dispersion ratio | Dispersion p | Paper mill % |
|---|---|---|---|---|---|---|---|---|
| Full machine cohort | 9161 | 491 | 267.5 | 0.0545 | < 1e-15 | 0.348 | < 1e-15 | 71.12 |
| Excluding notice A (657 articles) | 8504 | 478 | 225.3 | 0.0278 | 3.5e-5 | 0.293 | < 1e-15 | 68.93 |
| Excluding notice B (379 articles) | 8782 | 489 | 259.1 | 0.0492 | 1.1e-13 | 0.337 | < 1e-15 | 69.87 |
| Excluding both multi-article notices | 8125 | 475 | 214.1 | 0.0207 | 0.002 | 0.279 | < 1e-15 | 67.48 |
| Notice level, median lag per notice | 8127 | 475 | 214.1 | 0.0351 | 3.4e-7 | 0.279 | < 1e-15 |  |

## Mantel-Haenszel risk ratio with Greenland-Robins interval
Adjusted risk ratio 1.431 (95% CI 1.385 to 1.478).
Strata contributing to the pooled estimate: 23. Strata with all four cells non-zero, and therefore contributing to the Cochran Q test: 8, giving 7 degrees of freedom.

The Cochran Q test uses only strata in which all four cells are non-zero, because the stratum-specific log odds ratio and its variance are otherwise undefined. This is why the degrees of freedom are fewer than the number of strata contributing to the pooled estimate.

