Discover Artificial Intelligence — audited supplementary materials

Study window: 2020 Q1–2025 Q1 (21 quarterly FDA AEMS/FAERS extracts).

Outcome: report-level presence of OUTC code DE versus reports with another recorded OUTC outcome. Reports without an OUTC record are excluded. This is retrospective report classification, not prospective patient-level prediction, incidence estimation, or causal drug assessment.

Date-restricted cohort counts used in the audited manuscript: 4,534,679 report-level records; 667,062 with a recorded DE outcome; 3,867,617 with other recorded outcomes. The counts exclude retained records whose FDA receipt dates fell outside 2020-01-01 to 2025-03-31.

Model sample: 250,000 fit reports; 75,000 calibration reports; 150,000 test reports. The supplied model_sample.csv and reproduce_faers_models.py reproduce the model-stage metrics. We reran the script in this environment; metrics agree with the manuscript to rounding precision. The main model_metrics.csv contains the threshold-tuned metrics reported in the manuscript; thresholds were selected on the calibration set by maximizing F1.

Primary results (HistGradientBoosting): ROC-AUC 0.722; average precision 0.343; precision 0.304; recall 0.469; F1 0.369; balanced accuracy 0.645; Brier score 0.110. Logistic regression: ROC-AUC 0.657; average precision 0.247; precision 0.229; recall 0.528; F1 0.320; balanced accuracy 0.617; Brier score 0.118.

TreeSHAP was calculated on a 1,000-report test subsample. Categorical variables were ordinal-encoded in the gradient-boosting model, which can impose artificial ordering. Same-report predictors were not verified to be available before the outcome. SHAP values are model explanations, not causal effects or clinically validated risk factors.

Reproducibility boundary: model-stage results reproduce from the supplied sample and code. The full upstream extraction, deletion handling, and cohort-construction pipeline has not been independently rerun as part of this audit. The counts above are reconciled to the study-window note and audited manuscript, but full upstream data-cleaning reproducibility remains to be confirmed before making a claim of end-to-end reproducibility.

Official FDA quarterly ASCII source: https://fis.fda.gov/extensions/FPD-QDE-FAERS/FPD-QDE-FAERS.html
