# Verification Guide

This package supports two different verification tasks:

1. `Hash verification` answers: "Is this preserved file byte-identical to the
   archived file?"
2. `Numeric verification` answers: "Does a regenerated result reproduce the
   reported numerical quantities within an acceptable floating-point tolerance?"

They are not interchangeable.

## 1. Hash verification

Use SHA-256 hashes when checking files that are meant to be preserved exactly:

- raw input snapshots in `data/raw/`
- frozen scripts in `code/`
- frozen protocol files in `protocol/`
- preserved figures, tables, and result files in `results/`

Commands:

```bash
shasum -a 256 -c manifests/SHA256SUMS.txt
```

Hash verification is strict. A single byte difference changes the digest. This
is the correct check for archival integrity and editorial audit.

## 2. Numeric verification

Use numeric verification when you regenerate outputs from code. Recreated CSV or
JSON files may differ in byte representation because of:

- floating-point formatting,
- library-version differences,
- column-order or metadata formatting changes,
- newline conventions.

Those differences can change file hashes even when the substantive results are
the same to machine precision.

Recommended workflow:

```bash
python code/frozen/run_study1.py --output results/reproduced_study1
python code/run_study2_portable.py --data-dir data/raw --output results/reproduced_study2
python code/verify_numeric_results.py \
  --study1-dir results/reproduced_study1 \
  --study2-dir results/reproduced_study2
```

The numeric verifier checks selected frozen outputs column by column and uses
exact comparison for nonnumeric fields plus `numpy.allclose` for numeric fields.
Default tolerances are `rtol=1e-8` and `atol=1e-10`.

## Which check to report

- Report `hash verification` when stating that the archived package files match
  the deposited package.
- Report `numeric verification` when stating that rerun analyses reproduce the
  paper's results.

For manuscript exhibits, use `results/paper_results_crosswalk.csv` to identify
which result files underpin each figure and table.
