ONLINE RESOURCE 3: ANALYSIS DATA AND CODE
16 November 2026

This package contains de-identified analytic extracts, code and results for
new retrospective checks of the existing master records. It includes final
coder agreement, comparisons using each coder's own codes on the same events,
participant-level sensitivity analyses, and a verified worked example. These
are additional analyses of existing codes, not new coding. The package does
not contain the original video or interview recordings and does not establish
independent breach-detection accuracy or validate the proposed mechanism.

QUICK REPRODUCTION

Extract the archive and open a terminal in its top-level directory.
The checked environment was Python 3.14.6 and numpy 2.4.6. Python and numpy are
the only requirements for reproducing the analyses from the supplied CSVs.

python -m pip install -r requirements.txt
python code/reproduce.py

The second command runs both analysis scripts, recreates the coder matching
and exclusions, and checks all 14 expected output files byte for byte. It also
checks that the two main extracts contain identical IDs and original codes.
New outputs are written under reproduced/; existing expected results are not
overwritten. See reproduced/verification.json for the checks. Original
workbook access is not needed. The physical empty-row counts come from the
source metadata; all actual record-level exclusions are recalculated.

The analyses can also be run separately:

python code/coder_reanalysis.py --keys data/master_matching_keys.csv \
  --observers data/all_observer_records.csv \
  --flow data/matching_flow.json --out reproduced/coder

python code/participant_sensitivity.py --input data/publicmain_records.csv \
  --out reproduced/participant

Optional original-workbook modes in the scripts require openpyxl; the version
used was 3.1.5. See requirements-workbook-optional.txt. Private linkage files
and the original workbook are intentionally not included. SOURCE_PROVENANCE.json
gives the workbook filename, SHA-256 digest and source/attendance provenance.

FILES

data/master_matching_keys.csv: 3212 timestamped AO records; anonymous event,
participant and clip IDs; exact elapsed timestamps; original seven initial
flags and decision. The master worksheet row is included for provenance.
data/all_observer_records.csv: all 328 nonempty FH and 341 nonempty NL records,
including records later excluded. Source row, anonymous key and original
flags/decision are retained.
data/matching_flow.json: source and exclusion metadata. Wholly empty worksheet
formatting rows are reported separately from actual unmatched records.
data/publicmain_records.csv: the same 3212 main events with all original flags
and decision, plus verified RIW/RRIW attendance indicators.
code/: both analysis scripts and the portable reproduction/checking script.
expected_results/coder/: complete results, eligible paired records, exclusions,
binary contingency tables, overall agreement, same-event contrasts, individual
marker decision rates, and participant counts.
expected_results/participant/: complete results, individual participant rates
and all leave-one-participant-out results for the full and RRIW-only samples.
examples/: the verified worked example E0028/S001 at 03:44, in TXT/CSV/JSON.
checks/: reproduction and independent source-check summaries.
MANIFEST_SHA256.csv: checksums for package files, excluding the manifest itself.

DATA DICTIONARY AND IDENTIFIERS

S001-S060 are assigned by first appearance among timestamped master records;
E0001-E3212 follow the same event order. These analytical IDs are not the
participant numbers in the thesis. Clip IDs C001 onward follow first master
appearance; XC denotes a clip label present only in an observer source. No raw
participant or clip names, demographics, or private crosswalks are included.
The worked example explicitly links its thesis illustration identifier to its
analytical identifier; it contains a published thesis excerpt, not new audio
transcription. Original footage is not included.

Pause, Hesitation, Retry, Give up and Scan are the five initial markers. IDK and
None are the two auxiliary original flags. A 1 means the source flag is set;
0 means it is unset. These values are used without cognitive reinterpretation.
The main IDK value for E1393 is blank and stays blank. All five substantive
main flags and main Decision are complete and binary. Observer '-' cells are
preserved as source placeholders and fail the complete-binary-code criterion;
they are not imputed as zero. Decision is the existing final binary decision
code. RIW_completed and RRIW_completed are verified binary attendance flags.
All 60 participants completed RIW; 51 completed RRIW, and 9 did not.

Elapsed time is expressed as integer microseconds and, for readability, seconds.
Matching uses participant, clip and exact integer elapsed microseconds. Excel
time-of-day and timedelta storage are normalized to elapsed units without
rounding or a matching tolerance. One FH record stored as a timedelta thereby
matches its equivalent master time value. No source cell is changed.

SEQUENTIAL CODER MATCHING AND EXCLUSIONS

FH has 534 physical rows after the header, of which 206 are entirely empty
formatting rows and 328 are actual records. Among the 328, exclude 6 without
an exact master match, then 24 with incomplete binary initial flags in either
coder, then 7 with an all-zero seven-flag vector in either coder. None then
fails the binary-decision criterion. The final sample is 291 breaches from
9 participants.

NL has 341 actual records and no empty rows. Exclude 15 unmatched, then 23
incomplete binary-flag records, then 3 all-zero-vector records. None then fails
the decision criterion. The final sample is 300 breaches from 8 participants.

The samples share 106 breaches and 3 participants: their union is 485 unique
breaches from 14 participants. They are not independent replications. An
unmatched key does not necessarily mean a missed breach; an all-zero vector
does not establish confirmed absence of behaviour. Exclusion reasons are
sequential, so each excluded record is counted once at its first failed check.

ANALYSIS DEFINITIONS

PHS-exclusive: at least one initial Pause/Hesitation/Scan and no Retry/Give up.
RG-exclusive: at least one initial Retry/Give up and no Pause/Hesitation/Scan.
Remaining events include mixed and neither cases. They are kept and reported
separately; participant analyses combine them in Other. Each observer's
same-event analysis uses that observer's own initial flags and decision code.
Group membership can therefore change across coders within a fixed cohort.
Individual-marker rates count every event with that flag; marker sets may
overlap and their denominators should not be added.

Binary agreement tables include both marginals and the four cells ordered
AO0/observer0, AO0/observer1, AO1/observer0, AO1/observer1. Positive agreement
is 2a/(2a+b+c) and negative agreement is 2d/(2d+b+c), where a is both present
(11), d is both absent (00), b is AO absent/observer present (01), and c is
AO present/observer absent (10). Cohen's kappa adjusts
observed agreement for the two marginal distributions. Full-vector agreement
treats each distinct seven-bit vector as one nominal category; joint agreement
also requires the same decision code. Agreement refers to retained final
post-replay codes and should not be interpreted as video-only detection
validity or a test against a gold standard.

Participant analyses report pooled event rates, unweighted means of participant
rates, and the mean within-participant contrast among those with both groups.
Participants who have no events in a group do not receive a zero for that
group. Group-specific unweighted means can involve different participant sets;
the paired contrast is restricted to a common set. Full-data and RRIW-only
analyses are both provided. The RRIW restriction is a descriptive sensitivity
check, not a randomized comparison of interview procedures.

UNCERTAINTY AND LIMITS

Both scripts use 10000 replicates and seed 20261116. Pooled-rate and coder
analyses resample participants with replacement and retain all their events.
Coder comparisons use the same participant multiplicities for AO and observer
codes. The paired participant contrast resamples the participants contributing
both groups equally. Intervals are pointwise 2.5th-97.5th percentile intervals.
The scripts specify their random stream ordering; separate analysis streams
should not be interpreted as independent empirical replications.

Undefined estimates are excluded from interval quantile calculations and
their counts are reported. Empty groups can yield undefined rates; rare codes
can yield undefined kappa or positive agreement. A [0,0] or [1,1] bootstrap
interval reflects the support observed in this sample, not population
certainty. Coder intervals are exploratory because only 9 or 8 participant
clusters contribute. These checks were not preregistered confirmatory tests.

Report weak as well as stronger results. IDK and None have low positive
agreement despite high overall agreement. The grouped PHS-versus-RG contrast
has the same direction for AO, FH and NL on the matched samples, but effect
sizes differ. Give up is rare and often has a reported decision in these
subsets, so Retry largely drives their low grouped RG rate. Neither grouping
nor resampling removes common-source measurement, uncertainty in retrospective
reports, selection of matched events, or possible dependence between coding
and decision assessment.
