RPP73 and RPP154 criterion clarification
5 October 2026

Summary

The 105-source analysis remains the conservative baseline (48 positive,
57 negative). This is retention of the existing analysis, not a new claim
that these exclusions were prospectively specified. RPP73 can be added as
positive in an explicitly named 106-source selected-contrast sensitivity.
RPP154 should retain separate numerical, interval-rule and author-assessment
axes. Neither unresolved status is equivalent to a negative result.

RPP73

The archived analysis script selects the Staunton community. Its public
workbook contains 25 Democratic and 95 Republican stickers, tested against
voter proportions .30 and .70. Independent calculation gives n = 120,
Pearson chi-square(1) = 4.801587301587 and p = .028433528914. The effect is
in the predicted direction: Republican stickers exceed their expected share
by .091667. Signed phi is +.200033.

The frozen p = .028459737 follows from the rounded reported chi-square of
4.80. The unrounded Pearson p should not be called an exact-test p-value;
the exact two-sided binomial p is .028312275493. These distinctions do not
change the nominal classification.

The report preserved in the actual May 12, 2013 registration names
Charlottesville and Staunton first, with Albemarle and Waynesboro as secondary
locations. Both initially named communities show the predicted significant
pattern. Of all four sites, three show that pattern and Albemarle reverses it.
The final report retains all 437 observations and openly discusses that
reverse-direction result. The initial/secondary distinction does not itself
specify an aggregation rule for a single source-wide outcome.

The Albemarle and Charlottesville voter percentages in the report sum to 98.
Their printed statistics are closely reproduced with unnormalized expected
counts. Normalizing the two-category probabilities gives Pearson p = .039824
for Albemarle and .00008835 for Charlottesville, preserving their direction
and nominal significance. Staunton's probabilities already sum to 100 percent.

The scientifically supported axes are therefore:
- Archived selected Staunton contrast: positive
- Conjunction of the two initially named sites: positive as a scoped sensitivity
- Conjunction of all four sites: not met as a different scoped sensitivity
- Unique source-wide rule: unresolved
- Authors' broad assessment: positive

Definition for the 106-source sensitivity

Add RPP73 as the archived selected Staunton contrast, evaluated by
predicted-direction nominal Pearson chi-square significance against local
Democratic/Republican voter proportions .30/.70. Preserve the source-wide
aggregation uncertainty. This produces 49 positive and 57 negative sources.
It is an explicit sensitivity convention, not a newly established prospective
inclusion rule or a universal replication-success conclusion.

If the manuscript describes its outcome as the archived selected contrast,
Staunton meets that description. If it intends a uniquely nominated
source-wide conclusion, the stricter condition needs consistent application
across the full inventory. This distinction should be explicit in the Methods.

The original article's full Study 3 text was not independently recovered in
this review. Its identity and abstract were verified; the actual registered
wiki quotes the original p. 879 contrast, and the protocol, replication report,
count workbook and selected-site audit script were inspected. Do not describe
the entire original article as fully checked.

RPP73 sources
Registered protocol: https://osf.io/download/n6m5d/
Registration: https://osf.io/hctz9/
Report: https://osf.io/download/2jwi6/
Counts: https://osf.io/download/ftzbj/
Archived selected analysis: https://osf.io/download/hau4p/

RPP154

The named primary target is the average association between aggregate
self-report NEO-PI-R conscientiousness and five country-level validity
criteria. The original -.43 and replication -.11 are self-report averages.
Perceptions of national character (PNC) are a separate construct.

The original article's Table 1 and footnote 4 identify Fisher r-to-z-to-r
averaging. Its public data reproduce the original self-report mean
r = -.426107, rounding to -.43. The original does not report a primary
significance test or confidence interval for this average. The original
author's publicly archived note also describes uncertainty about a formal
replication criterion for these average correlations.

Historical criterion conflict

Wiki versions 3 and 4, dated September 29, 2014, define success through a
partial estimate lying within the original interval OR the two intervals
overlapping. This wording is preserved in the October 31, 2014 registration.
It also incorrectly describes the original -.43 as a partial correlation;
the original article uses -.43 for the unadjusted average.

The January 2015 report instead requires BOTH the average and partial average
to lie within the original intervals. The file labelled pre-data collection
uses the same joint criterion but was uploaded February 13, 2015, after the
final report. Its filename does not establish prospective precedence.
The October registration described safeguarding files before replacing a
Data folder; it does not by itself prove that the criterion preceded data
inspection. The presently accessible Google Doc includes results and cannot
establish what its contents were in 2014.

Numerical recovery

The Materials folder contains a transformed final country dataset and a
calculation workbook that reproduce the report's primary result:
- Self-report unadjusted mean r = -.106295597428
- Self-report GDP-adjusted mean r = -.212126784792
- 16 available self-report countries and 9 listwise-complete countries

These values agree with the report's -.11 and Table 2's -.21 at its reported
two-decimal precision. The workbook's rounded-input cached means are -.106062
and -.212029; exact numerical identity is not claimed. The report prose's
-.26 is not reproduced for the replication. No conclusion about how the
discrepancy arose is made.

The reported partials use a pairwise-deletion correlation matrix. Different
component correlations can have different country subsets. This differs from
residualizing both variables on GDP within their shared complete cases. That
alternative gives an average partial r = -.205289 and is retained separately.

The earlier dataset referenced by the public audit script gives unadjusted
back-transformed r = -.227315 instead of -.11. It contains older values and
17 available self-report countries. The script averages Fisher z values
without an explicit back-transform and states that the p-value is not
applicable. Dataset and estimand versions should therefore remain distinct.

Criterion-specific results

Using the reference intervals printed in the replication report:
- Unadjusted mean in [-.52, -.33]: not met
- Adjusted mean in [-.58, -.02]: met
- Later report's joint AND rule: not met
- Earlier OR rule, interpreted using the matching partial interval: met
- Earlier wording read literally against the interval around -.43: the
  point-in-interval component is not met, and the full OR rule remains
  unresolved without a specified replication-interval construction
- Unique historically justified native binary: unresolved
- Primary nominal p-value: not applicable
- Authors' broad narrative assessment: positive

These reference intervals were supplied in the replication report, rather
than in the original article. A dependency-aware construction for the mean
of overlapping correlations is not documented in the inspected sources.
Manufacturing an ordinary correlation p-value for the average would not
recover a missing primary test. The frozen p = .690466537 should not be
described as a reported primary significance test.

Using the earliest dated wording is a possible explicit analytic hierarchy,
but it still requires resolving its mislabeled estimand and interval method.
Using the later clearer conjunction is a valid final-report-rule sensitivity,
but does not make it the uniquely documented prospective criterion. The
authors' positive conclusion belongs on a separate conceptual-assessment axis.

RPP154 sources
Original article: https://doi.org/10.1111/j.1467-9280.2008.02085.x
Original public data: https://www2.psych.ubc.ca/~heine/personalitydata.xls
Dated criterion: https://api.osf.io/v2/wikis/rtsgj/versions/4/content/
Registration: https://osf.io/ufaxj/
Final report: https://osf.io/download/g4hn3/
Report-matching data: https://osf.io/download/kbwcf/
Calculation workbook: https://osf.io/download/qkb4t/
Original author's archived note: https://api.osf.io/v2/wikis/4h8fw/content/

Integration and uncertainty

Retain 105 as the conservative baseline and report the 106-source Staunton
sensitivity separately. Keep RPP154 outside the nominal-significance cohort.
If a heterogeneous native-criterion sensitivity is reported, distinguish
RPP154 negative under the later AND rule from positive under the matching
partial interpretation of the earlier OR rule. With Staunton included, these
would produce 49/58 or 50/57 among 107 sources. They are alternative analytic
conventions, not consensus outcome labels.

RPP19 remains outside this clarification. Worst-case sensitivity for all three
unresolved source-wide outcomes can enumerate all eight assignments without
selecting a convenient assignment as the primary result. Numerical recovery
does not remove criterion uncertainty, and uncertainty does not imply failure.
