%Version 3.1 December 2024
% See section 11 of the User Manual for version history
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%                                                                 %%
%% Please do not use \input{...} to include other tex files.       %%
%% Submit your LaTeX manuscript as one .tex document.              %%
%%                                                                 %%
%% All additional figures and files should be attached             %%
%% separately and not embedded in the \TeX\ document itself.       %%
%%                                                                 %%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%\documentclass[referee,sn-basic]{sn-jnl}% referee option is meant for double line spacing

%%=======================================================%%
%% to print line numbers in the margin use lineno option %%
%%=======================================================%%

%%\\documentclass[lineno,pdflatex,sn-basic]{sn-jnl}% Basic Springer Nature Reference Style/Chemistry Reference Style
%%\documentclass[iicol]{sn-jnl} % Emily added this line based on the JCSS instructions, "We recommend using Springer Nature’s LaTeX template and choosing the formatting option “[iicol]”, but then she commented the line out because other JCSS papers she looked at used a single column. Tommy: I'm curious what Lisa did.

%%=========================================================================================%%
%% the documentclass is set to pdflatex as default. You can delete it if not appropriate.  %%
%%=========================================================================================%%

%%\documentclass[sn-basic]{sn-jnl}% Basic Springer Nature Reference Style/Chemistry Reference Style

%%Note: the following reference styles support Namedate and Numbered referencing. By default the style follows the most common style. To switch between the options you can add or remove Numbered in the optional parenthesis. 
%%The option is available for: sn-basic.bst, sn-chicago.bst%  
 
%\documentclass[pdflatex,sn-nature]{sn-jnl}% Style for submissions to Nature Portfolio journals 
\documentclass[pdflatex,sn-basic,Numbered]{sn-jnl}

%%\documentclass[pdflatex,sn-basic]{sn-jnl}% Basic Springer Nature Reference Style/Chemistry Reference Style
%%\documentclass[pdflatex,sn-mathphys-num]{sn-jnl}% Math and Physical Sciences Numbered Reference Style
%\documentclass[pdflatex,sn-mathphys-ay]{sn-jnl}% Math and Physical Sciences Author Year Reference Style
%%\documentclass[pdflatex,sn-aps]{sn-jnl}% American Physical Society (APS) Reference Style
%%\documentclass[pdflatex,sn-vancouver-num]{sn-jnl}% Vancouver Numbered Reference Style
%%\documentclass[pdflatex,sn-vancouver-ay]{sn-jnl}% Vancouver Author Year Reference Style
%%\documentclass[pdflatex,sn-apa]{sn-jnl}% APA Reference Style
%%\documentclass[pdflatex,sn-chicago]{sn-jnl}% Chicago-based Humanities Reference Style

%\documentclass[pdflatex,sn-chicago,Numbered]{sn-jnl}% Chicago-based Humanities Reference Style
% Emily's note: I added "Numbered" to follow the instructions to put numbers in brackets for citations. It seems the journal actually wants APA style but the APA file they provided is not compatible with the numbers in brackets.



%%%% Standard Packages
%%<additional latex packages if required can be included here>

\usepackage{graphicx}%
\usepackage{multirow}%
\usepackage{amsmath,amssymb,amsfonts}%
\usepackage{amsthm}%
\usepackage{mathrsfs}%
\usepackage[title]{appendix}%
\usepackage{xcolor}%
\usepackage{textcomp}%
\usepackage{manyfoot}%
\usepackage{booktabs}%
\usepackage{algorithm}%
\usepackage{algorithmicx}%
\usepackage{algpseudocode}%
\usepackage{listings}%
\usepackage{caption}
\usepackage{longtable}
% I (Tommy) added the following lines here to reproduce what was seen in the template for the figures and the tables. I had to use captionsetup to make longtables look like tables, but for some reason using captionsetup for longtables also returned figure and table captions to default settings, so I reset those as well.
\usepackage[left]{lineno} % for line numbers on left side
\usepackage{placeins}
\usepackage{setspace}
\usepackage{makecell}
\DeclareCaptionFont{captionfont}{\fontsize{8bp}{9.5bp}\selectfont}
\DeclareCaptionLabelSeparator{twospace}{\ \ }
\captionsetup[figure]{font = {captionfont}, justification = raggedright, singlelinecheck = false, labelsep = twospace}
\captionsetup[table]{font = {captionfont}, skip = 10pt, justification = raggedright, singlelinecheck = false, labelsep = twospace}
\captionsetup[longtable]{font = {captionfont}, skip = 10pt, justification = raggedright, singlelinecheck = false, labelsep = twospace}
%%%%

%%%%%=============================================================================%%%%
%%%%  Remarks: This template is provided to aid authors with the preparation
%%%%  of original research articles intended for submission to journals published 
%%%%  by Springer Nature. The guidance has been prepared in partnership with 
%%%%  production teams to conform to Springer Nature technical requirements. 
%%%%  Editorial and presentation requirements differ among journal portfolios and 
%%%%  research disciplines. You may find sections in this template are irrelevant 
%%%%  to your work and are empowered to omit any such section if allowed by the 
%%%%  journal you intend to submit to. The submission guidelines and policies 
%%%%  of the journal take precedence. A detailed User Manual is available in the 
%%%%  template package for technical guidance.
%%%%%=============================================================================%%%%

%% as per the requirement new theorem styles can be included as shown below
\theoremstyle{thmstyleone}%
\newtheorem{theorem}{Theorem}%  meant for continuous numbers
%%\newtheorem{theorem}{Theorem}[section]% meant for sectionwise numbers
%% optional argument [theorem] produces theorem numbering sequence instead of independent numbers for Proposition
\newtheorem{proposition}[theorem]{Proposition}% 
%%\newtheorem{proposition}{Proposition}% to get separate numbers for theorem and proposition etc.

\theoremstyle{thmstyletwo}%
\newtheorem{example}{Example}%
\newtheorem{remark}{Remark}%

\theoremstyle{thmstylethree}%
\newtheorem{definition}{Definition}%


\raggedbottom
%%\unnumbered% uncomment this for unnumbered level heads

\begin{document}

\captionsetup[longtable]{justification=raggedright}

\title[Article Title]{%
  \parbox{\textwidth}{\centering
  You Have More Data Than You Think:\\
  Optimizing Machine Learning in Tabular Social Science Datasets Through Augmentation and Linkage\\
  \vspace{.75cm}
  \textbf{APPENDICES}}} % Emily tweaked formatting here arbitrarily, feel free to change it back if there was a specific requirement by the journal

%%=============================================================%%
%% GivenName	-> \fnm{Joergen W.}
%% Particle	-> \spfx{van der} -> surname prefix
%% FamilyName	-> \sur{Ploeg}
%% Suffix	-> \sfx{IV}
%% \author*[1,2]{\fnm{Joergen W.} \spfx{van der} \sur{Ploeg} 
%%  \sfx{IV}}\email{iauthor@gmail.com}
%%=============================================================%%

\author*[1]{\fnm{Hanzhang} \sur{Ren}}\email{ren1999@stanford.edu}
\equalcont{These authors contributed equally to this work.}

\author[2]{\fnm{Emily M.} \sur{Cantrell}}\email{e.cantrell@princeton.edu}
% Emily's note: I didn't like the semicolon after the second email, but I guess it's okay since the equal contribution line is after that. The formatting changes in the real publication anyway. 

\affil[1]{\orgdiv{Department of Sociology}, \orgname{Stanford University}, \orgaddress{\street{450 Jane Stanford Way, Building 120}, \city{Stanford}, \state{CA} \postcode{94305}, \country{USA}}} % Emily's note: the template asked for a specific street address, but that seems odd and I didn't see a street address in another example paper from the same journal. I also moved the postcode to be after the state and without a comma after the state. Tommy: it does seem like the last page of the pdf for Sivak et al. 2024 has some street addresses, but not all papers in JCSS have them. The submission guideline says "If address information is provided with the affiliation(s) it will also be published." So all this feels optional, feel free to delete!

\affil[2]{\orgdiv{Department of Sociology}, \orgname{Princeton University}, \orgaddress{\street{Wallace Hall}, \city{Princeton}, \state{NJ} \postcode{08544}, \country{USA}}}

\maketitle

\newpage

\tableofcontents

\backmatter

\newpage

\begin{appendices}

\doublespacing
\linenumbers

\section{Methodological details}\label{secA}

\subsection{Missingness}\label{secA1:missingness}

\subsubsection{Missingness in features}

Figure~\ref{figa1:missingness} shows the distribution of missingness rates across features in the winning model. We measured missingness rates after feature engineering but before one-hot encoding. During feature engineering, we inferred values for data that were only missing because of survey logic. For example, we inferred that people were not married if they previously responded ``no'' to a screener on whether they had either a married or unmarried partner. Also, if a respondent indicated that they did not know whether they expected to have a child in the future, we counted the answer “I don’t know” as nonmissing because we considered the response potentially informative. To avoid having too many sparse columns, we recoded some rare categorical responses with under 50 cases as missing if there was not an obvious way to meaningfully combine these categories with other categories.

As noted in the main text, the mean missingness rate for features in the time-shifted data was higher than for features in the original data. The higher missingness rate in the time-shifted data occurred in part because some participants recently joined the LISS panel during sample refreshment. These participants had outcomes available for 2018--2020 but had not yet joined the panel in some, if not all, of the earlier years up to 2017. The original PreFer training set excluded any participants who had not taken a core survey prior to the outcome period (2021--2023) \cite{sivak_combining_2024}. However, during time-shift data augmentation, we included 446 participants without core surveys prior to the time-shifted outcome period (2018--2020). The PreFer data had some background features like birth years retroactively available for these people, and sometimes we obtained additional information by linking in data from household members who did participate.

\begin{figure}[h]
\centering
\includegraphics[width=\textwidth]{Fig_a1.jpg}
\caption{Distribution of missingness rates for the 84 features in the winning model}\label{figa1:missingness}
\end{figure}

\subsubsection{Missingness in the outcome}\label{secA12_missingness_outcome}

Figure~\ref{figa2:outcome_flowchart_original} describes which LISS core survey participants had nonmissing outcomes for the time window between 2021 and 2023. Figure~\ref{figa3:outcome_flowchart_time_shifted} provides a similar description for time-shifted outcomes between 2018 and 2020. The LISS panel contained simple and stratified samples of an administrative register of all addresses and was refreshed over the years to represent residents aged 16 and above in the Netherlands. In 2007, around 48\% of the people contacted for the first wave of LISS actually joined the panel. Panel members attrited at a rate of about 12\% each year \cite{centerdata_methodology_2026}. From panel members who took at least one of LISS's core surveys between 2007 and 2020, the PreFer challenge organizers created a training dataset containing a subsample of people in the correct age range for the original outcome time window (18--45 years old in 2020), as well as a supplementary dataset containing people who were not in the correct age range, as described in Sivak et al. \cite{sivak_combining_2024}. Our time-shifted data consisted of members of the PreFer training dataset and supplementary dataset who were in the correct age range for the time-shifted outcome (18--45 years in 2017), and for whom sufficient data were available to compute the outcome for 2018--2020.

\begin{figure}[h]
\centering
\includegraphics[width=\textwidth]{Fig_a2.jpg}
\caption{Flowchart describing which members of the LISS panel have non-missing outcomes in the original, non-time-shifted data}\label{figa2:outcome_flowchart_original}
\end{figure}

\begin{figure}[h]
\centering
\includegraphics[width=\textwidth]{Fig_a3.jpg}
\caption{Flowchart describing which members of the LISS panel have non-missing outcomes in the time-shifted data}\label{figa3:outcome_flowchart_time_shifted}
\end{figure}

\FloatBarrier

\subsection{Hyperparameter tuning}\label{secA2:hyperparameter_tuning}

% Emily's note: I'm not sure how to link to section 4.5 from a separate document and I'm not sure if that's appropriate anyway. We can decide if linking across documents is appropriate when we're working on the final formatting details after acceptance.  
As stated in Section 4.5 of the main text, we used XGBoost with hyperparameter tuning via grid search. Here, we describe how we determined the values in the grid. We consulted examples and discussions in Hastie, Tibshirani, and Friedman \cite{hastie_elements_2009} to determine which hyperparameters to tune and to establish an initial set of hyperparameter values for the hyperparameter grid. We expanded the grid whenever cross-validation results showed that almost all top performances corresponded to a hyperparameter value on the edge of the grid. We occasionally reduced the number of values in the middle of the grid to keep computation times within practical limits. However, we ensured that default hyperparameter values from \texttt{tidymodels} \cite{kuhn_boosted_2026} always remained part of the grid.


\subsection{Features}\label{secA3:features}

Table~\ref{tablea1:winning_model_features} lists the features we used as inputs for the winning model we submitted in the PreFer challenge. Note that as a result of feature engineering and data preparation steps, some variables we fed into XGBoost were modified from their raw form in the PreFer challenge data files, even when they had the same name. The data processing code is available on our \href{https://github.com/HanzhangRen/fertility-prediction-challenge-2024}{GitHub repository}.


\begin{singlespace}
\tablebodyfont
\begin{longtable}[t]{p{.5cm}p{5cm}p{3cm}p{2cm}p{.5cm}p{5cm}p{3cm}p{2cm}p{.5cm}p{5cm}p{3cm}p{2cm}p{.5cm}p{5cm}p{3cm}p{2cm}}
\caption{Features in the Winning Model\label{tablea1:winning_model_features}}\\
\toprule
\# & Variable name & Description & Source\\
\midrule
\endfirsthead
\caption[]{Features in the Winning Model \textit{(Continued)}}\\
\toprule
\# & Variable name & Description & Source\\
\midrule
\endhead

\endfoot
\bottomrule
\endlastfoot
1 & birthyear\_bg & \raggedright Birth year, based on gebjaar in the LISS Archive & \raggedright Focal person data\\
2 & partner\_birth\_year20 & \raggedright Presumed birthyear of 2020 partner, constructed using partner’s birthyear\_bg and coalescing cf[08a--20m]026 & \raggedright Focal person data and partner data\\
3 & partner\_birth\_year19 & \raggedright Presumed birthyear of 2019 partner, coalescing cf[08a--19l]026 & \raggedright Focal person data\\
4 & partner\_birth\_year18 & \raggedright Presumed birthyear of 2018 partner, coalescing cf[08a--18k]026 & \raggedright Focal person data\\
5 & gender\_bg & \raggedright Gender, based on geslacht in the LISS Archive & \raggedright Focal person data\\
6 & belbezig\_2020 & \raggedright Primary occupation (example values: paid employment, takes care of the housekeeping), based on belbezig in the LISS Archive & \raggedright Focal person data\\
7 & hhinc\_per\_capita20 & \raggedright Household income per capita, constructed based on household income nettohh\_f and household size aantalhh in the LISS Archive.  & \raggedright Focal person data\\
8 & migration\_background\_bg & \raggedright Migration background (example values: western immigrant, non-western immigrant), based on herkomstgroep in the LISS Archive & \raggedright Focal person data\\
9 & nettohh\_f\_2020 & \raggedright Household net monthly income, based on nettoink\_f of household members. & \raggedright Focal person data\\
10 & nettoink\_f\_2020 & \raggedright Personal net monthly income, based on nettoink\_f in the LISS Archive, involves values imputed by LISS  & \raggedright Focal person data\\
11 & oplmet\_2020 & \raggedright Highest level of education with diploma, based on oplmet in the LISS Archive & \raggedright Focal person data\\
12 & sted\_2020 & \raggedright Degree of urban character of place of residence, based on sted in the LISS Archive & \raggedright Focal person data\\
13 & woning\_2020 & \raggedright Type of dwelling (example values: self-owned, rented), based on woning in the LISS Archive & \raggedright Focal person data\\
14 & ca20g012 & \raggedright Total balance in bank accounts, augmented with categorical information from ca20g013 and ca20g078 & \raggedright Focal person data\\
15 & cd20m034 & \raggedright Number of rooms in dwelling & \raggedright Focal person data\\
16 & cf20m024 & \raggedright Have a partner in 2020 & \raggedright Focal person data\\
17 & cf19l024 & \raggedright Have a partner in 2019 & \raggedright Focal person data\\
18 & cf18k024 & \raggedright Have a partner in 2018 & \raggedright Focal person data\\
19 & cf20m025 & \raggedright Cohabiting with partner in 2020 & \raggedright Focal person data\\
20 & cf19l025 & \raggedright Cohabiting with partner in 2019 & \raggedright Focal person data\\
21 & cf18k025 & \raggedright Cohabiting with partner in 2018 & \raggedright Focal person data\\
22 & cf20m029 & \raggedright In what year did you start living together with your partner? 2020 & \raggedright Focal person data\\
23 & cf19l029 & \raggedright In what year did you start living together with your partner? 2019 & \raggedright Focal person data\\
24 & cf18k029 & \raggedright In what year did you start living together with your partner? 2018 & \raggedright Focal person data\\
25 & cf20m030 & \raggedright Are you married to this partner? 2020 & \raggedright Focal person data\\
26 & cf19l030 & \raggedright Are you married to this partner? 2019 & \raggedright Focal person data\\
27 & cf18k030 & \raggedright Are you married to this partner? 2018 & \raggedright Focal person data\\
28 & cf20m031 & \raggedright In what year did you marry? 2020 & \raggedright Focal person data\\
29 & cf19l031 & \raggedright In what year did you marry? 2019 & \raggedright Focal person data\\
30 & cf18k031 & \raggedright In what year did you marry? 2018 & \raggedright Focal person data\\
31 & cf20m128 & \raggedright Do you think you will have (more) children in the future? 2020 & \raggedright Focal person data\\
32 & cf19l128 & \raggedright Do you think you will have (more) children in the future? 2019 & \raggedright Focal person data\\
33 & cf18k128 & \raggedright Do you think you will have (more) children in the future? 2018 & \raggedright Focal person data\\
34 & cf20m128\_PartnerSurvey & \raggedright Partner’s response to “Do you think you will have (more) children in the future?” 2020 & \raggedright Partner data\\
35 & cf19l128\_PartnerSurvey & \raggedright Partner’s response to “Do you think you will have (more) children in the future?” 2019 & \raggedright Partner data\\
36 & cf20m129 & \raggedright How many [more] children do you think you will have in the future? 2020 & \raggedright Focal person data\\
37 & cf19l129 & \raggedright How many [more] children do you think you will have in the future? 2019 & \raggedright Focal person data\\
38 & cf18k129 & \raggedright How many [more] children do you think you will have in the future? 2018 & \raggedright Focal person data\\
39 & cf20m129\_PartnerSurvey & \raggedright Partner’s response to “How many [more] children do you think you will have in the future?” 2020 & \raggedright Partner data\\
40 & cf19l129\_PartnerSurvey & \raggedright Partner’s response to “How many [more] children do you think you will have in the future?” 2019 & \raggedright Partner data\\
41 & cf20m130 & \raggedright Within how many years do you hope to have your (first/next) child? If [you and your partner / you] are currently expecting a child, please enter a zero (0). 2020 & \raggedright Focal person data\\
42 & cf19l130 & \raggedright Within how many years do you hope to have your (first/next) child? If [you and your partner / you] are currently expecting a child, please enter a zero (0). 2019 & \raggedright Focal person data\\
43 & cf18k130 & \raggedright Within how many years do you hope to have your (first/next) child? If [you and your partner / you] are currently expecting a child, please enter a zero (0). 2018 & \raggedright Focal person data\\
44 & cf20m130\_PartnerSurvey & \raggedright Partner’s response to “Within how many years do you hope to have your (first/next) child? If [you and your partner / you] are currently expecting a child, please enter a zero (0).” 2020 & \raggedright Partner data\\
45 & cf19l130\_PartnerSurvey & \raggedright Partner’s response to “Within how many years do you hope to have your (first/next) child? If [you and your partner / you] are currently expecting a child, please enter a zero (0).” 2019 & \raggedright Partner data\\
46 & cf20m166 & \raggedright How satisfied are you with your situation as a single? An 11-point scale & \raggedright Focal person data\\
47 & cf20m180 & \raggedright How satisfied are you with your current relationship? An 11-point scale from 2020 & \raggedright Focal person data\\
48 & cf19l180 & \raggedright How satisfied are you with your current relationship? An 11-point scale from 2019 & \raggedright Focal person data\\
49 & cf20m181 & \raggedright How satisfied are you with your family life? An 11-point scale from 2020 & \raggedright Focal person data\\
50 & cf19l181 & \raggedright How satisfied are you with your family life? An 11-point scale from 2019 & \raggedright Focal person data\\
51 & cf20m454 & \raggedright Did you ever have children? 2020 & \raggedright Focal person data\\
52 & cf19l454 & \raggedright Did you ever have children? 2019 & \raggedright Focal person data\\
53 & cf18k454 & \raggedright Did you ever have children? 2018 & \raggedright Focal person data\\
54 & cf20m454\_PartnerSurvey & \raggedright Partner’s response to “Did you ever have children?” 2020 & \raggedright Partner data\\
55 & cf19l454\_PartnerSurvey & \raggedright Partner’s response to “Did you ever have children?” 2019 & \raggedright Partner data\\
56 & cf20m455 & \raggedright Number of living children in 2020 & \raggedright Focal person data\\
57 & cf19l455 & \raggedright Number of living children in 2019 & \raggedright Focal person data\\
58 & cf18k455 & \raggedright Number of living children in 2018 & \raggedright Focal person data\\
59 & cf20m455\_PartnerSurvey & \raggedright Partner’s number of living children in 2020 & \raggedright Partner data\\
60 & cf19l455\_PartnerSurvey & \raggedright Partner’s number of living children in 2019 & \raggedright Partner data\\
61 & cf20m456 & \raggedright Birth year of first child & \raggedright Focal person data\\
62 & cf20m456\_PartnerSurvey & \raggedright Birth year of partner’s first child & \raggedright Partner data\\
63 & year\_relationship\_began20 & \raggedright Coalescing cf[08a--20m]028 & \raggedright Focal person data\\
64 & year\_relationship\_began19 & \raggedright Coalescing cf[08a--19l]028 & \raggedright Focal person data\\
65 & year\_relationship\_began18 & \raggedright Coalescing cf[08a--18k]028 & \raggedright Focal person data\\
66 & most\_recent\_child20 & \raggedright Coalescing birth years from cf20m[456--470] & \raggedright Focal person data\\
67 & most\_recent\_child19 & \raggedright Coalescing birth years from cf19l[456--470] & \raggedright Focal person data\\
68 & most\_recent\_child18 & \raggedright Coalescing birth years from cf18k[456--470] & \raggedright Focal person data\\
69 & most\_recent\_child20\_PartnerSurvey & \raggedright Coalescing birth years from cf20m[456--470] for partner & \raggedright Partner data\\
70 & most\_recent\_child19\_PartnerSurvey & \raggedright Coalescing birth years from cf19l[456--470] for partner & \raggedright Partner data\\
71 & child\_feeling & \raggedright Warm feelings toward one’s oldest child on a 7-point scale based on cf20m[513--521]. Example item: I am very fond of my [son/daughter]. & \raggedright Focal person data\\
72 & ch20m004 & \raggedright Self health description on a five-point scale. & \raggedright Focal person data\\
73 & ch20m219 & \raggedright Had contact with gynecologist over the last 12 months & \raggedright Focal person data\\
74 & ch20m219\_PartnerSurvey & \raggedright Partner had contact with gynecologist over the last 12 months & \raggedright Partner data\\
75 & cr20m162 & \raggedright Religiosity on a 4-point scale & \raggedright Focal person data\\
76 & gendered\_religiosity & \raggedright Measured in 2018, a 3-point scale based on cr18k[101--105]. Example item: Does a wife have to do what her husband asks of her? & \raggedright Focal person data\\
77 & traditional\_motherhood & \raggedright Level of belief in traditional views on motherhood, a 5-point scale based on cv20l[109--111]. Example item: A working mother’s relationship with her children can be just as close and warm as that of a non-working mother & \raggedright Focal person data\\
78 & traditional\_fatherhood & \raggedright Level of belief in traditional views on fatherhood, a 5-point scale based on cv20l[112--115]. Example item: Both father and mother should contribute to the family income. & \raggedright Focal person data\\
79 & traditional\_marriage & \raggedright Level of belief in traditional views on marriage, a 5-point scale based on cv20m[124--130]. Example item: Married people are generally happier than unmarried people. & \raggedright Focal person data\\
80 & traditional\_fertility & \raggedright Level of beliefs in traditional views on fertility. A 5-point scale based on cv10c[135--138]. Measured in 2010. Example item: Your life is not complete if you haven’t had children. & \raggedright Focal person data\\
81 & working\_mother & \raggedright Level of approval towards mothers working on a 3-point scale, based on cv20l[143--146]. Example item: Do you think that women, under the circumstances described below, should be able to have a full-time job, a part-time job, or no job at all? If she has a baby (a child younger than 1 year). & \raggedright Focal person data\\
82 & sexism & \raggedright Level of sexist beliefs, a 5-point scale based on cv20l[151--154]. Example item: A woman is more suited to rearing young children than a man. & \raggedright Focal person data\\
83 & time\_shifted\_data & \raggedright Indicator variable of time shift data augmentation & \raggedright\\
84 & partner\_survey\_available & \raggedright Indicator variable of partner linkage & \raggedright\\*
\end{longtable}
\end{singlespace}

\normalsize

\subsection{Pooled performance measure}\label{secA4:hyperparameter_tuning}

As stated in Section 4.7.3 of the main text, we are reserving the holdout set to evaluate performance at the end of the journal review process, so that we maintain a clean holdout set in case reviewer feedback alters any of our methodological decisions. Because of the small sample size of the holdout set ($n$ = 395), we may pool the holdout and cross-validation results together in the main paper for more precise performance estimates, provided that the holdout set results are consistent with the cross-validated test set results in terms of the impact of time-shift augmentation and partner linkage. We will use the following formula to aggregate cross-validated and holdout performance estimates.

\begin{equation}
R^2_{\text{CV, Holdout}} = 1 - 
\frac{\sum_{i \in \text{All Folds}} (y_i - \hat{y}_i)^2 + \sum_{i \in \text{Holdout}} (y_i - \hat{y}_i)^2}
{\sum_{i \in \text{All Folds}} (y_i - \bar{y}_{\text{Training}_j})^2 + \sum_{i \in \text{Holdout}} (y_i - \bar{y}_{\text{Training}})^2}
\end{equation}

{\fontsize{8}{9.5}\selectfont
\noindent\textbf{Eq. A1}\ \ Formula for $R^2$ computed jointly on the cross-validation folds and the holdout set
}


%%=============================================%%
%% For submissions to Nature Portfolio Journals %%
%% please use the heading ``Extended Data''.   %%
%%=============================================%%

%%=============================================================%%
%% Sample for another appendix section			       %%
%%=============================================================%%


\section{Model performance with other metrics}\label{secB:other_metrics}

The tables below display model performance with a variety of metrics.

\setcounter{table}{0} % This makes it start at B1 instead of B2

\tablebodyfont

\begin{sidewaystable}

\caption{Performance of Tuned XGBoost Models}
\centering
\begin{tabular}[t]{lcccccccc}
\toprule
  \parbox{1.5cm}{\centering Metric} & \parbox{1.5cm}{\centering Original} & \parbox{1.5cm}{\centering Original \\ 95\% CI} & \parbox{1.5cm}{\centering Partner} & \parbox{1.5cm}{\centering Partner \\ 95\% CI} & \parbox{1.5cm}{\centering Time shift} & \parbox{1.5cm}{\centering Time shift \\ 95\% CI} & \parbox{1.5cm}{\centering Partner + \\ time shift} & \parbox{1.5cm}{\centering Partner + \\ time shift \\ 95\% CI}\\
\midrule
$R^2_\text{CV}$ & .543 & {}[.477, .608] & .557 & {}[.484, .629] & .581 & {}[.512, .648] & \textbf{.591} & \textbf{[.528, .655]}\\
$R^2_\text{Traditional}$ & .542 & {}[.475, .607] & .556 & {}[.482, .628] & .579 & {}[.510, .647] & \textbf{.590} & \textbf{[.526, .653]}\\
AUC & .912 & {}[.886, .937] & .926 & {}[.903, .948] & .935 & {}[.914, .953] & \textbf{.937} & \textbf{[.916, .955]}\\
Log loss & .270 & {}[.234, .307] & .269 & {}[.221, .320] & .247 & {}[.207, .291] & \textbf{.241} & \textbf{[.206, .275]}\\
MSE & .077 & {}[.065, .090] & .075 & {}[.062, .089] & .071 & {}[.058, .084] & \textbf{.069} & \textbf{[.057, .081]}\\
F1 & .773 & {}[.724, .819] & .784 & {}[.736, .828] & .798 & {}[.755, .841] & \textbf{.803} & \textbf{[.759, .846]}\\
Precision & .820 & {}[.766, .874] & .851 & {}[.796, .902] & .847 & {}[.792, .898] & \textbf{.864} & \textbf{[.814, .914]}\\
Recall & .731 & {}[.670, .793] & .726 & {}[.665, .788] & \textbf{.755} & \textbf{[.697, .815]} & .750 & {}[.689, .809]\\
Accuracy & .908 & {}[.889, .926] & .914 & {}[.896, .931] & .918 & {}[.901, .935] & \textbf{.921} & \textbf{[.904, .938]}\\
\bottomrule
\multicolumn{9}{l}{\makecell[l]{%
Note: The best performance of each row is in bold.\\
F1, precision, recall, and accuracy were computed by classifying predicted probabilities of 0.5 or greater as positive.
}}
\end{tabular}
\end{sidewaystable}




\begin{sidewaystable}

\caption{Performance of engineered data \textbf{relative to original data}, tuned XGBoost models}
\centering
\begin{tabular}[t]{lcccccc}
\toprule
  \parbox{1.5cm}{\centering Metric} & \parbox{1.5cm}{\centering Partner} & \parbox{1.5cm}{\centering Partner \\ 95\% CI} & \parbox{1.5cm}{\centering Time shift} & \parbox{1.5cm}{\centering Time shift \\ 95\% CI} & \parbox{1.5cm}{\centering Partner + \\ time shift} & \parbox{1.5cm}{\centering Partner + \\ time shift \\ 95\% CI}\\
\midrule
$R^2_\text{CV}$ & .013 & {}[$-$.023, .048] & .037 & {}[$-$.002, .074] & \textbf{.047} & \textbf{[.012, .081]}\\
$R^2_\text{Traditional}$ & .013 & {}[$-$.023, .048] & .037 & {}[$-$.002, .075] & \textbf{.048} & \textbf{[.012, .081]}\\
AUC & .014 & {}[.001, .027] & .023 & {}[.006, .040] & \textbf{.025} & \textbf{[.010, .040]}\\
Log loss & $-$.001 & {}[$-$.022, .023] & $-$.023 & {}[$-$.044, .000] & \textbf{$-$.030} & \textbf{[$-$.047, $-$.013]}\\
MSE & $-$.002 & {}[$-$.008, .004] & $-$.006 & {}[$-$.013, .000] & \textbf{$-$.008} & \textbf{[$-$.014, $-$.002]}\\
F1 & .011 & {}[$-$.020, .041] & .025 & {}[$-$.003, .057] & \textbf{.030} & \textbf{[.000, .059]}\\
Precision & .031 & {}[$-$.013, .073] & .026 & {}[$-$.015, .070] & \textbf{.044} & \textbf{[.004, .085]}\\
Recall & $-$.005 & {}[$-$.043, .034] & \textbf{.024} & \textbf{[$-$.013, .065]} & .019 & {}[$-$.018, .056]\\
Accuracy & .006 & {}[$-$.006, .018] & .010 & {}[$-$.002, .022] & \textbf{.013} & \textbf{[.001, .025]}\\
\bottomrule
\multicolumn{7}{l}{\makecell[l]{%
Note: The best performance of each row is in bold.\\
F1, precision, recall, and accuracy were computed by classifying predicted probabilities of 0.5 or greater as positive.
}}
\end{tabular}
\end{sidewaystable}


\begin{sidewaystable}

\caption{Performance of Logistic Regression Models on Seven Baseline Features}
\centering
\begin{tabular}[t]{lcccccccc}
\toprule
  \parbox{1.5cm}{\centering Metric} & \parbox{1.5cm}{\centering Original} & \parbox{1.5cm}{\centering Original \\ 95\% CI} & \parbox{1.5cm}{\centering Partner} & \parbox{1.5cm}{\centering Partner \\ 95\% CI} & \parbox{1.5cm}{\centering Time shift} & \parbox{1.5cm}{\centering Time shift \\ 95\% CI} & \parbox{1.5cm}{\centering Partner + \\ time shift} & \parbox{1.5cm}{\centering Partner + \\ time shift \\ 95\% CI}\\
\midrule
$R^2_\text{CV}$ & .516 & {}[.449, .578] & .536 & {}[.465, .604] & .519 & {}[.459, .579] & \textbf{.567} & \textbf{[.508, .625]}\\
$R^2_\text{Traditional}$ & .515 & {}[.447, .577] & .535 & {}[.463, .602] & .518 & {}[.457, .577] & \textbf{.566} & \textbf{[.506, .623]}\\
AUC & .922 & {}[.899, .942] & .926 & {}[.902, .946] & .922 & {}[.899, .942] & \textbf{.933} & \textbf{[.912, .952]}\\
Log loss & .292 & {}[.243, .350] & .324 & {}[.249, .415] & .271 & {}[.235, .308] & \textbf{.250} & \textbf{[.211, .290]}\\
MSE & .082 & {}[.070, .095] & .078 & {}[.066, .091] & .081 & {}[.070, .093] & \textbf{.073} & \textbf{[.062, .084]}\\
F1 & .704 & {}[.654, .753] & .735 & {}[.686, .783] & .689 & {}[.633, .744] & \textbf{.750} & \textbf{[.703, .797]}\\
Precision & .760 & {}[.701, .822] & .828 & {}[.768, .882] & .818 & {}[.758, .879] & \textbf{.860} & \textbf{[.806, .911]}\\
Recall & .656 & {}[.595, .718] & .660 & {}[.599, .724] & .594 & {}[.529, .662] & \textbf{.665} & \textbf{[.606, .728]}\\
Accuracy & .881 & {}[.861, .901] & .898 & {}[.879, .916] & .884 & {}[.864, .904] & \textbf{.905} & \textbf{[.886, .922]}\\
\bottomrule
\multicolumn{9}{l}{\makecell[l]{%
Note: The best performance of each row is in bold.\\
F1, precision, recall, and accuracy were computed by classifying predicted probabilities of 0.5 or greater as positive.
}}
\end{tabular}
\end{sidewaystable}


\begin{sidewaystable}

\caption{Performance of Engineered Data \textbf{Relative to Original Data}, Logistic Regression Models on Seven Baseline Features}
\centering
\begin{tabular}[t]{lcccccc}
\toprule
  \parbox{1.5cm}{\centering Metric} & \parbox{1.5cm}{\centering Partner} & \parbox{1.5cm}{\centering Partner \\ 95\% CI} & \parbox{1.5cm}{\centering Time shift} & \parbox{1.5cm}{\centering Time shift \\ 95\% CI} & \parbox{1.5cm}{\centering Partner + \\ time shift} & \parbox{1.5cm}{\centering Partner + \\ time shift \\ 95\% CI}\\
\midrule
$R^2_\text{CV}$ & .020 & {}[$-$.022, .061] & .003 & {}[$-$.023, .030] & \textbf{.051} & \textbf{[.009, .092]}\\
$R^2_\text{Traditional}$ & .020 & {}[$-$.023, .061] & .003 & {}[$-$.023, .030] & \textbf{.051} & \textbf{[.009, .093]}\\
AUC & .004 & {}[$-$.009, .015] & $-$.001 & {}[$-$.008, .008] & \textbf{.011} & \textbf{[.000, .022]}\\
Log loss & .033 & {}[$-$.016, .094] & $-$.021 & {}[$-$.060, .004] & \textbf{$-$.042} & \textbf{[$-$.084, $-$.011]}\\
MSE & $-$.003 & {}[$-$.010, .004] & .000 & {}[$-$.005, .004] & \textbf{$-$.009} & \textbf{[$-$.016, $-$.002]}\\
F1 & .031 & {}[$-$.009, .070] & $-$.015 & {}[$-$.050, .019] & \textbf{.046} & \textbf{[.005, .089]}\\
Precision & .069 & {}[.018, .117] & .059 & {}[.015, .103] & \textbf{.100} & \textbf{[.046, .153]}\\
Recall & .005 & {}[$-$.043, .053] & $-$.061 & {}[$-$.101, $-$.020] & \textbf{.009} & \textbf{[$-$.043, .062]}\\
Accuracy & .016 & {}[.001, .030] & .003 & {}[$-$.010, .015] & \textbf{.023} & \textbf{[.007, .040]}\\
\bottomrule
\multicolumn{7}{l}{\makecell[l]{%
Note: The best performance of each row is in bold.\\
F1, precision, recall, and accuracy were computed by classifying predicted probabilities of 0.5 or greater as positive.
}}
\end{tabular}
\end{sidewaystable}



\begin{sidewaystable}

\caption{Performance of Logistic Regression Models on All Features in Winning Model}
\centering
\begin{tabular}[t]{lcccccccc}
\toprule
  \parbox{1.5cm}{\centering Metric} & \parbox{1.5cm}{\centering Original} & \parbox{1.5cm}{\centering Original \\ 95\% CI} & \parbox{1.5cm}{\centering Partner} & \parbox{1.5cm}{\centering Partner \\ 95\% CI} & \parbox{1.5cm}{\centering Time shift} & \parbox{1.5cm}{\centering Time shift \\ 95\% CI} & \parbox{1.5cm}{\centering Partner + \\ time shift} & \parbox{1.5cm}{\centering Partner + \\ time shift \\ 95\% CI}\\
\midrule
$R^2_\text{CV}$ & .361 & {}[.270, .445] & .365 & {}[.262, .456] & .446 & {}[.380, .512] & \textbf{.494} & \textbf{[.426, .558]}\\
$R^2_\text{Traditional}$ & .359 & {}[.268, .443] & .363 & {}[.261, .454] & .444 & {}[.377, .511] & \textbf{.492} & \textbf{[.424, .557]}\\
AUC & .875 & {}[.847, .900] & .896 & {}[.871, .918] & .892 & {}[.867, .916] & \textbf{.906} & \textbf{[.881, .930]}\\
Log loss & .412 & {}[.337, .514] & .458 & {}[.363, .577] & .313 & {}[.277, .352] & \textbf{.288} & \textbf{[.250, .328]}\\
MSE & .108 & {}[.094, .124] & .107 & {}[.092, .124] & .094 & {}[.081, .107] & \textbf{.086} & \textbf{[.074, .099]}\\
F1 & .670 & {}[.615, .720] & .687 & {}[.636, .736] & .693 & {}[.638, .745] & \textbf{.719} & \textbf{[.667, .768]}\\
Precision & .701 & {}[.636, .763] & .685 & {}[.623, .745] & .773 & {}[.709, .837] & \textbf{.811} & \textbf{[.751, .868]}\\
Recall & .642 & {}[.578, .705] & \textbf{.689} & \textbf{[.627, .751]} & .627 & {}[.562, .692] & .646 & {}[.583, .708]\\
Accuracy & .864 & {}[.842, .884] & .865 & {}[.845, .887] & .880 & {}[.860, .901] & \textbf{.892} & \textbf{[.873, .910]}\\
\bottomrule
\multicolumn{9}{l}{\makecell[l]{%
Note: The best performance of each row is in bold.\\
F1, precision, recall, and accuracy were computed by classifying predicted probabilities of 0.5 or greater as positive.
}}
\end{tabular}
\end{sidewaystable}

\begin{sidewaystable}

\caption{Performance of Engineered Data \textbf{Relative to Original Data}, Logistic Regression Models on All Features in Winning Model}
\centering
\begin{tabular}[t]{lcccccc}
\toprule
  \parbox{1.5cm}{\centering Metric} & \parbox{1.5cm}{\centering Partner} & \parbox{1.5cm}{\centering Partner \\ 95\% CI} & \parbox{1.5cm}{\centering Time shift} & \parbox{1.5cm}{\centering Time shift \\ 95\% CI} & \parbox{1.5cm}{\centering Partner + \\ time shift} & \parbox{1.5cm}{\centering Partner + \\ time shift \\ 95\% CI}\\
\midrule
$R^2_\text{CV}$ & .004 & {}[$-$.055, .058] & .085 & {}[.039, .135] & \textbf{.133} & \textbf{[.075, .194]}\\
$R^2_\text{Traditional}$ & .004 & {}[$-$.055, .059] & .085 & {}[.039, .135] & \textbf{.133} & \textbf{[.076, .195]}\\
AUC & .021 & {}[.008, .035] & .017 & {}[.002, .033] & \textbf{.031} & \textbf{[.013, .050]}\\
Log loss & .045 & {}[$-$.007, .102] & $-$.100 & {}[$-$.182, $-$.045] & \textbf{$-$.124} & \textbf{[$-$.210, $-$.065]}\\
MSE & $-$.001 & {}[$-$.010, .009] & $-$.014 & {}[$-$.023, $-$.007] & \textbf{$-$.022} & \textbf{[$-$.032, $-$.013]}\\
F1 & .017 & {}[$-$.016, .051] & .023 & {}[$-$.018, .063] & \textbf{.049} & \textbf{[.006, .095]}\\
Precision & $-$.016 & {}[$-$.060, .032] & .072 & {}[.020, .126] & \textbf{.110} & \textbf{[.056, .167]}\\
Recall & \textbf{.047} & \textbf{[.011, .086]} & $-$.014 & {}[$-$.063, .030] & .005 & {}[$-$.049, .058]\\
Accuracy & .001 & {}[$-$.014, .017] & .016 & {}[.000, .032] & \textbf{.027} & \textbf{[.010, .046]}\\
\bottomrule
\multicolumn{7}{l}{\makecell[l]{%
Note: The best performance of each row is in bold.\\
F1, precision, recall, and accuracy were computed by classifying predicted probabilities of 0.5 or greater as positive.
}}
\end{tabular}
\end{sidewaystable}




\begin{sidewaystable}

\caption{Performance of Final and Less Engineered XGBoost Models}
\centering
\begin{tabular}[t]{lcccccccc}
\toprule
  \parbox{1.5cm}{\centering Metric} & \parbox{1.5cm}{\centering Final model} & \parbox{1.5cm}{\centering Final model \\ 95\% CI} & \parbox{1.5cm}{\centering No tuning} & \parbox{1.5cm}{\centering No tuning \\ 95\% CI} & \parbox{1.5cm}{\centering No partner + \\ no time shift} & \parbox{1.5cm}{\centering No partner + \\ no time shift \\ 95\% CI} & \parbox{1.5cm}{\centering Fertility \\ intentions} & \parbox{1.5cm}{\centering Fertility \\ intentions \\ 95\% CI}\\
\midrule
$R^2_\text{CV}$ & \textbf{.591} & \textbf{[.528, .655]} & .568 & {}[.500, .635] & .543 & {}[.477, .608] & .434 & {}[.373, .495]\\
$R^2_\text{Traditional}$ & \textbf{.590} & \textbf{[.526, .653]} & .567 & {}[.498, .634] & .542 & {}[.475, .607] & .433 & {}[.369, .492]\\
AUC & \textbf{.937} & \textbf{[.916, .955]} & .928 & {}[.906, .948] & .912 & {}[.886, .937] & .884 & {}[.856, .913]\\
Log loss & \textbf{.241} & \textbf{[.206, .275]} & .253 & {}[.212, .294] & .270 & {}[.234, .307] & .326 & {}[.294, .360]\\
MSE & \textbf{.069} & \textbf{[.057, .081]} & .073 & {}[.060, .086] & .077 & {}[.065, .090] & .096 & {}[.084, .108]\\
F1 & \textbf{.803} & \textbf{[.759, .846]} & .765 & {}[.717, .811] & .773 & {}[.724, .819] & .700 & {}[.645, .750]\\
Precision & \textbf{.864} & \textbf{[.814, .914]} & .825 & {}[.770, .881] & .820 & {}[.766, .874] & .800 & {}[.739, .860]\\
Recall & \textbf{.750} & \textbf{[.689, .809]} & .712 & {}[.650, .776] & .731 & {}[.670, .793] & .623 & {}[.556, .687]\\
Accuracy & \textbf{.921} & \textbf{[.904, .938]} & .906 & {}[.888, .924] & .908 & {}[.889, .926] & .886 & {}[.865, .905]\\
\bottomrule
\multicolumn{9}{l}{\makecell[l]{%
Note: The best performance of each row is in bold.\\
F1, precision, recall, and accuracy were computed by classifying predicted probabilities of 0.5 or greater as positive.
}}
\end{tabular}
\end{sidewaystable}






\begin{sidewaystable}

\caption{Performance of Less Engineered XGBoost Models \textbf{Relative to Final Model}}
\centering
\begin{tabular}[t]{lcccccc}
\toprule
  \parbox{1.5cm}{\centering Metric} & \parbox{1.5cm}{\centering No tuning} & \parbox{1.5cm}{\centering No tuning \\ 95\% CI} & \parbox{1.5cm}{\centering No partner + \\ no time shift} & \parbox{1.5cm}{\centering No partner + \\ no time shift \\ 95\% CI} & \parbox{1.5cm}{\centering Fertility \\ intentions} & \parbox{1.5cm}{\centering Fertility \\ intentions \\ 95\% CI}\\
\midrule
$R^2_\text{CV}$ & \textbf{$-$.023} & \textbf{[$-$.041, $-$.006]} & $-$.047 & {}[$-$.081, $-$.012] & $-$.156 & {}[$-$.196, $-$.116]\\
$R^2_\text{Traditional}$ & \textbf{$-$.023} & \textbf{[$-$.041, $-$.006]} & $-$.048 & {}[$-$.081, $-$.012] & $-$.157 & {}[$-$.197, $-$.116]\\
AUC & \textbf{$-$.008} & \textbf{[$-$.015, $-$.002]} & $-$.025 & {}[$-$.040, $-$.010] & $-$.052 & {}[$-$.071, $-$.034]\\
Log loss & \textbf{.012} & \textbf{[.002, .022]} & .030 & {}[.013, .047] & .085 & {}[.067, .104]\\
MSE & \textbf{.004} & \textbf{[.001, .007]} & .008 & {}[.002, .014] & .026 & {}[.019, .034]\\
F1 & $-$.038 & {}[$-$.063, $-$.017] & \textbf{$-$.030} & \textbf{[$-$.059, .000]} & $-$.103 & {}[$-$.147, $-$.061]\\
Precision & \textbf{$-$.039} & \textbf{[$-$.072, $-$.010]} & $-$.044 & {}[$-$.085, $-$.004] & $-$.064 & {}[$-$.110, $-$.021]\\
Recall & $-$.038 & {}[$-$.068, $-$.011] & \textbf{$-$.019} & \textbf{[$-$.056, .018]} & $-$.127 & {}[$-$.185, $-$.074]\\
Accuracy & $-$.015 & {}[$-$.024, $-$.006] & \textbf{$-$.013} & \textbf{[$-$.025, $-$.001]} & $-$.035 & {}[$-$.051, $-$.020]\\
\bottomrule
\multicolumn{7}{l}{\makecell[l]{%
Note: The best performance of each row is in bold.\\
F1, precision, recall, and accuracy were computed by classifying predicted probabilities of 0.5 or greater as positive.
}}
\end{tabular}
\end{sidewaystable}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\clearpage

\normalsize

\section{REFORMS checklist}\label{secC:REFORMS}

To ensure this paper follows best practices and documents our methods and claims clearly, we completed the ``Recommendations for Machine-learning-based Science" (REFORMS) checklist below. For details on the REFORMS guidelines, see Kapoor et al. \cite{kapoor_reforms_2024}.

\subsection{Module 1: Study goals}

\textbf{1a. Population or distribution about which the scientific claim is made.} 

\noindent Our claim that time-shift data augmentation and partner linkage have the potential to improve predictive performance in a variety of data settings is not specific to a demographic population. However, there is a population of interest for the case study we use to demonstrate these methods. We wrote: “The population of interest for this outcome was Dutch residents of childbearing age, aged 18--45.”

\vspace{6pt}
\noindent \textbf{1b. Motivation for choosing this population or distribution (1a.).}

\noindent Participating in the PreFer challenge allowed us to compare the effectiveness of our time-shift data augmentation and partner linkage strategies to the effectiveness of strategies used by other competitors. Therefore, our population of interest is the population chosen by the organizers of the PreFer challenge. They chose this population due to interest in the Dutch context and availability of exceptional data sources in the Dutch context: a detailed survey dataset and a full-population administrative register for which results can be compared to understand differences in predictability with different data characteristics. The age range 18--45 was chosen due to being a typical age range for having children.

\vspace{6pt}
\noindent \textbf{1c. Motivation for the use of ML methods in the study.} 

\noindent We wrote: ``Machine learning offers new tools to improve our description, understanding, and forecasting of the social world by identifying complex interactions and nonlinear patterns \cite{grimmer_machine_2021,hofman_integrating_2021,verhagen_pragmatists_2022}. However, even with increasingly sophisticated machine learning techniques, predictive performance remains low in many social science applications \cite{salganik_measuring_2020,kuikka_predictability_2024,van_de_rijt_luck_2026,zheng_social_2025,sivak_births_2026}. This is especially true when working with small sample sizes, as is common in many social science data sources such as surveys, experiments, and studies of small subgroups or rare events. Given these limitations, what strategies can improve predictive accuracy in social science contexts?"

\subsection{Module 2: Computational reproducibility}

\noindent \textbf{2a. Dataset used for training and evaluating the model along with link or DOI to uniquely identify the dataset.} 

\noindent We wrote: ``We used the LISS dataset, a panel survey representative of the Dutch population aged 16 and older (see Appendix Section~\ref{secA12_missingness_outcome} [Additional file 1] for sampling details) that covers a variety of social scientific topics \cite{centerdata_home_2026}." Data access instructions are available in the \texttt{README} of the project's \href{https://github.com/HanzhangRen/fertility-prediction-challenge-2024}{GitHub repository}. This link is also included in the paper.

\vspace{6pt}
\noindent \textbf{2b. Code used to train and evaluate the model and produce the results reported in the paper along with link or DOI to uniquely identify the version of the code used.}

\noindent Our code is available in the the project's \href{https://github.com/HanzhangRen/fertility-prediction-challenge-2024}{GitHub repository}.

\vspace{6pt}
\noindent \textbf{2c. Description of the computing infrastructure used.}

\noindent A description of our computing infrastructure is available in the \texttt{README} of the project's \href{https://github.com/HanzhangRen/fertility-prediction-challenge-2024}{GitHub repository}.

\vspace{6pt}
\noindent \textbf{2d. README file which contains instructions for generating the results using the provided dataset and code.}

\noindent Instructions are available in the \texttt{README} of the project's \href{https://github.com/HanzhangRen/fertility-prediction-challenge-2024}{GitHub repository}.

\vspace{6pt}
\noindent \textbf{2e. Reproduction script to produce all results reported in the paper.}

\noindent The user will need to install the correct version of \texttt{R} and the correct version of the version control package \texttt{groundhog}. Then all results can be generated from a single command line entry that initiates a script called \texttt{00\_run\_all.R}. Instructions are available in the \texttt{README} of the project's \href{https://github.com/HanzhangRen/fertility-prediction-challenge-2024}{GitHub repository}.

\subsection{Module 3: Data quality}\label{module3}

\vspace{6pt}
\noindent \textbf{3a. Source(s) of data, separately for the training and evaluation datasets (if applicable), along with the time when the dataset(s) are collected, the source and process of ground-truth annotations, and other data documentation.}

\noindent We wrote: ``We used the LISS dataset, a panel survey representative of the Dutch population aged 16 and older (see Appendix Section~\ref{secA12_missingness_outcome} [Additional file 1] for sampling details) that covers a variety of social scientific topics \cite{centerdata_home_2026}... We used a version of the LISS data provided by the PreFer challenge organizers; additional details are available in the PreFer challenge announcement paper \cite{sivak_combining_2024}."

Additionally, we wrote: ``The outcome was a binary indicator for whether a respondent had at least one new child via birth or adoption in a three-year period (2021--2023), as calculated by the organizers of the PreFer challenge based on respondents’ answers to several survey questions."

\vspace{6pt}
\noindent \textbf{3b. Distribution or set from which the dataset is sampled (i.e., the sampling frame).}

\noindent We wrote: ``We used the LISS dataset, a panel survey representative of the Dutch population aged 16 and older (see Appendix Section~\ref{secA12_missingness_outcome} [Additional file 1] for sampling details) that covers a variety of social scientific topics \cite{centerdata_home_2026}." Additionally, information about the LISS survey sampling methodology is available in our supplemental materials in Section~\ref{secA12_missingness_outcome}. In that section we also provide flowcharts showing the relationship between the full LISS sample and our training and holdout sets for both the original and time-shifted data.  

\vspace{6pt}
\noindent \textbf{3c. Justification for why the dataset is useful for the modeling task at hand.}

\noindent Sivak et al. (the organizers of the PreFer challenge) explain their reasons for choosing this dataset in the paper that introduces the challenge \cite{sivak_combining_2024}. See their section titled “Novel opportunities for fertility research.”

\vspace{6pt}
\noindent \textbf{3d. The definition of the outcome variable of the model along with descriptive statistics, if applicable.}

\noindent We wrote: “The outcome was a binary indicator for whether a respondent had at least one new child via birth or adoption in a three-year period (2021--2023), as calculated by the organizers of the PreFer challenge based on respondents’ answers to several survey questions.” For breakdown by class, see Table 1 in the main text.

\vspace{6pt}
\noindent \textbf{3e. Number of samples in the dataset.}

\noindent We wrote: ``The organizers of the PreFer challenge created a dataset of LISS panel members aged 18--45 with a non-missing “new child” outcome, and split this data into a 70\% training set and 30\% holdout set (training set: $n = 987$, holdout set: $n = 395$)."

\vspace{6pt}
\noindent \textbf{3f. Percentage of missing data, split by class for a categorical outcome variable.}

\noindent Figure~\ref{figa1:missingness} provides the distribution of missingness rates across the 84 features in the winning model, split by outcome class.

\vspace{6pt}
\noindent \textbf{3g. Justification for why the distribution or set from which the dataset is drawn (3b.) is representative of the one about which the scientific claim is being made (1a.).}

\noindent The LISS survey was designed to be representative of the Dutch population aged 16 and older.

\subsection{Module 4: Data preprocessing}\label{module4}

\vspace{6pt}
\noindent \textbf{4a. Identification of whether any samples are excluded with a rationale for why they are excluded.}

\noindent Not applicable. No samples are excluded unless they were excluded by the PreFer organizers due to missing outcome data.

\vspace{6pt}
\noindent \textbf{4b. How impossible or corrupt samples are dealt with.}

\noindent One respondent answered a question in the 2020 survey, “Within how many years do you hope to have your [first / next] child?” by saying “2025.” We edited this response to say “5,” representing 5 years. This change is documented in our code.

\vspace{6pt}
\noindent \textbf{4c. All transformations of the dataset from its raw form (3a.) to the form used in the model, for instance, treatment of missing data and normalization.}

\noindent See Section 4.4, “Feature engineering” and Section 4.6, “Data preparation,” in the main text.

\subsection{Module 5: Modeling}\label{module5}

\vspace{6pt}
\noindent \textbf{5a. Detailed descriptions of all models trained, including: all features used in the model (including any feature selection); types of models implemented (e.g., Random Forests, Neural Networks); loss function used.}

\noindent We address each of the above criteria as follows:

\begin{itemize}
    \item Features are listed in Table~\ref{tablea1:winning_model_features} of the supplemental materials with descriptions. 
    \item Model types are clearly named (XGBoost is the main model; there is a short section on logistic regression).
    \item We wrote: “Our algorithm employed a logistic loss function.”
\end{itemize}

\vspace{6pt}
\noindent \textbf{5b. Justification for the choice of model types implemented.}

\noindent We wrote: ``We chose XGBoost because it performed well in other settings and offered a simple out-of-the-box model for testing our time-shift and partner linkage strategies \cite{chen_xgboost_2016}." When illustrating the potential applicability of our data engineering strategies to other model types, we chose logistic regression because of this model type's widespread use in social science. An additional benefit of logistic regression is that it involves little hyperparameter tuning, which alleviates concerns that our data engineering strategies only appeared to work because we overfit the model across the validation folds during hyperparameter tuning. 

\vspace{6pt}
\noindent \textbf{5c. Method for evaluating the model(s) reported in the paper, including details of train-test splits or cross-validation folds.}

\noindent See Section 4.7, “Train-test splits” and Section 4.8, ``Metrics."

\vspace{6pt}
\noindent \textbf{5d. Method for selecting the model(s) reported in the paper.}

\noindent We only tested XGBoost for our main analysis, so we did not need to select among model classes. Section 4.5, “Model,” explains that we tuned the model via grid search.

Note: We tested performance with logistic regression as a secondary analysis in Section 5.2, “Improvements in logistic regression performance.” This was added after our analysis with XGBoost was complete.

\vspace{6pt}
\noindent \textbf{5e. For the model(s) reported in the paper, specify details about the hyperparameter tuning.}

\noindent See Section 4.5, ``Model," in the main text, and Section~\ref{secA2:hyperparameter_tuning} in the supplementary materials.

\vspace{6pt}
\noindent \textbf{5f. Justification that model comparisons are against appropriate baselines.}

\noindent Performance on the original PreFer data was the natural baseline for testing how time-shift augmentation and partner linkage improve performance compared to a dataset that has not been expanded with these methods. We took another approach in Section 5.3, ``Putting the magnitude of the improvements in context." Instead of adding rows and columns to our original data, we removed components from our final model in so-called ablation tests. For each test, we removed one aspect of our final model to measure the predictive impact of that aspect.

\subsection{Module 6: Data leakage}

\vspace{6pt}
\noindent \textbf{6a. Justification that pre-processing (\hyperref[module4]{Module 4}) and modeling (\hyperref[module5]{Module 5}) steps only use information from the training dataset (and not the test dataset).}

\noindent We discussed steps to avoid leakage in Section 4.7.2, ``Cross-validation without drift or leakage," as well as in Section 2.4, ``Train-test splits with time-shift data augmentation," since time-shift data augmentation requires special considerations to avoid leakage.

\vspace{6pt}
\noindent \textbf{6b. Methods to address dependencies or duplicates between the training and test datasets (e.g., different samples from the same patients are kept in the same dataset partition).}

\noindent We wrote: “To prevent leakage, we applied two additional constraints in the creation of these splits (not depicted in the diagram). First, we assigned all members of the same household to the same fold. Second, since the time-shift augmentation introduced earlier observations of the same individuals, we excluded any time-shifted row from a training set if it was an earlier observation of someone in that fold’s test set.”

\vspace{6pt}
\noindent \textbf{6c. Justification that each feature or input used in the model is legitimate for the task at hand and does not lead to leakage.}

\noindent All features collected prior to the outcome time window were considered in-bounds for the PreFer challenge, as the goal was to predict births and adoptions in 2021--2023 based on data from 2020 and earlier. This meant even features about fertility intentions and gynecologist visits (which might indicate the respondent is already pregnant) were appropriate, since they were collected in 2020 or earlier. Likewise, in our time-shifted data, all features came from before the relevant outcome time window.

\subsection{Module 7: Metrics and uncertainty}

\vspace{6pt}
\noindent \textbf{7a. All metrics used to assess and compare model performance (e.g., accuracy, AUROC etc.). Justify that the metric used to select the final model is suitable for the task.}

\noindent We explained our metric choice in Section 4.8, “Metrics.”

\vspace{6pt}
\noindent \textbf{7b. Uncertainty estimates (e.g., confidence intervals, standard deviations), and details of how these are calculated.}

\noindent Bootstrapping for 95\% CIs was explained in Section 4.8, “Metrics.”

\vspace{6pt}
\noindent \textbf{7c. Justification for the choice of statistical tests (if used) and a check for the assumptions of the statistical test.}

\noindent Not applicable.

\subsection{Module 8: Generalizability and limitations}

\vspace{6pt}
\noindent \textbf{8a. Evidence of external validity.}

\noindent In Section 5.2, ``Improvements in logistic regression performance," we tested our data engineering strategies on logistic regression models with two different feature sets, showing that the strategies worked for more than just one model type and for more than just one set of features. In Section 7, ``Discussion," we described other contexts where our methods could be applied: “Both of our data engineering strategies can be implemented in a variety of contexts. Time-shift data augmentation can be used to \textit{increase sample size} in any tabular dataset where the same constructs are repeatedly measured at regular time intervals, such as government administrative records or longitudinal panel surveys, or in continuously updated event data, such as administrative records that are updated every time a birth (or other event of interest) occurs. Meanwhile, partner linkage (or linkage with other household members) can be used to \textit{increase the number of features} in any dataset where members of the same household are stored in separate rows or files, which is common in government administrative records and surveys sampled at the household level.”

\vspace{6pt}
\noindent \textbf{8b. Contexts in which the authors do not expect the study’s findings to hold.}

\noindent We wrote: ``In studies of processes that change substantially over time, temporal drift may cause models to capture patterns not applicable to the future \cite{logeswaran_its_2025,yao_wild-time_2023}. If the sample size is already large, time-shift augmentation is also unlikely to greatly improve performance \cite{cantrell_2026_more, viering_shape_2023}. For outcomes where partner characteristics are less relevant, partner linkage might lead models to overfit on noise in the additional variables, possibly reducing predictive performance \cite{james_introduction_2021,viering_shape_2023,hughes_mean_1968}."



%%===========================================================================================%%
%% If you are submitting to one of the Nature Portfolio journals, using the eJP submission   %%
%% system, please include the references within the manuscript file itself. You may do this  %%
%% by copying the reference list from your .bbl file, paste it into the main manuscript .tex %%
%% file, and delete the associated \verb+\bibliography+ commands.                            %%
%%===========================================================================================%%

\nolinenumbers

\bibliography{bibliography}% common bib file
%% if required, the content of .bbl file can be included here once bbl is generated
%%\input sn-article.bbl

\end{appendices}

\end{document}
