{
 "paper": {
  "package": "S1",
  "working_title": "What the expert labels of an open milling anomaly benchmark encode: microphone loudness, tool identity and chatter signatures",
  "title_note": "Working title from Phase 2. The submitted title is chosen in Phase 4 after reading recent IJAMT titles, as the pipeline asks.",
  "venue": "The International Journal of Advanced Manufacturing Technology (Springer)",
  "article_type": "Original Article"
 },
 "rq": "What do the expert path labels of the MSM milling dataset encode on its main laboratory machine, imi_vm20i, and how should detectors trained on them be evaluated? The loudness and clipping findings of Phase 2 enter the paper as results seen before registration. The plan tests, before computing them, whether detectors lose ranking performance on tool families they have not seen (each family comes with its own materials and spindle speeds; H1, H2), and whether the spindle accelerometer and a sensor on the base column mark the same label 1 paths with an off-harmonic spectral line more often than spindle speed explains (H3). Whether that line differs between tool families is examined without a confirmatory test (E11), because the families other than A hold label 1 in too few effective runs.",
 "negatives": [
  "The headline numbers are already known. Loudness, clipping, tool identity and the spectral shares that motivate S1 were computed in Phase 2, so the paper cannot present them as confirmatory. {{n_predictions|wordcap}} predictions remain open, and each rests on a quantity no script has computed yet (section 11).",
  "Only imi_vm20i can carry a confirmatory test. imi_vmx30ui has no accelerometers, and its label 1 paths sit in {{pop_imi_vmx30ui_a_runs_l1|int}} runs of family A, {{pop_imi_vmx30ui_e_runs_l1|int}} of family E and {{pop_imi_vmx30ui_d_runs_l1|int}} of family D. Everything on that machine is exploratory.",
  "Tool family, material and spindle speed cannot be separated. On imi_vm20i only A and E hold many label 1 paths ({{pop_imi_vm20i_a_l1|int}} and {{pop_imi_vm20i_e_l1|int}}); D holds {{pop_imi_vm20i_d_l1|int}}, F1 holds {{pop_imi_vm20i_f1_l1|int}}, and B and C1 hold none. Family A was cut almost only on AA 6061 and at the highest spindle speeds, so H1 and H2 measure the cost of an unseen family together with its material and speed range, and the fold that holds A out carries {{pop_imi_vm20i_a_l1|int}} of the {{pop_h_highrate_primary_n_l1|int}} label 1 paths.",
  "The planned test of whether the off-harmonic line differs by tool family was dropped before any number was computed. The label 1 paths of the families other than A sit in {{pop_bytool_primary_other_runs|int}} runs, but {{pop_l1_runs_other_single|int}} of them hold a single path and a few large runs hold most of the rest, so the effective number of runs is {{pop_kish_l1_runs_other|f1}}, below the floor of {{min_runs_interpretable|int}}. The comparison, in practice family A against family E, stays as exploratory analysis E11.",
  "The spectral indicator has no ground truth. The dataset has no force, displacement or frequency response measurements. H3 asks whether a spindle accelerometer and a stethoscope sensor on the base column, both picking up structure-borne vibration, mark the same label 1 paths; agreement supports a machine-level origin of the off-harmonic line but cannot prove that the line is chatter.",
  "H1 and H2 may fail on precision even if the effect is real. The run-held-out AUC of the Phase 2 screen model has a family-stratified run-bootstrap standard error of {{prec_auc_run_highrate_se|f3}} for the high-rate set and {{prec_auc_run_mtc_se|f3}} for the MTConnect set, and the family-held-out AUC will be less precise because a single fold carries most label 1 paths."
 ],
 "format": {
  "choice": "LaTeX with the Springer Nature journal article template ({{v_template_version|text}} package) and the [iicol] option that the guidelines recommend; bibliography style sn-basic.bst from the same package unless Adam decides otherwise at Q1. One format only: the guidelines accept LaTeX or Word and require neither both nor Word at first submission.",
  "build": "Compiled with pdflatex from {{env_texlive|text}}, which is installed in this workspace (pdflatex {{env_pdflatex|text}} on the path). The class and style files come from the downloaded template package and nowhere else. Figures are EPS files, as the guidelines prefer, included through graphicx and converted by epstopdf during the build; the submission system compiles the sources itself and its TeX set-up was not captured, so the build is checked here and the risk is stated at submission. If the download is not allowed (Q3), Phase 4 cannot compile a faithful PDF and will say so; it will not imitate the class.",
  "files": "The source file, the bibliography database, the class and style files of the template, the figure files named as the guidelines ask, and the compiled PDF, all in one flat folder."
 },
 "budget": {
  "tables": [
   {
    "id": "Table 1",
    "title": "Analysis populations",
    "content": "Runs, label 0 paths and label 1 paths by tool family on both laboratory machines",
    "layout": "Rows: tool families. Columns: runs, label 0 paths and label 1 paths for each machine under a machine header. No other merged cells.",
    "width": "page",
    "items": [
     "D1 (seen)",
     "Phase 3 population counts"
    ]
   },
   {
    "id": "Table 2",
    "title": "Pre-registered predictions and outcomes",
    "content": "For H1 to H3: the prediction in brief, its threshold, the estimate with its interval, the Holm-adjusted p-value and the outcome class",
    "layout": "Rows: predictions. Columns: prediction, threshold, estimate with interval, Holm-adjusted p-value, class.",
    "width": "page",
    "items": [
     "H1",
     "H2",
     "H3"
    ]
   },
   {
    "id": "Table 3",
    "title": "AUC by feature set and evaluation scheme on imi_vm20i",
    "content": "Pooled AUC with its interval for the high-rate, MTConnect, high-rate without Sound0, clipping-only and design-only models under random (Phase 2), run-held-out, family-held-out and material-held-out evaluation",
    "layout": "Rows: feature sets. Columns: evaluation schemes. Empty cells marked as not computed.",
    "width": "page",
    "items": [
     "H1",
     "H2",
     "E2",
     "E3",
     "E4",
     "E7",
     "E10"
    ]
   },
   {
    "id": "Table 4",
    "title": "Robustness of each prediction",
    "content": "For every variant, whether its interval lies wholly below the threshold of each prediction it applies to",
    "layout": "Rows: variants. Columns: H1 to H3. Cells: consistent, contradicts, not applicable.",
    "width": "col",
    "items": [
     "V1 to V9"
    ]
   }
  ],
  "figures": [
   {
    "id": "Fig. 1",
    "title": "Label 1 share of paths by tool family and material",
    "layout": "Grouped bars, hatching by material, a panel per laboratory machine",
    "width": "v_fig_width_col_mm",
    "height": "fig1_height_mm",
    "items": [
     "D1 (seen)"
    ]
   },
   {
    "id": "Fig. 2",
    "title": "Share of Sound0 samples at full scale by label",
    "layout": "Empirical cumulative distributions with a line style per label, a panel per laboratory machine",
    "width": "v_fig_width_col_mm",
    "height": "fig2_height_mm",
    "items": [
     "E1",
     "D9 (seen)"
    ]
   },
   {
    "id": "Fig. 3",
    "title": "Acc0 and Sound2 spectra of a label 1 and a label 0 path from the same run, with the harmonic comb shaded",
    "layout": "A grid of panels, sensors in rows and labels in columns, shared frequency axis. Paths chosen by a fixed rule: among imi_vm20i runs holding both labels in the agreement population, the run with the most label 1 paths (ties: the lower run number); in it, for each label, the path whose Acc0 RMS is closest to the median Acc0 RMS of that label in the run",
    "width": "v_fig_width_page_mm",
    "height": "fig3_height_mm",
    "items": [
     "method of H3",
     "E11"
    ]
   },
   {
    "id": "Fig. 4",
    "title": "AUC by evaluation scheme with run-bootstrap intervals",
    "layout": "Dot-and-whisker plot, a marker shape per scheme, feature sets on the vertical axis",
    "width": "v_fig_width_col_mm",
    "height": "fig4_height_mm",
    "items": [
     "H1",
     "H2",
     "E2",
     "E4"
    ]
   },
   {
    "id": "Fig. 5",
    "title": "Off-harmonic indicators",
    "layout": "Panel a: stacked bars of the combinations of the Acc0 and Sound2 indicators among label 1 paths, by speed class. Panel b: share of label 1 paths with an off-harmonic Acc0 line by tool family, with interval whiskers (exploratory, E11)",
    "width": "v_fig_width_page_mm",
    "height": "fig5_height_mm",
    "items": [
     "H3",
     "E6",
     "E11"
    ]
   }
  ],
  "esm": "Online Resources (electronic supplementary material): the split assignments, the analysis code with the values file (a ZIP archive) and a CSV table of the per-path indicators, each named ESM with its number, captioned and cited in the text (R21).",
  "style": "Every figure uses Arial or Helvetica lettering at {{fig_font_pt|int}} pt without effects, lines of at least {{fig_line_pt|f1}} pt, markers or hatching in addition to colour, no title inside the artwork, and content that stays readable in black and white so that no caption depends on colour (R15). The sizes in the table above are checked against R11 and R12 by the check script."
 },
 "quantities": {
  "h1_auc_run": {
   "unit": "AUC",
   "filter": "imi_vm20i, labels 0 and 1, highrate features present",
   "statistic": "q_confirm.auc_drop, auc_run: mean over repeats of the pooled out-of-fold AUC of each repeat (s1_repeats repeats of s1_folds folds of whole runs), logistic model with balanced class weights on the log RMS of Acc0, Acc1, Sound0, Sound1 and Sound2"
  },
  "h1_auc_family": {
   "unit": "AUC",
   "filter": "imi_vm20i, labels 0 and 1, highrate features present",
   "statistic": "q_confirm.auc_drop, auc_family: pooled out-of-fold AUC, a fold per tool family, same model"
  },
  "h1_drop": {
   "unit": "AUC",
   "filter": "imi_vm20i, labels 0 and 1, highrate features present",
   "statistic": "h1_auc_run minus h1_auc_family"
  },
  "h1_drop_ci": {
   "unit": "AUC",
   "filter": "imi_vm20i, labels 0 and 1, highrate features present",
   "statistic": "percentile interval of h1_drop at ci_level over boot_b family-stratified run resamples, predictions held fixed (interval record with lo and hi)"
  },
  "h1_p": {
   "unit": "probability",
   "filter": "imi_vm20i, labels 0 and 1, highrate features present",
   "statistic": "one-sided p-value of the null that the drop is at or below zero, same resamples, plus-one rule"
  },
  "h1_nonfinite": {
   "unit": "fraction",
   "filter": "imi_vm20i, labels 0 and 1, highrate features present",
   "statistic": "share of resamples in which a statistic of H1 is undefined"
  },
  "h1_p_holm": {
   "unit": "probability",
   "filter": "tested predictions",
   "statistic": "Holm-adjusted h1_p"
  },
  "h2_auc_run": {
   "unit": "AUC",
   "filter": "imi_vm20i, labels 0 and 1, mtc features present",
   "statistic": "as h1_auc_run, on the MTConnect spindle features"
  },
  "h2_auc_family": {
   "unit": "AUC",
   "filter": "imi_vm20i, labels 0 and 1, mtc features present",
   "statistic": "as h1_auc_family, on the MTConnect spindle features"
  },
  "h2_drop": {
   "unit": "AUC",
   "filter": "imi_vm20i, labels 0 and 1, mtc features present",
   "statistic": "h2_auc_run minus h2_auc_family"
  },
  "h2_drop_ci": {
   "unit": "AUC",
   "filter": "imi_vm20i, labels 0 and 1, mtc features present",
   "statistic": "as h1_drop_ci"
  },
  "h2_p": {
   "unit": "probability",
   "filter": "imi_vm20i, labels 0 and 1, mtc features present",
   "statistic": "as h1_p"
  },
  "h2_nonfinite": {
   "unit": "fraction",
   "filter": "imi_vm20i, labels 0 and 1, mtc features present",
   "statistic": "as h1_nonfinite"
  },
  "h2_p_holm": {
   "unit": "probability",
   "filter": "tested predictions",
   "statistic": "Holm-adjusted h2_p"
  },
  "h3_kappa": {
   "unit": "kappa",
   "filter": "imi_vm20i, label 1, Acc0 and Sound2 spectra",
   "statistic": "q_confirm.cooccurrence, kappa: Cohen's kappa between the Acc0 and Sound2 off-harmonic indicators (D5 settings)"
  },
  "h3_kappa_ci": {
   "unit": "kappa",
   "filter": "imi_vm20i, label 1, Acc0 and Sound2 spectra",
   "statistic": "percentile interval of h3_kappa, family-stratified run bootstrap (interval record)"
  },
  "h3_kappa_null": {
   "unit": "kappa",
   "filter": "imi_vm20i, label 1, Acc0 and Sound2 spectra",
   "statistic": "kappa expected if the indicators were independent within the spindle-speed classes set by speed_class_edge_rpm"
  },
  "h3_kappa_max": {
   "unit": "kappa",
   "filter": "imi_vm20i, label 1, Acc0 and Sound2 spectra",
   "statistic": "largest kappa the per-class shares of the two indicators allow"
  },
  "h3_thr": {
   "unit": "kappa",
   "filter": "imi_vm20i, label 1, Acc0 and Sound2 spectra",
   "statistic": "h3_kappa_null plus thr_kappa_frac of the distance to h3_kappa_max"
  },
  "h3_excess": {
   "unit": "kappa",
   "filter": "imi_vm20i, label 1, Acc0 and Sound2 spectra",
   "statistic": "h3_kappa minus h3_kappa_null"
  },
  "h3_p": {
   "unit": "probability",
   "filter": "imi_vm20i, label 1, Acc0 and Sound2 spectra",
   "statistic": "one-sided p-value of the null that h3_excess is at or below zero, the null recomputed in every resample, plus-one rule"
  },
  "h3_nonfinite": {
   "unit": "fraction",
   "filter": "imi_vm20i, label 1, Acc0 and Sound2 spectra",
   "statistic": "share of resamples in which a statistic of H3 is undefined"
  },
  "h3_p_holm": {
   "unit": "probability",
   "filter": "tested predictions",
   "statistic": "Holm-adjusted h3_p"
  }
 },
 "variant_naming": "Each variant registers the names of its prediction with the variant code inserted after the prediction code (for example `h1_v3_drop`, `h1_v3_drop_ci` and `h1_v3_p` for V3 of H1; `h3_v9_kappa`, `h3_v9_kappa_ci` and `h3_v9_thr` for V9 of H3), computed by the same function in q_confirm.py with the single change the variant defines. A name ending in `__hi` reads the upper bound of an interval record. Variant p-values are reported unadjusted and do not enter the verdict.",
 "predictions": [
  {
   "id": "H1",
   "name": "Unseen tool families, high-rate sensors",
   "plain": "Amplitude-based detectors of the labels rank label 1 above label 0 paths of imi_vm20i less well on tool families they have not seen than on runs they have not seen. Each family brings its own materials and spindle speeds, so the fall measures how much performance depends on having seen the family. Test: for a logistic model on the {{s1_nfeat_imi_vm20i_highrate|word}} high-rate RMS amplitudes, the estimated fall in pooled out-of-fold AUC is at least {{thr_auc_drop|f2}} and exceeds zero at the Holm-adjusted level.",
   "threshold_why": "At the Phase 2 screen's run-held-out AUC of {{s1_auc_imi_vm20i_highrate_run|f3}} (the median over repeats), a fall of {{thr_auc_drop|f2}} would multiply the share of misordered label 1 and label 0 pairs by {{err_mult_highrate|f1}}.",
   "rationale": "Label 1 sits mostly with family A (D1), and amplitude levels differ between tools, materials and speeds. A model that has seen other runs of a family learns where that family's paths lie on the feature scale; a model that has never seen the family must place it by the levels of other families. Both the ranking within a family and the ranking across families can suffer; exploratory analysis E10 tells them apart.",
   "against": "Within a run and a parameter set, the Acc0 and Sound0 amplitudes already separate the labels with AUCs of {{d4_auc_imi_vm20i_acc0|f2}} and {{d4_auc_imi_vm20i_s0|f2}} (D4), so loudness may carry over to unseen families and the fall may stay small.",
   "direction": "h1_auc_run > h1_auc_family",
   "magnitude": "h1_drop >= thr_auc_drop",
   "precision": "h1_p_holm < alpha_family",
   "rule": "h1_drop >= thr_auc_drop and h1_p_holm < alpha_family",
   "testable": "pop_h_highrate_primary_min_train_l1_family >= min_pos_testable and pop_h_highrate_primary_min_train_l0_family >= min_pos_testable and pop_h_highrate_primary_min_train_l1_run >= min_pos_testable and pop_kish_l1_runs_primary >= min_runs_interpretable",
   "testable_runtime": "h1_nonfinite <= max_nonfinite_share",
   "satisfiable": [
    {
     "expr": "prec_auc_run_highrate > thr_auc_drop + auc_chance",
     "why": "A pass needs a family-held-out AUC at or below the run-held-out AUC minus the threshold; that bound lies above chance, so the rule can pass without the family-held-out model failing outright, and it can fail because the family-held-out AUC may lie anywhere up to a perfect ranking."
    }
   ],
   "variants": [
    {
     "id": "V1",
     "applicable": "pop_h_highrate_no_twins_min_train_l1_family >= min_pos_testable and pop_h_highrate_no_twins_min_train_l1_run >= min_pos_testable and pop_kish_l1_runs_no_twins >= min_runs_interpretable",
     "contradicts": "h1_v1_drop_ci__hi < thr_auc_drop"
    },
    {
     "id": "V2",
     "applicable": "pop_h_highrate_no_early_min_train_l1_family >= min_pos_testable and pop_h_highrate_no_early_min_train_l1_run >= min_pos_testable and pop_kish_l1_runs_no_early >= min_runs_interpretable",
     "contradicts": "h1_v2_drop_ci__hi < thr_auc_drop"
    },
    {
     "id": "V3",
     "applicable": "pop_h_highrate_primary_min_train_l1_family >= min_pos_testable",
     "contradicts": "h1_v3_drop_ci__hi < thr_auc_drop"
    },
    {
     "id": "V5",
     "applicable": "pop_h_highrate_primary_min_train_l1_family >= min_pos_testable",
     "contradicts": "h1_v5_drop_ci__hi < thr_auc_drop"
    }
   ]
  },
  {
   "id": "H2",
   "name": "Unseen tool families, MTConnect signals",
   "plain": "Detectors of the labels built on the controller's spindle signals rank label 1 above label 0 paths of imi_vm20i less well on tool families they have not seen than on runs they have not seen. Test: for a logistic model on the {{s1_nfeat_imi_vm20i_mtc|word}} MTConnect spindle features, the estimated fall in pooled out-of-fold AUC is at least {{thr_auc_drop|f2}} and exceeds zero at the Holm-adjusted level.",
   "threshold_why": "At the Phase 2 screen's run-held-out AUC of {{s1_auc_imi_vm20i_mtc_run|f3}}, a fall of {{thr_auc_drop|f2}} would multiply the share of misordered pairs by {{err_mult_mtc|f1}}.",
   "rationale": "Spindle load and power scale with the cutting force, which depends on tool diameter and workpiece material, and the spindle-motor vibration levels depend on the tool as well. I expect the controller signals to carry more family-specific level than the high-rate amplitudes.",
   "against": "The spindle-motor vibration RMS alone separates the labels with an AUC of {{d3_auc_imi_vm20i_smarms|f2}} across all paths and {{d4_auc_imi_vm20i_smarms|f2}} within strata (D3, D4), so part of the MTConnect signal may carry over as well.",
   "direction": "h2_auc_run > h2_auc_family",
   "magnitude": "h2_drop >= thr_auc_drop",
   "precision": "h2_p_holm < alpha_family",
   "rule": "h2_drop >= thr_auc_drop and h2_p_holm < alpha_family",
   "testable": "pop_h_mtc_primary_min_train_l1_family >= min_pos_testable and pop_h_mtc_primary_min_train_l0_family >= min_pos_testable and pop_h_mtc_primary_min_train_l1_run >= min_pos_testable and pop_kish_l1_runs_primary >= min_runs_interpretable",
   "testable_runtime": "h2_nonfinite <= max_nonfinite_share",
   "satisfiable": [
    {
     "expr": "prec_auc_run_mtc > thr_auc_drop + auc_chance",
     "why": "As for H1, with the MTConnect screen AUC."
    }
   ],
   "variants": [
    {
     "id": "V1",
     "applicable": "pop_h_mtc_no_twins_min_train_l1_family >= min_pos_testable and pop_h_mtc_no_twins_min_train_l1_run >= min_pos_testable and pop_kish_l1_runs_no_twins >= min_runs_interpretable",
     "contradicts": "h2_v1_drop_ci__hi < thr_auc_drop"
    },
    {
     "id": "V2",
     "applicable": "pop_h_mtc_no_early_min_train_l1_family >= min_pos_testable and pop_h_mtc_no_early_min_train_l1_run >= min_pos_testable and pop_kish_l1_runs_no_early >= min_runs_interpretable",
     "contradicts": "h2_v2_drop_ci__hi < thr_auc_drop"
    },
    {
     "id": "V3",
     "applicable": "pop_h_mtc_primary_min_train_l1_family >= min_pos_testable",
     "contradicts": "h2_v3_drop_ci__hi < thr_auc_drop"
    },
    {
     "id": "V5",
     "applicable": "pop_h_mtc_primary_min_train_l1_family >= min_pos_testable",
     "contradicts": "h2_v5_drop_ci__hi < thr_auc_drop"
    }
   ]
  },
  {
   "id": "H3",
   "name": "Acc0 and Sound2 mark the same label 1 paths",
   "plain": "Among label 1 paths of imi_vm20i, the spindle accelerometer (Acc0) and the stethoscope sensor on the base column (Sound2) show an off-harmonic dominant line on the same paths more often than their dependence on spindle speed explains, which points to a vibration of the machine and away from an artefact of either sensor. Test: Cohen's kappa between the indicators is at least the threshold set by rule, the point {{thr_kappa_frac|pct0}} of the way from the kappa expected under independence within the spindle-speed classes to the largest kappa the per-class shares allow, and it exceeds that independence expectation at the Holm-adjusted level.",
   "threshold_why": "The rule asks for agreement well beyond what speed alone produces without asking for identical indicators. Its value depends on per-class shares within label 1, which would partly reveal the family comparison of E11 (family A dominates the upper class), so it is computed at execution, after the freeze.",
   "rationale": "Chatter-like vibration of the machine structure should reach both sensors on the same paths, while an artefact of a single sensor (clipping, interference, a mounting resonance) should not. Label 1 is broader than chatter, so within label 1 the paths with and without the line should split the same way in both sensors.",
   "against": "The sensors sit at different places and see different modes; the per-sensor shares in label 1 differ ({{d5_nhdom_imi_vm20i_acc0_l1|pct0}} against {{d5_nhdom_imi_vm20i_s2_l1|pct0}}, D5), and the comb leaves a larger share of the band off-harmonic at higher speeds ({{comb_offfrac_acc0_at_d5_min|pct0}} of the Acc0 band at {{d5_min_rpm|int}} r/min against {{comb_offfrac_acc0_at_speed_class_edge|pct0}} at {{speed_class_edge_rpm|int}} r/min).",
   "direction": "h3_kappa > h3_kappa_null",
   "magnitude": "h3_kappa >= h3_thr",
   "precision": "h3_p_holm < alpha_family",
   "rule": "h3_kappa >= h3_thr and h3_p_holm < alpha_family",
   "testable": "pop_h3_primary_lo_n >= min_pos_testable and pop_h3_primary_hi_n >= min_pos_testable and pop_h3_primary_lo_runs >= min_runs_interpretable and pop_h3_primary_hi_runs >= min_runs_interpretable and pop_kish_l1_runs_primary >= min_runs_interpretable",
   "testable_runtime": "h3_nonfinite <= max_nonfinite_share and h3_kappa_max > h3_kappa_null",
   "satisfiable": [
    {
     "expr": "kappa_max_l1_pooled > kappa_independent",
     "why": "With the speed classes pooled, the per-sensor shares in label 1 (D5) allow a kappa well above independence. Within classes the same holds unless the classes separate the indicators completely, which the runtime condition catches; the threshold then lies strictly between the null and the maximum by construction."
    }
   ],
   "variants": [
    {
     "id": "V1",
     "applicable": "pop_h3_no_twins_lo_n >= min_pos_testable and pop_h3_no_twins_hi_n >= min_pos_testable and pop_kish_l1_runs_no_twins >= min_runs_interpretable",
     "contradicts": "h3_v1_kappa_ci__hi < h3_v1_thr"
    },
    {
     "id": "V2",
     "applicable": "pop_h3_no_early_lo_n >= min_pos_testable and pop_h3_no_early_hi_n >= min_pos_testable and pop_kish_l1_runs_no_early >= min_runs_interpretable",
     "contradicts": "h3_v2_kappa_ci__hi < h3_v2_thr"
    },
    {
     "id": "V6a",
     "applicable": "pop_h3_primary_lo_n >= min_pos_testable and pop_h3_primary_hi_n >= min_pos_testable",
     "contradicts": "h3_v6a_kappa_ci__hi < h3_v6a_thr"
    },
    {
     "id": "V6b",
     "applicable": "pop_h3_primary_lo_n >= min_pos_testable and pop_h3_primary_hi_n >= min_pos_testable",
     "contradicts": "h3_v6b_kappa_ci__hi < h3_v6b_thr"
    },
    {
     "id": "V6c",
     "applicable": "pop_h3_primary_lo_n >= min_pos_testable and pop_h3_primary_hi_n >= min_pos_testable",
     "contradicts": "h3_v6c_kappa_ci__hi < h3_v6c_thr"
    },
    {
     "id": "V7",
     "applicable": "pop_h3_primary_lo_n >= min_pos_testable and pop_h3_primary_hi_n >= min_pos_testable",
     "contradicts": "h3_v7_kappa_ci__hi < h3_v7_thr"
    },
    {
     "id": "V9",
     "applicable": "pop_h3_primary_lo_n >= min_pos_testable and pop_h3_primary_hi_n >= min_pos_testable",
     "contradicts": "h3_v9_kappa_ci__hi < h3_v9_thr"
    }
   ]
  }
 ],
 "variants": [
  {
   "id": "V1",
   "name": "Without defective-tool twins",
   "change": "Drop every path cut with an asterisk tool (A*, B*, D*, E*).",
   "why": "A twin's label 1 may mark the defect rather than the cutting condition."
  },
  {
   "id": "V2",
   "name": "Without early-stopped runs",
   "change": "Drop the runs whose summary remark records an early stop ({{pop_early_runs_lab|int}} laboratory runs).",
   "why": "The event that stopped a run may have shaped the labels near it."
  },
  {
   "id": "V3",
   "name": "Random forest",
   "change": "Replace the logistic model by a random forest of {{rf_trees|int}} trees with leaves of at least {{rf_min_leaf|int}} paths and balanced class weights.",
   "why": "Tests whether the fall depends on the logistic model's linear logit."
  },
  {
   "id": "V5",
   "name": "Winsorised features",
   "change": "Clip every feature at the training-fold quantiles {{winsor_q|pct0}} from each end.",
   "why": "Tests whether a few extreme paths drive the fall."
  },
  {
   "id": "V6a",
   "name": "Narrow comb",
   "change": "Treat {{comb_bins_v6a|int}} bin on each side of a harmonic as harmonic, instead of {{d5_comb_bins|int}}.",
   "why": "Tests the comb width in one direction."
  },
  {
   "id": "V6b",
   "name": "Wide comb",
   "change": "Treat {{comb_bins_v6b|int}} bins on each side of a harmonic as harmonic, instead of {{d5_comb_bins|int}}.",
   "why": "Tests the comb width in the other direction."
  },
  {
   "id": "V6c",
   "name": "Relative comb",
   "change": "Widen each harmonic's zone to {{rel_speed_tol|pct1}} of its frequency wherever that exceeds {{d5_comb_bins|int}} bins.",
   "why": "Absorbs harmonics displaced by a spindle speed error that grows with harmonic order (A3). The MTConnect speed cannot serve instead, because it equals N on {{pop_mtc_speed_equals_n_share|pct1}} of paths."
  },
  {
   "id": "V7",
   "name": "Lower band edge",
   "change": "Search the dominant line up to {{band_hi_v7_hz|khz}} instead of {{d5_band_hi_hz|khz}}.",
   "why": "High harmonic orders are the most sensitive to speed error, and the upper band holds weaker lines."
  },
  {
   "id": "V9",
   "name": "Null within family and speed class",
   "change": "Compute the null and the attainable maximum within strata of tool family and spindle-speed class instead of speed class alone.",
   "why": "Removes agreement that the tool family could produce, so the prediction cannot pass on a difference between families."
  }
 ],
 "exploratory": [
  {
   "id": "E1",
   "idea": "I07",
   "question": "How common is full-scale clipping, by machine, sound channel and label, and how does the Sound0 RMS level off as its full-scale share grows?",
   "output": "Fig. 2 and text",
   "seen": "Clipping shares by machine, channel and label (D9)."
  },
  {
   "id": "E2",
   "idea": "I06",
   "question": "How well does the high-rate set rank the labels without Sound0, under run-held-out and family-held-out evaluation, next to the clipping-only score?",
   "output": "Table 3, Fig. 4",
   "seen": "Run-held-out AUC of the clipping-only score (Phase 2 screen)."
  },
  {
   "id": "E3",
   "idea": "I06",
   "question": "How far do the design variables alone go (spindle speed, depths of cut, feed per tooth, cutting direction and workpiece, with and without tool family) under run-held-out evaluation?",
   "output": "Table 3",
   "seen": "AUC of tool identity alone (S3)."
  },
  {
   "id": "E4",
   "idea": "I06",
   "question": "How do random path-level, run-held-out and family-held-out evaluation compare for every feature set on imi_vm20i, all with the balanced model of H1 and H2?",
   "output": "Table 3, Fig. 4",
   "seen": "Random and run-held-out AUCs of the unweighted Phase 2 screen model."
  },
  {
   "id": "E5",
   "idea": "I03",
   "question": "How does the share of paths with an off-harmonic dominant line, by label and channel (Acc0, Acc1, Sound1, Sound2), change across the comb settings of V6a, V6b and V6c and the band edges of {{band_hi_v7_hz|khz}} and {{d5_band_hi_hz|khz}}?",
   "output": "Text and supplementary table",
   "seen": "The shares at the Phase 2 settings (D5)."
  },
  {
   "id": "E6",
   "idea": "I03",
   "question": "Which label 0 paths carry off-harmonic lines in both Acc0 and Sound2 (candidate chatter the label missed), and which label 1 paths carry neither (candidate anomalies without such a line), by run and family?",
   "output": "Text, Fig. 5",
   "seen": "Nothing."
  },
  {
   "id": "E7",
   "idea": "I06",
   "question": "What happens to the AUC when whole workpiece materials are held out? The smallest training set keeps {{pop_material_min_train_l1|int}} label 1 paths; material and tool are confounded, so this cannot separate them.",
   "output": "Table 3",
   "seen": "Nothing."
  },
  {
   "id": "E8",
   "idea": "I06, I07, I03",
   "question": "What do clipping, the Sound2 off-harmonic shares, tool identity and a transfer of the MTConnect model trained on imi_vm20i show on imi_vmx30ui?",
   "output": "Text",
   "seen": "Clipping, Sound2 shares and tool identity on imi_vmx30ui (D9, D5, S3)."
  },
  {
   "id": "E9",
   "idea": "I03",
   "question": "Among label 1 paths where both indicators fire, how often do the Acc0 and Sound2 off-harmonic lines lie within {{freq_match_tol_hz|f1}} Hz of each other, against the rate when the Sound2 frequencies are paired with other paths of the same run?",
   "output": "Text",
   "seen": "Nothing."
  },
  {
   "id": "E10",
   "idea": "I06",
   "question": "How much of the pooled AUC under each scheme comes from pairs within a family and how much from pairs across families, and what is the within-family AUC of each qualifying family ({{pop_within_families_primary_list|text}}) when that family is held out?",
   "output": "Text, Table 3",
   "seen": "Nothing."
  },
  {
   "id": "E11",
   "idea": "I03",
   "question": "Among label 1 paths, how does the share with an off-harmonic dominant Acc0 line in family A compare with family E, which holds {{pop_imi_vm20i_e_l1|int}} of the {{pop_bytool_primary_other_n|int}} other label 1 paths? Estimate and family-stratified interval only; the effective number of runs is {{pop_kish_l1_runs_a|f1}} for A and {{pop_kish_l1_runs_e|f1}} for E, and spindle speeds differ between the families.",
   "output": "Text, Fig. 5",
   "seen": "The share in family A's label 0 paths on AA 6061 (D5)."
  }
 ],
 "methods": [
  {
   "id": "M1",
   "topic": "Unit and population",
   "choice": "The labelled cutting path (a label.csv row) on imi_vm20i, labels 0 and 1: {{pop_h_highrate_primary_n|int}} paths in {{pop_h_highrate_primary_runs|int}} runs, {{pop_h_highrate_primary_n_l1|int}} of them label 1 in {{pop_h_highrate_primary_runs_l1|int}} runs.",
   "why": "The labels are given per path. imi_vm20i is the only machine with accelerometers and with label 1 outside family A in more than a few runs.",
   "alternatives": "Fixed-length windows would multiply dependent samples under one path label. Pooling machines would confound every contrast with the machine. Label 2 marks defective tools, a different question, and stays out."
  },
  {
   "id": "M2",
   "topic": "Features",
   "choice": "The high-rate set: log RMS of Acc0, Acc1, Sound0, Sound1 and Sound2. The MTConnect set: logs of the spindle-motor vibration RMS, peak, crest factor and velocity RMS and of the true power, with the spindle load as recorded (it is not strictly positive). Both exactly as in the Phase 2 screen.",
   "why": "The question is what simple amplitude models learn from these labels. Logs suit positive, multiplicative levels.",
   "alternatives": "Spectral or learned features would turn the question into detector design."
  },
  {
   "id": "M3",
   "topic": "Model",
   "choice": "Standardised logistic regression with inverse penalty {{lr_c|f1}} and balanced class weights.",
   "why": "Few features and an expected monotone relation; the Phase 2 screen used the same model without class weights. Weights are needed because the label 1 share of the family-held-out training sets ranges from {{pop_h_highrate_primary_train_l1_share_min_family|pct0}} to {{pop_h_highrate_primary_train_l1_share_max_family|pct0}}. Balancing removes the part of each fold model's intercept that follows its own base rate; it does not remove the shifts that unseen feature levels cause, which are part of what H1 and H2 measure (E10 separates them).",
   "alternatives": "Random forest (V3). Boosted trees or neural networks add tuning choices without changing the question."
  },
  {
   "id": "M4",
   "topic": "Evaluation schemes",
   "choice": "Run-held-out: {{s1_folds|int}} folds of whole runs, repeated {{s1_repeats|int}} times with new fold assignments. Tool-family-held-out: a fold per family, {{pop_h_highrate_primary_families|int}} folds.",
   "why": "Families rather than tools, because a defective twin shares its good tool's geometry: holding A out while training on A* would leak the tool. Both schemes keep every path of a run on the same side of a split.",
   "alternatives": "Leave-material-out is confounded with the tool (E7, exploratory). Leave-one-run-out is close to the run scheme with many more fits."
  },
  {
   "id": "M5",
   "topic": "Metric",
   "choice": "Pooled out-of-fold AUC over all paths of the population, tied scores counted at the midrank. For the run scheme, the mean over repeats of each repeat's pooled AUC.",
   "why": "Threshold-free and comparable with the Phase 2 screen. Averaging AUCs rather than scores keeps the comparison like for like: in both schemes every score comes from a single fitted model, whereas averaging scores over repeats would give the run scheme an ensemble that the family scheme lacks.",
   "alternatives": "Within-family AUC answers a narrower question and is reported in E10. Average precision and balanced accuracy depend on a base rate or a threshold and are not used."
  },
  {
   "id": "M6",
   "topic": "Uncertainty",
   "choice": "Run bootstrap with predictions held fixed: {{boot_b|int}} resamples in which the runs of each tool family are drawn with replacement within the family, a single weight matrix shared by both schemes (q_confirm.boot_weights); percentile intervals at {{ci_level|pct0}}; one-sided p-values as the share of resamples at or below the null (zero for an AUC difference, the recomputed independence kappa for H3) with the plus-one rule; seed {{seed_analysis|raw}}.",
   "why": "Labels cluster within runs (the largest within-run share of label 1 paths is {{max_run_l1_path_share_imi_vm20i|pct0}}, Phase 2), so path-level methods such as DeLong's test would understate the uncertainty. Stratifying by family keeps every family in every resample, so the family-held-out estimand keeps its composition (family B has a single run).",
   "alternatives": "An unstratified run bootstrap would drop single-run families from many resamples. Refitting inside every resample would put copies of a run in training and test folds at once. Holding predictions fixed leaves out the variability of refitting; the text says so."
  },
  {
   "id": "M7",
   "topic": "Spectral indicator",
   "choice": "The indicator is on when the largest line of the path's Welch spectrum between {{d5_band_lo_hz|int}} Hz and {{d5_band_hi_hz|khz}} lies more than {{d5_comb_bins|int}} bins from every harmonic of the commanded spindle frequency, and off otherwise; defined for spindle speeds of at least {{d5_min_rpm|int}} r/min; the spectra of the Phase 2 device pass, over the whole path.",
   "why": "Identical to D5, so the per-label shares already reported stay valid. It has no tuned threshold and treats every sensor alike.",
   "alternatives": "Energy-ratio or entropy chatter indices need thresholds calibrated against a ground truth that does not exist here."
  },
  {
   "id": "M8",
   "topic": "Co-occurrence (H3)",
   "choice": "Among label 1 paths with both spectra, Cohen's kappa between the Acc0 and Sound2 indicators; null: the kappa expected if the indicators were independent within the spindle-speed classes (below, or at and above, {{speed_class_edge_rpm|int}} r/min), recomputed in every resample; threshold rule {{thr_kappa_frac|pct0}} of the way from that null to the attainable maximum, evaluated at execution; all of it in q_confirm.cooccurrence.",
   "why": "Restricting to label 1 removes the agreement that both indicators' link to the label would produce, including the rare label 0 lines that cluster in family A at high speed. The speed classes remove agreement that the comb geometry alone produces. The class edge leaves more than one family and more than {{min_pos_testable|int}} paths in each class ({{pop_h3_primary_lo_n|int}} and {{pop_h3_primary_hi_n|int}}).",
   "alternatives": "Agreement over both labels would be dominated by the label (the reviewed draft did this; RV1). A frequency-matched co-occurrence is stricter but needs a matching rule without a validated tolerance, so it is exploratory (E9). Sound1 has a weak indicator ({{d5_nhdom_imi_vm20i_s1_l1|pct0}} of label 1 paths) and stays exploratory (E5)."
  },
  {
   "id": "M9",
   "topic": "Frozen estimators",
   "choice": "`q_confirm.py` holds one function per prediction family (`auc_drop` for H1 and H2, `cooccurrence` for H3), every variant as the single setting it changes (`VARIANTS`, `VARIANTS_FOR`, `population`, `h3_strata`), the stratified bootstrap (`boot_weights`) and the Holm step (`holm_adjust`); the Phase 4 script only loops over them with the registered settings.",
   "why": "The confirmatory computation is fixed before any result exists, and the check exercises it on synthetic data.",
   "alternatives": "Freezing only building blocks would leave pairing, strata and missing-value handling to Phase 4 (the reviewed draft did this; RV7)."
  },
  {
   "id": "M10",
   "topic": "Multiplicity",
   "choice": "Holm over the predictions that pass both testability conditions, at a family-wise level of {{alpha_family|f2}}.",
   "why": "Controls the family-wise error without assuming independence between the predictions.",
   "alternatives": "Bonferroni is uniformly more conservative; no correction would inflate the error rate."
  },
  {
   "id": "M11",
   "topic": "Software",
   "choice": "Python {{env_python|text}}, scikit-learn {{env_sklearn|text}}, NumPy {{env_numpy|text}}, pandas {{env_pandas|text}} and Matplotlib {{env_matplotlib|text}}; analysis functions in q_common.py, estimators in q_confirm.py and rule evaluation in q_rules.py, all frozen with this plan.",
   "why": "The Phase 4 check can then assert that the confirmatory code is the code registered here.",
   "alternatives": "None considered."
  }
 ],
 "assumptions": [
  {
   "id": "A1",
   "assumption": "Runs are independent units, exchangeable within the machine.",
   "for": "Every interval and p-value",
   "test": "No test can settle it. Phase 4 reports, for every confirmatory statistic, the largest change when a single run is left out.",
   "action": "Reported with each prediction. A rule that flips when a single run is left out is flagged in the text; the class does not change."
  },
  {
   "id": "A2",
   "assumption": "A linear logit in the log features is adequate.",
   "for": "H1, H2",
   "test": "V3 (random forest).",
   "action": "If V3's interval lies wholly below the threshold while H1 or H2 is confirmed, the prediction is CONFIRMED_NOT_ROBUST."
  },
  {
   "id": "A3",
   "assumption": "The commanded spindle speed places the harmonic comb within its tolerance.",
   "for": "H3",
   "test": "Done in Phase 3 without labels: the dominant line lies above harmonic order {{comb_check_order_floor|int}} and between {{d5_comb_bins|int}} and {{comb_check_window_bins|int}} bins from a harmonic in {{comb_check_acc0_share_displaced|pct1}} of paths for Acc0 and {{comb_check_s2_share_displaced|pct1}} for Sound2. A harmonic displaced further (a speed error of a fraction of a percent moves a high-order harmonic by several bins) would escape this count, so V6c widens each harmonic's zone in proportion to its frequency, and V6b and V7 test the comb and band.",
   "action": "Reported; a V6b, V6c or V7 interval wholly below its threshold makes a confirmed H3 CONFIRMED_NOT_ROBUST."
  },
  {
   "id": "A4",
   "assumption": "The dominant line describes steady cutting.",
   "for": "H3",
   "test": "The stored spectra cover whole paths, entry and exit included, and no trimmed spectra exist. Phase 4 reports H3 again on paths of at least {{d7_min_dur_s|int}} s, where entry and exit weigh less.",
   "action": "Reported as a check; accepted as a limitation."
  },
  {
   "id": "A5",
   "assumption": "The Acc0 and Sound2 spectra of a path cover the same time span.",
   "for": "H3",
   "test": "Done in Phase 2 (D8): acceleration and sound envelopes peak within {{d8_lag_abs_max_s|f1}} s of each other in {{d8_share_lag_le01|pct0}} of the pairings tested.",
   "action": "None needed."
  },
  {
   "id": "A6",
   "assumption": "Sound2 clipping does not drive its indicator.",
   "for": "H3",
   "test": "Sound2 has more than {{d9_threshold|pct0}} of samples at full scale in {{d9_paths_gt1pct_imi_vm20i_s2|pct1}} of imi_vm20i paths (D9). Phase 4 reports H3 again without those paths.",
   "action": "Reported as a check."
  }
 ],
 "missing": [
  "Every labelled path on imi_vm20i has all high-rate and all MTConnect features ({{pop_h_highrate_primary_n|int}} paths for the high-rate set, {{pop_h_mtc_primary_n|int}} for the MTConnect set, out of {{pop_h_highrate_primary_n|int}} labelled paths). A path that loses a feature in Phase 4 leaves that analysis and is counted.",
  "The spectral indicator is undefined below {{d5_min_rpm|int}} r/min. That removes {{pop_agree_excluded_slow|int}} paths from the spectral analyses, and the count of label 1 paths among them is {{pop_agree_excluded_slow_l1|int}}, so H3 keeps every label 1 path. This is exclusion by definition, and the text says so.",
  "Label 2 marks defective tools and stays outside every analysis.",
  "A bootstrap draw in which a statistic is undefined is counted, never dropped silently. If more than {{max_nonfinite_share|pct0}} of the draws of a prediction are undefined, or its estimate is not a finite number, the prediction is NOT_TESTED (q_rules.testable).",
  "No imputation anywhere."
 ],
 "outliers": [
  "No path is removed as an outlier. Amplitudes enter as logs, which compresses their range; V5 clips every feature at the training-fold quantiles {{winsor_q|pct0}} from each end to test whether extreme paths drive H1 or H2.",
  "Short paths stay in; A4 reports H3 again on paths of at least {{d7_min_dur_s|int}} s."
 ],
 "multiplicity": [
  "Confirmatory family: H1 to H3, Holm step-down at a family-wise level of {{alpha_family|f2}} over the predictions that pass both testability conditions (q_confirm.holm_adjust).",
  "Variants: no significance test enters their verdict. A variant contradicts its prediction only when its whole interval lies below the prediction's threshold; its p-value is reported unadjusted, as a description.",
  "Exploratory analyses: estimates and run-bootstrap intervals only, no p-values."
 ],
 "preregistration": {
  "statement": "Confirmatory: the predictions H1 to H3, each with the decision rule in section 5, the Holm correction in section 8 and the variants in section 9, computed by the functions in q_confirm.py. Everything else in the paper is exploratory or restates Phase 2. The plan (inputs/plan.json), the analysis functions (code/q_common.py), the estimators (code/q_confirm.py) and the rule evaluator (code/q_rules.py) are frozen with their sha256 at this stop. Any change in Phase 4 goes into the deviations log, with its reason, before the affected result is computed.",
  "seen": [
   {
    "what": "Label structure by tool, material and run, with label shares by path and by time",
    "phase": "Phase 2 (D1)",
    "values": "d1_*, lab_*, path_share_*, time_share_*, l1_*, max_run_l1_path_share_*"
   },
   {
    "what": "Cutting direction and within-run parameter AUCs",
    "phase": "Phase 2 (D2)",
    "values": "d2_*"
   },
   {
    "what": "AUC of single features for label 1 against label 0, across all paths and within strata",
    "phase": "Phase 2 (D3, D4)",
    "values": "d3_*, d4_*"
   },
   {
    "what": "Off-harmonic shares by label and channel, and in family A's label 0 paths on AA 6061",
    "phase": "Phase 2 (D5)",
    "values": "d5_*, d5x_*"
   },
   {
    "what": "Clipping shares by machine, channel and label",
    "phase": "Phase 2 (D9)",
    "values": "d9_*"
   },
   {
    "what": "Random and run-held-out AUCs of the screen's feature sets (high-rate, MTConnect, Sound2 only, clipping only) on both laboratory machines",
    "phase": "Phase 2 screen",
    "values": "s1_*"
   },
   {
    "what": "AUC of tool identity alone",
    "phase": "Phase 2 screen",
    "values": "s3_*"
   },
   {
    "what": "Instrument, timing and matched-tool checks, and the reproduction of the descriptor's tables",
    "phase": "Phase 2 (D6 to D8, D10)",
    "values": "d6_*, d7_*, d8_*, d10_*, doc_*, t4_*"
   },
   {
    "what": "Population counts, training-set counts and shares, effective run counts, speed-class counts, the pooled attainable kappa (from the D5 shares), the label-free comb check, comb geometry, and the run-held-out AUC of the screen model with its precision and the error multipliers",
    "phase": "Phase 3 (this plan)",
    "values": "pop_*, kappa_max_l1_pooled, comb_check_*, comb_offfrac_*, prec_*, err_mult_*"
   }
  ],
  "not_seen": [
   "Any score from a model trained without a tool family (H1, H2, V3, V5, E2, E4, E10) or without a material (E7)",
   "Any AUC of the logistic model with balanced class weights",
   "Any cross-tabulation of the off-harmonic indicators of different sensors on the same paths (H3, V9, E6, E9)",
   "Any off-harmonic share by tool family within label 1, or by speed class within label 1 (H3 threshold, E11)",
   "Any statistic of the design-only model (E3) or of the high-rate set without Sound0 (E2)"
  ],
  "seen_note": "Every value in values.json (Phase 2) was seen; the table groups the ones this plan relies on, and the check lists the rest by prefix. One Phase 2 sentence needs a correction: it says that most label 1 paths carry an off-harmonic line in both accelerometers and in Sound2. That describes the per-channel shares; no joint count across channels was ever computed, and the paper will not use the sentence."
 },
 "taxonomy": {
  "order": "q_rules.classify applies the tests in this order: testable (the population condition, then the runtime condition), a finite value for every name the rule reads, direction, magnitude, precision, then the applicable variants. The first test that fails fixes the class.",
  "classes": [
   {
    "id": "CONFIRMED",
    "definition": "The rule holds and no applicable variant contradicts the prediction.",
    "reporting": "Stated as confirmed, with the estimate and interval."
   },
   {
    "id": "CONFIRMED_NOT_ROBUST",
    "definition": "The rule holds, but the whole interval of at least one applicable variant lies below the prediction's threshold.",
    "reporting": "This is the prediction that satisfies its decision rule but fails the looser plain-language wording, which speaks of detectors or sensors in general and not of one model and one set of settings. The text says the rule was met and names every contradicting variant; the abstract may not state the plain-language claim without that qualifier. An applicable variant that could not be computed is reported as missing, with its reason, next to the class."
   },
   {
    "id": "FAILED_DIRECTION",
    "definition": "The estimate lies on the wrong side of the null.",
    "reporting": "Stated as failed, with the estimate and interval, in the abstract and results."
   },
   {
    "id": "FAILED_MAGNITUDE",
    "definition": "The estimate lies on the predicted side but short of the threshold.",
    "reporting": "Stated as failed; the text may say the direction held but may not call the effect confirmed."
   },
   {
    "id": "FAILED_PRECISION",
    "definition": "The estimate reaches the threshold but the Holm-adjusted p-value is not below the family-wise level.",
    "reporting": "Stated as failed for lack of precision, with the interval."
   },
   {
    "id": "NOT_TESTED",
    "definition": "A testability condition is false at execution, or a value the rule needs is missing or not finite.",
    "reporting": "Stated as not tested, with the reason; never counted as confirmed or failed."
   }
  ],
  "rule_vs_wording": "Failing the decision rule means one of the FAILED classes (direction, magnitude, precision). Meeting the rule while failing the plain-language wording means CONFIRMED_NOT_ROBUST. No other distinction will be introduced later.",
  "tally": "The Phase 4 check script classifies every prediction with q_rules.classify and writes `tally_confirmed`, `tally_confirmed_not_robust`, `tally_failed` (and its parts `tally_failed_direction`, `tally_failed_magnitude`, `tally_failed_precision`), `tally_not_tested` and `tally_total` into the values file, with each prediction's class as `h1_class` to `h3_class`. The abstract, introduction, results, conclusions and appendix cite these tokens and nothing else."
 },
 "structure": [
  {
   "section": "Whole manuscript",
   "content": "Abbreviations defined at first use, SI units, a main text within the self-imposed length ceiling, a neutral tone about every named work and group.",
   "rules": [
    "R06",
    "R07",
    "R26",
    "R27",
    "R30",
    "R31",
    "R32",
    "R33"
   ]
  },
  {
   "section": "Title page",
   "content": "Title, authors in the pipeline's order with affiliations (institution, department, city, state, country) and ORCIDs, the corresponding author with an e-mail address, abstract, keywords, acknowledgments if any.",
   "rules": [
    "R02",
    "R03",
    "R04",
    "R18",
    "R22",
    "R24",
    "R28"
   ]
  },
  {
   "section": "1 Introduction",
   "content": "Why the content of benchmark labels matters; the MSM dataset and its published users U1 and U2, described neutrally, with U3 as decided at Q4; the gap; contributions.",
   "rules": [
    "R06",
    "R08",
    "R09",
    "R26",
    "R29"
   ]
  },
  {
   "section": "2 Dataset and populations",
   "content": "Machines, sensors, tools, materials and labels; Table 1 and Fig. 1.",
   "rules": [
    "R07",
    "R11",
    "R16"
   ]
  },
  {
   "section": "3 Methods",
   "content": "Features and models; evaluation schemes and metric; spectral indicator with Fig. 3; statistical analysis and pre-registration (predictions, Holm, taxonomy); use of language models.",
   "rules": [
    "R05",
    "R07",
    "R19"
   ]
  },
  {
   "section": "4 Results",
   "content": "What was seen before registration (loudness, clipping, tool identity, spectral shares; Fig. 2); the pre-registered predictions (Table 2, Fig. 4, Fig. 5 panel a); robustness (Table 4); exploratory analyses, the family comparison E11 included (Table 3, Fig. 5 panel b).",
   "rules": [
    "R11",
    "R12",
    "R13",
    "R14",
    "R15",
    "R16"
   ]
  },
  {
   "section": "5 Discussion",
   "content": "What the labels encode; how to evaluate detectors on this dataset (split by run and by tool family, report clipping); limitations (no ground truth, a single machine for the tests, tool family, material and spindle speed confounded).",
   "rules": [
    "R26"
   ]
  },
  {
   "section": "6 Conclusions",
   "content": "Findings with the computed tally.",
   "rules": []
  },
  {
   "section": "Statements and Declarations",
   "content": "Funding, competing interests, ethics approval and consent (not applicable: public data, no human or animal participants), data availability (the MSM dataset, CC BY 4.0, with its DOI), code availability, author contributions (the pipeline's CRediT statement unchanged). Placement pending Q2.",
   "rules": [
    "R17",
    "R20"
   ]
  },
  {
   "section": "References",
   "content": "Numbered in square brackets, full DOI links, the dataset cited by its DOI.",
   "rules": [
    "R08",
    "R09",
    "R29"
   ]
  },
  {
   "section": "Electronic supplementary material",
   "content": "Online Resources: split assignments, code with the values file, per-path indicators; each named ESM with its number, captioned, cited in the text and carrying the title-page details.",
   "rules": [
    "R20",
    "R21"
   ]
  },
  {
   "section": "Submission files",
   "content": "A flat folder with the source, the bibliography database, the template's class and style files, the figure files and the compiled PDF.",
   "rules": [
    "R10",
    "R13"
   ]
  }
 ],
 "failure_modes": [
  {
   "id": "W1",
   "title": "The tool result reads as a property of one experimental design",
   "probability": "p_fail_tool",
   "mechanism": "Family A holds {{pop_imi_vm20i_a_l1|int}} of the {{pop_h_highrate_primary_n_l1|int}} label 1 paths and was cut almost only on AA 6061 at the highest spindle speeds. Whatever H1 and H2 show, a reviewer can say it describes how this dataset was built, since family, material and speed move together, and the recommendation to hold tool families out then rests on the single fold that holds A out.",
   "signal": "The fold that holds A out carries most of the fall, or E10 shows the fall comes only from pairs across families.",
   "response": "Present the result as a property of this benchmark that every user has to handle, which is the purpose of an audit, name material and speed with the family everywhere, and report the within-family AUCs of A and E."
  },
  {
   "id": "W2",
   "title": "The spectral indicator is judged unvalidated",
   "probability": "p_fail_indicator",
   "mechanism": "Without ground truth, a failed H3 leaves the off-harmonic line as an unexplained feature of each sensor and the chatter part of the title cannot stand. A passed H3 can still be attributed to shared causes that the speed classes and V9 do not remove.",
   "signal": "H3 fails, or V9 contradicts it.",
   "response": "Drop chatter from the title and the claims, and report the indicator as a spectral descriptor."
  },
  {
   "id": "W3",
   "title": "Nothing confirmatory survives",
   "probability": "p_fail_null",
   "mechanism": "If H1 to H3 all fail or are not robust, the paper rests on the Phase 2 findings. A reviewer can call the loudness result obvious if the annotators judged by ear, and an audit of a single dataset then reads as a comment.",
   "signal": "The computed tally.",
   "response": "Stop at the Phase 4 self-audit and bring Adam the choice between a short communication and abandoning S1."
  }
 ],
 "confidence": {
  "value": "conf_plan",
  "target": "conf_plan_target",
  "raise": "A confirmed and robust H1 or H2 together with a confirmed H3 would raise it; so would a reading of the public Kaggle notebook and a Scopus or Web of Science search that finds no competing audit (this pipeline has no access to either database).",
  "lower": "A published audit of these labels, or a corrected release of the dataset, would lower it."
 },
 "decisions": [
  {
   "decision": "Paper S1 (what the expert labels encode) for The International Journal of Advanced Manufacturing Technology, as an Original Article",
   "reason": "Adam's choice at the Phase 2 stop; the journal published U1 on this dataset under that type."
  },
  {
   "decision": "A single submission format: LaTeX with the Springer Nature template and the [iicol] option, figures as EPS converted by epstopdf",
   "reason": "The guidelines recommend LaTeX with that option and prefer EPS; they do not require Word."
  },
  {
   "decision": "Confirmatory scope: H1 to H3 on imi_vm20i; imi_vmx30ui exploratory only",
   "reason": "Accelerometers, and label 1 outside family A in more than a few runs, exist only on imi_vm20i."
  },
  {
   "decision": "The family comparison of the off-harmonic line moved from a confirmatory prediction to exploratory analysis E11",
   "reason": "The label 1 paths of the families other than A sit in {{pop_kish_l1_runs_other|f1}} effective runs, below the floor of {{min_runs_interpretable|int}}; decided before any share by family was computed (review item RV4)."
  },
  {
   "decision": "H3 restricted to label 1, with a null of independence within spindle-speed classes and the threshold computed at execution",
   "reason": "Agreement over both labels would be dominated by the label and by the label 0 lines of family A at high speed (review item RV1)."
  },
  {
   "decision": "Primary metric: pooled out-of-fold AUC of a logistic model with balanced class weights; the within-family and across-family decomposition as exploratory analysis E10",
   "reason": "The label 1 share of the family-held-out training sets ranges from {{pop_h_highrate_primary_train_l1_share_min_family|pct0}} to {{pop_h_highrate_primary_train_l1_share_max_family|pct0}}; the within-family AUC answers a narrower question (review item RV2)."
  },
  {
   "decision": "A variant contradicts its prediction only when its whole interval lies below the threshold; only variants that change a nuisance choice enter the verdict",
   "reason": "A point-estimate rule would turn the lost power of smaller subsets into verdicts (review item RV3)."
  },
  {
   "decision": "Run bootstrap stratified by tool family, one weight matrix shared by both schemes",
   "reason": "Keeps single-run families in every resample (review item RV9)."
  },
  {
   "decision": "Confirmatory estimators frozen as code (q_confirm.py) and tested on synthetic data in the check",
   "reason": "Leaves no pairing, strata or missing-value choice to Phase 4 (review item RV7)."
  },
  {
   "decision": "Spectral indicator kept exactly as in D5, with comb and band variants, the relative comb V6c included",
   "reason": "Keeping D5 keeps the reported shares valid; the MTConnect speed equals N and cannot correct the comb."
  },
  {
   "decision": "{{budget_n_tables|wordcap}} tables and {{budget_n_figures|word}} figures in simple layouts, with the Fig. 3 examples chosen by a fixed rule",
   "reason": "Each item maps to a prediction or an exploratory question; a fixed rule prevents picking a flattering spectrum."
  },
  {
   "decision": "Checklist rebuilt section by section after the review: {{rules_n|int}} rules on {{quotes_n_cited|int}} verified quotes, every heading of the guidelines page accounted for",
   "reason": "The first extraction missed rules on references, the title page, figures and supplementary files (review item RV8)."
  }
 ],
 "questions": [
  {
   "id": "Q1",
   "topic": "Reference style",
   "question": "The pipeline asks for Vancouver references. IJAMT asks for numbered citations in square brackets and gives its own entry format (authors, then the year in parentheses, then the title and the abbreviated journal name), which its style file sn-basic.bst produces. Which should Phase 4 use?",
   "options": [
    "The journal's format through sn-basic.bst (recommended; it is what the editors expect, and it is what you chose for the PHMSA paper, as the decisions log in mechfin/project_state.md records)",
    "Vancouver entries with numbered square-bracket citations, against the template"
   ]
  },
  {
   "id": "Q2",
   "topic": "Placement of the declarations",
   "question": "The guidelines place a 'Statements and Declarations' section after the references (phrased as must), a 'Declarations' section for funding and competing interests before the reference list, and also name a 'Compliance with Ethical Standards' section. Where should Phase 4 put them?",
   "options": [
    "One 'Statements and Declarations' section after the references, covering every required statement (recommended; the only instruction phrased as must)",
    "Wherever the downloaded template places its declarations section"
   ]
  },
  {
   "id": "Q3",
   "topic": "Template download",
   "question": "May Phase 4 download the Springer Nature LaTeX template package ({{v_template_version|text}} version, a ZIP file of {{v_template_zip_kb|f2}} KB) from the Springer Nature author support page ({{v_sn_latex_url|text}})?",
   "options": [
    "Yes, at the start of Phase 4",
    "No; you will supply the template yourself"
   ]
  },
  {
   "id": "Q4",
   "topic": "The unpublished repository U3",
   "question": "U3 (the HBAF repository with an unpublished manuscript) is the closest prior use of this dataset with experiment-disjoint splits. The guidelines say unpublished works should only be mentioned in the text, and in the numbered reference rules that they should not appear in the text at all. How should Phase 4 handle it?",
   "options": [
    "Mention it in the text with its URL and keep it out of the reference list (recommended; honest about prior work, and follows the first instruction)",
    "Leave it out of the paper and describe the split argument without it",
    "Ask the editorial office before submission"
   ]
  },
  {
   "id": "Q5",
   "topic": "Title page details",
   "question": "The title page needs every affiliation as institution, department, city, state and country, and a named corresponding author with an e-mail address. The pipeline gives institutions and e-mail addresses only and names no corresponding author. Who is the corresponding author, and what are the departments and locations?",
   "options": [
    "Adam Park as corresponding author, with the e-mail address the pipeline gives and the departments and locations you supply (recommended; the PHMSA paper assumed the same author)",
    "Another author as corresponding author"
   ]
  }
 ],
 "declared_not_applicable": [],
 "variant_note": "V4, V8 and V10 of the reviewed draft are gone: V4 became exploratory analysis E10 and V8 and V10 left with the family comparison (review items RV2 to RV4). The remaining ids are kept so that the review record stays readable."
}