# Establishing Objective Ground-Truth for Pediatric ADHD Engagement

This repository contains the official implementation and methodological framework for generating an objective, clinically grounded benchmark dataset for pediatric ADHD engagement, as presented in our paper: *"Establishing Objective Ground-Truth for Pediatric ADHD Engagement: A Methodological Framework and Benchmark Dataset."*

## Overview
The framework bridges the gap between subjective human labeling and objective clinical performance. It anchors engagement labels to standardized metrics derived from the Integrated Visual and Auditory Continuous Performance Test (IVA-2).

## Repository Structure

```text
├── annotation/             # Clinical Data Processing & Label Generation
│   ├── extracted_iva2_.../ # Anonymized clinical scales (CSV)
│   ├── results/            # Generated Engagement Scores and Validation Plots
│   └── main.py             # 15-step validation & annotation pipeline
├── feature_extraction/     # Video Feature Processing (OpenFace 2.0)
│   ├── results/            # Filtered, non-identifiable time-series features
│   └── main.py             # Feature cleaning and final dataset (.pkl) creation
│   └── datasets            # Generated final dataset
├── baseline_models/        # Benchmark AI Experiments
│   ├── results/            # Performance logs for RF, LSTM, and etc.
│   └── main.py             # Leave-One-Subject-Out (LOSO) benchmarks
├── log.py                  # Centralized logging configuration
└── README.md               # Documentation

```

## Ethical Note on Data Availability

Due to the sensitive nature of pediatric clinical data:

1. **Raw Videos:** Not included to protect participant privacy.
2. **Biometric Data:** Raw HOG features and absolute landmarks are excluded.
3. **Provided Features:** We provide non-identifiable time-series data: Head Pose (6), Eye Gaze (8), and Facial Action Units (17).
4. **Clinical Data:** All IVA-2 scores are provided as anonymized CSV files.

## Getting Started

### 1. Installation

#### Data & Code Setup
To evaluate this framework, the code and dataset are provided as two separate Supplementary files due to size constraints:

1. **Extract** `Supplementary_Data_S1_Code.zip` and `Supplementary_Data_S2_Dataset.zip` to your local directory.
2. **Place** the `final_dataset_window_size_10.pkl` (from `Supplementary_Data_S2_Dataset.zip`) into the following directory:
   `feature_extraction/datasets/`
   
Once the file is placed, you can proceed directly to running the validation (Step 15 in `annotation/main.py`) or the benchmarks (`baseline_models/main.py`).

**Note on Manual Generation:**
If you prefer to generate the `final_dataset_window_size_10.pkl` file manually rather than using the provided `Supplementary_Data_S2_Dataset.zip`, please follow these steps:
1. Run `annotation/main.py` (Steps 1–14) to generate the clinical labels in `annotation/results/`.
2. Run `feature_extraction/main.py` (Step 3) to merge these labels with the non-identifiable features.
3. The dataset will then be automatically generated at: `./feature_extraction/results/final_dataset_window_size_10.pkl`.


Ensure you have Python 3.9+ and required packages in requirements.txt

### 2. Execution Order

The scripts are interdependent and must be run in the following sequence:

1. **Run `annotation/main.py` (Steps 1–14):** Generates clinical labels. Note: Step 8 is disabled by default to maintain participant privacy (requires raw video input)
2. **Run `feature_extraction/main.py`:** This combines your OpenFace features with the labels from 1.
3. **Run `annotation/main.py` (Step 15):** Now that the dataset exists, this step can perform final ML-to-clinical analysis.
4. **Run `baseline_models/main.py`:** Final benchmarks.

#### Step A: Generate Clinical Labels

Processes the IVA-2 reports to create the objective ground truth.

```bash
cd annotation
python3 main.py

```

#### Step B: Feature Preparation & Dataset Creation

Merges the filtered OpenFace features with the labels generated in Step A.

```bash
cd ../feature_extraction
python3 main.py

```

#### Step C: Run Benchmarks

Train and evaluate the baseline models (Random Forest, LSTM, etc) using LOSO cross-validation.

```bash
cd ../baseline_models
python3 main.py

```

## Reproducing Sensitivity Analysis
The study differentiates between the full cohort (N=21) and the cohort used for specific clinical validations (N=18 after excluding participants with missing clinical IVA-2 global scales).

To reproduce these specific analyses in `annotation/main.py`:
- **For Main Manuscript Results:** Ensure the `excluded_list` in `main.py` is active: 
  `excluded_list = ['ruser2', 'ruser16', 'ruser21']`
- **For Full Cohort Analysis:** Comment out the list: 
  `excluded_list = []`

The logs in the `/results` folder provide the outputs for both configurations as reported in the manuscript.

## Key Results

* **Convergent Validity:** AUC = 0.97 against clinical IVA-2 global scales.
* **Reliability:** Internal consistency (r = 0.84).
* **Benchmarks:** Static Random Forest currently outperforms temporal models, highlighting a "Complexity Gap" in pediatric engagement monitoring.

## Citation

If you use this framework or dataset in your research, please cite:
*(Citation details upon publication)*

## License

* **Code:** Licensed under the MIT License.
* **Dataset Features:** Provided under Creative Commons Attribution 4.0 International (CC BY 4.0).



