\documentclass[fleqn,10pt]{wlscirep}

\usepackage{multirow}
\usepackage{subcaption}
\usepackage{algorithm}
\usepackage{algpseudocode}
\newcommand{\increase}{\textcolor{green!70!black}{$\boldsymbol{\uparrow}$}}
\newcommand{\decrease}{\textcolor{red}{$\boldsymbol{\downarrow}$}}
\newcommand{\neutral}{\textcolor{gray}{$-$}} % For negligible 
%
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\title{SOD-FE: A Supervised Outlier Detection and Feature Engineering Approach for Student Dropout Prediction}

\author[1]{Sabah Saad}
\author[1]{Önder Yakut}
\author[2,3,*]{Eissa Alzabidi}
\author[3]{Oğuz Fındık}

\affil[1]{Department of Information System Engineering, Kocaeli University, Kocaeli, Turkey}
\affil[2]{Department of Computer Engineering, Hodeidah University, Hodeidah, Yemen}
\affil[3]{Department of Computer Engineering, Karabük University, Karabük, Turkey}

\affil[*]{eissaalzabidi@hoduniv.net.ye}

\begin{abstract}
Student dropout in higher education creates academic and socio-economic challenges for institutions and students. Effective early prediction models are essential to identify at-risk students and implement timely interventions. This paper proposes SOD-FE, a supervised machine learning approach that combines label-aware outlier detection with feature engineering to enhance dropout prediction. The approach integrates interquartile range (IQR) based outlier detection with mutual information and Pearson correlation to identify and mitigate the impact of outliers before constructing the final model. Then, a feature selection strategy is applied to refine the dataset. The approach is evaluated through experiments on two real-world datasets (Portugal and Slovakia) utilizing five classification algorithms, including Random Forest (RF) and Extreme Gradient Boosting (XGB). Performance increased greatly with the RF classifier, achieving F1 scores of 98.09\% and 98.33\% on two benchmark datasets, using 5-fold cross-validation. The proposed approach also incorporates explainable AI techniques (SHAP) to enhance model transparency and support data-driven educational policy. These findings show the significant potential of the SOD-FE method for improving student retention and early intervention systems in educational institutions.

\end{abstract}

% \keywords{Student Dropout Prediction, Supervised Outlier Detection, Feature Engineering, Machine Learning, XAI}

\begin{document}

\flushbottom
\maketitle

\thispagestyle{empty}

\textbf{Keywords:} Student Dropout Prediction, Supervised Outlier Detection, Feature Engineering, Machine Learning, XAI.

\section*{Introduction}

Student dropout in higher education is a serious problem that affects not just the students but also educational institutions \cite{alhazmi2023early}. High dropout rates have long-term negative consequences, including financial losses, loss of face for the school, and drastic ramifications in the lives of those who fail to finish their studies. At-risk dropouts typically possess issues that may lead to antisocial behaviors or difficulties in the labor market \cite{chung2019dropout}. The ability to predict student dropout is critical to help improve student retention and academic performance in educational institutions \cite{schmitt2021wwh}. In recent years, the application of machine learning (ML) techniques in the education sector has garnered significant attention, particularly in the context of predicting student academic success \cite{mulyani2019dropout}.The application of these techniques can be used to aid in the development of early warning systems and identify at-risk students using demographic, socioeconomic, macroeconomic and academic datasets that have been developed \cite{Rabelo2024AMF}. Several ML methods were used in previous studies to predict student performance, such as decision tree (DT), logistic regression (LR), SVM, and ensemble methods such as RF, XGB, etc. In addition, various resampling techniques such as SMOTE \cite{han2005borderline}, ADASYN \cite{he2008adasyn}, etc., were utilized. These algorithms proved to be effective in the binary classification of students as graduates or dropouts \cite{10498520, ZEINEDDINE2021106903}.

This paper presents a proposed supervised outlier detection and feature engineering (SOD-FE) approach developed to improve the effectiveness of student dropout prediction models. The primary objective of this approach is to enhance data quality and model performance by applying target-aware outlier filtering and informed feature selection prior to training the final classifier. Unlike traditional unsupervised outlier detection techniques, the proposed method utilizes label information in conjunction with mutual information and Pearson correlation to identify the most relevant features and eliminate instances that are likely to introduce noise or bias \cite{fernandez2022supervised}. Outliers are detected using an Interquartile Range (IQR)-based statistical filter, which is tuned concerning class distribution, ensuring that critical patterns, particularly those related to minority dropout classes, are preserved \cite{vinutha2018detection}.

In addition to its foundational elements, the SOD-FE approach offers several key contributions. First, it introduces a supervised outlier score as an additional predictive feature, allowing models to benefit from label-informed anomaly detection. Second, a parametric optimization strategy is adopted to fine-tune the filtering process based on both class relevance and feature informativeness. Third, it employs a dual-stage feature selection mechanism, combining Random Forest Importance (RFI) and Recursive Feature Elimination (RFE) to improve interpretability and reduce redundancy. The approach further incorporates explainable AI techniques (SHAP) to highlight influential features and support transparent, data-driven decisions \cite{salih2025perspective}. Moreover, by integrating resampling methods such as SMOTE and ADASYN, SOD-FE enhances model robustness against class imbalance, achieving state-of-the-art F1 scores exceeding 98\% on two benchmark datasets (Portugal and Slovakia).

These contributions are compatible with the main stages of the predictive modeling process for student academic success, as shown in Figure \ref{fig_proposed_approach}. The key stages include data acquisition, data preprocessing, feature engineering, model building, model evaluation and best model selection. Data acquisition involves gathering relevant datasets from institutional sources. The acquired data then undergoes data preprocessing, which includes handling missing values, encoding categorical variables and using the proposed Supervised Outlier Detection (SOD) method. Following this, we move on to the feature engineering stage, which focuses on enhancing the dataset and improving the model’s predictive capabilities by selecting, transforming, and creating new features that enhance the models predictive \cite{Rizwan2025IdentifyingTF}. Next, model building involves training several classification models, including Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), Extreme Gradient Boosting (XGB), and Multi-layer Perceptron (MLP). The performance of these models is assessed through the supplied test set, and cross-validation, using hold-out and 5-fold cross-validation settings, was performed. Finally, the results are analyzed in detail, with performance metrics visualized to facilitate interpretation.

\section*{Related Work}

Recent advances in ML have significantly improved the ability to predict student dropout by leveraging diverse datasets, algorithms, and evaluation methodologies. Traditional ML algorithms such as Logistic Regression (LR), Support Vector Machines (SVM), and K-nearest neighbors (KNN) have been extensively used in student dropout prediction. A1\cite{ZEINEDDINE2021106903} applied AutoML, incorporating KNN and ANN, achieving strong classification accuracy. A2\cite{dewi2023machine} used Lasso-regularized LR, reaching an F1 score of 97\% in online learning environments . Similarly, A3\cite{huynh2025early} reported a 93.5\% F1 score, while A4\cite{su2022applying} found NN models outperforming SVM and RF for assignment submission prediction.

Tree-based ensemble techniques such as Random Forest (RF), Gradient Boosting (XGB, GBoost), LightGBM, and CatBoost have consistently demonstrated superior performance across various contexts. RF was the top performer in several studies, including those by A5\cite{kabathova2021towards}, A6\cite{deb2024predicting}, A7\cite{sulak2024predicting}, A8\cite{bezek2024classification}, A9\cite{kok2024novel}, and A10\cite{10498520}, with F1 scores ranging from 76.2\% to 96\%. Gradient Boosting models have also achieved high accuracy. A11\cite{martinez2024prediction} reached 95.4\% using academic and demographic data, while A12\cite{syed2023predictive} applied GBoost with Boruta feature selection for minimal error. A13\cite{kruger2023explainable} employed XGB to predict potential dropouts across 19 schools in Brazil, achieving a recall of 72.2\%. LightGBM and CatBoost demonstrated excellent results in multiple studies. A14\cite{cho2023study} achieved an F1 score of 84\% at Sahmyook University in Seoul using academic records of 20,050 students, while A15\cite{seo2024predictive} achieved 87.4\% in a South Korean distance university. A16\cite{Villar2024SupervisedML} showed LightGBM and CatBoost achieving up to 88\% F1 scores with hyperparameter optimization and SMOTE resampling.

Hybrid models and deep learning techniques are increasingly employed to address data complexity and imbalance. A17\cite{talebi2024ensemble} used CNN-LSTM models in MOOCs, achieving 94\% accuracy. A18\cite{mustofa2025novel} proposed an interpretable hybrid LR–NN model with SHAP and LIME, reporting a 96\% F1 score. A19\cite{beseiso2025enhancing} introduced a CNN–Attention–XGB hybrid reaching a 97\% F1 score. A20\cite{deng2024multidimensional} presented an OSO-DDNN model for academic performance prediction, achieving 93.8\% F1 on a large-scale elementary education dataset.

Numerous studies emphasized data preprocessing, including class balancing, feature engineering and selection A3\cite{huynh2025early}, A16\cite{Villar2024SupervisedML}, A18\cite{mustofa2025novel}, and A21\cite{ghorbani2020comparing} also compared various resampling techniques for imbalanced data. Feature selection was another crucial aspect investigated. A12\cite{syed2023predictive} employed Boruta, RFE, and Lasso for feature selection. A10\cite{10498520} highlighted the impact of pre-processing, including outlier removal, on model performance. Explainable AI (XAI) has gained traction to improve model transparency. Tools such as SHAP and LIME were used by A18\cite{mustofa2025novel}, A22\cite{nagy2024interpretable}, A23\cite{islam2025integration}, and A24\cite{johora2025explainable}, confirming the role of interpretability in educational decision-making.

The evaluation of models varied across studies through performance metrics including accuracy, precision, recall, F1 score and ROC-AUC. The Performance was compared to existing studies using the F1 score, which is a more appropriate measure for imbalanced data. Different data splitting strategies were employed, including holdout validation with varying ratios and k-fold cross-validation. A comparative summary of selected studies is shown in Table \ref{tab:dropout_studies}, focusing on the used method, dataset, and performance.

\begin{table}[ht]
\centering
\footnotesize
\scriptsize
% \tiny
\begin{tabular}{|l|l|l|l|l|l|l|}
\hline
\textbf{Author} & \textbf{Year} & \textbf{Method} & \textbf{Data Source} & \textbf{Splitting Data} & \textbf{F1 score} & \textbf{XAI utilized?} \\
\hline
{A1\cite{ZEINEDDINE2021106903}} & 2021 & AutoML & UAE dataset & 10-fold & 90\% accuracy & No \\
\hline
{A2\cite{dewi2023machine}} & 2023 & LR with Lasso & Binus University in Indonesia & holdout 70-30\% & 97\% & No \\
\hline
{A3\cite{huynh2025early}} & 2025 & kNN + SMOTE & ASUPG University in Undiksha & 10-fold & 93.5\% & No \\
\hline
{A4\cite{su2022applying}} & 2022 & NN & Taiwan & - & 88\% & No \\
\hline
{A5\cite{kabathova2021towards}} & 2021 & RF & Philosopher University in Slovakia & 10-fold & 91\% & No \\
\hline
{A6\cite{deb2024predicting}} & 2024 & RF & Major university in Bangladesh & holdout 80-20\% & 78\% & No \\
\hline
{A7\cite{sulak2024predicting}} & 2024 & RF & SATDAP in Portugal & - & 76.2\% & No \\
\hline
{A8\cite{bezek2024classification}} & 2024 & RF & SATDAP in Portugal & holdout 70-30\% & 87.8\% & No \\
\hline
{A9\cite{kok2024novel}} & 2024 & RF & Hons University in Malaysia & holdout 80-20\% & 81\% & No \\
\hline
{A10\cite{10498520}} & 2024 & RF, XGB & SATDAP in Portugal & - & 96\% & No \\
\hline
{A11\cite{martinez2024prediction}} & 2024 & GBoost & UNADECA data & - & 95.4\% accuracy & No \\
\hline
{A12\cite{syed2023predictive}} & 2023 & GBoost & OULA dataset & - & 78\% accuracy & No \\
\hline
{A13\cite{kruger2023explainable}} & 2023 & XGB & 19 schools in Brazil & - & 72.2\% Recall & No \\
\hline
{A14\cite{cho2023study}} & 2023 & LightGBM & Sahmyook University in Seoul & holdout 80-20\% & 84\% & No \\
\hline
{A15\cite{seo2024predictive}} & 2024 & LightGBM & Distance University in South Korean & - & 87.4\% & No \\
\hline
{A16\cite{Villar2024SupervisedML}} & 2024 & LightGBM, CatBoost & SATDAP in Portugal & - & 88\% & No \\
\hline
{A17\cite{talebi2024ensemble}} & 2024 & CNN-LSTM & MOOCs in China & - & 98\% & No \\
\hline
{A18\cite{mustofa2025novel}} & 2025 & HLRNN model & SATDAP in Portugal & - & 96\% & Yes \\
\hline
{A19\cite{beseiso2025enhancing}} & 2025 & CNN + Atten + XGB & SATDAP in Portugal & holdout 80-20\% & 97\% & No \\
\hline
{A20\cite{deng2024multidimensional}} & 2024 & OSO-DDNN & Elementary school & holdout 80-20\% & 92\% & No \\
\hline
{A21\cite{ghorbani2020comparing}} & 2020 & Ensemble RF and SVM-SMOTE & Iran and Portugal & holdout and 5-fold & 84\% & No \\
\hline
{A22\cite{nagy2024interpretable}} & 2024 & CatBoost & BME University in Hungary & holdout 75-25\% & 77.4\% AUC & Yes \\
\hline
{A23\cite{islam2025integration}} & 2025 & XGB & SATDAP in Portugal & holdout 80-20\% & 83\% & Yes \\
\hline
{A24\cite{johora2025explainable}} & 2025 & GBoost, RF & Several institutions in Bangladesh & holdout 80-20\% & 86.49\% & Yes \\
\hline
\multirow{2}{*}{Our} & \multirow{2}{*}{2025} & \multirow{2}{*}{Proposed SOD-FE model} & SATDAP in Portugal & \multirow{2}{*}{holdout and 5-fold} & \textbf{98.09\%} & \multirow{2}{*}{Yes} \\
\cline{4-4} \cline{6-6}
&  &  & Philosopher University in Slovakia &  & \textbf{98.33\%} &  \\
\hline
\end{tabular}
\caption{\label{tab:dropout_studies}Summary of Student Dropout Prediction Studies}
\end{table}

Across the literature, tree-based ensemble models like RF, XGB, LightGBM, and CatBoost consistently demonstrate high predictive accuracy in dropout prediction tasks. Additionally, hybrid approaches and the integration of deep learning techniques are emerging as promising directions. Several pieces of literature also underscore the importance of addressing data characteristics such as class imbalance and employing appropriate evaluation metrics. However, the pre-processing stage is often neglected despite its vital role in enhancing model performance and reliability. Moreover, only a limited number of works have incorporated explainable AI (XAI) techniques to identify the most influential features and offer valuable insights for decision support systems. A further challenge lies in the limited availability and lack of public access to comprehensive educational datasets. This limitation, combined with the presence of biased or non-representative data, can result in skewed predictions, particularly if the training data disproportionately represents specific populations. The proposed SOD-FE model achieved an F1 score of 98.09\% on the Portugal dataset. To ensure the generalizability of our findings, the model has also been applied to the Slovakia dataset, where the proposed model achieved an F1 score of up to 98.33\%. Additionally, the SOD-FE model incorporates explainable artificial intelligence (XAI) techniques to enhance transparency and foster trust in its predictions, thereby supporting informed decision-making by educational stakeholders.

\section*{Materials and Methods}

This study adopts the proposed SOD-FE approach to develop predictive models for the early identification of at-risk students through the use of a dataset sourced from a higher education institution. Our study focuses on optimizing the student dropout prediction by assessing the efficacy of six widely used ML classification algorithms. The proposed approach, termed SOD-FE, introduces two primary methodological contributions: supervised outlier detection (SOD) and feature engineering (FE). Structurally as shown in Figure \ref{fig_proposed_approach}, these contributions are smoothly combined in a process that extends from obtaining the initial data acquisition to the final model selection. In particular, this section provides comprehensive coverage of every aspect of the application of data pre-processing and feature engineering methods. Each stage aims to convert the original dataset into the optimal dataset to serve as an input for classification algorithms to generate dependable predictive models for decision support systems.

\begin{figure}[ht]
\centering
\includegraphics[width=0.50\textwidth]{figures/proposed approach.pdf}
\caption{The proposed approach (SOD-FE)}\label{fig_proposed_approach}
\end{figure}

\subsection*{Data Acquisition}

The Portugal dataset was created by a higher education institution program SATDAP in Portugal between the academic years 2008/2009 to 2018/2019 \cite{Martins2021EarlyPO}. The Portugal dataset consists of information about students enrolled in 17 undergraduate programs, including fields such as agronomy, design, education, nursing, journalism, management, social service and technologies \cite{realinho2022predicting}. The Portugal dataset contains 4424 entries with 34 features along with a target variable. These features cover demographic factors, social-economic factors, and student academic factors in Table \ref{tab:feature_summary}. The target variable classifies students into one of three categories: \textit{enrolled} (students continuing their studies), \textit{dropout} (students who discontinued), and \textit{graduate} (students who completed their studies) \cite{Villar2024SupervisedML}. To address the problem of student dropout prediction, we excluded the \textit{enrolled} class from the target variable and redefined the \textit{graduate} class as \textit{non-dropout}. The final dataset comprises 39\% \textit{dropout} cases and 61\% \textit{non-dropout} cases (see Figure~\ref{fig:class-before})~\cite{mustofa2025novel}. 

\begin{table}
\centering
\footnotesize
\scriptsize
\tiny
\begin{tabular}{|l|c|c|c|c|c|p{9cm}|}
\hline
\multirow{2}{*}{\textbf{Feature}} & \multirow{2}{*}{\textbf{$\#$Cls}} & \multicolumn{4}{c|}{\textbf{Statistical Distribution}} & \multirow{2}{*}{\textbf{Interpretation}} \\
\cline{3-6}
  &  & \textbf{$\mu$} & \textbf{$\sigma$} & \textbf{$\min$} & \textbf{$\max$} & \\
\hline
Curricular Units (Approved) & - & ~4.0 & ~2.9 & 0.0 & ~19 & Clear separation, strong predictive potential, widespread, students vary significantly in success rate and strongly associated with \textit{dropout}.\\
\hline
Curricular Units (Grades) & - & ~9.5 & 5.4 & 0.0 & ~18 & Wide ranges \textbf{$\mu$}, high \textbf{$\sigma$}, \textit{dropout} students skewed left, and \textit{non-dropout} students skewed right, indicating performance disparity across the student population.\\
\hline
Curricular Units (Evaluations) & - & ~7.5 & ~3.8 & 0.0 & 25 & \textit{Dropouts} tend at low evaluations, \textit{non-dropouts} more balanced, substantial variability (\textbf{$\sigma$}=~3.8), suggesting a diverse academic performance landscape.\\
\hline
Curricular Units (Enrolled) & - & 6.1 & ~1.8 & 0.0 & 21.0 & \textit{Non-dropout} counts are modestly lower among \textit{dropouts}, potentially signaling moderate relevance.\\
\hline
Curricular Units (Credited) & - & ~0.3 & ~1.5 & 0.0 & 17.0 & Low mean implies crediting is rare, mostly zero-valued, \textit{dropouts} slightly sparser, hinting at lack of progress.\\
\hline
Curricular Units (without Eval.) & - & 0.1 & 0.7 & 0.0 & 10.0 & Mostly zeros, \textit{non-dropout} students slightly sparser.\\
\hline
Tuition fees up to date & 2 & 0.8 & 0.4 & 0.0 & 1.0 & 20\% late payers, unpaid tuition linked to \textit{dropout}. \\
\hline
Scholarship holder & 2 & 0.3 & 0.4 & 0.0 & 1.0 & 30\% receive aid, scholarship holders are less likely to \textit{dropout}. \\
\hline
Debtor & 2 & 0.1 & 0.3 & 0.0 & 1.0 & Correlates with \textit{dropout}. \\
\hline
GDP & - & -0.1 & 2.2 & -4.1 & 3.5 & \multirow{3}{9cm}{No strong separation, low individual correlations, distributions similar across \textit{dropout} and \textit{non-dropout} groups, low direct effect.} \\
\cline{1-6}
Inflation rate & - & 1.3 & 1.4 & -0.8 & 3.7 & \\
\cline{1-6}
Unemployment rate & - & 11.6 & 2.7 & 7.6 & 16.2 & \\
\hline
Age at enrollment & - & 22.9 & 7.4 & 17.0 & 70.0 & Most in early 20s, older students more likely to \textit{dropout} (right-skewed). \\
\hline
Gender & 2 & 0.3 & 0.4 & 0.0 & 1.0 & Balanced, low predictive value. \\
\hline
Marital status & 6 & 1.1 & 0.6 & 1.0 & 6.0 & Mostly single students, certain statuses are more common among \textit{dropout}. \\
\hline
Displacement status & 2 & 0.6 & 0.5 & 0.0 & 1.0 & Modest association with \textit{dropout}. \\
\hline
Nationality & 21 & 1.2 & 1.7 & 1.0 & 21.0 & Mainly one nationality, some diversity. \\
\hline
International & 2 & 0.0 & 0.1 & 0.0 & 1.0 & Highly sparse, slightly higher \textit{dropout} rate. \\
\hline
Application mode & 18 & 6.4 & 5.0 & 1.0 & 17.0 & Wide spread, useful categorical indicator. \\
\hline
Application order & 7 & 1.9 & 1.4 & 1.0 & 6.0 & \textit{Dropouts} clustered in lower application types. \\
\hline
Previous qualification & 17 & 2.1 & 3.4 & 1.0 & 17.0 & \textit{Dropouts} concentrated at lower qualifications. \\
\hline
Daytime/evening attendance & 2 & 0.9 & 0.3 & 0.0 & 1.0 & \textit{Dropouts} are more in evening classes. \\
\hline
Course & 17 & 10.4 & 4.1 & 1.0 & 17.0 & \textit{Dropout} rates vary by course. \\
\hline
Educational special needs & 2 & 0.0 & 0.1 & 0.0 & 1.0 & Almost all coded 0, sparse, low relevance. \\
\hline
Father’s qualification & 34 & 16.5 & 11.0 & 1.0 & 34.0 & \multirow{2}{9cm}{Higher dropout among students with less educated parents.} \\
\cline{1-6}
Mother’s qualification & 34 & 12.6 & 8.9 & 1.0 & 29.0 &\\
\hline
Father’s occupation & 46 & 7.5 & 3.7 & 1.0 & 46.0 & \multirow{2}{9cm}{Some \textit{dropout} clustering by occupation.}\\
\cline{1-6}
Mother’s occupation & 46 & 7.1 & 3.4 & 1.0 & 32.0 &\\
\hline
\end{tabular}
\caption{\label{tab:feature_summary}Statistical Summary and Feature Interpretation of the Student Dropout Dataset}
\end{table}

\begin{figure}[ht]
\centering
\includegraphics[width=0.63\textwidth]{figures/data_dist_before.jpeg}
\caption{\label{fig_data_dist_before}Distributional Comparison of Features Between \textit{Dropout} and \textit{Non-dropout} Students}
\end{figure}

To ensure the generalizability of our study, we additionally employed a second dataset, referred to as the Slovakia dataset. This dataset was extracted from an online course hosted on the Moodle Learning Management System (LMS) at Constantine the Philosopher University in Slovakia, covering four academic years \cite{kabathova2021towards}. The Slovakia dataset contains 261 unique students and consists of five features that reflect their participation in various types of course-related activities.

\subsection*{Data Preprocessing}

Data preprocessing is a critical initial stage that involves cleaning the dataset, handling missing values, removing irrelevant features, and addressing outliers to improve data quality and structure \cite{10498520}. In this study, Exploratory Data Analysis (EDA) techniques were employed to identify patterns, trends, and correlations between features using statistical summaries and visualizations \cite{beseiso2025enhancing}. In addition, the proposed Supervised Outlier Detection (SOD) method was utilized during preprocessing to uncover hidden relationships between features, detect anomalous instances, and identify potential biases that may negatively influence the performance of dropout prediction models. This label-aware filtering step ensures that relevant patterns, particularly those associated with minority classes, are preserved during subsequent modeling stages.

% \subsubsection*{Missing Values}

To prepare the data for classification, it was emphasized that there were no missing values \cite{syed2023predictive}. Fortunately, the missing values have previously been identified and processed appropriately. However, additional minimal cleaning procedures are needed, and they include renaming columns for a better working experience during future operations \cite{realinho2022predicting}.

% \subsubsection*{Data Analysis}

Categorical features were converted into a numerical format via label encoding to ensure compatibility with ML algorithms, thereby preparing the dataset for further analysis. A comprehensive statistical examination was conducted to understand the distributional characteristics of the dataset, as summarized in Table~\ref{tab:feature_summary}. and Figure~\ref{fig_data_dist_before} presents the probability density functions of all 34 features, stratified by \textit{dropout} status. Among academic-related features such as curricular\_units\_approved, curricular\_units\_grades, and curricular\_units\_evaluations, distinct separability is evident. \textit{Non-dropout} students cluster around higher values, whereas \textit{dropout} students are concentrated in lower ranges, including zero values. This observation aligns with the summary statistics, where these features generally exhibit moderate mean values but substantial variability and skewness (e.g., curricular\_units\_grades: mean $\approx$ 9--10, standard deviation $>$ 5), suggesting the potential need for normalization or transformation when employing models sensitive to feature scale or distributional skew.

In particular, features such as curricular\_units\_approved show stark contrasts, with \textit{non-dropout} students concentrated at higher approval counts (e.g., >10), while \textit{dropout} students are concentrated near zero, reinforcing the relationship between academic underperformance and \textit{dropout} risk. Binary features, including tuition\_fees, debtor, and scholarship\_holder, display bimodal distributions. \textit{Dropout} students are disproportionately represented in categories associated with financial instability (e.g., unpaid tuition), whereas \textit{non-dropout} students dominate the opposite ends. For instance, the mean value for tuition\_fees is 0.8, indicating a considerable portion of students are at financial risk. Conversely, several categorical and binary features such as nationality, international, and educational\_special\_needs exhibit extremely imbalanced distributions, with over 95\% of values concentrated in a single category. Due to their low variance and limited discriminatory power, these features were excluded from subsequent modeling stages to reduce noise and enhance model performance. Macroeconomic indicators (GDP, unemployment\_rate, and inflation\_rate) present more uniform or mixed distributions, showing no strong differentiation between \textit{dropout} and \textit{non-dropout} students. This suggests a limited direct influence of macroeconomic conditions on individual dropout behavior. Some categorical variables, such as application\_mode (18 categories) and nationality (21 categories), exhibit broad distributional variability. Although subtle, differences in categorical preferences between groups were observed, for instance, \textit{dropout} students tend to apply earlier in the application\_order, while \textit{non-dropout} students are more evenly distributed. The age variable shows a right-skewed distribution, with most students in their early twenties. However, \textit{dropout} students span a wider age range, including a noticeable tail extending into older age groups (e.g., over 40 years), indicating that mature students may face higher \textit{dropout} risks. The binary target variable \textit{dropout} exhibits a mean of 0.6 and a standard deviation of 0.5, confirming that 60\% of students in the dataset discontinued their studies. This moderate class imbalance necessitates the application of resampling techniques or class-weight adjustments during model training to address potential bias and improve minority class representation.


\subsection*{Supervised Outlier Detection (SOD)}\label{sec:supervised-outlier-detection}

The proposed approach integrates SOD with label-aware feature relevance assessment and statistical filtering. Specifically, it combines mutual information and Pearson correlation to rank features based on their association with the target variable \cite{vergara2014review}. A subset of the most informative features is then selected according to these rankings. Outlier detection is performed using an interquartile range (IQR) based statistical filter, denoted as $D(\mathbf{x}_i; \alpha, \beta)$, guided by class-specific conditions $\alpha$ and the number of selected features $\beta$. The cleaned dataset, denoted by $D^{cl}((\mathbf{x}_i, y_i); \alpha, \beta)$, is then used to train a supervised model $M$. Optimal values of $\alpha$ and $\beta$ are selected based on the F1-score during cross-validation. This ensures that the retained features are both relevant and non-redundant, while outliers that may degrade classification performance are filtered out in a label-aware manner.

Let $\{X, Y\} = \{(\mathbf{x}_i, y_i)\}_{i=1}^n$ denote a dataset of $n$ labeled student records, where $\mathbf{x}_i \in \mathbb{R}^d$ is a feature vector and $y_i \in \{0,1\}$ is a binary dropout label. The feature matrix $X$ may be contaminated with noise or outliers. The goal is to clean $X$ before training the final supervised model. The SOD method helps to determine the optimal filtering configuration.

Let the pairs $\{X_{tr}, Y_{tr}\}$ and $\{X_{ts}, Y_{ts}\}$ denote the training and testing subsets obtained via stratified splitting. The SOD method is applied exclusively to the training data matrix $X_{tr}$ to identify and filter potential outliers, resulting in a cleaned dataset $X^{cl}$ and its corresponding target vector $Y^{cl}$.

To identify the most informative features, we computed two supervised relevance scores for each feature $x_j$:

\begin{align}
\rho_j = \mathrm{corr}(x_j, y) \\
\mu_j = I(x_j; y)
\end{align}

Where $\rho_j$ is the absolute Pearson correlation and $\mu_j$ is the mutual information between $x_j$ and the target $y$. The combined rank for feature $x_j$ is computed as:

\begin{equation}
R_j = \frac{1}{2} \left( \mathrm{rank}_{\rho}(x_j) + \mathrm{rank}_{\mu}(x_j) \right)
\end{equation}

We select the top-$\beta$ features with the lowest $R_j$ values, denoted by $\mathcal{F}_\beta$. IQR-based outlier detection is applied to $X_{tr}$ using only the selected feature subset $\mathcal{F}_\beta$. For each $x_j \in \mathcal{F}_\beta$, we compute:

\begin{align}
\mathrm{IQR}_j &= Q3_j - Q1_j \\
\mathrm{Lower}_j &= Q1_j - 1.5 \cdot \mathrm{IQR}_j \\
\mathrm{Upper}_j &= Q3_j + 1.5 \cdot \mathrm{IQR}_j
\end{align}

An instance $\mathbf{x}_i$ is considered an inlier if:

\begin{equation}
x_{ij} \in [\mathrm{Lower}_j, \mathrm{Upper}_j], \quad \forall j \in \mathcal{F}_\beta
\end{equation}

To address class imbalance, the outlier detection procedure is evaluated under different label conditions $(\alpha)$. We define the detection rule $D(\mathbf{x}_i; \alpha, \beta) \in \{\text{inlier}, \text{outlier}\}$, where $\alpha \in \{0, 1, 2\}$ denotes the filtering focus: only \textit{non-dropout} class (0), only \textit{dropout} class (1), or \textit{all} data (2), and $\beta$ is the number of selected features. The cleaned training set is given by:

\begin{equation}
X_{tr}^{cl} = \left\{ \mathbf{x}_i \in X_{tr} \mid D(\mathbf{x}_i; \alpha, \beta) = \text{inlier} \right\}
\end{equation}

To determine the optimal configuration $(\alpha^*, \beta^*)$, a Random Forest classifier $M$ is trained and evaluated on $\{X_{tr}^{cl}, Y_{tr}^{cl}\}$ using the F1 score as the evaluation metric:

\begin{equation}
(\alpha^*, \beta^*) = \arg\max_{\alpha, \beta} \mathrm{F1}\left( M(\mathcal{D}^{cl}(\alpha, \beta)) \right)
\end{equation}

Once the optimal hyperparameters are determined, the training and test sets are merged:

\begin{equation}
\{X, Y\} = \{X_{tr}, Y_{tr}\} \cup \{X_{ts}, Y_{ts}\}
\end{equation}

The final outlier detection function $D(\cdot; \alpha^*, \beta^*)$ is then applied to all instances in $X$ to assign a binary outlier score:

\begin{equation}
\mathrm{outlier\_score}(\mathbf{x}_i) = 
\begin{cases}
1 & \text{if } D(\mathbf{x}_i; \alpha^*, \beta^*) = \text{inlier} \\
-1 & \text{if } D(\mathbf{x}_i; \alpha^*, \beta^*) = \text{outlier}
\end{cases}
\end{equation}

This score serves as a supervised outlier indicator and is appended to the feature vector of each instance. The final dataset is:

\begin{equation}
\hat{X} = \left\{ \hat{\mathbf{x}}_i = [\mathbf{x}_i, \ \mathrm{outlier\_score}(\mathbf{x}_i)] \right\}_{i=1}^{n}
\end{equation}

Where each instance $\hat{\mathbf{x}}_i$ is formed by appending the supervised outlier score to the original feature vector $\mathbf{x}_i$.

\begin{algorithm}
\caption{\label{alg:sod}Proposed Supervised Outlier Detection (SOD) Method}
\begin{algorithmic}[1]
\Require Dataset $\{X, Y\} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{n}$, Outlier Detector $D$, Classifier $M$, Hyperparameters $\alpha$, $\beta$
\Ensure New dataset $\{\hat{X}, Y\}$ with outlier score, optimal Hyperparameters $(\alpha^*, \beta^*)$
\State Split dataset into $\{X_{tr}, Y_{tr}\}$ and $\{X_{ts}, Y_{ts}\}$
\For{$(\alpha, \beta) \in R_{\alpha} \times R_{\beta}$}
\State Select top-$\beta$ features based on combined ranking
\State $X_{tr}^{cl} = \mathcal{D}^{cl}(X_{tr};\alpha;\beta)$ \Comment{Apply SOD to clean $X_{tr}$}
\State $Y_{tr}^{cl} = select(Y_{tr}, X_{tr}^{cl})$
\State $F_{1}(\alpha, \beta) = M(Y_{tr}^{cl}, \hat{Y}_{tr}^{cl})$ \Comment{Train $M$ and compute F1 score}
\EndFor
\State$(\alpha^{*}, \beta^{*}) = \{F_{1}(\alpha, \beta)\}.argmax()$ \Comment{Select best configuration}
\State $X = \bigcup \{X_{tr},X_{ts}\}$, $Y = \bigcup \{Y _{tr},Y_{ts}\}$ \Comment{Merge the training and test sets} 
\State $\hat{X} = \mathcal{D}(X;\alpha^{*};\beta^{*})$  \Comment{Compute and append outlier scores} 
\State \textbf{return} $\hat{X}$
\end{algorithmic}
\end{algorithm}

The optimal hyperparameters $(\alpha^*, \beta^*)$ are determined via K-fold cross-validation over the search space $R_{\alpha} \times R_{\beta}$. The SOD method identifies relevant features, removes outliers and optimizes classifier performance using the F1-score as the evaluation criterion. As shown in Figure~\ref{fig:outlier_filtering_performance}, this procedure enhances model robustness and generalization. The F1 score initially increases with the threshold, indicating improved model performance due to effective outlier removal. The F1 score peaks at 96.27\% when the optimal hyperparameters $\alpha^{*}=1$ and $\beta^{*}=11$ are selected.

\begin{figure}[ht]
\centering
\includegraphics[width=0.5\textwidth]{figures/best_feature_selection_Combined_Rank_Method_0_1.pdf}
\caption{\label{fig:outlier_filtering_performance}Performance of SOD with combined feature Ranking. The plot illustrates the F1 score (\%) (left y-axis, blue line with circular markers) and outlier data ratio (\%) (right y-axis, red dashed line with 'x' markers) as a function of the determining the optimal hyperparameters target class ($\alpha^*$), and number of selected features ($\beta^*$) with the maximum F1 score.}
\end{figure}

While the SOD method applies IQR-based statistical filtering guided by Pearson correlation and label information, we acknowledge that such techniques may be limited in capturing non-linear feature interactions. To address this, we incorporated mutual information alongside Pearson correlation during feature relevance assessment, allowing the detection of both linear and non-linear dependencies. Additionally, the risk of inadvertently removing legitimate yet atypical dropout cases, such as students with unusually high or low attendance, was carefully mitigated through a parametric tuning strategy. Specifically, the outlier detection process is controlled by two parameters, $\alpha$ (class focus) and $\beta$ (number of features), which are optimized using cross-validation to preserve patterns relevant to the minority class. This targeted adjustment ensures that difficult but valid cases are retained, rather than being misclassified as noise. Furthermore, a \hyperref[sec:ablation_study]{Component-wise Analysis section} confirmed that the selected configuration achieved the best balance between noise reduction and dropout pattern preservation. The effect of applying the SOD method can be seen in Figures \ref{fig:pca-before}, \ref{fig:pca-after} and \ref{fig:InOut-after} show the data distribution before and after the SOD-based outlier removal. These visualizations confirm the impact of the SOD method on improving data quality and model readiness.

\begin{figure}[ht]
\centering
\begin{subfigure}{0.48\textwidth}
\centering
\includegraphics[width=0.5\textwidth]{figures/2d_PCA_before.pdf}
\caption{\label{fig:pca-before}PCA data before SOD}
\end{subfigure}
\hfill
\begin{subfigure}{0.48\textwidth}
\centering
\includegraphics[width=0.5\textwidth]{figures/2d_PCA_after22.pdf}
\caption{\label{fig:pca-after}PCA filtered data after optimal SOD}
\end{subfigure}
% \vskip\baselineskip
% \vspace{-0.95em}
\begin{subfigure}{0.48\textwidth}
\centering
\includegraphics[width=.50\textwidth]{figures/class_pie_before.pdf}
\caption{\label{fig:class-before}Class distribution}
\end{subfigure}
\hfill
\begin{subfigure}{0.48\textwidth}
\centering
\includegraphics[width=.50\textwidth]{figures/class_pie_after33.pdf}
\caption{\label{fig:InOut-after}Inliers and outliers identified by SOD}
\end{subfigure}
\caption{\label{fig:data-distribution}Data and class distribution used in our study.}
\end{figure} 

\subsection*{Feature Engineering}

During the feature engineering phase, the dataset receives enhancement through assessments of feature interrelationships, choosing relevant variables through apply proper transformations and feature generation techniques \cite{Rizwan2025IdentifyingTF}. An extensive correlation analysis was performed to identify and discard weakly or redundant features which improved model predictive capabilities and interpretation.

\subsubsection*{Correlation Analysis}

The Pearson correlation heatmap Figure \ref{fig_Correlation_Heatmap} demonstrated the relationship between all features including the target variable through a visual representation of their pairwise correlation coefficients. The heatmap uses color intensity to display linear relationship strength while light colors represent weak to nonexistent associations. Each feature shows a strong correlation with itself since the value on the diagonal is equal to 1. The visual representation allowed for the detection of multicollinearity while showing which factors are strongly linked to the target variable. The analysis implemented absolute values of correlation coefficient measurements to study linear associations with positive or negative mathematical relationships. The absolute correlations between features and the target variable are classified into strong ($\leq$ 0.4), moderate ($\leq$ 0.2 and $<$ 0.4), weak ($\leq$ 0.1 and $<$ 0.2) and negligible ($<$ 0.1) categories. A specific color scheme distinguished each group in the heatmap (blue, green, purple, and brown respectively) while red indicated the target variable.  

\begin{figure}[ht]
\centering
\includegraphics[width=0.7\textwidth]{figures/Correlation_Heatmap.pdf}
\caption{\label{fig_Correlation_Heatmap}Correlation Heatmap}
\end{figure}

Academic performance-related features displayed powerful relationships with the target during the analysis which indicated their strong predictive potential. The combination of financial indicators with other features or their application in non-linear models can derive additional predictive value despite their moderate and weak correlations. A comprehensive analysis should evaluate the combined impact of macroeconomic indicators on high school dropout rates even though each metric measured separately has minimal individual correlation. Educational\_special\_needs, international and nationality features displayed minimal correlation with student dropout instead of enhancing prediction accuracy therefore these features could safely be omitted to reduce model complexity and overfitting risk.

\subsubsection*{Random Forest Importance (RFI)} 

RFI techniques evaluated the importance of features through a ranking system for features. As shown in Figure \ref{fig_RFI_before}. To find out the most important features, we applied the test using F1 score as the error measure, which works better for cases where the data are not balanced \cite{realinho2022predicting}. This result enabled the recognition of non-essential features which proved either insignificant or unimportant for the prediction model. The computational efficiency and generalizability were enhanced by excluding these features. This feature selection process allows the model to train using only important variables, thereby achieving high accuracy, maintaining robustness, and preventing overfitting caused by irrelevant inputs \cite{8074494}. The top twenty-five features representing demographic, academic, and socio-economic factors were chosen from the initial dataset. These selected features are further discussed and explained in the \hyperref[sec:Model_Interpretability]{Model Interpretability section}.

\begin{figure}[ht]
\centering
\includegraphics[width=0.50\textwidth]{figures/Output_RFI_before.pdf}
\caption{\label{fig_RFI_before}Feature importance analysis using the RFI method}
\end{figure}

\subsubsection*{Recursive Feature Elimination (RFE)} 

The RFE technique functions as a ML procedure that performs repeated eliminations of unimportant features from datasets to achieve better model performance. RFE approach assigns weights to each feature for its predictive power in determining the target variable before removing the least important features successive times until the specified number of features remains, while considering their impact on the output variable prediction. RFE serves as an enhancement tool for other ML methods to boost their operational efficiency \cite{syed2023predictive}. The proposed model displays different performance results with the inclusion of the twenty-eight most important features. With the previous twenty-five features, the Previous\_qualification, Inflation\_rate and GDP features were selected. This inclusion emphasizes the important role that economic factors and broader environmental conditions can play in influencing students' motivation and ability to complete their studies. The RFI method was adopted to evaluate our proposed SOD-FE approach for predicting students academic success. The impact of employing both the RFI and RFE feature selection methods is further discussed in the \hyperref[sec:feature_analysis]{Feature Selection Analysis section}.

\subsection*{Data Normalization}

Data normalization or feature scaling was performed before starting to model building stage. This is important because most ML models depend on Euclidean distance which may not work well unless features are scaled properly. The standardization method was used to make the model more efficient. All values were adjusted so they behaved as a standard normal distribution, having a mean ($\mu$) of 0 and a standard deviation ($\sigma$) of 1. The scaling is performed using the following formula:

\begin{equation}
z = \frac{x - \mu}{\sigma}
\end{equation}

Where $x$ represents the existing value of the feature, $\mu$ is the average of the feature, and $\sigma$ represents the standard deviation. As a result, every standardized feature will be centered at 0 and will have a standard deviation of 1.

\subsection*{Data Splitting}

The normalized dataset is divided into training and testing subsets to ensure balanced label distributions and prevent bias in model evaluation. This study employs two widely used cross-validation strategies: random hold-out and shuffled k-fold cross-validation. Initially, randomly 80\% of the total dataset is included in the training set and the remaining 20\% is assigned to the test set. To obtain more reliable and generalizable results, we also apply 5-fold cross-validation. This method ensures the model performs well on unseen data by systematically splitting the dataset into five equal subsets. In every iteration, a portion of the data is used for testing and the rest for training. 

\subsection*{Machine Learning (ML) Methods}

This paper focuses on LR, SVM, RF, XGB, and MLP models to analyze factors affecting and predict student performance. The selected methods encompass both traditional and ensemble techniques, enabling robust comparative evaluation. The ensemble method or ensemble learning is a model that makes predictions based on the number of different models. The goal of ensemble learning is to combine the predictions made by several estimators built using a particular algorithm so that they can produce better and more effective results. The two families of ensemble methods are generally known as averaging and boosting methods. When using averaging, different estimators are built first and then their estimates are averaged. As a result, the combined estimator has a lower variance and usually performs better than any of the original estimators, e.g. RF method. Boosting methods involve creating base estimators one after another and attempting to reduce the bias in the final Averages learning with Genetic Algorithms. The goal is to use a number of weak models to make a more powerful group model, such as XGB \cite{goren2024early}.

\subsubsection*{Logistic Regression (LR)}

The LR model is a widely used statistical and ML algorithm for binary classification problems. To assign an input to a specific class, this algorithm takes a logistic function of a linear combination of the input features. The logistic function (sigmoid) transforms values into a range between 0 and 1, which can be interpreted as probabilities, and based on a chosen threshold (typically 0.5), the model assigns the input to one of the two categories. Its simplicity, interpretability, and effectiveness in handling categorical outcomes make it widely used in fields such as healthcare, finance, and social sciences \cite{hosmer2013applied}.

\subsubsection*{Support Vector Machine (SVM)}

The SVM classifies with a hyperplane that defines classes in the feature space so that there is a large margin between them. The support vectors are the points closest to the hyperplane where the decision is made. Considering that SVM uses kernel tricks, it is effective for both high-dimensional inputs and sparse data \cite{cortes1995support}. The algorithm has demonstrated strong performance in classification tasks, including predicting student dropout rates and achieving high precision, accuracy, and sensitivity \cite{ZEINEDDINE2021106903}. 

\subsubsection*{Random Forest (RF)} 

The RF model is an ensemble learning method that uses the bagging technique by generating bootstrap samples to build multiple decision trees during training to enhance predictive accuracy and reduce overfitting. For classification, the algorithm outputs the mode of prediction, while for regression, it calculates the mean, demonstrating its versatility across different problem types. It achieves diversity among trees by selecting random subsets of features and data points, making it well-suited for handling complex large datasets with high dimensionality. Additionally, RF provides insights into feature importance, helping users identify key features that influence predictions and improving model interpretability \cite{breiman2001random}.

\subsubsection*{Extreme Gradient Boosting (XGB)} 

The XGB model works well and is easy to use in different contexts. It works by sequentially producing multiple decision trees, while each new tree tries to correct mistakes made by the previous ones. XGB relies on a Taylor expansion of the loss function. It also includes a regularization to prevent overfitting often leading to high predictive power and scalability in various ML tasks \cite{chen2016xgboost}.

\subsubsection*{Multi-Layer Perceptron (MLP)}

The MLP model is an artificial neural network with several layers that are all connected. Usually, a neural network includes an input layer, one or several hidden layers, and an output layer. MLP uses a non-linear activation function in its hidden layers to understand difficult patterns between what is input and the output. The network is trained using backpropagation to adjust the weights of the connections between neurons \cite{rumelhart1986learning}.

\subsection*{Performance Evaluation}

The reliability of model results is verified through performance metrics including \textbf{accuracy}, \textbf{precision}, \textbf{recall}, \textbf{F1 score}, and \textbf{ROC-AUC}, as well as confusion matrices. As the most widely used performance metric, \textbf{accuracy} determines the ratio of successful predictions in relation to the total number of samples. This metric is most effective when applied to datasets with relatively balanced class distributions. Both \textbf{precision} and \textbf{recall} provide greater insight than accuracy when analyzing unevenly distributed data. The \textbf{F1 score} is a crucial metric for evaluating model performance, especially in the context of imbalanced datasets. It represents the harmonic mean of precision and recall, offering a unified measure of performance.

The sequence of Equations~\eqref{eq:accuracy}--\eqref{eq:f1} defines the computations for these performance metrics:

\begin{equation}
\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}
\label{eq:accuracy}
\end{equation}

\begin{equation}
\text{Precision} = \frac{TP}{TP + FP}
\label{eq:precision}
\end{equation}

\begin{equation}
\text{Recall} = \frac{TP}{TP + FN}
\label{eq:recall}
\end{equation}

\begin{equation}
\text{F1\_score} = 2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}
\label{eq:f1}
\end{equation}

ROC-AUC is commonly relied upon as a diagnostic metric for binary classification tasks. It calculates the difference between how much the model identifies both the positive and the negative classes. It shows a graph of TPR against FPR by using various threshold values. All scores in evaluation are between 0 and 1, and higher numbers show that the performance is stronger.

\section*{Results}\label{sec2}

An experimental validation process was carried out to test the SOD-FE approach for student academic success prediction and dropout risk assessment. The Python 3.9 experimental pipeline included Pandas, NumPy, and Scikit-learn among other essential libraries. A Windows 11 workstation with dual Intel Core I7 CPUs (2.9 GHz) running 16 GB of RAM served as the platform for all testing activities. The dataset exists under ”Dataframe” and serves as the primary data format for our entire research process. Table \ref{tab:hyperparameters} presents the ML models and their hyperparameter settings used in this paper.

\begin{table}[ht]
\centering
\begin{tabular}{|l|l|}
\hline
\textbf{Method} & \textbf{Hyperparameters} \\
\hline
LR & C=100, penalty='l2', solver='lbfgs', max\_iter=1000 \\
\hline
SVM & kernel='linear' \\
\hline
RF & n\_estimators=100, min\_samples\_leaf=2 \\
\hline
XGB & 'n\_estimators'=500, scale\_pos\_weight=0.82, 'max\_depth'=8, 'gamma'=0.8, 'learning\_rate'=0.05 \\
\hline
MLP & activation='relu', solver='adam', max\_iter=500, learning\_rate\_init=0.001, early\_stopping=True \\
\hline
\end{tabular}
\caption{\label{tab:hyperparameters}Hyperparameters of machine learning (ML) methods used in our study}
\end{table}

Our results in Table \ref{tab:performance_comparison_holdout} and \ref{tab:performance_comparison_k_fold} demonstrate the significant impact of SOD-FE on classifier performance. Across both hold-out and 5-fold cross-validation strategies, and on Portugal and Slovakia datasets, SOD-FE consistently improved accuracy, F1 Score, and AUC for Logistic Regression, SVM, Random Forest, XGB, and MLP models. Specifically, when analyzing the hold-out validation results presented in Table \ref{tab:performance_comparison_holdout}, without SOD-FE, AUC values ranged from 89.66\% to 99.00\%. With SOD-FE, the RF and XGB classifiers demonstrated the most substantial improvements. For example, RF accuracy increased from 90.08\% to 98.02\% and AUC from 94.55\% to 99.43\% on the Portugal dataset with SOD-FE+SMOTE. Similarly, XGB showed an increase in F1 Score from 85.87\% to 97.72\% on the Portugal dataset with SOD-FE+SMOTE. Notably, the MLP performance on the Slovakia dataset improved marginally or even decreased slightly in terms of F1 Score and AUC (from 83.87\% to 84.62\% and from 97.86\% to 97.34\%, respectively, with SOD-FE+SMOTE). This may reflect the sensitivity of neural networks to feature transformations when data size or quality is variable. The tables also include performance with SOD-FE+ADASYN, which generally showed strong results comparable to SOD-FE+SMOTE, with RF and XGB reaching 100\% AUC on the Slovakia dataset. 

\begin{table}[ht]
\centering
\footnotesize
\scriptsize
\begin{tabular}{|l|l|c|c|c|c|c|c|c|c|c|}
\hline
\multirow{2}{*}{\textbf{Model}} & \multirow{2}{*}{\textbf{Dataset}} & \multicolumn{3}{c|}{\textbf{Without SOD-FE}} & \multicolumn{3}{c|}{\textbf{With SOD-FE+SMOTE}} & \multicolumn{3}{c|}{\textbf{With SOD-FE+ADASYN}} \\
\cline{3-5} \cline{6-8} \cline{9-11}
 & & {\textbf{Accuracy}} & {\textbf{F1 Score}} & {\textbf{AUC}} & {\textbf{Accuracy}} & {\textbf{F1 Score}} & {\textbf{AUC}} & {\textbf{Accuracy}} & {\textbf{F1 Score}} & {\textbf{AUC}} \\
\hline
\multirow{2}{*}{LR} & Portugal & 90.08 & 87.67 & \textbf{95.56} & 96.15 & 96.15 & 99.43 & 95.24 & 95.24 & 99.01 \\
\cline{2-11}
& Slovakia & \textbf{96.88} & \textbf{92.86} & 97.71 & 95.79 & 95.74 & 98.51 & 94.79 & 94.77 & 99.12 \\
\hline
\multirow{2}{*}{SVM} & Portugal & 89.26 & 86.91 & 95.45 & 96.85 & 96.85 & 99.40 & 95.13 & 95.12 & 98.95 \\
\cline{2-11}
& Slovakia & 95.31 & 89.66 & 96.14 & 95.79 & 95.74 & 98.23 & 94.79 & 94.77 & 98.59 \\
\hline
\multirow{2}{*}{RF} & Portugal & 90.08 & 87.32 & 94.55 & \textbf{98.25} & \textbf{98.25} & 99.55 & 97.80 & 97.79 & 99.48 \\
\cline{2-11}
& Slovakia & \textbf{96.88} & \textbf{92.86} & \textbf{98.71} & 95.79 & 95.74 & \textbf{99.72} & \textbf{100.00} & \textbf{100.00} & \textbf{100.00} \\
\hline
\multirow{2}{*}{XGB} & Portugal & 88.98 & 85.87 & 94.38 & {97.78} & {97.78} & \textbf{99.67} & \textbf{98.14} & \textbf{98.14} & \textbf{99.64} \\
\cline{2-11}
& Slovakia & 95.31 & 89.66 & {99.00} & \textbf{97.89} & \textbf{97.88} & 99.58 & \textbf{100.00} & \textbf{100.00} & \textbf{100.00} \\
\hline
\multirow{2}{*}{MLP} & Portugal & \textbf{91.18} & \textbf{89.00} & 95.27 & 96.15 & 96.15 & 99.41 & 95.59 & 95.58 & 99.09 \\
\cline{2-11}
& Slovakia & 92.19 & 83.87 & 97.86 & 87.37 & 87.47 & 97.34 & 82.29 & 82.36 & 91.67 \\
\hline
\end{tabular}
\caption{\label{tab:performance_comparison_holdout}Performance of the classifiers based on the hold-out strategy on SOD-FE with SMOTE and ADASYN balanced datasets.}
\end{table}

To assess the robustness and generalizability of the models, shuffle 5-fold cross-validation was conducted (Table \ref{tab:performance_comparison_k_fold}). The standard deviations reported indicate stability in model performance. Again, SOD-FE yielded consistent improvements across most models and metrics. For instance, RF on the Portugal dataset improved from an accuracy of 90.74±1.3\% to 98.09±0.6\% with SOD-FE+SMOTE, and XGB from 90.63±0.9\% to 97.83±0.5\% with SOD-FE+SMOTE. The cross-validation results further solidify the efficacy of SOD-FE, demonstrating its ability to enhance model performance across different balancing strategies and provide more generalized results.

\begin{table}[ht]
\centering
\footnotesize
\scriptsize
\begin{tabular}{|l|l|c|c|c|c|c|c|c|c|c|}
\hline
\multirow{2}{*}{\textbf{Model}} & \multirow{2}{*}{\textbf{Dataset}} & \multicolumn{3}{c|}{\textbf{Without SOD-FE}} & \multicolumn{3}{c|}{\textbf{With SOD-FE+SMOTE}} & \multicolumn{3}{c|}{\textbf{With SOD-FE+ADASYN}} \\
\cline{3-5} \cline{6-8} \cline{9-11} 
& & {\textbf{Accuracy}} & {\textbf{F1 Score}} & {\textbf{AUC}} & {\textbf{Accuracy}} & {\textbf{F1 Score}} & {\textbf{AUC}} & {\textbf{Accuracy}} & {\textbf{F1 Score}} & {\textbf{AUC}} \\
\hline
\multirow{2}{*}{LR} & Portugal & {91.07$\pm$1.1\%} & {90.98$\pm$1.1\%} & \textbf{95.39$\pm$0.7\%} & {96.69$\pm$0.7\%} & {96.68$\pm$0.7\%} & {99.51$\pm$0.1\%} & {94.71$\pm$1.1\%} & {94.70$\pm$1.1\%} & {98.85$\pm$0.2\%} \\
\cline{2-11}
& Slovakia    & {95.61$\pm$0.6\%} & {95.53$\pm$0.6\%} & {97.87$\pm$1.2\%} & {96.41$\pm$1.8\%} & {96.40$\pm$1.9\%} & {99.15$\pm$0.6\%} & {94.15$\pm$2.2\%} & {94.13$\pm$2.2\%} & {98.31$\pm$1.1\%} \\
\hline
\multirow{2}{*}{SVM} & Portugal & \textbf{91.13$\pm$0.9\%} & \textbf{91.01$\pm$0.9\%} & {95.15$\pm$0.7\%} & {96.80$\pm$0.6\%} & {96.80$\pm$0.6\%} & {99.51$\pm$0.1\%} & {94.92$\pm$1.1\%} & {94.91$\pm$1.1\%} & {98.81$\pm$0.2\%} \\
\cline{2-11}
& Slovakia    & \textbf{95.61$\pm$1.2\%} & \textbf{95.56$\pm$1.2\%} & \textbf{98.11$\pm$1.2\%} & \textbf{97.04$\pm$1.8\%} & \textbf{97.04$\pm$1.8\%} & {99.09$\pm$0.7\%} & {94.36$\pm$1.4\%} & {94.34$\pm$1.4\%} & {98.42$\pm$1.0\%} \\
\hline
\multirow{2}{*}{RF} & Portugal & {90.74$\pm$1.3\%} & {90.63$\pm$1.4\%} & {95.22$\pm$1.0\%} & \textbf{98.09$\pm$0.6\%} & \textbf{98.09$\pm$0.6\%} & \textbf{99.58$\pm$0.1\%} & \textbf{97.82$\pm$0.6\%} & \textbf{97.82$\pm$0.6\%} & \textbf{99.47$\pm$0.2\%} \\
\cline{2-11}
& Slovakia    & {94.99$\pm$1.5\%} & {94.96$\pm$1.5\%} & {98.23$\pm$1.5\%} & {96.62$\pm$1.6\%} & {96.62$\pm$1.6\%} & \textbf{99.67$\pm$0.4\%} & \textbf{98.32$\pm$1.4\%} & \textbf{98.33$\pm$1.4\%} & \textbf{99.86$\pm$0.2\%} \\
\hline
\multirow{2}{*}{XGB} & Portugal & {90.63$\pm$0.9\%} & {90.55$\pm$0.9\%} & {94.82$\pm$1.0\%} & {97.83$\pm$0.5\%} & {97.83$\pm$0.5\%} & {99.57$\pm$0.2\%} & {97.79$\pm$0.5\%} & {97.79$\pm$0.5\%} & {99.61$\pm$0.2\%} \\
\cline{2-11}
& Slovakia    & {94.67$\pm$1.2\%} & {94.63$\pm$1.3\%} & {97.17$\pm$1.9\%} & {95.35$\pm$2.6\%} & {95.36$\pm$2.6\%} & {98.59$\pm$0.9\%} & {97.28$\pm$2.0\%} & {97.28$\pm$2.0\%} & {99.78$\pm$0.2\%} \\
\hline
\multirow{2}{*}{MLP} & Portugal & {90.85$\pm$1.0\%} & {90.74$\pm$1.0\%} & {94.80$\pm$0.8\%} & {96.99$\pm$0.5\%} & {96.99$\pm$0.5\%} & {99.57$\pm$0.1\%} & {94.57$\pm$0.8\%} & {94.56$\pm$0.8\%} & {98.88$\pm$0.2\%} \\
\cline{2-11}
& Slovakia    & {93.72$\pm$1.8\%} & {93.44$\pm$1.9\%} & {96.94$\pm$3.7\%} & {85.42$\pm$6.0\%} & {85.26$\pm$6.2\%} & {97.85$\pm$0.7\%} & {88.94$\pm$8.2\%} & {88.94$\pm$8.1\%} & {94.90$\pm$5.2\%} \\
\hline
\end{tabular}
\caption{\label{tab:performance_comparison_k_fold}Performance of the classifiers based on the shuffle 5-fold cross-validation on SOD-FE with SMOTE and ADASYN balanced datasets.}
\end{table}

This clear enhancement in AUC, as visualized in the ROC curves in Figure \ref{fig:ROCs}, specifically comparing the "Without SOD-FE" panels (\ref{fig:roc-before} and \ref{fig:roc-before2} for Portugal and Slovakia datasets respectively) against the implied "With SOD-FE" scenarios from the overall performance improvements in Figure \ref{fig:roc-after} and \ref{fig:roc-after2}. It suggests that SOD-FE significantly improves the models' ability to distinguish between classes, leading to better overall performance and robustness. The consistently higher true positive rates at lower false positive rates in the "With SOD-FE" cases (as discussed in the context of the tables) directly correlate with the shift of the ROC curves towards the top-left corner. The reduced variance in performance metrics observed with SOD-FE, especially in the cross-validation results, further indicates the stability and reliability of the enhanced feature set. This demonstrates that SOD-FE effectively captures underlying patterns, leading to more generalizable models.

\begin{figure}[ht]
\centering
\begin{subfigure}[c]{0.40\textwidth}
\centering
\includegraphics[width=.7\textwidth]{figures/ROC_before.pdf}
\caption{\label{fig:roc-before}Without SOD-FE on the Portugal dataset}
\end{subfigure}
\hfill
\begin{subfigure}[c]{0.40\textwidth}
\centering
\includegraphics[width=.7\textwidth]{figures/ROC.pdf}
\caption{\label{fig:roc-after}Without SOD-FE on the Portugal dataset}
\end{subfigure}
\hfill
\begin{subfigure}[c]{0.40\textwidth}
\centering
\includegraphics[width=.7\textwidth]{figures/ROC_before2.pdf}
\caption{\label{fig:roc-before2}Without SOD-FE on the Slovakia dataset}
\end{subfigure}
\hfill
\begin{subfigure}[c]{0.40\textwidth}
\centering
\includegraphics[width=.7\textwidth]{figures/ROC2.pdf}
\caption{\label{fig:roc-after2}Without SOD-FE on the Slovakia dataset}
\end{subfigure}
\caption{\label{fig:ROCs}ROC Curve Comparison for Classifiers on Portugal and Slovakia Datasets, Before and After SOD-FE Integration.}
\end{figure}

The confusion matrices, presented in Figure \ref{fig:matrix_Portugal} for the Portugal dataset and Figure \ref{fig:matrix_Slovakia} for the Slovakia dataset, provide detailed insights into the prediction results and misclassifications for \textit{dropout} and \textit{non-dropout} students across the evaluated ML models. On the Portugal dataset (Figure \ref{fig:matrix_Portugal}), the RF and XGB models demonstrated strong performance. The RF classifier correctly identified 460 \textit{non-dropout} students and 383 \textit{dropout} students, with only 7 false positives and 12 false negatives. XGB showed similar efficacy, correctly predicting 461 \textit{non-dropout} students and 385 \textit{dropout} students, with 6 false positives and 10 false negatives. LR also exhibited high reliability, with 454 correct predictions for \textit{non-dropout} students and 367 for \textit{dropout} students, resulting in 13 false positives and 20 false negatives. SVM and MLP maintained low misclassification rates. SVM correctly identified 456 \textit{non-dropout} and 364 \textit{dropout} students, with 11 false positives and 31 false negatives. MLP showed 456 correct \textit{non-dropout} predictions and 363 correct \textit{dropout} predictions, with 11 false positives and 27 false negatives.

\begin{figure}[ht]
\centering
% Subfigure 1
\begin{subfigure}[b]{0.19\textwidth} % 0.19 * 5 = 0.95, leaving some space
\centering
\includegraphics[width=\textwidth]{figures/CM_ LR.pdf} % Replace with your image filename
\caption{\label{fig:matrix11}}
\end{subfigure}
\hfill % This command distributes the subfigures evenly
% Subfigure 2
\begin{subfigure}[b]{0.19\textwidth}
\centering
\includegraphics[width=\textwidth]{figures/CM_SVM.pdf} % Replace with your image filename
\caption{}
\label{fig:matrix12}
\end{subfigure}
\hfill
% Subfigure 3
\begin{subfigure}[b]{0.19\textwidth}
\centering
\includegraphics[width=\textwidth]{figures/CM_ RF.pdf} % Replace with your image filename
\caption{\label{fig:matrix13}}
\end{subfigure}
\hfill
% Subfigure 4
\begin{subfigure}[b]{0.19\textwidth}
\centering
\includegraphics[width=\textwidth]{figures/CM_XGB.pdf} % Replace with your image filename
\caption{\label{fig:matrix14}}
\end{subfigure}
\hfill
% Subfigure 5
\begin{subfigure}[b]{0.19\textwidth}
\centering
\includegraphics[width=\textwidth]{figures/CM_MLP.pdf} % Replace with your image filename
\caption{\label{fig:matrix15} }
\end{subfigure}
\caption{\label{fig:matrix_Portugal}Confusion matrices of ML models for dropout prediction on the Portugal dataset}
\end{figure}

On the Slovakia dataset (Figure \ref{fig:matrix_Slovakia}), the RF and XGB models achieved remarkably high accuracy, both correctly identifying 54 \textit{non-dropout} students and 42 \textit{dropout} students, with zero false positives and zero false negatives. LR also performed very well, correctly predicting 53 \textit{non-dropout} students and 38 \textit{dropout} students, with only 1 false positive and four false negatives. SVM showed similar robust performance, correctly identifying 53 \textit{non-dropout} students and 38 \textit{dropout} students, with one false positive and four false negatives. MLP, while still effective, had a slightly higher number of misclassifications, correctly predicting 42 \textit{non-dropout} students and 37 \textit{dropout} students, with 12 false positives and 5 false negatives. These results validate the ensemble-based classifiers (RF and XGB) as highly effective solutions for identifying students at risk of dropping out, as they consistently minimize both false positive and false negative errors across both datasets.

\begin{figure}[ht]
\centering

% Subfigure 1
\begin{subfigure}[b]{0.19\textwidth} % 0.19 * 5 = 0.95, leaving some space
\centering
\includegraphics[width=\textwidth]{figures/CM_ LR2.pdf} % Replace with your image filename
\caption{\label{fig:matrix1}}
\end{subfigure}
\hfill % This command distributes the subfigures evenly
% Subfigure 2
\begin{subfigure}[b]{0.19\textwidth}
\centering
\includegraphics[width=\textwidth]{figures/CM_SVM2.pdf} % Replace with your image filename
\caption{\label{fig:matrix2}}
\end{subfigure}
\hfill
% Subfigure 3
\begin{subfigure}[b]{0.19\textwidth}
\centering
\includegraphics[width=\textwidth]{figures/CM_ RF2.pdf} % Replace with your image filename
\caption{\label{fig:matrix3}}
\end{subfigure}
\hfill
% Subfigure 4
\begin{subfigure}[b]{0.19\textwidth}
\centering
\includegraphics[width=\textwidth]{figures/CM_XGB2.pdf} % Replace with your image filename
\caption{\label{fig:matrix4}}
\end{subfigure}
\hfill
% Subfigure 5
\begin{subfigure}[b]{0.19\textwidth}
\centering
\includegraphics[width=\textwidth]{figures/CM_MLP2.pdf} % Replace with your image filename
\caption{\label{fig:matrix5}}
\end{subfigure}
\caption{\label{fig:matrix_Slovakia}Confusion matrices of ML models for dropout prediction on the Slovakia dataset}
\end{figure}

Figures \ref{fig:matrix_Portugal} and \ref{fig:matrix_Slovakia} show that most classification models achieve strong accuracy when predicting \textit{non-dropout} students. False negatives that predict students to remain enrolled while they leave school represent crucial concerns for educational applications because they prevent important intervention efforts. The RF and XGB classifiers demonstrated the lowest false negative rate which demonstrates their strong potential for operational early warning systems.

\subsection*{Model Interpretability}\label{sec:Model_Interpretability}

This study employed a combination of feature ranking and explainability techniques, including correlation heatmap, RFI, and SHAP to improve predictive model interpretability while finding influential features that lead to student dropout. The correlation and RFI offer global insights into feature importance and help understand model effects but they lack the ability to explain single prediction outcomes. In contrast, the SHAP creates interpretability on both a global and local scale by providing numerical values, allowing stakeholders to assess the impact of each feature on a specific prediction outcome. This capability enables not only transparency and model auditing before deployment, but also actionable insight into student-level risk factors. Figure \ref{fig:feature_importance} shows an assessment of important features via RFI (Figure \ref{fig:RFI_after}) compared against the results obtained from SHAP (Figure \ref{fig:shap_after}). For example, SHAP visualizations revealed that students with a high number of curricular\_units\_approved are significantly less likely to drop out, reinforcing educational theories that associate academic achievement with retention. Other influential features identified through SHAP include gender, age, tuition\_fees, debtor, unemployment\_rate, and scholarship\_holder, emphasizing the role of both demographic and financial factors. 

\begin{figure}[ht]
\centering
\begin{subfigure}[c]{0.5\textwidth}
\centering
\includegraphics[width=0.90\textwidth]{figures/Output_RFI_after.pdf}
\caption{\label{fig:RFI_after}Feature importance ranking using RFI}
\end{subfigure}
\hfill
\begin{subfigure}[c]{0.40\textwidth}
\centering
\includegraphics[width=0.90\textwidth]{figures/Output_shap_after.pdf}
\caption{\label{fig:shap_after}SHAP summary}
\end{subfigure}
\caption{\label{fig:feature_importance}Comparison of Global and Local Feature Importance analysis using the RFI and SHAP methods.}
\end{figure}

The SHAP method surpasses other methods because it demonstrates how each unique situation responds to different feature inputs. For example, a higher number of curricular\_units\_approved proves to be an extremely effective indicator for lower dropout risk. The results support educational theory that academic excellence during previous semesters leads students to persist in their education. Younger students and the Female exhibit higher continued enrollment probabilities according to the SHAP feature analysis. Students who maintain stable financial conditions, which include scholarship\_holder status and not identified as a debtor tend to achieve greater academic success. Predictive socioeconomic factors like Mother\_qualification and Father\_occupation revealed lower positions on the plot and a small impact on prediction results. RFI provides insufficient detail to reveal this degree of information. SHAP analysis grants increased insight through visual representations that display how features affect each particular case. Such insights can guide targeted interventions like financial aid counseling or academic mentoring.

\subsection*{Feature Selection Analysis}\label{sec:feature_analysis}

This subsection analyzes the impact of feature selection methods, Random Forest Importance (RFI) and Recursive Feature Elimination (RFE) on the performance of various machine learning models, compared to using all features, on the Portugal dataset. The analysis in Table \ref{tab:feature_selection_comparison} demonstrates that both RFI and RFE feature selection methods produce comparable high-performing models, with minor differences between them. RF showed the best overall performance, with RFI- selected features yielding slightly higher accuracy and F1 score (98.09\%) and AUC (99.58\%) compared to RFE-selected features (98.02\% accuracy/F1, 99.57\% AUC) and the baseline (97.78\% accuracy/F1, 99.50\% AUC). XGB performed exceptionally well even without feature selection, achieving the highest AUC (99.66\%), though RFE also resulted in a strong AUC (99.62\%). Multilayer Perceptron (MLP) performance was positively impacted by RFI, showing an improved F1 score (96.99\%) and AUC (99.57\%) over the baseline (96.64\% F1, 99.51\% AUC), but RFE led to a slight decrease in performance compared to RFI. SVM and LR models showed robust and consistent high performance across all three feature sets, indicating their less sensitivity to feature selection methods.

\begin{table}[ht]
\centering
\footnotesize
\scriptsize
% \tiny
\begin{tabular}{|l|c|c|c|c|c|c|c|c|c|}
\hline
\multirow{2}{*}{\textbf{Model}} & \multicolumn{3}{c|}{\textbf{Without-Selected Features}} & \multicolumn{3}{c|}{\textbf{RFI-Selected Features}} & \multicolumn{3}{c|}{\textbf{RFE-Selected Features}} \\
\cline{2-4} \cline{5-7} \cline{8-10}
& {\textbf{Accuracy}} & {\textbf{F1 Score}} & {\textbf{AUC}} & {\textbf{Accuracy}} & {\textbf{F1 Score}} & {\textbf{AUC}} & {\textbf{Accuracy}} & {\textbf{F1 Score}} & {\textbf{AUC}} \\
\hline
LR & {96.62} & {96.61} & {99.50} & {96.69} & {96.68} & {99.51 } & {96.50} & {96.50} & {99.49} \\
\hline
SVM & {96.90} & {96.89} & {99.50} & {96.80} & {96.80} & {99.51} & {96.87} & {96.87} & {99.49} \\
\hline
RF & \textbf{97.78} & \textbf{97.78} & {99.50} & \textbf{98.09} & \textbf{98.09} & \textbf{99.58} & \textbf{98.02} & \textbf{98.02} & {99.57} \\
\hline
XGB & {97.71} & {97.71} & \textbf{99.66} & {97.83} & {97.83} & {99.57} & {97.83} & {97.83} & \textbf{99.62} \\
\hline
MLP & {96.64} & {96.64} & {99.51} & {96.99} & {96.99} & {99.57} & {96.71} & {96.71} & {99.45} \\
\hline
\end{tabular}
\caption{\label{tab:feature_selection_comparison}Performance Comparison of Models Using Without-Selected, RFI-Selected, and RFE-Selected Features on the Portugal dataset.}
\end{table}

The feature engineering strategy employed in this study combined correlation analysis, Random Forest Importance (RFI), and Recursive Feature Elimination (RFE). These methods were chosen due to their proven efficiency in identifying relevant and non-redundant features in classification tasks, particularly in educational data mining contexts. The combination provided a structured approach for dimensionality reduction and interpretability. However, this approach is not without limitations. Correlation analysis primarily captures linear dependencies, which may not fully reflect the complex, non-linear interactions often present in dropout-related variables. Similarly, RFE can be sensitive to the initial feature set, potentially leading to suboptimal subsets, especially when informative features only become significant in interaction with others. While the integration of these methods resulted in modest performance improvements, the gains were not substantial across all models. Therefore, we recommend further exploration of non-linear feature selection techniques, such as embedded deep learning-based selection, to better capture complex relationships in educational datasets. Future work may also benefit from incorporating feature construction based on domain knowledge, and conducting ablation studies to more rigorously assess the contribution of each feature to model performance.

\subsection*{\label{sec:ablation_study}Component-wise Analysis}

This subsection mainly discusses the effects of the component-wise performance analysis of the SOD-FE approach (including supervised outlier detection (SOD) and feature engineering (FE)) and balancing techniques (including SMOTE and ADASYN). Tables \ref{tab:Portugal_performance_comparison} and \ref{tab:Slovakia_performance_comparison} present a detailed ablation study using the RF classifier, examining the impact of different data preprocessing and feature engineering strategies on student dropout prediction in the Portugal and Slovakia datasets, respectively. The performance is evaluated using standards like accuracy, precision, recall, F1 score, and AUC based on the 5-fold cross-validation.

\begin{table}[ht]
\centering
\small % Keep the font size small to try and fit everything
\footnotesize
\scriptsize
% \tiny
% \begin{adjustbox}{width=\textwidth,center}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
\hline
\textbf{SMOTE} & \textbf{ADASYN} & \textbf{SOD} & \textbf{FE} & {\textbf{Accuracy}} & {\textbf{Precision}} & {\textbf{Recall}} & {\textbf{F1}} & {\textbf{AUC}} \\
\hline
 &  &  &  & 90.74 & 90.91 & 90.74 & 90.63 & 95.22 \\
\hline
\checkmark &  &  &  & 90.08\decrease & 90.14\decrease & 90.08\decrease & 90.01\decrease & 94.55\decrease \\
\hline
 & \checkmark &  &  & 90.08\neutral & 90.12\decrease & 90.08\neutral & 90.01\neutral & 94.98\increase \\
\hline
 &  & \checkmark &  & 97.99\increase & 98.01\increase & 97.99\increase & 97.98\increase & 99.21\increase \\
\hline
\checkmark &  & \checkmark &  & 97.78\decrease & 97.81\decrease & 97.78\decrease & 97.78\decrease & 99.50\increase \\
\hline
 & \checkmark & \checkmark &  & 97.54\decrease & 97.56\decrease & 97.54\decrease & 97.54\decrease & 99.51\increase \\
\hline
 &  & \checkmark & \checkmark & 98.07\increase & 98.09\increase & 98.07\increase & 98.07\increase & 99.34\decrease \\
\hline
 & \checkmark & \checkmark & \checkmark & 97.82\decrease & 97.84\decrease & 97.82\decrease & 97.82\decrease & 99.47\increase \\
\hline
\checkmark &  & \checkmark & \checkmark & \textbf{98.09}\increase & \textbf{98.10}\increase & \textbf{98.09}\increase & \textbf{98.09}\increase & \textbf{99.58}\increase \\
\hline
\end{tabular}
\caption{\label{tab:Portugal_performance_comparison}Comparison of performance metrics of SOD-FE with Random Forest classifier on the Portugal dataset when using different components}
\end{table}

According to the results, the proposed method gave the best accuracy, precision, recall, F1 score, and AUC on both datasets.For instance, on the Portugal dataset, combining SOD and FE with SMOTE achieved an AUC of 99.58\%, compared to baselines of around 95.22\%. Similarly, for the Slovakia dataset, the combined SOD-FE with ADASYN reached an AUC of 99.86\%, demonstrating superior performance over baselines around 98.23\%. Furthermore, the choice of balancing technique also influences performance. Regardless of the balance strategy, the results indicate strong performance when applying both SOD and FE, which supports SOD-FE as a strong approach to enhance classification performance in complex and different datasets.

\begin{table}[ht]
\centering
\footnotesize
\scriptsize
\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
\hline
\textbf{SMOTE} & \textbf{ADASYN} & \textbf{SOD} & \textbf{FE} & {\textbf{Accuracy}} & {\textbf{Precision}} & {\textbf{Recall}} & {\textbf{F1 Score}} & {\textbf{AUC}} \\
\hline
 &  &  &  & 94.99 & 95.12 & 94.99 & 94.96 & 98.23 \\
\hline
\checkmark &  &  &  & 96.88\increase & 96.88\increase & 96.88\increase & 96.88\increase & 98.71\increase \\
\hline
 & \checkmark &  &  & 96.88\increase & 96.88\increase & 96.88\increase & 96.88\increase & 99.14\increase \\
\hline
 &  & \checkmark &  & 95.62\increase & 95.71\increase & 95.62\increase & 95.53\increase & 99.26\increase \\
\hline
\checkmark &  & \checkmark &  & 96.62\increase & 96.66\increase & 96.62\increase & 96.62\increase & 99.67\increase \\
\hline
 & \checkmark & \checkmark &  & 98.32\increase & 98.42\increase & 98.32\increase & 98.33\increase & 99.86\increase \\
\hline
 &  & \checkmark & \checkmark & 95.62\neutral & 95.71\neutral & 95.62\neutral & 95.53\neutral & 99.26\neutral \\
\hline
\checkmark &  & \checkmark& \checkmark & 96.62\neutral & 96.66\neutral & 96.62\neutral & 96.62\neutral & 99.67\neutral \\
\hline
 & \checkmark & \checkmark & \checkmark & \textbf{98.32}\neutral & \textbf{98.42}\neutral & \textbf{98.32}\neutral & \textbf{98.33}\neutral & \textbf{99.86}\neutral \\
\hline
\end{tabular}
\caption{\label{tab:Slovakia_performance_comparison}Comparison of performance metrics of SOD-FE with Random Forest classifier on the Slovakia dataset when using different components}
\end{table}

SOD-FE operates within the proposed approach to boost model performance when working with the imbalanced datasets often found in educational contexts. The modular design enables effortless integration with institutional analytics systems while supporting interpretability. However, this study contains limitations. Although the SOD-FE method offers various benefits, its analysis remains univariate because it depends on Pearson correlation to identify linear relationships. The analysis method has the potential to miss complicated non-linear patterns between features. The data preprocessing phase needs further investigation to develop multivariate outlier detection approaches and non-linear feature relevance measures for enhancing its robustness and accuracy.

\section*{Discussion}

The experimental evaluation demonstrates that the proposed SOD-FE method significantly enhances student dropout prediction across various machine learning models and datasets. When combined with the Random Forest classifier, the SOD-FE approach achieved high F1 scores on both datasets—98.09\% for the Portugal dataset and 98.33\%. SOD played a crucial role in preserving informative outlier cases related to dropout by eliminating anomalies based on the relationships between features and the target variable. A dual validation strategy, comprising hold-out validation and stratified 5-fold cross-validation, was employed to mitigate the risk of overfitting and ensure reliable model performance across different data partitions. To assess generalizability, the model was evaluated on two distinct datasets originating from Portugal and Slovakia. These datasets differ in student demographics, institutional structures, and socio-economic contexts. The consistent performance of the SOD-FE framework across both datasets underscores its potential applicability in diverse educational settings. Nevertheless, further research is recommended to evaluate the model on larger and more heterogeneous datasets, particularly those from underrepresented regions and educational systems, to better assess its scalability and adaptability. Moreover, model interpretability techniques such as SHAP provided valuable insights into the factors influencing dropout predictions. Key features identified included academic performance, gender, and financial status, factors that are crucial for early intervention. These insights support the practical deployment of predictive models to assist educational institutions in identifying and supporting at-risk students. However, the current approach relies primarily on linear correlation measures and may be sensitive to dataset-specific characteristics. Future work should explore the integration of non-linear feature analysis methods to enhance robustness and generalizability.

\section*{Conclusion}

This study developed a new predictive system named supervised outlier detection with feature engineering (SOD-FE) for recognizing at-risk students during their higher education period. The SOD-FE approach improved multiple machine learning (ML) models through its targeted methods for dealing with class imbalance problems and handling irrelevant or noisy data using outlier filtering and systematic feature engineering approaches. Random Forest (RF) proves the most effective classifier according to thorough evaluations since it achieved maximum accuracy alongside F1 score and AUC values across random hold-out and 5-fold cross-validation tests. The combination of outlier filtering using the SOD and improved feature selection mechanisms based on RFI and RFE has created better performing and understandable models. The use of SHAP as an explainable AI tool delivered important predictive insights about academic metrics and financial data as well as significant demographic factors which enhanced the transparency and trustworthiness of the models for education decision-makers. The results using the suggested method showed that educational predictive modeling requires integration between data preprocessing techniques with balanced filtering methods along with interpretability features as fundamental success factors. The SOD-FE approach presents an accessible solution for educational organizations to create early warning systems that lead to better student retention rates and educational success. However, future academic research needs to collect data from multiple higher education institutions and apply advanced multivariable and non-linear analysis methods to improve prediction accuracy and the generalizability of the results. Moreover, our proposed study has developed an approach that incorporates ML and neural network-based techniques. Embedding with advanced models such as CNNs, LSTM networks, and vision transformers can also be researched. These models are useful in sequential dependency modeling patterns, which can be more robust, adaptable, and comprehensive for analyzing the risk of student dropout.

\section*{Data availability}

The data presented in this study are publicly available at: \\
\url{https://archive.ics.uci.edu/dataset/697/predict+students+dropout+and+academic+success} (accessed on 13 June 2025). \\
Additional data are available at: \\
\url{https://github.com/JKabathova/Students-Dropout-Prediction} (accessed on 13 June 2025).

\section*{Funding}

No funding.

% \bibliography{sample}
\begin{thebibliography}{10}
\urlstyle{rm}
\expandafter\ifx\csname url\endcsname\relax
  \def\url#1{\texttt{#1}}\fi
\expandafter\ifx\csname urlprefix\endcsname\relax\def\urlprefix{URL }\fi
\expandafter\ifx\csname doiprefix\endcsname\relax\def\doiprefix{DOI: }\fi
\providecommand{\bibinfo}[2]{#2}
\providecommand{\eprint}[2][]{\url{#2}}

\bibitem{alhazmi2023early}
\bibinfo{author}{Alhazmi, E.} \& \bibinfo{author}{Sheneamer, A.}
\newblock \bibinfo{journal}{\bibinfo{title}{Early predicting of students performance in higher education}}.
\newblock {\emph{\JournalTitle{Ieee Access}}} \textbf{\bibinfo{volume}{11}}, \bibinfo{pages}{27579--27589} (\bibinfo{year}{2023}).

\bibitem{chung2019dropout}
\bibinfo{author}{Chung, J.~Y.} \& \bibinfo{author}{Lee, S.}
\newblock \bibinfo{journal}{\bibinfo{title}{Dropout early warning systems for high school students using machine learning}}.
\newblock {\emph{\JournalTitle{Children and Youth Services Review}}} \textbf{\bibinfo{volume}{96}}, \bibinfo{pages}{346--353} (\bibinfo{year}{2019}).

\bibitem{schmitt2021wwh}
\bibinfo{author}{Schmitt, J.}, \bibinfo{author}{Fini, M.~I.}, \bibinfo{author}{Bailer, C.}, \bibinfo{author}{Fritsch, R.} \& \bibinfo{author}{Andrade, D. F.~d.}
\newblock \bibinfo{journal}{\bibinfo{title}{Wwh-dropout scale: when, why and how to measure propensity to drop out of undergraduate courses}}.
\newblock {\emph{\JournalTitle{Journal of Applied Research in Higher Education}}} \textbf{\bibinfo{volume}{13}}, \bibinfo{pages}{540--560} (\bibinfo{year}{2021}).

\bibitem{mulyani2019dropout}
\bibinfo{author}{Mulyani, E.}, \bibinfo{author}{Hidayah, I.} \& \bibinfo{author}{Fauziati, S.}
\newblock \bibinfo{title}{Dropout prediction optimization through smote and ensemble learning}.
\newblock In \emph{\bibinfo{booktitle}{2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI)}}, \bibinfo{pages}{516--521} (\bibinfo{organization}{IEEE}, \bibinfo{year}{2019}).

\bibitem{Rabelo2024AMF}
\bibinfo{author}{Rabelo, A.~M.} \& \bibinfo{author}{Z{\'a}rate, L.~E.}
\newblock \bibinfo{journal}{\bibinfo{title}{A model for predicting dropout of higher education students}}.
\newblock {\emph{\JournalTitle{Data Science and Management}}}  (\bibinfo{year}{2024}).

\bibitem{han2005borderline}
\bibinfo{author}{Han, H.}, \bibinfo{author}{Wang, W.-Y.} \& \bibinfo{author}{Mao, B.-H.}
\newblock \bibinfo{title}{Borderline-smote: a new over-sampling method in imbalanced data sets learning}.
\newblock In \emph{\bibinfo{booktitle}{International conference on intelligent computing}}, \bibinfo{pages}{878--887} (\bibinfo{organization}{Springer}, \bibinfo{year}{2005}).

\bibitem{he2008adasyn}
\bibinfo{author}{He, H.}, \bibinfo{author}{Bai, Y.}, \bibinfo{author}{Garcia, E.~A.} \& \bibinfo{author}{Li, S.}
\newblock \bibinfo{title}{Adasyn: Adaptive synthetic sampling approach for imbalanced learning}.
\newblock In \emph{\bibinfo{booktitle}{2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence)}}, \bibinfo{pages}{1322--1328} (\bibinfo{organization}{Ieee}, \bibinfo{year}{2008}).

\bibitem{10498520}
\bibinfo{author}{Sabbir, W.}, \bibinfo{author}{Abdullah-Al-Kafi, M.}, \bibinfo{author}{Afridi, A.~S.}, \bibinfo{author}{Rahman, M.~S.} \& \bibinfo{author}{Karmakar, M.}
\newblock \bibinfo{title}{Improving predictive analytics for student dropout: A comprehensive analysis and model evaluation}.
\newblock In \emph{\bibinfo{booktitle}{2024 11th International Conference on Computing for Sustainable Global Development (INDIACom)}}, \bibinfo{pages}{951--956}, \doiprefix\url{10.23919/INDIACom61295.2024.10498520} (\bibinfo{year}{2024}).

\bibitem{ZEINEDDINE2021106903}
\bibinfo{author}{Zeineddine, H.}, \bibinfo{author}{Braendle, U.} \& \bibinfo{author}{Farah, A.}
\newblock \bibinfo{journal}{\bibinfo{title}{Enhancing prediction of student success: Automated machine learning approach}}.
\newblock {\emph{\JournalTitle{Computers \& Electrical Engineering}}} \textbf{\bibinfo{volume}{89}}, \bibinfo{pages}{106903}, \doiprefix\url{https://doi.org/10.1016/j.compeleceng.2020.106903} (\bibinfo{year}{2021}).

\bibitem{fernandez2022supervised}
\bibinfo{author}{Fern{\'a}ndez, {\'A}.}, \bibinfo{author}{Bella, J.} \& \bibinfo{author}{Dorronsoro, J.~R.}
\newblock \bibinfo{journal}{\bibinfo{title}{Supervised outlier detection for classification and regression}}.
\newblock {\emph{\JournalTitle{Neurocomputing}}} \textbf{\bibinfo{volume}{486}}, \bibinfo{pages}{77--92} (\bibinfo{year}{2022}).

\bibitem{vinutha2018detection}
\bibinfo{author}{Vinutha, H.}, \bibinfo{author}{Poornima, B.} \& \bibinfo{author}{Sagar, B.}
\newblock \bibinfo{title}{Detection of outliers using interquartile range technique from intrusion dataset}.
\newblock In \emph{\bibinfo{booktitle}{Information and decision sciences: Proceedings of the 6th international conference on ficta}}, \bibinfo{pages}{511--518} (\bibinfo{organization}{Springer}, \bibinfo{year}{2018}).

\bibitem{salih2025perspective}
\bibinfo{author}{Salih, A.~M.} \emph{et~al.}
\newblock \bibinfo{journal}{\bibinfo{title}{A perspective on explainable artificial intelligence methods: Shap and lime}}.
\newblock {\emph{\JournalTitle{Advanced Intelligent Systems}}} \textbf{\bibinfo{volume}{7}}, \bibinfo{pages}{2400304} (\bibinfo{year}{2025}).

\bibitem{Rizwan2025IdentifyingTF}
\bibinfo{author}{Rizwan, S.}, \bibinfo{author}{Nee, C.~K.} \& \bibinfo{author}{Garfan, S.}
\newblock \bibinfo{journal}{\bibinfo{title}{Identifying the factors affecting student academic performance and engagement prediction in mooc using deep learning: A systematic literature review}}.
\newblock {\emph{\JournalTitle{IEEE Access}}} \textbf{\bibinfo{volume}{13}}, \bibinfo{pages}{18952--18982} (\bibinfo{year}{2025}).

\bibitem{dewi2023machine}
\bibinfo{author}{Dewi, M.~A.}, \bibinfo{author}{Kurniadi, F.~I.}, \bibinfo{author}{Murad, D.~F.}, \bibinfo{author}{Rabiha, S.~G.} \& \bibinfo{author}{Romli, A.}
\newblock \bibinfo{title}{Machine learning algorithms for early predicting dropout student online learning}.
\newblock In \emph{\bibinfo{booktitle}{2023 IEEE 9th International Conference on Computing, Engineering and Design (ICCED)}}, \bibinfo{pages}{1--4} (\bibinfo{organization}{IEEE}, \bibinfo{year}{2023}).

\bibitem{huynh2025early}
\bibinfo{author}{Huynh-Cam, T.-T.}, \bibinfo{author}{Chen, L.-S.} \& \bibinfo{author}{Lu, T.-C.}
\newblock \bibinfo{journal}{\bibinfo{title}{Early prediction models and crucial factor extraction for first-year undergraduate student dropouts}}.
\newblock {\emph{\JournalTitle{Journal of Applied Research in Higher Education}}} \textbf{\bibinfo{volume}{17}}, \bibinfo{pages}{624--639} (\bibinfo{year}{2025}).

\bibitem{su2022applying}
\bibinfo{author}{Su, Y.-S.}, \bibinfo{author}{Lin, Y.-D.} \& \bibinfo{author}{Liu, T.-Q.}
\newblock \bibinfo{journal}{\bibinfo{title}{Applying machine learning technologies to explore students’ learning features and performance prediction}}.
\newblock {\emph{\JournalTitle{Frontiers in Neuroscience}}} \textbf{\bibinfo{volume}{16}}, \bibinfo{pages}{1018005} (\bibinfo{year}{2022}).

\bibitem{kabathova2021towards}
\bibinfo{author}{Kabathova, J.} \& \bibinfo{author}{Drlik, M.}
\newblock \bibinfo{journal}{\bibinfo{title}{Towards predicting student’s dropout in university courses using different machine learning techniques}}.
\newblock {\emph{\JournalTitle{Applied Sciences}}} \textbf{\bibinfo{volume}{11}}, \bibinfo{pages}{3130} (\bibinfo{year}{2021}).

\bibitem{deb2024predicting}
\bibinfo{author}{Deb, S.} \emph{et~al.}
\newblock \bibinfo{title}{Predicting student dropout: A machine learning approach}.
\newblock In \emph{\bibinfo{booktitle}{2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT)}}, \bibinfo{pages}{1--7} (\bibinfo{organization}{IEEE}, \bibinfo{year}{2024}).

\bibitem{sulak2024predicting}
\bibinfo{author}{SULAK, S.~A.} \& \bibinfo{author}{KOKLU, N.}
\newblock \bibinfo{journal}{\bibinfo{title}{Predicting student dropout using machine learning algorithms}}.
\newblock {\emph{\JournalTitle{Intelligent Methods In Engineering Sciences}}} \textbf{\bibinfo{volume}{3}}, \bibinfo{pages}{91--98} (\bibinfo{year}{2024}).

\bibitem{bezek2024classification}
\bibinfo{author}{BEZEK~G{\"U}RE, {\"O}.}
\newblock \bibinfo{journal}{\bibinfo{title}{Classification of students'dropout status using the random forest method.}}
\newblock {\emph{\JournalTitle{International Journal of Eurasia Social Sciences/Uluslararasi Avrasya Sosyal Bilimler Dergisi}}} \textbf{\bibinfo{volume}{15}} (\bibinfo{year}{2024}).

\bibitem{kok2024novel}
\bibinfo{author}{Kok, C.~L.}, \bibinfo{author}{Ho, C.~K.}, \bibinfo{author}{Chen, L.}, \bibinfo{author}{Koh, Y.~Y.} \& \bibinfo{author}{Tian, B.}
\newblock \bibinfo{journal}{\bibinfo{title}{A novel predictive modeling for student attrition utilizing machine learning and sustainable big data analytics}}.
\newblock {\emph{\JournalTitle{Applied Sciences}}} \textbf{\bibinfo{volume}{14}}, \bibinfo{pages}{9633} (\bibinfo{year}{2024}).

\bibitem{martinez2024prediction}
\bibinfo{author}{Mart{\'\i}nez, J.} \& \bibinfo{author}{Castillo, D.}
\newblock \bibinfo{journal}{\bibinfo{title}{Prediction of student dropout using artificial intelligence algorithms}}.
\newblock {\emph{\JournalTitle{Procedia Computer Science}}} \textbf{\bibinfo{volume}{251}}, \bibinfo{pages}{764--770} (\bibinfo{year}{2024}).

\bibitem{syed2023predictive}
\bibinfo{author}{Syed~Mustapha, S.}
\newblock \bibinfo{journal}{\bibinfo{title}{Predictive analysis of students’ learning performance using data mining techniques: A comparative study of feature selection methods}}.
\newblock {\emph{\JournalTitle{Applied System Innovation}}} \textbf{\bibinfo{volume}{6}}, \bibinfo{pages}{86} (\bibinfo{year}{2023}).

\bibitem{kruger2023explainable}
\bibinfo{author}{Kr{\"u}ger, J. G.~C.}, \bibinfo{author}{de~Souza Britto~Jr, A.} \& \bibinfo{author}{Barddal, J.~P.}
\newblock \bibinfo{journal}{\bibinfo{title}{An explainable machine learning approach for student dropout prediction}}.
\newblock {\emph{\JournalTitle{Expert Systems with Applications}}} \textbf{\bibinfo{volume}{233}}, \bibinfo{pages}{120933} (\bibinfo{year}{2023}).

\bibitem{cho2023study}
\bibinfo{author}{Cho, C.~H.}, \bibinfo{author}{Yu, Y.~W.} \& \bibinfo{author}{Kim, H.~G.}
\newblock \bibinfo{journal}{\bibinfo{title}{A study on dropout prediction for university students using machine learning}}.
\newblock {\emph{\JournalTitle{Applied Sciences}}} \textbf{\bibinfo{volume}{13}}, \bibinfo{pages}{12004} (\bibinfo{year}{2023}).

\bibitem{seo2024predictive}
\bibinfo{author}{Seo, E.-Y.}, \bibinfo{author}{Yang, J.}, \bibinfo{author}{Lee, J.-E.} \& \bibinfo{author}{So, G.}
\newblock \bibinfo{journal}{\bibinfo{title}{Predictive modelling of student dropout risk: Practical insights from a south korean distance university}}.
\newblock {\emph{\JournalTitle{Heliyon}}} \textbf{\bibinfo{volume}{10}} (\bibinfo{year}{2024}).

\bibitem{Villar2024SupervisedML}
\bibinfo{author}{Villar, A.~S.} \& \bibinfo{author}{de~Andrade, C. R.~V.}
\newblock \bibinfo{journal}{\bibinfo{title}{Supervised machine learning algorithms for predicting student dropout and academic success: a comparative study}}.
\newblock {\emph{\JournalTitle{Discov. Artif. Intell.}}} \textbf{\bibinfo{volume}{4}} (\bibinfo{year}{2024}).

\bibitem{talebi2024ensemble}
\bibinfo{author}{Talebi, K.}, \bibinfo{author}{Torabi, Z.} \& \bibinfo{author}{Daneshpour, N.}
\newblock \bibinfo{journal}{\bibinfo{title}{Ensemble models based on cnn and lstm for dropout prediction in mooc}}.
\newblock {\emph{\JournalTitle{Expert Systems with Applications}}} \textbf{\bibinfo{volume}{235}}, \bibinfo{pages}{121187} (\bibinfo{year}{2024}).

\bibitem{mustofa2025novel}
\bibinfo{author}{Mustofa, S.}, \bibinfo{author}{Emon, Y.~R.}, \bibinfo{author}{Mamun, S.~B.}, \bibinfo{author}{Akhy, S.~A.} \& \bibinfo{author}{Ahad, M.~T.}
\newblock \bibinfo{journal}{\bibinfo{title}{A novel ai-driven model for student dropout risk analysis with explainable ai insights}}.
\newblock {\emph{\JournalTitle{Computers and Education: Artificial Intelligence}}} \textbf{\bibinfo{volume}{8}}, \bibinfo{pages}{100352} (\bibinfo{year}{2025}).

\bibitem{beseiso2025enhancing}
\bibinfo{author}{Beseiso, M.}
\newblock \bibinfo{journal}{\bibinfo{title}{Enhancing student success prediction: A comparative analysis of machine learning technique}}.
\newblock {\emph{\JournalTitle{TechTrends}}} \bibinfo{pages}{1--13} (\bibinfo{year}{2025}).

\bibitem{deng2024multidimensional}
\bibinfo{author}{Deng, J.}, \bibinfo{author}{Huang, X.} \& \bibinfo{author}{Ren, X.}
\newblock \bibinfo{journal}{\bibinfo{title}{A multidimensional analysis of self-esteem and individualism: A deep learning-based model for predicting elementary school students' academic performance}}.
\newblock {\emph{\JournalTitle{Measurement: Sensors}}} \textbf{\bibinfo{volume}{33}}, \bibinfo{pages}{101147} (\bibinfo{year}{2024}).

\bibitem{ghorbani2020comparing}
\bibinfo{author}{Ghorbani, R.} \& \bibinfo{author}{Ghousi, R.}
\newblock \bibinfo{journal}{\bibinfo{title}{Comparing different resampling methods in predicting students’ performance using machine learning techniques}}.
\newblock {\emph{\JournalTitle{IEEE access}}} \textbf{\bibinfo{volume}{8}}, \bibinfo{pages}{67899--67911} (\bibinfo{year}{2020}).

\bibitem{nagy2024interpretable}
\bibinfo{author}{Nagy, M.} \& \bibinfo{author}{Molontay, R.}
\newblock \bibinfo{journal}{\bibinfo{title}{Interpretable dropout prediction: towards xai-based personalized intervention}}.
\newblock {\emph{\JournalTitle{International Journal of Artificial Intelligence in Education}}} \textbf{\bibinfo{volume}{34}}, \bibinfo{pages}{274--300} (\bibinfo{year}{2024}).

\bibitem{islam2025integration}
\bibinfo{author}{Islam, M.~M.}, \bibinfo{author}{Sojib, F.~H.}, \bibinfo{author}{Mihad, M. F.~H.}, \bibinfo{author}{Hasan, M.} \& \bibinfo{author}{Rahman, M.}
\newblock \bibinfo{journal}{\bibinfo{title}{The integration of explainable ai in educational data mining for student academic performance prediction and support system}}.
\newblock {\emph{\JournalTitle{Telematics and Informatics Reports}}} \bibinfo{pages}{100203} (\bibinfo{year}{2025}).

\bibitem{johora2025explainable}
\bibinfo{author}{Johora, F.~T.}, \bibinfo{author}{Hasan, M.~N.}, \bibinfo{author}{Rajbongshi, A.}, \bibinfo{author}{Ashrafuzzaman, M.} \& \bibinfo{author}{Akter, F.}
\newblock \bibinfo{journal}{\bibinfo{title}{An explainable ai-based approach for predicting undergraduate students academic performance}}.
\newblock {\emph{\JournalTitle{Array}}} \textbf{\bibinfo{volume}{26}}, \bibinfo{pages}{100384} (\bibinfo{year}{2025}).

\bibitem{Martins2021EarlyPO}
\bibinfo{author}{Martins, M.~V.}, \bibinfo{author}{Tolledo, D.}, \bibinfo{author}{Machado, J.}, \bibinfo{author}{Baptista, L. M.~T.} \& \bibinfo{author}{Realinho, V.}
\newblock \bibinfo{title}{Early prediction of student's performance in higher education: A case study}.
\newblock In \emph{\bibinfo{booktitle}{WorldCIST}} (\bibinfo{year}{2021}).

\bibitem{realinho2022predicting}
\bibinfo{author}{Realinho, V.}, \bibinfo{author}{Machado, J.}, \bibinfo{author}{Baptista, L.} \& \bibinfo{author}{Martins, M.~V.}
\newblock \bibinfo{journal}{\bibinfo{title}{Predicting student dropout and academic success}}.
\newblock {\emph{\JournalTitle{Data}}} \textbf{\bibinfo{volume}{7}}, \bibinfo{pages}{146} (\bibinfo{year}{2022}).

\bibitem{vergara2014review}
\bibinfo{author}{Vergara, J.~R.} \& \bibinfo{author}{Est{\'e}vez, P.~A.}
\newblock \bibinfo{journal}{\bibinfo{title}{A review of feature selection methods based on mutual information}}.
\newblock {\emph{\JournalTitle{Neural computing and applications}}} \textbf{\bibinfo{volume}{24}}, \bibinfo{pages}{175--186} (\bibinfo{year}{2014}).

\bibitem{8074494}
\bibinfo{author}{Jaiswal, J.~K.} \& \bibinfo{author}{Samikannu, R.}
\newblock \bibinfo{title}{Application of random forest algorithm on feature subset selection and classification and regression}.
\newblock In \emph{\bibinfo{booktitle}{2017 World Congress on Computing and Communication Technologies (WCCCT)}}, \bibinfo{pages}{65--68}, \doiprefix\url{10.1109/WCCCT.2016.25} (\bibinfo{year}{2017}).

\bibitem{goren2024early}
\bibinfo{author}{Goren, O.}, \bibinfo{author}{Cohen, L.} \& \bibinfo{author}{Rubinstein, A.}
\newblock \bibinfo{title}{Early prediction of student dropout in higher education using machine learning models}.
\newblock In \emph{\bibinfo{booktitle}{Proceedings of the 17th International Conference on Educational Data Mining}}, \bibinfo{pages}{349--359} (\bibinfo{year}{2024}).

\bibitem{hosmer2013applied}
\bibinfo{author}{Hosmer~Jr, D.~W.}, \bibinfo{author}{Lemeshow, S.} \& \bibinfo{author}{Sturdivant, R.~X.}
\newblock \emph{\bibinfo{title}{Applied logistic regression}} (\bibinfo{publisher}{John Wiley \& Sons}, \bibinfo{year}{2013}).

\bibitem{cortes1995support}
\bibinfo{author}{Cortes, C.} \& \bibinfo{author}{Vapnik, V.}
\newblock \bibinfo{journal}{\bibinfo{title}{Support-vector networks}}.
\newblock {\emph{\JournalTitle{Machine learning}}} \textbf{\bibinfo{volume}{20}}, \bibinfo{pages}{273--297} (\bibinfo{year}{1995}).

\bibitem{breiman2001random}
\bibinfo{author}{Breiman, L.}
\newblock \bibinfo{journal}{\bibinfo{title}{Random forests}}.
\newblock {\emph{\JournalTitle{Machine learning}}} \textbf{\bibinfo{volume}{45}}, \bibinfo{pages}{5--32} (\bibinfo{year}{2001}).

\bibitem{chen2016xgboost}
\bibinfo{author}{Chen, T.} \& \bibinfo{author}{Guestrin, C.}
\newblock \bibinfo{title}{Xgboost: A scalable tree boosting system}.
\newblock In \emph{\bibinfo{booktitle}{Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining}}, \bibinfo{pages}{785--794} (\bibinfo{year}{2016}).

\bibitem{rumelhart1986learning}
\bibinfo{author}{Rumelhart, D.~E.}, \bibinfo{author}{Hinton, G.~E.} \& \bibinfo{author}{Williams, R.~J.}
\newblock \bibinfo{journal}{\bibinfo{title}{Learning representations by back-propagating errors}}.
\newblock {\emph{\JournalTitle{nature}}} \textbf{\bibinfo{volume}{323}}, \bibinfo{pages}{533--536} (\bibinfo{year}{1986}).

\end{thebibliography}


\section*{Author Contributions}

Sabah Saad: Conceptualization, data acquisition, methodology, writing – original draft preparation;  
Eissa Alzabidi: Software implementation, methodology, validation, writing – review and editing;  
Önder Yakut: Supervision, manuscript review, and final approval;
Oğuz Fındık: Methodology, manuscript review, and final approval.

\section*{Declarations}

\section*{Competing interests}

The authors declare no competing interests.

\section*{Additional information}

\textbf{Correspondence} and requests for materials should be addressed to E.A.\\
\textbf{Reprints and permissions information} is available at \url{www.nature.com/reprints}.\\
\textbf{Publisher’s note} Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

\end{document}