%Version 3.1 December 2024
% See section 11 of the User Manual for version history
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%                                                                 %%
%% Please do not use \input{...} to include other tex files.       %%
%% Submit your LaTeX manuscript as one .tex document.              %%
%%                                                                 %%
%% All additional figures and files should be attached             %%
%% separately and not embedded in the \TeX\ document itself.       %%
%%                                                                 %%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%\documentclass[referee,sn-basic]{sn-jnl}% referee option is meant for double line spacing

%%=======================================================%%
%% to print line numbers in the margin use lineno option %%
%%=======================================================%%

%%\documentclass[lineno,pdflatex,sn-basic]{sn-jnl}% Basic Springer Nature Reference Style/Chemistry Reference Style

%%=========================================================================================%%
%% the documentclass is set to pdflatex as default. You can delete it if not appropriate.  %%
%%=========================================================================================%%

%%\documentclass[sn-basic]{sn-jnl}% Basic Springer Nature Reference Style/Chemistry Reference Style

%%Note: the following reference styles support Namedate and Numbered referencing. By default the style follows the most common style. To switch between the options you can add or remove Numbered in the optional parenthesis. 
%%The option is available for: sn-basic.bst, sn-chicago.bst%  
 
%%\documentclass[pdflatex,sn-nature]{sn-jnl}% Style for submissions to Nature Portfolio journals
%%\documentclass[pdflatex,sn-basic]{sn-jnl}% Basic Springer Nature Reference Style/Chemistry Reference Style
% \documentclass[pdflatex,sn-mathphys-num]{sn-jnl}% Math and Physical Sciences Numbered Reference Style
%%\documentclass[pdflatex,sn-mathphys-ay]{sn-jnl}% Math and Physical Sciences Author Year Reference Style
%%\documentclass[pdflatex,sn-aps]{sn-jnl}% American Physical Society (APS) Reference Style
%%\documentclass[pdflatex,sn-vancouver-num]{sn-jnl}% Vancouver Numbered Reference Style
%%\documentclass[pdflatex,sn-vancouver-ay]{sn-jnl}% Vancouver Author Year Reference Style
%%\documentclass[pdflatex,sn-apa]{sn-jnl}% APA Reference Style
%%\documentclass[pdflatex,sn-chicago]{sn-jnl}% Chicago-based Humanities Reference Style
\documentclass[iicol,sn-mathphys-num]{sn-jnl}

%%%% Standard Packages
%%<additional latex packages if required can be included here>

\usepackage{graphicx}%
\usepackage{multirow}%
\usepackage{amsmath,amssymb,amsfonts}%
\usepackage{amsthm}%
\usepackage{mathrsfs}%
\usepackage[title]{appendix}%
\usepackage{xcolor}%
\usepackage{textcomp}%
\usepackage{manyfoot}%
\usepackage{booktabs}%
\usepackage{algorithm}%
\usepackage{algorithmicx}%
\usepackage{algpseudocode}%
\usepackage{listings}%
\usepackage{tabularx}%
% \usepackage[final]{graphicx}%
% \usepackage{float}%
\usepackage[caption = false]{subfig}%
%%%%

%%%%%=============================================================================%%%%
%%%%  Remarks: This template is provided to aid authors with the preparation
%%%%  of original research articles intended for submission to journals published 
%%%%  by Springer Nature. The guidance has been prepared in partnership with 
%%%%  production teams to conform to Springer Nature technical requirements. 
%%%%  Editorial and presentation requirements differ among journal portfolios and 
%%%%  research disciplines. You may find sections in this template are irrelevant 
%%%%  to your work and are empowered to omit any such section if allowed by the 
%%%%  journal you intend to submit to. The submission guidelines and policies 
%%%%  of the journal take precedence. A detailed User Manual is available in the 
%%%%  template package for technical guidance.
%%%%%=============================================================================%%%%

%% as per the requirement new theorem styles can be included as shown below
\theoremstyle{thmstyleone}%
\newtheorem{theorem}{Theorem}%  meant for continuous numbers
%%\newtheorem{theorem}{Theorem}[section]% meant for sectionwise numbers
%% optional argument [theorem] produces theorem numbering sequence instead of independent numbers for Proposition
\newtheorem{proposition}[theorem]{Proposition}% 
%%\newtheorem{proposition}{Proposition}% to get separate numbers for theorem and proposition etc.

\theoremstyle{thmstyletwo}%
\newtheorem{example}{Example}%
\newtheorem{remark}{Remark}%

\theoremstyle{thmstylethree}%
\newtheorem{definition}{Definition}%

\raggedbottom
%%\unnumbered% uncomment this for unnumbered level heads

\begin{document}

\title[Article Title]{A Hybrid Pipeline for Aircraft Role Classification in Satellite Imagery}

%%=============================================================%%
%% GivenName	-> \fnm{Joergen W.}
%% Particle	-> \spfx{van der} -> surname prefix
%% FamilyName	-> \sur{Ploeg}
%% Suffix	-> \sfx{IV}
%% \author*[1,2]{\fnm{Joergen W.} \spfx{van der} \sur{Ploeg} 
%%  \sfx{IV}}\email{iauthor@gmail.com}
%%=============================================================%%
\author*[1]{\fnm{Tugba} \sur{Tandogan}}\email{tandogantb@gmail.com}
\author[1]{\fnm{Gorkem} \sur{Saygili}}\email{G.Saygili@tilburguniversity.edu}

\affil*[1]{\orgdiv{Department of Intelligent Systems, Tilburg School of Humanities and Digital Sciences}, \orgname{Tilburg University}, 
\orgaddress{\street{Warandelaan 2}, \city{Tilburg}, \postcode{5037 AB}, \state{North Brabant}, \country{Netherlands}}}

%%\pacs[JEL Classification]{D8, H51}

%%\pacs[MSC Classification]{35A01, 65L10, 65L12, 65L20, 65L70}

\maketitle

\section{Experimental Tools \& Metadata Features} \label{secA1}
The Table~\ref{tab:tools} shows the experimental configurations and tool used.

\begin{table}[ht]
\caption{Experimental configuration and tools.}
\label{tab:tools}
\centering
\begin{tabular}{lc}
\toprule
\textbf{Tool} & \textbf{Version} \\
\midrule
OS                   & Ubuntu 22.04.6 LTS \\
CPU                  & x86\_64 \\
GPU                  & NVIDIA A100-SXM4-40GB \\
Framework            & PyTorch 2.6.0 \\
Programming Language & Python 3.11.12 \\
CUDA                 & cu12.4 \\
Google Colab Pro     & -- \\
\botrule
\end{tabular}
\end{table}

The following extracted geometric features are used as metadata:
\begin{itemize}
    \item \textbf{Width (Wingspan)}: The width of the polygon (Meters).
    \item \textbf{Height (Length)}: The length of the polygon (Meters).
    \item \textbf{Area}: The area of the polygon (Square meters).
    \item \textbf{Aspect Ratio}: The ratio of the width and length of the polygon.
    \item \textbf{Perimeter}: The total length of all polygon edges (Meters).
    \item \textbf{bbox\_diagonal}: Maximum straight-line distance, i.e., the diagonal of the bounding box (Meters).
    \item \textbf{Compactness}: A shape descriptor calculated as $\frac{\text{Perimeter}^2}{4\pi \times \text{Area}}$. It indicates how irregular the shape is and can be used to eliminate background objects.
    \item \textbf{Wingspan Class}: A categorical class based on wingspan thresholds of the Federal Aviation Administration (FAA) in categories (A–F) \citep{RarePlanes_Paper}.
\end{itemize}

\section{YOLO} \label{secA2}

An example of YOLO predictions are shown in Figure~\ref{fig:yolo_example}.
\begin{figure*}[ht]    
\includegraphics[width=\textwidth]{fig5.png}
    \caption{YOLO11n-seg predictions on satellite imagery. The figure compares three model variants: Baseline, Pretrained, and Tuned \& Pretrained. Predictions are shown as bounding boxes labeled by class—civil (blue) and military (cyan) with confidence scores. Images are created by YOLO \citep{ultralytics}.}
    \label{fig:yolo_example}
    \centering
\end{figure*}

\paragraph{Augmentation Details}\label{sec:augmentation} To deal with imbalance the following augmentation details were used using Ultralytics YOLO.
\begin{itemize}
    \item Degrees: Aircraft can appear at different orientations; minor rotations (±5°) simulate realistic satellite view shifts without distorting structure.
    \item Scale: Simulates varying flight size and resolutions. A scale of 0.7 allows modest resizing by zoom-in and zoom-outs on the image
    \item HSV: Hue shift, saturation, and brightness applied separately. Thus, satellite imagery's lighting, seasonal, and atmospheric changes were simulated.
    \item Horizontal flip is useful for aerial imagery. It does not change aircraft identity but increases sample diversity. Vertical flip was not a rational choice because it is unnatural in aircraft orientation.
\end{itemize}

\paragraph{Hyperparameters for YOLO} YOLO model was trained with the following parameters and they are optimized using Optuna \citep{optuna}. Final results are showed in Table~\ref{tab:final-hyperparameters}
\begin{itemize}
    \item Learning rate and weight decay on a logarithmic scale to explore small, sensitive ranges.
    \item Optimizer selection between SGD and Adam, balancing generalization and adaptive learning.
    \item Loss weights for box, class, and distance focal loss: These YOLO specific values are used to balance training emphasis between localization and classification. They were added especially because of class imbalance.
    \item Although the default YOLO epoch count is 100, the tuning was restricted to 30. Optimization converged after approximately 25 trials, so it can be claimed that it returns diminishing results beyond this point. 
    \item While 5 startup trials were used to collect performance benchmarks. In addition, one warm-up step was chosen to apply at least one fold before pruning to increase efficiency. This was essential when the cost of training across three full folds was considered.
The hyperparameter configuration is shown in Table~\ref{tab:final-hyperparameters}.
\end{itemize}

\begin{table}
\caption{Final optimization hyperparameters for YOLO11n-seg training.}
\label{tab:final-hyperparameters}
\centering
\begin{tabular}{lc}
\toprule
\textbf{Hyperparameter} & \textbf{Value} \\
\midrule
optimizer                        & AdamW \\
lr0                              & 0.00032895255738669336 \\
momentum                         & 0.9028846020228083 \\
weight\_decay                    & 8.529955332033192e-05 \\
dfl (Distribution Focal Loss)    & 1.4302711202961447 \\
box (IoU loss scaling)           & 0.15523343180287605 \\
cls (Classification loss scaling)& 1.4090935659936386 \\
\botrule
\end{tabular}
\end{table}

\section{XGBoost} \label{secA3}
Details of XGBoost configurations are given in this section

\paragraph{Hyperparameters for XGBoost} \label{hp_xgb}
The same hyperparameter configuration were used for each experiment using Optuna. The optimization space was created with the following parameters which are number of estimators, subsample, maximum depth, learning rate, colsample\_bytree to prevent the model from relying too much on certain features, min\_child\_weight to prevent overfitting in imbalanced sets, and gamma to avoid unnecessary growth for split regularization. According to the official XGBoost documentation \citep{xgboost}, these are recommended parameters for regularization, controlling overfitting, and improving generalization. The best found hyperparameters were used to retrain the final model on the entire training set (See Table~\ref{best_hp_xgb} and Table~\ref{best_hp_xgb_2}).

\begin{table}[ht]
\caption{The best hyperparameters found using Optuna for XGBoost across all scenarios (without SMOTE).}
\label{best_hp_xgb}
\centering
\begin{tabular}{lccc}
\toprule
\textbf{Hyperparameter} & \textbf{emb+meta} & \textbf{emb} & \textbf{meta} \\
\midrule
n\_estimators       & 300   & 300   & 100   \\
max\_depth          & 8     & 6     & 8     \\
learning\_rate      & 0.082 & 0.231 & 0.927 \\
subsample           & 0.835 & 0.989 & 0.910 \\
colsample\_bytree   & 0.624 & 0.711 & 0.693 \\
min\_child\_weight  & 1     & 1     & 4     \\
gamma               & 0.036 & 0.005 & 1.479 \\
\textbf{Macro F1}   & 0.8880 & 0.8878 & 0.7577 \\
\botrule
\end{tabular}
\end{table}

\begin{table}[ht]
\caption{Best hyperparameters found using Optuna for XGBoost across all scenarios for the model S+HP (with SMOTE).}
\label{best_hp_xgb_2}
\centering
\begin{tabular}{lccc}
\toprule
\textbf{Hyperparameter} & \textbf{emb+meta} & \textbf{emb} & \textbf{meta} \\
\midrule
n\_estimators       & 200   & 200   & 150   \\ 
max\_depth          & 5     & 7     & 8     \\
learning\_rate      & 0.047 & 0.097 & 0.199 \\
subsample           & 0.721 & 0.732 & 0.765 \\
colsample\_bytree   & 0.901 & 0.773 & 0.923 \\
min\_child\_weight  & 2     & 5     & 4     \\
gamma               & 0.396 & 0.555 & 0.247 \\
\textbf{Macro F1}   & 0.8986 & 0.8990 & 0.7006 \\
\botrule
\end{tabular}
\end{table}

\section{Interpretability and Feature Importance}\label{secA4}

\begin{figure*}[ht]
\centering
    \subfloat[SHAPLEY: Emb+Meta HP]{\includegraphics[width = 0.48\textwidth]{fig6.png}}%
    \subfloat[SHAPLEY: Meta S]{\includegraphics[width = 0.48\textwidth]{fig7.png}}%
    \caption{SHAP summary plots for the \textit{emb+meta} (top) baseline (B) model and \textit{meta} (bottom) SMOTE (S) models. Bars represent mean SHAP values, with feature contributions color-coded by class}
    \label{shapley_together}
\end{figure*}

The SHAP analysis in Figure~\ref{shapley_together} shows the most influential features for the best and worst resulted models, and it supports the fusion model’s interpretability. Among these, YOLO’s confidence scores dominate model decisions, highlighting its role as a primary error signal used by XGBoost for refining predictions, particularly in distinguishing aircraft from background. Embeddings contribute to discriminating fine inter-class differences, but they lack semantic transparency and thus offer limited interpretability. In contrast, geometric metadata offers greater interpretability but contributes less to predictive power.

In the \textit{emb+meta} HP model, embeddings and confidence scores account for the majority of the model's influence. However, metadata-derived features like aspect ratio and compactness still provide information, particularly for Class 2 (background), where geometric cues may be more prominent. In the \textit{meta} S model, decisions rely almost entirely on YOLO confidence, with shape descriptors offering only limited class-specific discrimination. This result aligns with earlier findings that metadata-only configurations underperform, especially for the minority Class 1 (military).

While metadata alone cannot match the predictive accuracy of embeddings, it offers complementary and interpretable signals that become especially valuable in fused models. As a result, even though geometric features do not independently outperform embeddings, they are complementary and interpretable outputs. This interpretability is mainly valuable in safety-critical applications like aerial surveillance or defense, where validation of predictions might be needed based on physical attributes. The combination of accurate classification and meaningful feature attribution strengthens the practicality of the model.

The Receiver-operating characteristic curve (ROC) and Precision-Recall (PR) curves in Figure~\ref{fig:roc_curves} of \textit{emb+meta} HP model demonstrate discriminative performance across all classes. While all classes show high AUCs, the PR curve for Class 1 (military) reveals weaker precision-recall trade-offs due to class imbalance. This highlights the necessity of PR-based evaluation under skewed distributions.

\begin{figure*}[t]
\centering
    \subfloat[YOLO]{\includegraphics[width=0.5\textwidth]{fig8.png}}
    \subfloat[Emb+Meta HP]{\includegraphics[width = 0.5\textwidth]{fig9.png}}
    \caption{ROC and Precision-Recall (PR) curves for the Embeddings+Metadata hyper-tuned model (HP) which is the best model. }
    \label{fig:roc_curves}
\end{figure*}

Consequently, the fusion of YOLO extractions with XGBoost’s refinement capability significantly improves classification, but its success depends on setup-specific tuning and careful imbalance handling. SHAP analysis confirms that while metadata aids interpretability, embeddings and confidence are the primary drivers of performance. Future efforts should focus on more advanced features and exploring robust imbalance techniques.

\clearpage{}
\bibliography{sn-bibliography}% common bib file
%% if required, the content of .bbl file can be included here once bbl is generated
%%\input sn-article.bbl

\end{document}
