\documentclass[fleqn,10pt]{wlscirep}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}

\title{AdvYOLO: Advanced YOLOv8 Application for Bone Pathology Localization and Classification in Wrist X-ray Images}

\author[1]{Rongxiang Wang}
\author[1,*]{Jingwen Zhao}
\author[1]{Xiang Liu}
\author[1]{Xian Tang}
\author[1]{Yunyu Shi}
\author[2]{Lina Wei}
\affil[1]{School of Electronic and Electric Engineering, Shanghai University of Engineering Science, Shanghai, China}
\affil[2]{School of Computer and Computing Science, Hangzhou City University, China}

\affil[*]{jingwen\_echo@outlook.com}

\keywords{Artificial intelligence, Bone fractures, YOLOv8, Pathology localization, Wrist fractures, X-ray}
% \keywords{Keyword1, Keyword2, Keyword3}

\begin{abstract}
In the field of pediatric orthopedics, accurate and timely identification of wrist fractures is vital for effective treatment and recovery. Fractures significantly affect daily activities and can lead to long-term health issues. Especially in areas with limited medical resources, or for doctors with less experience, interpreting X-ray images accurately is challenging. This paper introduces the AdvYOLO algorithm, an enhanced version of YOLOv8, trained on the GRAZPEDWRI-DX dataset to diagnose wrist bone pathologies. The integration of the Dilation-wise Residual (DWR) and Large Separable Kernel Attention (LSKA) modules is critical for improving feature extraction and classification, as they allow more effective processing of complex patterns in X-ray images, leading to more accurate diagnostics. The mAP 50 value of AdvYOLO improved from 63.8\% to 68.7\%, achieving state-of-the-art performance in wrist detection. Additionally, the paper presents BoneVisionAI, a tool to assist doctors, particularly those with less experience, in accurately interpreting children's wrist X-rays, aiming to reduce diagnostic errors in the healthcare sector.
\end{abstract}
\begin{document}

\flushbottom
\maketitle
% * <john.hammersley@gmail.com> 2015-02-09T12:07:31.197Z:
%
%  Click the title above to edit the author information and abstract
%
\thispagestyle{empty}


\section*{Introduction}

In the medical diagnostic field, especially in pediatric orthopedics, quickly and accurately identifying wrist fractures is crucial for timely treatment and functional recovery. Fractures not only directly impact the patient's daily activities but can also lead to long-term health issues \cite{Rimmerj4683}. In some advanced areas, experienced radiologists can usually analyze X-ray images precisely. However, in less developed areas, this task often falls to inexperienced young surgeons. Due to their lack of experience, they sometimes struggle to accurately interpret X-ray images, directly affecting the patient's ability to receive timely and correct treatment. For example, in some hospitals in Africa, it is difficult to obtain expert diagnostic reports, significantly reducing the chances of surgical success. Studies show that the error rate in interpreting X-ray images in these areas could be as high as 26\% \cite{Erhan2013Overlooked,Mounts2011Most}. Currently, X-ray imaging is the primary tool for detecting wrist fractures, but its complexity and the high demand for expertise, especially in resource-limited environments, pose a challenge. Therefore, developing an efficient and easy-to-use orthopedic diagnostic tool is crucial for improving the overall quality of medical services.

Recent advancements in machine learning, especially deep learning frameworks, have shown great potential in image analysis, notably achieving significant success in the medical diagnostic field. For example, Yong Pi and his team utilized a deep neural network enhanced with an attention mechanism to develop an automated multi-view bone scanning diagnostic model for detecting bone metastasis, achieving performance on par with three medical experts \cite{PI2020101784}. Meanwhile, Zhihao Su et al. \cite{diagnostics13203245} explored the application of deep learning in fracture detection, identifying, classifying, and locating fractures through the analysis of a large number of X-ray images. They emphasized the importance of integrating multimodal information, providing treatment suggestions, and developing advanced visualization technologies, envisioning the future direction of this field.

This paper presents the AdvYOLO system, representing the latest application of deep learning technology in the field of medical imaging. By improving the YOLOv8 algorithm \cite{yolov8} and data processing techniques, AdvYOLO offers a new perspective on bone pathology detection in wrist X-ray images. This includes not only the efficient identification of fractures but also extends to the precise localization and classification of a wider range of bone pathology types, providing a more comprehensive diagnostic view. To address the efficiency issue of acquiring multi-scale contextual information, we were inspired by the DWRSeg network structure to simplify the feature extraction process, thus improving computational efficiency. Simultaneously, drawing from the LSKA attention module, the model enhances focus on object shapes rather than textures, increasing the accuracy and robustness of bone pathology detection. Aimed at improving large kernel attention designs in convolutional neural networks (CNNs), our extensive experiments on the improved GRAZPEDWRI-DX \cite{GRAZPEDWRI} wrist X-ray image dataset have demonstrated significant advantages of our proposed method. The research results show a significant improvement in the accuracy of bone pathology detection and classification compared to existing techniques.

The contributions of this study can be summarized as follows, with the main innovations and contributions including:
\begin{enumerate}
\item We have significantly enhanced the performance of YOLOv8 in detecting and classifying bone pathologies in wrist X-ray images by integrating the DWR module. The DWR module's unique approach of Region Residualization and Semantic Residualization effectively captures multi-scale contextual information, simplifies the learning process, and ensures the orderly and efficient extraction of relevant features.
\item We incorporated the LSKA module into YOLOv8, which decomposes a 2D convolutional kernel into cascaded horizontal and vertical 1D kernels, efficiently processing large-scale images while maintaining effective feature extraction capabilities. This method enhances the efficiency of extracting complex features in medical images, which is crucial for accurate diagnosis of bone pathologies.
\item We developed a multi-platform application named "BoneVisionAI", specifically designed to assist doctors in remote areas and those with limited experience in accurately interpreting children's wrist X-ray images. The application aims to reduce the error rate in X-ray image analysis, providing more accurate diagnostic support.
\end{enumerate}

\section*{Related Works}

In recent years, machine learning algorithms, especially those based on deep learning, have garnered increasing attention in medical image processing \cite{cai2020review,latif2019medical}. Deep learning algorithms, with their inherent capability for automatic feature extraction \cite{fourcade2019deep,singh2020explainable,Jo2023DeepLS}, stand out compared to traditional methods. They are extensively applied in processing X-ray and CT images, particularly for detecting and classifying fractures \cite{wong2019artificial,Wu2021TheFA}.

For instance, Guan et al. \cite{GUAN2019521} employed the Extended Convolutional Feature Pyramid Network (DCFPN) on 3,842 thigh fracture X-ray images, achieving an average accuracy of 82.1\%. Jinman Wei et al. adopted a semi-supervised object detection approach using a single-stage detector, which, when tested on 3484 training and 358 test samples, yielded a leading accuracy of 86.2\% in detecting thigh fractures \cite{wei2023semi}. Chung et al. \cite{Chung} utilized the ResNet-152 CNN model on 1,891 shoulder X-ray images, achieving a remarkable 96\% accuracy. Olczak et al. \cite{Olczak} analyzed 256,000 wrist, hand, and ankle X-rays and reported the VGG16 network's highest accuracy at 93\%. Furthermore, Sha et al. used YOLOv2 \cite{9182582} and Faster R-CNN model \cite{9194863} for detecting spine fractures in 5134 CT images, while Xue et al. \cite{2021Detection} employed the Faster R-CNN model on 3067 X-ray images of hand trauma, achieving an average accuracy of 70.0\%. However, there is not much research on wrist fractures.

In computer vision tasks, particularly in real-time object detection, the YOLO (You Only Look Once) model family has made notable advancements. The YOLOv5 and YOLOv7 models are significant in this context. The YOLOv5 model, noted for its application in real-time medical image analysis on mobile devices, was used by Yuan et al. for detecting skull fractures in CT images. Their improved YOLOv5 method, incorporating external attention and 3D feature fusion, demonstrated superior performance in skull fracture detection with an average precision (AP) of 88.08\% \cite{Yuan2021AnIY}. YOLOv7 is known for its efficient single-stage detection and neural network structure. A. Priya et al. combined YOLOv7 with a Shifted Window Transformer (Swin) to focus on precise localization and classification of bone pathologies \cite{dibo2023deeploc}.

The latest iteration in the YOLO series, YOLOv8, is designed to provide faster and more accurate object detection capabilities. Building on the success of its predecessors, YOLOv8 introduces a new backbone network, a novel anchor-free detection head, and a new loss function, enhancing its performance \cite{ju2023fracture}. Previous studies have seldom focused on wrist fracture detection, and the accuracy of bone pathology detection still needs improvement. The GRAZPEDWRI-DX dataset, released in 2022, offers more detailed and complete annotations, providing a scientific basis for our research. We have improved upon the YOLOv8 model by integrating the DWR and LSKA modules into the YOLOv8 algorithm. Compared to other methods of bone pathology detection, our study effectively captures multi-scale contextual information, simplifies the learning process, and ensures the orderly and efficient extraction of relevant features. By decomposing the two-dimensional convolutional kernel into cascading horizontal and vertical one-dimensional kernels, our method effectively processes large-scale images while maintaining efficient feature extraction capabilities. Our approach achieves optimal accuracy in wrist bone pathology detection. Compared to visualization programs from other studies, our visualization program can be deployed on multiple platforms, enhancing the model's compatibility and robustness.

\section*{Materials and methods}

\subsection*{Datasets}
In our research, we considered two major medical imaging datasets: MURA (Musculoskeletal Radiographs) and GRAZPEDWRI-DX. MURA is an extensive public dataset developed by Stanford University, comprising over 40,000 musculoskeletal radiographic images from various patients, covering seven different body parts. These images are annotated by radiologists as normal or abnormal, providing a valuable resource for the detection of skeletal abnormalities (see Fig. \ref{mura} for MURA dataset images).
\begin{figure}[ht]
	\centering 
		\includegraphics[width=0.7\textwidth]{imgs/MURA.jpg}
	  \caption{The seven categories of the MURA dataset.}
        \label{mura}
\end{figure}

On the other hand, the GRAZPEDWRI-DX dataset is focused on pediatric trauma wrist radiographs, containing 6091 images from cases treated at the Department of Pediatric Surgery, University Hospital Graz, between 2008 and 2018. This dataset includes a variety of detailed annotations, such as bone anomalies, lesions, foreign bodies, fractures, metal, periosteal reactions, pronator signs, soft tissue, text, and centerlines. Notably, it contains 13,550 images with one or more fracture marking boxes (see Figure \ref{grazpedwri-dx} for an example image from GRAZPEDWRI-DX).

\begin{figure}[ht]
	\centering 
		\includegraphics[width=0.7\textwidth]{imgs/grazpedwri.jpg}
	  \caption{Example image from GRAZPEDWRI-DX dataset.}
        \label{grazpedwri-dx}
\end{figure}

After evaluating the characteristics of both datasets and the specific requirements of our study, we ultimately decided to select the GRAZPEDWRI-DX dataset. This decision was driven by several pivotal factors:

\begin{enumerate}
    \renewcommand{\labelenumi}{\Roman{enumi}.}
    \item The specificity of the GRAZPEDWRI-DX dataset aligns more closely with our research focus on pediatric wrist trauma.
    \item Compared to the MURA dataset, GRAZPEDWRI-DX provides more detailed annotations related to fractures, which are essential for our study.
    \item The high quality of annotations and the concentrated focus on a specific type of trauma in GRAZPEDWRI-DX make it a superior resource for our research objectives.
\end{enumerate}

In conclusion, while the MURA dataset offers benefits in terms of size and diversity, the specialized and in-depth approach of GRAZPEDWRI-DX in pediatric wrist radiographic analysis is more suited to the needs of our research.

\subsection*{Data Augmentation}
In the field of deep learning, especially in computer vision tasks, data enhancement is widely recognized as an effective means to improve model generalization. This method can significantly improve the recognition ability of the model to new and unseen data by diversifying the training data. In this study, we used a specific data enhancement technique for image recognition tasks to mimic different ambient light conditions by adjusting the contrast and brightness of images.
Specifically, we applied the following enhancement formula to each image:
% \begin{equation}
% \centering
%  \text{Enhanced Image} = \text{Original Image} \times \alpha + \beta, \label{eq1}
% \end{equation}
\begin{equation}
\text{Enhanced Image} = \text{Original Image} \times \alpha + \beta, \label{eq1}
\end{equation}
where \(\alpha\) is the contrast adjustment factor, and \(\beta\) is the brightness adjustment value. In our experiment, \(\alpha\) was set to 1.1, which means we increased the contrast of the image by 10\%, and \(\beta\) was set to 20, which means the brightness was increased by 20 units. This adjustment not only enhances the features in the image, but also simulates what might occur when the image is captured under different lighting conditions, thus improving the robustness of the model to changes in lighting.

In deep learning, this enhancement technique is particularly important because neural network models tend to be sensitive to new situations that do not appear in the training data. By introducing slight variations, the model can learn the ability to extract key features in different situations, rather than just memorizing specific patterns in the training set. This method is helpful to reduce overfitting, improve the performance of the model in practical application, and improve the recognition ability and robustness of the model. The image after adjusting the contrast and brightness is shown in Fig. \ref{dataaug}.
\begin{figure}[ht]
\centering \includegraphics[width=0.5\textwidth]{imgs/dataaug.jpg}
	  \caption{Examples of pediatric wrist trauma X-ray images using data augmentation. (a) the original images; (b) the adjusted
images.}
        \label{dataaug}
\end{figure}

\subsection*{Related Principles}
\subsubsection*{Introduction to YOLOv8 Algorithm}
The You-Only-Look-Once (YOLO) algorithm divides an image into multiple grids, predicting bounding boxes and their contained object categories within each grid, and employs the Non-Maximum Suppression (NMS) algorithm to eliminate overlapping bounding boxes. YOLOv1 \cite{RedmonDGF15} detects relatively quickly but is less effective for closely spaced objects and smaller targets. YOLOv2 \cite{RedmonF16} adopts the Darknet19 as the feature extraction network, adapting to images of different sizes and improving detection accuracy for small objects. YOLOv3 \cite{abs-1804-02767} introduces the Feature Pyramid Network (FPN) and Spatial Pyramid Pooling (SPP) modules to enhance the detection of different scales and semantic information. YOLOv4 \cite{abs-2004-10934} incorporates the Mish activa  tion function to increase accuracy. YOLOv5 introduces the C3 module and the SPPF module, improving feature perturbation and detection capabilities. YOLOv7 \cite{wang2023yolov7} brings in scalable and efficient layer aggregation networks E-ELAN, innovative transitional modules, and re-parameterization structures, enhancing feature extraction and semantic information representation, further optimizing object detection performance \cite{ju2023fracture}.

As one of the classic single-stage detection algorithms, YOLO has been upgraded to YOLOv8. This paper proposes an improved method for YOLOv8, which not only enhances detection accuracy and reduces computational load but also achieves model visualization. Fig. \ref{yolov8struct}, taking the structure of YOLOv8n as an example, describes the various components of the network. The backbone network consists of CBS, C2f, and SPPF modules, still using the CSP concept. However, the backbone neck network borrows the ELAN structure design idea from YOLOv7, replacing the C3 module in YOLOv5 with a more gradient-rich C2f structure. The C2f structure enhances the feature fusion ability of convolutional neural networks, improves inference speed, and achieves further lightweight, while YOLOv8 still uses the SPPF module from the YOLOv5 architecture. The SPPF module is a spatial pyramid pooling layer that can expand the receptive field, achieving local and global feature fusion and enriching feature information. The Head part, compared to YOLOv5, has undergone significant changes, adopting the currently popular decoupled head structure, separating the classification and detection heads. YOLOv8 changes from the traditional Anchor-Based approach to an Anchor-Free concept; in terms of loss functions, YOLOv8 uses BCE Loss for classification and DFLLoss+CIOU Loss for regression. In sample matching, YOLOv8 changes from the traditional IOU matching or unilateral ratio distribution method, using the Task-Aligned Assigner matching approach.
\begin{figure}[ht]
\centering
\includegraphics[width=0.9\textwidth]{imgs/yolov8struct.jpg}
\caption{YOLOv8 Network Architecture.\label{yolov8struct}}
\end{figure}  

% \begin{figure*}[ht]
%     \centering 
%     \includegraphics[width=0.8\textwidth]{imgs/yolov8struct.jpg}
%     \caption{YOLOv8 Network Architecture}
%     \label{yolov8struct}
% \end{figure*}

\subsubsection*{Algorithmic Improvements}
To enhance the accuracy of locating and classifying bone pathologies in wrist X-ray images, the AdvYOLO model made key improvements upon the YOLOv8 framework. Fig. \ref{advyolo} illustrates the revised YOLOv8 structure, where the last two C2f modules of the YOLOv8 backbone network were replaced with DWR modules \cite{wei2022dwrseg}. The DWR module's uniqueness lies in its efficient multi-scale feature extraction ability. Utilizing regional and semantic residualization in the feature extraction process, the DWR module captures more detailed information at a lower computational cost. This is crucial for improving the recognition of bone pathologies, especially in images with complex backgrounds or details. The SPPF module \cite{He2014SpatialPP} was substituted with the LSKA module, effectively capturing long-distance dependencies while reducing computational complexity. Through this integration of technologies, AdvYOLO achieves more precise and rapid localization and classification of bone pathologies in wrist X-ray images.

\begin{figure}[ht]
\centering
\includegraphics[width=0.9\textwidth]{imgs/advyolo.jpg}
\caption{AdvYOLO Network Architecture.\label{advyolo}}
\end{figure}  

% \begin{figure*}[ht]
%     \centering 
%     \includegraphics[width=0.8\textwidth]{imgs/advyolo.jpg}
%     \caption{AdvYOLO Network Architecture}
%     \label{advyolo}
% \end{figure*}

\begin{itemize}
\item DWR Module 
\end{itemize}  

In our research, we enhanced the YOLOv8 network by integrating the Dilation-wise Residual (DWR) module, introduced in December 2022, a pivotal innovation for improving the detection and classification of bone pathologies in wrist X-ray images, particularly for wrist X-ray analysis. The DWR module's unique approach to acquiring multi-scale contextual information via two distinct steps—Region Residualization and Semantic Residualization—enables a more efficient and focused feature extraction process. Region Residualization simplifies feature maps of various sizes, optimizing them for the subsequent Semantic Residualization, where semantic-based morphological filtering is applied using specifically chosen receptive fields. This methodology not only simplifies the learning process but also ensures a more orderly and efficient extraction of relevant features, vital for accurate medical diagnosis (refer to Fig. \ref{multi-scale} and Fig. \ref{DWR} from the document for visual representation).

The effectiveness of DWR is underscored by its performance on datasets like Cityscapes, where it achieved an impressive mIoU of 72.7\% at a speed of 319.5 FPS on an NVIDIA GeForce GTX 1080 Ti card, surpassing existing methods. This significant improvement in both accuracy and computational efficiency is expected to be mirrored in the "AdvYOLO" system, particularly in the real-time analysis of wrist X-rays, where the nuanced detection of bone pathologies is crucial.
\begin{figure}[ht]
    \centering 
\includegraphics[width=0.7\textwidth]{imgs/multi-scale contextual information.jpg}
    \caption{multi-scale contextual information.}
    \label{multi-scale}
\end{figure}

\begin{figure}[ht]
    \centering 
\includegraphics[width=0.35\textwidth]{imgs/DWR.jpg}
    \caption{Illustration of the structure of a 3-branch DWR module. Conv denotes convolution, DConv denotes depth-wise convolution, D-n denotes dilated convolution with a dilation rate of n, + in a circle denotes the addition operation.}
    \label{DWR}
\end{figure}
\begin{itemize}
\item LSKA Module
\end{itemize}  

The LSKA (Large Separable Kernel Attention) module, published in September 2023, and cited as \cite{lau2024large}, represents a transformative development in the field of convolutional neural networks (CNNs), particularly for applications in medical imaging. This novel module is designed to address the challenges posed by traditional large convolutional kernels, primarily their substantial computational and memory demands. LSKA ingeniously decomposes a 2D convolutional kernel into cascaded horizontal and vertical 1-D kernels, addressing the computational challenges of processing large-scale images. This ingenious approach significantly reduces complexity and resource consumption, boosting efficiency in medical imaging applications. The detailed network structure of LSKA, as illustrated in Fig. \ref{LSKA}, exemplifies this architectural innovation, showcasing the module's capacity for effective feature extraction in complex medical images.
\begin{figure}[ht]
    \centering 
\includegraphics[width=0.2\textwidth]{imgs/LSKA.jpg}
    \caption{Large Separable Kernel Attention.Notice that {\small\textcircled{x}} represents Hadamard product, k represents the maximum receptive field, and d represents the dilation rate.}
    \label{LSKA}
\end{figure}

The LSKA module, an evolution of the LKA, achieves a more efficient configuration by splitting the 2D weight kernels of depth-wise convolution into two cascaded 1D separable weight kernels. This modification leads to a significant improvement in computational efficiency without compromising the effectiveness of the network. The operational mechanism of LSKA demonstrates this innovative configuration. Following the methodology outlined in reference \cite{guo2023visual}, the output of LSKA can be obtained as follows.

\begin{equation}
\bar{Z}^C = \sum_{H,W} W^C_{(2d-1) \times 1} * \left(  \sum_{H,W} W^C_{1 \times (2d-1)} * F^C \right) ,\label{eq4}
\end{equation}

\begin{equation}
{Z}^C = \sum_{H,W} W^C_{\left\lfloor \frac{k}{d} \right\rfloor \times 1} * \left( \sum_{H,W} W^C_{1 \times \left\lfloor \frac{k}{d} \right\rfloor} * \bar{Z}^C \right), \label{eq5}
\end{equation}


\begin{equation}
A^C = W_{1 \times 1} * \bar{Z}^C, \label{eq6}
\end{equation}

\begin{equation}
F^C = A^C \otimes F^C, \label{eq7}
\end{equation}
where $d$ is the dilation rate. $\bar{Z}^C$ in Eq. \ref{eq4} denotes the output of depthwise-convolution with kernel size $(2d - 1) \times (2d - 1)$ which captures the local spatial information and compensates gridding effect \cite{WangCYLHHC17} of the following depth-wise dilated convolution (Eq. \ref{eq5}) with kernel size $\lfloor \frac{k}{d} \rfloor \times \lfloor \frac{k}{d} \rfloor$. Note that $\lfloor \cdot \rfloor$ represents the floor operation. The dilated depth-wise convolution is responsible for capturing the global spatial information of the output $\bar{Z}^C$ of depth-wise convolution.

In my research focusing on the detection and classification of bone pathologies in wrist X-ray images, the integration of the LSKA module into the YOLOv8 network has been a groundbreaking step. This integration elevates the model's ability to discern intricate features within the X-rays, a task that is both computationally demanding and crucial for accurate diagnosis. The LSKA module, with its large separable kernels, enhances the network's feature extraction capabilities. 

The practical application of LSKA in my study has shown substantial improvements in the model's performance, evidenced by enhanced accuracy in identifying subtle and complex bone anomalies. This is particularly significant in the field of medical imaging, where the precision of diagnostic tools directly impacts patient outcomes. 

The adoption of LSKA in the YOLOv8 framework for medical imaging is a testament to the module's versatility and efficiency. This enhancement not only aligns with the ongoing evolution in deep learning and medical imaging technologies but also sets a new benchmark in the application of CNNs for medical diagnostics. The LSKA module, therefore, stands out as a critical component in my research, contributing significantly to the advancement of automated diagnostic tools in radiology.

\subsection*{Extracting Heatmaps and Predicting Bounding Boxes}
Our method utilizes Gradient-weighted Class Activation Mapping (Grad-CAM) \cite{SelvarajuDVCPB16} with the YOLOv8 model to highlight areas of interest in images. This technique computes gradients of class scores against feature maps in the last convolutional layer, assigning weights to each map based on gradient strength. These weighted maps are then aggregated to create a heatmap, which is normalized between 0 (lowest activation) and 255 (highest activation), visualized in a jet color map with blue indicating low and red high activation areas. Regions exceeding a specific activation threshold (0.6 in our case) are marked as significant, potentially indicating key features like fractures. These marked regions are compared with expert annotations to verify accuracy.As depicted in Fig. \ref{heatmap}, Our results are presented in a sequence of images: the original, the heatmap overlay, and the final prediction highlighting potential areas of interest \cite{diagnostics13182927}.
\begin{figure}[ht]
    \centering 
    \includegraphics[width=0.4\textwidth]{imgs/heatmap1.jpg}
    \caption{Heat-map extracted from the GradCAM model, with the greenbox indicating the ground truth fracture region.}
    \label{heatmap}
\end{figure}


\section*{Experiments and results}
The GRAZPEDWRI-DX dataset we used includes 20,327 X-ray images of pediatric wrist injuries. We divided the GRAZPEDWRI-DX dataset randomly into three groups: training, validation, and testing sets. The sizes of these groups are 70\%, 20\%, and 10\% of the original dataset, respectively, amounting to 14,243, 4,069, and 2,015 images. The training set is expanded to 28,486 images through data augmentation to improve the model's robustness.

\subsection*{Evaluation Metric}
\textbf{Intersection over Union (IoU)}

Intersection over Union (IoU) \cite{6909475} is a prevalent metric in computer vision tasks, notably in object detection and segmentation. It measures the overlap between the predicted bounding box (or segmentation area) and the ground truth bounding box (or segmentation area). The formula is given by:

\begin{equation}
    IoU = \frac{\text{Area of Intersection}}{\text{Area of Union}},
\end{equation}
here, the Area of Intersection is the area where the predicted and ground truth bounding boxes overlap, and the Area of Union is the combined area of both boxes. IoU values range from 0 (no overlap) to 1 (perfect overlap).

\textbf{Precision-Recall Curve}

The Precision-Recall Curve \cite{Boyd2013AreaUT} is an essential tool for evaluating classification model performance, especially in class imbalance scenarios. Precision (P) and Recall (R) are defined as:

\begin{equation}
    Precision (P) = \frac{TP}{TP + FP},
\end{equation}

\begin{equation}
   Recall (R) = \frac{TP}{TP + FN} ,
\end{equation}
where TP is True Positives, FP is False Positives, and FN is False Negatives. The Precision-Recall Curve shows the trade-off between these two metrics across different thresholds.

\textbf{F1 Score}

The F1 Score is the harmonic mean of Precision and Recall, a critical measure of a model's accuracy. It is calculated as:

\begin{equation}
   F1 = 2 \times \frac{P \times R}{P + R},
\end{equation}
the F1 Score ranges from 0 to 1, where 1 denotes optimal performance. This metric is particularly valuable in scenarios where both Precision and Recall are of equal importance.

\textbf{Mean Average Precision (mAP)}

Mean Average Precision (mAP) is a widely used metric in evaluating the performance of object detection models, particularly in scenarios involving multiple object classes. The mAP is calculated by first computing the Average Precision (AP) for each class and then taking the mean of these AP values across all classes. The AP for a single class is computed as the area under the Precision-Recall Curve for that class, reflecting the model's ability to correctly identify objects of that class across different confidence thresholds. The formula for AP is given by integrating the Precision (P) as a function of Recall (R):

\begin{equation}
    AP = \int_{0}^{1} p(r) dr,
\end{equation}
where \(p(r)\) is the precision as a function of recall \(r\). The mAP score ranges from 0 to 1, with 1 indicating perfect precision and recall for all classes. This metric is particularly useful for tasks involving multiple classes where it is important to assess the model's performance across all categories, providing a comprehensive overview of its detection capabilities.

\subsection*{Experiment Setup}
In our model training process, we built upon the YOLOv8s pre-trained model, initially setting a total training period of 200 epochs with an interim evaluation every 50 epochs. Training would cease if no significant improvement was observed after these evaluation periods. Our study on different optimizers revealed that peak performance usually occurred within the first 100 epochs, particularly between 10 to 50 epochs. Hence, we reduced the training period to 100 epochs for computational efficiency. 

Following Glenn's advice \cite{yolov8}, we learned that the Adam optimizer \cite{kingma2014adam} is more suitable for small custom datasets, while SGD \cite{Ruder16} excels with larger datasets. We trained the AdvYOLO model using both Adam and SGD optimizers to observe their impact on performance. As shown in Table \ref{tab1}, we opted for an SGD optimizer with an initial learning rate of 0.01, weight decay of 0.0005, and momentum of 0.937. Experiments were conducted with input image sizes of 640 and 1024 on a single GeForce RTX 3080Ti 12GB GPU with a batch size of 16, using Python 3.8 and PyTorch 1.8.2. Due to GPU memory constraints, we employed eight worker threads during training, indicating that GPUs with greater memory and computing power could significantly hasten training speed.

\begin{table}[htbp]
\centering
\caption{Model performance comparison of AdvYOLO models using SGD and Adam optimizers. For training with the SGD optimizer, the initial learning rate is $1\times10^{-2} $; for training with the Adam optimizer, the initial learning rate is $1\times10^{-3} $.}
\label{tab1}
% \begin{tabular*}{\textwidth}{@{}ccccccc@{}}
\begin{tabular}{ccccccc}
\toprule
Model & Size & Optimizer & Best Epoch & \( mAP^{val}_{50} \) & \( mAP^{val}_{50-95} \) & Speed \\ 
\midrule
AdvYOLO & 640  & SGD   & 12 & 0.582 & 0.369 & 12.0ms \\
AdvYOLO & 640  & Adam  & 83 & 0.484 & 0.279 & 10.2ms \\
AdvYOLO & 1024 & SGD   & 43 & 0.687 & 0.443 & 11.1ms \\
AdvYOLO & 1024 & Adam  & 127 & 0.492 & 0.292 & 10.8ms \\
\bottomrule
\end{tabular}
\end{table}

\subsection*{Ablation Study}
To validate the accuracy of the improved algorithm presented in this paper, experiments were conducted using several models including YOLOv8s, YOLOv8s-D, YOLOv8s-L, and YOLOv8s-D. YOLOv8s-D, which incorporates the DWR module, replaces the last two C2f layers of the YOLOv8 backbone. YOLOv8s-L, realized by replacing the SPPF module of the YOLOv8 backbone with the LSKA module. AdvYOLO is the algorithm proposed in this study. An initial ablation study on the YOLOv8s model calculated various evaluation metrics for each category without data augmentation, as shown in Table \ref{tab2}. In all categories, the YOLOv8s model demonstrated high precision in detecting fractures, metal, and text, with an mAP 50 of over 0.9. Conversely, its ability to detect bone anomalies was relatively poor, with an mAP 50 of 0.0591. Therefore, we increased the contrast and brightness of the X-ray images through data augmentation, enhancing the detectability of bone anomalies and the model's robustness.

\begin{table}[htbp]
\centering
\caption{Validation results of YOLOv8 for each class on the GRAZPEDWRI-DX dataset when the input image size is 1024.}
\label{tab2}
\begin{tabular}{ccccccc}
\toprule
Class & Images & Instances & Box(P) & Box(R) & \( mAP^{val}_{50} \) & \( mAP^{val}_{50-95} \)\\ 
\midrule
all & 4069 & 9483 & 0.716 & 0.502 & 0.599 & 0.387 \\
boneanomaly & 4069 & 61 & 0.223 & 0.011 & 0.0591 & 0.0206 \\
bonelesion & 4069 & 5 & 0.69 & 0.2 & 0.317 & 0.173 \\
fracture & 4069 & 3597 & 0.892 & 0.887 & 0.94 & 0.559 \\
metal & 4069 & 160 & 0.852 & 0.894 & 0.91 & 0.688 \\
periostealreaction & 4069 & 677 & 0.657 & 0.624 & 0.659 & 0.317 \\
pronatorsign & 4069 & 112 & 0.496 & 0.688 & 0.58 & 0.284 \\
softtissue & 4069 & 114 & 0.671 & 0.233 & 0.356 & 0.181 \\
text & 4069 & 4755 & 0.962 & 0.982 & 0.991 & 0.74 \\
\bottomrule
\end{tabular}
\end{table}


Table \ref{tab3} shows the evaluation metrics for each category obtained using our AdvYOLO algorithm model with data augmentation. We analyzed and compared the performance of the traditional YOLOv8 and our improved AdvYOLO model on the GRAZPEDWRI-DX dataset. Overall, the AdvYOLO model showed a significant improvement in mAP@50. The mAP@50 of the AdvYOLO model increased from 0.599 in YOLOv8 to 0.687, indicating significant improvements in processing complex images and precise target localization, as well as an overall advantage in recognition effectiveness. Further analysis of specific categories revealed that the AdvYOLO model was particularly effective in processing some challenging categories. For instance, in the Bone Lesion category, the mAP@50 of AdvYOLO increased from 0.0591 in YOLOv8 to 0.158, an increase of nearly 167.24\%, demonstrating the effectiveness of our data augmentation and improvements. These significant enhancements prove our model to be more precise and reliable in processing specific types of images. 

As shown in Table \ref{tab4}, the introduction of the DWR module replacing the last two C2f layers of the backbone network significantly improved precision to 0.84, although the recall rate slightly decreased to 0.539, but the mAP 50 increased to 0.642. The model's parameter count modestly reduced to 10.8M and computational load decreased to 28.3 GFLOPs. The introduction of the LSKA module replacing the SPPF module enhanced the ability to process complex medical images. As can be seen from Fig. \ref{results}, our model demonstrates superior performance in the detection of various wrist pathologies, resulting in an overall improvement in model performance. AdvYOLO, integrating both the DWR and LSKA modules, not only achieves a significant balance in performance but also maintains high precision and recall rates with lower computational requirements. This effectively reduces the error rate in medical image analysis, making a substantial contribution to technological advancements in the field of medical imaging diagnostics.


\begin{table}[htbp]
\centering
\caption{Validation results of AdvYOLO for each class on the GRAZPEDWRI-DX dataset when the input image size is 1024.}
\label{tab3}
% \begin{tabular*}{\textwidth}{@{}ccccccc@{}}
\begin{tabular}{ccccccc}
\toprule
Class & Images & Instances & Box(P & R & \( mAP^{val}_{50} \) & \( mAP^{val}_{50-95} \)\\ 
\midrule
all & 4069 & 9483 & 0.795 & 0.584 & 0.687 & 0.443 \\
boneanomaly & 4069 & 61 & 0.809 & 0.139 & 0.158 & 0.0718 \\
bonelesion & 4069 & 5 & 0.899 & 0.6 & 0.425 & 0.309 \\
fracture & 4069 & 3597 & 0.888 & 0.902 & 0.943 & 0.572 \\
metal & 4069 & 160 & 0.829 & 0.925 & 0.950 & 0.763 \\
periostealreaction & 4069 & 677 & 0.648 & 0.678 & 0.651 & 0.334 \\
pronatorsign & 4069 & 112 & 0.494 & 0.75 & 0.649 & 0.327 \\
softtissue & 4069 & 114 & 0.625 & 0.281 & 0.422 & 0.211 \\
text & 4069 & 4755 & 0.959 & 0.982 & 0.991 & 0.744 \\
\bottomrule
\end{tabular}
\end{table}


\begin{table}[htbp]
\caption{Ablation Experiment Results}
\label{tab4}
\begin{tabular}{cccccccccc}
\toprule
\textbf{Algorithms} & \textbf{DWR} & \textbf{LSKA} & \textbf{Optimizer} & \textbf{Precision} & \textbf{Recall} & \textbf{F1} & \textbf{map50} & \textbf{Params/M} & \textbf{GFLOPs} \\
\midrule
YOLOv8s & {} & {} & SGD & 0.768 & 0.579 & 0.660 & 0.608 & 11.1 & 28.7 \\
YOLOv8s & {} & {} & Adam & 0.678 & 0.392 & 0.497 & 0.431 & 11.1 & 28.7 \\
YOLOv8s-D & $\checkmark$ & {} & SGD & 0.84 & 0.539 & 0.657 & 0.642 & 10.8 & 28.3 \\
YOLOv8s-D & $\checkmark$ & {} & Adam & 0.651 & 0.527 & 0.582 & 0.528 & 10.8 & 28.3 \\
YOLOv8s-L & {} & $\checkmark$ & SGD & 0.775 & 0.578 & 0.662 & 0.598 & 12.2 & 29.5 \\
YOLOv8s-L & {} & $\checkmark$ & Adam & 0.694 & 0.491 & 0.575 & 0.524 & 12.2 & 29.5 \\
AdvYOLO & $\checkmark$ & $\checkmark$ & SGD & 0.795 & 0.584 & 0.673 & \textbf{0.687} & 11.9 & 29.1 \\
AdvYOLO & $\checkmark$ & $\checkmark$ & Adam & 0.775 & 0.475 & 0.589 & 0.492 & 11.9 & 29.1 \\
\bottomrule
\end{tabular}
\end{table}

\begin{table}[htbp]
\centering
\caption{Comparison of Studies.\label{tab5}}
\begin{tabular}{cccc}
\toprule
Studies & Methods Used & Dataset & Result \\
\midrule
Kim et al. \cite{Kim2017ArtificialII} & Inception V3 & Royal Devon and Exeter Hospital & 0.954(AUROC) \\
Lindsey et al. \cite{Lindsey2018DeepNN} & U-Net & Hospital for Special Surgery & 0.967(AUC) \\
Ananda et al. \cite{9264285} & Inception-ResNet-V2 & MURA & 0.869(Ac) \\
Nagy et al. & Yolo5 & GRAZPEDWRI-DX & 0.917(Precision) \\
Joonho Oh et al. & HyperColumn-Convolutional Block Attention & GRAZPEDWRI-DX and MURA & 0.968(Precison) \\
J Glenn et al. & YOLOv8 & GRAZPEDWRI-DX & 0.623(mAP 50) \\
Ju et al. & Yolov8 improvements & GRAZPEDWRI-DX & 0.638(mAP 50) \\
Ours & \textbf{AdvYOLO} & GRAZPEDWRI-DX & \textbf{0.687(mAP 50)} \\
\bottomrule
\end{tabular}
\end{table}


\begin{table}[htbp]
\centering
\caption{Performance metrics of various object detection models on the GRAZPEDWRI-DX dataset.}
\label{tab6}
\begin{tabular}{ccccc}
\toprule
Methods Used & Precision & Recall & \( mAP^{val}_{50-95} \)&  \( mAP^{val}_{50} \) \\ 
\midrule
YOLOv8 & 0.704 & 0.579 & 0.385 & 0.599 \\
YOLOv7 & 0.749 & 0.542 & 0.337 & 0.566 \\
YOLOv5 & 0.712 & 0.580 & 0.331 & 0.566 \\
Faster R-CNN & 0.730 & 0.560 & 0.350 & 0.575 \\
RTDETR & 0.719 & 0.517 & 0.343 & 0.540 \\
SSD & 0.700 & 0.550 & 0.320 & 0.560 \\
\textbf{AdvYOLO} & 0.795 & 0.584 & 0.443 & \textbf{0.687} \\
\bottomrule
\end{tabular}
\end{table}



\subsection*{Experimental Results}
Before training our model, we initially conducted training on the GRAZPEDWRI-DX dataset using the YOLOv8 algorithm. As indicated in Table \ref{tab2}, we identified significant room for improvement in accuracy from these training experiments. Subsequent research revealed considerable differences in the choice of optimizers. Table \ref{tab4} demonstrates that the SGD optimizer significantly outperformed Adam, leading us to select SGD as the optimizer for our experiments.

To further enhance our model, as shown in Tables \ref{tab2} and \ref{tab4}, we employed data augmentation to enrich the dataset and improve the model's accuracy in recognizing different types of bone pathology in images. Through extensive experimentation, we discovered that integrating the DWR module into the YOLOv8 algorithm significantly improved the detection and classification of bone pathologies in wrist X-ray images. The DWR module's unique process of Region Residualization and Semantic Residualization effectively captures multi-scale contextual information, simplifies the learning process, and ensures the orderly and efficient extraction of relevant features. Further integration of the LSKA module into YOLOv8, which decomposes a 2D convolutional kernel into cascaded horizontal and vertical 1D kernels, effectively handles large-scale images while maintaining efficient feature extraction capabilities. These two module enhancements led to further improvements in the accuracy of the YOLOv8 algorithm, with the mAP 50 increasing from 0.599 to 0.687, significantly enhancing the performance in detecting and classifying bone pathologies in wrist X-ray images. We referenced the data provided by Joonho Oh et al \cite{diagnostics13182927}. as shown in Table \ref{tab5}, and compared the performance of various mainstream object detection algorithms on the GRAZPEDWRI-DX dataset, as presented in Table \ref{tab6}, our model has achieved the best performance to date.

This paper presents the design of an application aimed at detecting pediatric wrist fractures, utilizing our model for fracture detection. Figure \ref{predict} illustrates both the manual annotations made by a radiologist and the predictions generated by our model. We have developed "BoneVisionAI", a multi-platform application specifically designed to aid doctors in remote areas and those with limited experience in accurately interpreting children's wrist X-ray images. The application aims to lower the error rate in X-ray image analysis and offer more precise diagnostic support, marking a significant advancement in the field of medical imaging.

\begin{figure*}[htbp]
\centering
    % 左侧组的三个子图 
    \begin{minipage}[b]{0.4\textwidth}
        \includegraphics[width=0.8\textwidth]{imgs/1P_curve.png}
        
        \includegraphics[width=0.8\textwidth]{imgs/1PR_curve.png}
        
        \includegraphics[width=0.8\textwidth]{imgs/1R_curve.png}
        \centerline{(a)}
    \end{minipage}
    % \hspace{\fill} % 用于在两组图之间填充空间
    % 右侧组的三个子图
    \begin{minipage}[b]{0.4\textwidth}
        \includegraphics[width=0.8\textwidth]{imgs/2P_curve.png}
        
        \includegraphics[width=0.8\textwidth]{imgs/2PR_curve.png}
        
        \includegraphics[width=0.8\textwidth]{imgs/2R_curve.png}
        \centerline{(b)}
    \end{minipage}
    \caption{Detailed illustration of the validation at the input image size of 1024. (a) YOLOv8 model; (b) our model.}
    \label{results}
\end{figure*}

\begin{figure*}[htbp]
\centering
    % 左侧组的三个子图
    \begin{minipage}[b]{0.4\textwidth}
        \includegraphics[width=0.9\textwidth]{imgs/val_batch1_labels.jpg}
        \centerline{(a)}
    \end{minipage}
    % \hspace{\fill} % 用于在两组图之间填充空间
    % 右侧组的三个子图
    \begin{minipage}[b]{0.4\textwidth}
        \includegraphics[width=0.9\textwidth]{imgs/val_batch1_pred.jpg}
        \centerline{(b)}
    \end{minipage}
    \caption{Examples of pediatric wrist fracture detection on X-ray images. (a) manually labeled images; (b) predicted images.}
    \label{predict}
\end{figure*}

\section*{Application}

To facilitate the application of wrist X-ray imaging in the localization and classification of bone pathology, we developed a graphical interface application using PySide6. PySide6 is a Python library for creating cross-platform desktop applications. It is the Python binding of the Qt framework, providing a wealth of GUI components and functionalities, enabling developers to create a variety of user interfaces. The software interface designed in this article is overall simple and elegant, with main features including selection of the training model; adjustment of Intersection over Union (IOU) threshold and Confidence; image uploading, detection, visualization of results, and result saving; total categories of detection targets, total targets, and the model used. The initial interface is as shown in Fig. \ref{init}:
\begin{figure}[ht]
    \centering 
    \includegraphics[width=0.6\textwidth]{imgs/init.jpg}
    \caption{The initialization interface of "BoneVisionAI".}
    \label{init}
\end{figure}

Changing the value in the adjustment box below the Intersection over Union (IOU) or Confidence will simultaneously change the progress of the slider bar, and vice versa. Changes in the IOU or Confidence values are synchronized to the model's configuration, altering the detection IOU threshold or Confidence threshold. Users can upload a single image for detection and recognition by clicking the local file selection. Clicking the image detection button completes the target detection function of the input image. The system then displays labels such as fracture, metal, text, and periosteal reaction in the detection results, along with their mAP values, and identifies the number of target categories in the picture. The detection results are as shown in Fig. \ref{BoneVisionAl}:
\begin{figure}[ht]
    \centering 
    \includegraphics[width=0.6\textwidth]{imgs/BoneVisionAl.jpg}
    \caption{Example of using the application "BoneVisionAl".}
    \label{BoneVisionAl}
\end{figure}

PySide6 visualization enables developers to create powerful, user-friendly applications with cross-platform compatibility, a rich library of functionalities, rapid development and iteration, strong interactivity, customizability, and extensibility. It assists surgeons in the localization and classification of bone pathology, reducing the likelihood of erroneous analysis and providing more useful information for surgery.

\section*{Conclusions}
This paper introduces the AdvYOLO algorithm model, a second modification based on the YOLOv8 model. Building on the YOLOv8 algorithm, it significantly enhances the localization and classification of pediatric wrist X-ray pathologies. This enhancement is achieved through the integration of the Dilated Wide Residual (DWR) module into the YOLOv8 network. The DWR module is a key innovation for improving the detection and classification of skeletal pathologies in wrist X-ray images, particularly in wrist X-ray analysis. It employs a unique method of acquiring multi-scale contextual information through two different steps: regional residualization and semantic residualization, making the feature extraction process more efficient and focused. The Large-Scale Kernel Attention (LSKA) module represents a transformative development in the field of Convolutional Neural Networks (CNN), especially for applications in medical imaging. Incorporating the LSKA module into the Yolov8 network is a groundbreaking step. This integration enhances the model's ability to recognize complex features in X-rays, a task that is both computationally demanding and crucial for accurate diagnosis. The LSKA module, with its larger separable kernels, augments the network's feature extraction capabilities. Additionally, our model has achieved a significant performance improvement, with the mAP50 value increasing from 0.638 to 0.687, further validating the effectiveness of the model. Finally, we have developed an application named "BoneVisionAI" using Pyside6. This application is deployable across multiple platforms, such as Windows, Linux, and iOS, and is designed to assist surgeons in resource-limited remote areas and those with less experience in the localization and classification of bone pathologies. It reduces the likelihood of erroneous analyses and provides more useful information for surgeries.

\section*{Data availability}

The datasets analysed during the current study are available at Figshare under \url{https://figshare.com/articles/dataset/GRAZPEDWRI-DX/14825193/2}. The implementation code and the trained model for this study can be found on GitHub at \url{https://github.com/Happy-Xiang/AdvYOLO}.

\bibliography{sample}



\section*{Author contributions statement}

Conceptualization, Rongxiang Wang and Jingwen Zhao; methodology, Rongxiang Wang; software, Xiang Liu; validation, Xiang Liu, Xian Tang, and Yunyu Shi; formal analysis, Xian Tang; investigation, Rongxiang Wang; resources, Lina Wei; data curation, Rongxiang Wang; writing—original draft preparation, Rongxiang Wang; writing—review and editing, Rongxiang Wang; visualization, Rongxiang Wang; supervision, Jingwen Zhao; project administration, Lina Wei; funding acquisition, Lina Wei. All authors have read and agreed to the published version of the manuscript.

\section*{Funding}

This research was supported in part by the National Natural Science Foundation of China (Grant No. 62006123), in part by the Natural Science Foundation of Jiangsu Province under Grant BK20200808.

\section*{Competing interests}

The authors have no financial or proprietary interests in any material discussed in this article.

\section*{Ethics approval}

This research does not involve human participants and/or animals.



\end{document}