The global population reached nearly 8 billion as of 2025, thereby significantly increasing the demand for food production. The global crop yield suffers on an average of 10% to 28% of loss annually due to pests and plant diseases [1, 2]. Therefore, early identification of crop diseases is crucial to prevent large-scale crop loss and to maintain a stable and sustainable food supply for the growing population. Continuous monitoring of crops and timely predictions of diseases are essential to prevent crop damage [3]. Physically inspecting the crops for monitoring crop health is time consuming and tedious process. Therefore, there is a growing demand for an accurate and timely disease prediction system. There has been a notable rise in the adoption of AI in agriculture over recent years, especially for crop health monitoring and management.

Convolution Neural Network (CNN) was the widely used deep learning technique for crop disease prediction. In CNN, convolutional layers are used to extract low level features such as edges, corners, textures and color gradients from the images. More convolution layers are stacked to learn the complex features in the deeper layers. Several pre-trained models such as AlexNet, VGGNet, InceptionV3, ResNet, DenseNet, and MobileNet have been used by many researchers to identify and classify crop images [4]. These models can capture local spatial features and edges effectively through hierarchical convolutional operations. Despite their success, CNN-based architectures possess inherent limitations, they cannot capture long-range dependencies and global contextual relationships between spatially distant regions of an image. In addition, CNNs has limitation such as computational inefficiency, vanishing gradients, and overfitting issues when applied to real world dataset.

Transformer models, which are mainly used in Natural Language Processing (NLP), have recently been adapted for a variety of tasks in computer vision and other domains [5]. The Vision Transformer (ViT) model uses self-attention mechanism to capture long range global contextual dependencies in the image without using the convolutional layers [6, 7]. This global perspective makes the transformer models perform better than the deep learning models thus improving their performance in complex visual recognition tasks. Swin Transformer, on the other hand, introduces hierarchical representation and local window attention that shifts between layers [8]. This design allows the model to capture both the local and global dependencies efficiently while reducing computational cost. There are other variants of transformer models such as DeiT (Data-efficient Image Transformer) [9], and PVT (Pyramid Vision Transformer) have further improved upon ViT by introducing hierarchical and efficient attention mechanisms. In the agricultural domain, these transformer-based models have shown great performance improvement in handling complex backgrounds, varying illumination, and high intra-class similarity among diseased leaves. However, their performance is often limited by quantity and the quality of the dataset.

Many hybrid models [10, 11, 12, 13] are available for plant disease classification and it mostly uses CNN as the base model for feature extraction and transformer models for classification for example, ViT-ResNet, Dense-ViT, and ConvNeXt-ViT. In spite, of significant progress made through the deep learning models and transformer models there are critical research gaps persist in the context of plant disease classification.

Firstly, CNN lacks generalization when applied to real-world datasets with complex backgrounds, varied lighting, and diverse disease stages, leading to overfitting. Secondly, the use of a single deep learning model or transformer model limits the model capability in capturing the features for instance, CNN focus locally, missing global context, while ViTs captures global dependencies but often misses fine-grained details. This leads to one of the critical features of spaces remaining unexplored. Thirdly, CNN and Transformer hybrid model employs shallow fusion strategies that combine features at the final layer, resulting in loss of important features.

To bridge these gaps, the current research focuses on building a hybrid architecture, a multi-transformer integration combining the global attention mechanism of Vision Transformer (ViT) and hierarchical and localized feature extraction capability of Swin Transformers, thereby exploring the strengths of both models to achieve more accurate and efficient feature representation.

The objective of this research is to develop an efficient hybrid transformer model for accurate classification of crop diseases. The primary contribution of this study is summarized as follows:

There are several works that exist on crop disease prediction and classification, ranging from deep learning models, transformer-based models, and ensemble learning models. The first group of researchers used CNN and pre-trained models for efficient classification of crop diseases. Pal et al. [14] developed a lightweight mobile application Mob-Res for precise detection of crop disease. The author used MobileNetV2 for feature extraction and ResNet for classification. The model achieved an accuracy of 97.73% on plant disease dataset and 99.47% on plant village dataset. A. Y. Ashurov et al. [15] proposed a novel customized CNN deep-learning model integrating squeeze-and-excitation block with a improved residual skip connection and achieved an accuracy of 98% on plant village dataset. Oni et al. [16] developed a custom CNN deep-learning model for the tomato leaf dataset collected from a local farm and achieved an accuracy of 95.2% when compared to the pre-trained models such as YOLOv5, MobileNetV2, ResNet18 with accuracy 77%, 89.38% and 71.88% respectively. Rezaei et al.’s [17] novel few-shot learning (FSL) technique is suituable when the dataset is limited to as few as five images per class. The speciality of the model is it uses pre-training coupled with meta-learning and feature attention (FA) mechanism. ResNet50 and ViT are used as feature learners. Krisha et. al [18] developed series of deep learning models such as EfficientNet-B0, EfficientNet-B3, DenseNet201 and ResNet50 on PlantDoc dataset and achieved an accuracy of 76.77% using EfficientNetB3 model.

The second group of researchers used vision transformer models for crop disease prediction. Barman et. al [7] proposed a smartphone – based solution for plant disease prediction using ViT. They also compared the performance of ViT with InceptionV3 and found ViT performs much better than deep learning models. Li et. al [19] proposed a real-time light-weight plant-based MobileViT(PMVT) in which the convolution blocks were replaced with residual structure that effectively captures long-distance dependencies between different images. The author also included convolutional block attention module to focus on essential features. The model was evaluated on wheat dataset and achieved an accuracy of 93.6% which is 1.6% higher than MobileNetV3.

The third group of researchers used hybrid models for crop disease prediction. Vallabhajosyula et al. [20] proposed a Hierarchical Residual Vision Transformer (HRVT) that combines an Improved Vision Transformer (IVT) for feature extraction and a ResNet-9 network for classification. This hybrid Transformer–CNN model effectively captures both global contextual and local spatial features from leaf images. It achieves an accuracy of 68.53% compared to large models like ResNet50 or InceptionV3. The model performs exceptionally well across three datasets: Local Crop Dataset (13 classes), Plant Village Dataset (38 classes), Extended Plant Village Dataset (51 classes). Tunio et al. [12] proposed a hybrid framework that combines CNN for local feature extraction and MViT for global feature extraction. Further, the author used feature-fusion method to align channel dimensions and enhance the local details of global representation. The author evaluated the model with three datasets: plant village, plantDoc and AC-plantDoc and obtained a performance improvement of 13.67% compared with SOTA and baseline model. Aboelenin et al. [10] proposed a hybrid model using VGG16, Inception-V3 and DenseNet for global feature extraction and ViT model for classification. The author evaluated the model on two datasets: Apple and Corn dataset and obtained an accuracy of 99.24% and 98% respectively. Yang et al. [21] proposes a novel leaf disease identification network (LDI-NET) that uses a multi-label method to simultaneously identify plant type, leaf disease, and severity in a single branch model. LDI-NET architecture leverages the strengths of both Convolutional Neural Networks (CNNs) and transformers for extracting local and long-range global features. Zeng et al. [11] introduces a novel deep – learning framework for plant disease prediction. In this model, CNN is used for local feature extraction, ViT with self-attention mechanism is used to capture global dependencies and the classification is done by the fully connected layer. The model is evaluated on three different datasets: cucumber, banana and plant village dataset. The model achieved 97% accuracy with increased brightness and achieved 93% accuracy with 30% decreased brightness. The model achieved a peak accuracy of 97% with a 30% increase in image brightness and maintaining an accuracy of 93% even with a 30% decrease in brightness. Beak et al. [22] introduces attention-score based multi-ViT model to detect crop disease. The author has used multiple pre-trained ViT to learn diverse representation of the input image. The model is tested on apple leaf and grape leaf dataset. Zhang et al. [23] proposed a hybrid model for identifying plant leaf disease in a distributed agriculture data source. The author used Swin transformer for feature extraction with federated learning to enable privacy-preserving model training

The model performance many wary based on the quality of the dataset used in real-world scenarios. Plant leaf images often contain significant noise, background objects, and other distortions. Traditional deep learning (DL) models primarily capture local features, which limits their ability to handle such complex variations. In contrast, transformer-based models can capture global dependencies across the image. However, each transformer architecture has its own limitations. To address this limitation, we propose a hybrid transformer model that combines the strengths of the ViT and the Swin transformer. The Swin transformer is well-suited for real-world applications, as it is designed to perform effectively on images captured in natural, uncontrolled environments rather than in lab settings.

This research introduces a hybrid transformer model Swin-HViT to efficiently identify the plant disease. The hybrid transformer model combines two transformer models: ViT and Swin transformer as shown in Fig. 1. ViT with the self-attention mechanism captures the global context of an image thus, it has a strong global feature modeling and simpler architecture than the CNN. Unlike ViT, the Swin transformer follows a hierarchical feature representation. Swin transformer uses window-based and shifted window self-attention mechanisms to capture both local and global features effectively.

The model was developed and tested in Google Colab Pro environment with python version 3.10. We used T4 GPU with high RAM session. The dataset was partitioned into training, validation, and testing subsets with proportions of 80%, 10% and 10% respectively, as summarized in Table 1.

Model performance is assessed using standard evaluation metrics, including accuracy (4), precision (5), recall (6) and F1-score (7) as defined below:

In this context, TP, TN, FP, and FN represent true positive, true negative, false positive, and false negative outcomes respectively. The model was trained for 15 epochs using the AdamW optimization algorithm with a categorical cross-entropy loss. A learning rate of 0.0001 and a batch size of 32 were employed as summarized in Table 2. The embedding dimensions were set to 768 for the Vision Transformer and 128 for the Swin Transformer.

The classification metrics for the Corn dataset with four classes are shown in Table 3. The proposed Swin-HViT Model demonstrates the best overall performance with a weighted average accuracy of 98.81%. It achieved the highest values for precision, recall, and F1-score in nearly every class, especially performing perfectly for the Healthy and Common Rust class and nearly perfect for Blight and Gray Leaf Spot class. The evaluation results on the more diverse, real-time PlantDoc Dataset, with 27 classes across 13 crop species, achieved an accuracy of 81.81%. There is a drop in accuracy due to the characteristics of the dataset. The PlantDoc dataset contains real-time images scraped from the internet, which include noise, diverse backgrounds, varying lighting conditions, and inconsistent image quality, all of which make the classification task more challenging.

The graphs in Fig. 6. a summarizes the training and validation loss of the hybrid model over 15 epochs on the Corn dataset. The validation loss drops steadily but has a slight increase in epoch 6 and after epoch 7 it stabilizes. This behavior suggests that the model generalizes well overall, with no severe overfitting. The training and the validation accuracy remains consistently high except in epoch 6 as shows in Fig. 10. b indicates good generalization and stable learning across the epoch.

Figure 7 Training and Validation Loss and Accuracy for PlanDoc Dataset

The training loss as shown in Fig. 7.a for the PlantDoc dataset shows a sharp reduction in the early epochs, and it converges effectively in the later epochs. In contrast, the validation loss decreases initially but gradually increases and fluctuates in the later epochs. The training accuracy as shown in Fig. 7. b rises rapidly confirming excellent performance on training dataset. But the validation accuracy remains stable in the range 75% − 82% with some oscillations across the epochs.

The confusion matrix for the hybrid model on Corn dataset is shown in Fig. 8. It demonstrates strong overall classification performance, with most predictions falling along the main diagonal, indicating correct classifications for each class. The three classes namely: "Blight", "Common_Rust", and "Healthy" samples are classified correctly, with only a minimal number of misclassifications. The primary confusion occurs with "Gray_Leaf_Spot," where a few samples are occasionally misidentified as "Blight" or "Common_Rust," though the number of such errors is minimal. The confusion matrix of the hybrid model for PlantDoc dataset is shown in Fig. 9. Most of the predictions are correctly aligned across the diagonal with few misclassifications noted off diagonal in some classes, specially classes with few samples and with similar symptoms. The hybrid model mis-classifies the class “squash powdery mildew leaf” as “tomato leaf bacterial spot”, “tomato mold leaf” and “grape leaf”. The AUC-ROC curve for the Corn Dataset is shown in Fig. 10. The hybrid model Swin-HViT achieved an AUC of 1.00 for all the classes on the Corn dataset. This indicates a perfect or near-perfect discrimination capability of the model across all the classes. The Fig. 11 presents class-wise ROC curves for PlantDoc dataset, where most classes achieve AUC values close to 1.0, indicating excellent discrimination performance. A few classes with slightly lower AUCs reflect higher visual similarity or data imbalance, but overall, the model demonstrates strong and reliable multi-class classification capability.

The proposed hybrid Swin-HViT model is compared with the existing SOTA models in Table 5. Swin-HViT outperforms existing methods on both the Corn and PlantDoc datasets, achieving the highest reported accuracies of 98.81% and 81.81% respectively. The existing deep learning models referred in [20, 27] and hybrid models in [17, 26] reported accuracies ranging from 94% to 98% with the Corn dataset. The proposed Swin-HViT reported 0.81% more than the existing model [20]. The existing deep learning models [19] and hybrid models [8, 18] reported accuracy ranging from 70% to 80.02% on PlantDoc dataset. The proposed Swin-HViT reported 81.8% which is 1.8% more than the existing model [8]. Thus, the proposed model Swin-HViT sets a new benchmark for crop disease classification.