The proposed system was deployed at two urban river sites in Shah Alam, Malaysia, namely Seksyen 2 and Seksyen 7, as shown in Fig. 1 and Fig. 2 respectively. The deployed monitoring units are presented in Fig. 3 and Fig. 4. These locations were strategically selected due to their current history of flood events and the frequent accumulation of floating debris, which obstructs water flow, reduces drainage capacity, and exacerbates flood risks.

The deployment was conducted in collaboration with local authorities to ensure adherence to safety protocols and to facilitate long-term system monitoring. Site selection criteria included vulnerability to flooding, accessibility for regular maintenance, and representativeness of typical urban river conditions in Malaysia. This field implementation provided an opportunity to evaluate the system’s capacity to detect, classify, and quantify floating debris in real-world conditions, thereby linking AI-based monitoring to practical flood risk management and riverine environmental assessment. Each monitoring unit integrates a high-definition surveillance camera connected to the NVIDIA Jetson device, running the YOLO models in real time to detect, classify, and count floating debris.

The overall process flow of the system is illustrated in Fig. 5. The first stage involves data collection, conducted by acquiring images and videos from online platforms as well as capturing footage from multiple river locations across the state. Following acquisition, the data undergo a series of image-processing steps to extract frames and ensure quality standards suitable for training. This pre-processing phase is essential to reduce training time and enhance detection accuracy. It includes resizing images, normalizing pixel values, and converting annotations into machine-readable formats. To further improve robustness, data augmentation techniques, such as rotation, flipping, and scaling, are applied to artificially expand dataset diversity. Once pre-processing is complete, the dataset is divided into training and testing subsets. These subsets are then used to train and evaluate the YOLOv7-based model for the detection and classification of floating debris.

Data collection involved acquiring images of rivers containing floating debris under both normal and extreme conditions. These images were obtained from primary sources, including video recordings and photographs captured using surveillance cameras, as well as from publicly available internet sources. To ensure diversity and robustness, the dataset included images taken at various times of the day and under different weather conditions. To improve dataset quality, duplicate images, both within and across datasets, were filtered out using the difPy library (Duplicate Image Finder) developed by Landman (2024).

Additionally, images with minimal flood coverage (i.e., less than 5% of flood pixels) were also excluded from the dataset. Manually curated filtering was applied to remove mislabeled debris images and irrelevant images of natural water bodies not associated with urban waterways. While most images featured binary labels (flooded vs. non-flooded), some portions of the DeepFlood dataset contained multi-class annotations (e.g., humans, vehicles, buildings, etc.). For consistency and relevance to the study’s focus on urban flooding, these labels were converted to binary format as well. The inclusion of diverse datasets in this study was not intended for comparative analysis but to improve the generalizability of the model by ensuring variability in data sources and quality, thereby reducing potential biases during training and evaluation.

Following data collection, the next phase involves data preprocessing. This begins with the extraction of video frames and their conversion into still images. Given that the collected dataset comprises images from multiple sources, the image dimensions vary significantly. Therefore, all images were resized to a uniform resolution of 640×640 pixels to ensure consistency and compatibility for model development, training, and validation. A custom Python script was developed to perform batch resizing of the images. Next, data augmentation was applied to the collected images.

Subsequently, data augmentation techniques were applied to enhance the diversity and robustness of the dataset. Data augmentation involves generating additional training samples by applying a range of transformations to the original images. These transformations include rotations, skewing, random zoom, flipping, and the addition of salt-and-pepper noise. The goal of these augmentations is to expose the proposed model to a wider range of visual events that it may experience in real-world circumstances. By introducing these variations, the model’s ability to generalize its comprehension of objects is enhanced, allowing it to better handle diverse environmental conditions and demonstrate improved resilience during inference. During augmentations, a random image is selected from the dataset and subjected to random combinations of the transformations, thereby expanding both the size and variability of the dataset.

Next, the image labeling process was conducted to assign specific labels to objects within each image. During this stage, objects visible in the images were annotated using bounding boxes, each defined by its coordinates (x and y), and associated class number. The process is one of the most labor-intensive aspects of constructing an AI-based detection model. Labeling was performed using LabelImg software, which facilitates the manual annotation of images and automatically exports the data in a format suitable for training object detection models. The labeling classes consist of 0:Bottle, 1:Branch, 2:Can, 3:Cup, 4:Styrofoam, 5:Plastic Bag, 6:Clustered Trash, 7:Plastic Container, 8:Cardboard, 9:Canopy, 10:Table, 11: Chair, and 12: Big Umbrella. These categories were specifically selected based on their potential hazards and prevalence in urban riverine environments. While objects such as canopy, table, chair, and big umbrella are not typically found in the river, they were included in the dataset due to their frequent presence along the riverbank, particularly in temporary business areas where small-scale vendors set up stalls during weekly morning and night markets in Malaysia. Each labeled image was accompanied by a corresponding .txt file containing the coordinates and class identifiers of all labeled floating debris. The text file follows the format:

<object-class-id > < x> <y > < width > < height>

Following the labeling process, the dataset comprised paired camera image files and their corresponding annotation files for floating debris. These datasets were subsequently subjected to a structured preprocessing workflow and partitioned into three distinct subsets: training, validation, and testing. A randomized allocation strategy was applied to divide the labeled data into 70% for training, 20% for validation, and 10% for testing. The training subset was utilized to optimize the model's parameters, while the validation subset was employed to monitor learning progress and guide hyperparameter tuning. The testing subset, held out during model development, was employed to objectively evaluate the model’s performance and generalization capability on unseen data.

The system employed the YOLO object detection algorithm, a state-of-the-art model renowned for its real-time object detection capabilities. The implementation was carried out on a Linux server platform to ensure efficient access to computational resources. The training and evaluation processes were conducted using a curated dataset comprising images of floating debris on river surfaces. Comparative performance assessments were performed between YOLOv7 and YOLOv9. To establish the training environment, source files from both YOLOv7 and YOLOv9 were obtained from their respective official repositories and installed on a local high-performance machine. Separate virtual environments were configured for each version to ensure isolated and conflict-free development. This approach supports full local deployment, offering better control and data privacy than containerized cloud-based alternatives.

Both YOLO algorithms require a file that defines the dataset's structure, including class labels and file paths. The file is referred to as a .yaml file and is provided as input for the training parameter using the --data option. After all necessary files have been created, the training process is ready to begin. Several crucial parameters require adjustment. Executing training with the default parameter settings without adjustments may lead to errors or inaccurate outcomes. Therefore, it is necessary to modify certain crucial parameters to attain the intended outcome. The parameters utilized for this project encompass both the training and formatting aspects and are outlined in Table 3.

The detect.py module in the original YOLO has an incorporated capability to detect objects and mark them with bounding boxes. By modifying the source code of detect.py, it is possible to further improve the program's capabilities. This enables the display of additional data overlaid on the existing detection frames. This change enhances the program's ability to provide a more comprehensive and useful visual display, beyond the limitations of traditional bounding boxes. The improved iteration of detect.py can be tailored to incorporate additional information, such as object labels, counts, and relevant metadata, right onto the visual output. This enhancement not only improves the clarity of the detection results but also enables a more thorough comprehension of the identified items within the entire scene, hence increasing the effectiveness and user-friendliness of the YOLOv7-based detection system.

In this project, an additional counting technique is implemented, and its results are displayed on the screen in conjunction with the detection procedure. By implementing these modifications, a greater amount of useful data becomes accessible for analytical purposes. The detect.py code has been modified to implement these changes. This code snippet creates a dictionary to store all the identified data in the current frame of detection. During this process, the detect.py script retains the data. Once the data is accessible, additional code is used to compute and store the data for display on the current detection frame. Finally, the code will display all the processed data on the frame and then move on to the next frame. The detect.py script requires two inputs: (1) the trained custom weights, obtained from prior model training, and (2) the input video or image files for object detection.

Next, the program initiates the detection process by retrieving input from a video source, which may consist of either pre-recorded footage or a real-time streaming link. The video stream is processed frame by frame using the object detection algorithm, thereby converting the continuous feed into a sequence of individual images for analysis. For each frame, the detection module identifies floating debris items and records the number of objects detected per class. Before detecting the next frame, the program compares the current count of detected debris items per class against the maximum recorded counts from previous frames. If a higher count is identified, the program updates the count accordingly. If the count is lower or unchanged, it continues with the next frame. This iterative comparison ensures that the maximum number of floating debris objects per class is retained throughout the detection process.

Once all frames have been analyzed and no further data remains to be processed, the system executes a termination command to conclude the detection session. Notably, the original detect.py script in the YOLO framework is limited to basic object detection. In this study, the script was enhanced to incorporate object counting functionality. Algorithm 1 illustrates the pseudocode detailing the modified detection and counting process for floating debris in riverine environments.

During the preliminary test, this study conducted a comparative analysis between two datasets: one comprising 3,000 images (hereafter referred to as the 3K dataset) and another containing 15,000 images (referred to as the 15K dataset). The 15K dataset was primarily constructed through frame extraction from video sources, followed by extensive preprocessing techniques, including flipping/mirroring, rotation, image sharpening, the addition of Gaussian noise, and adjustments to contrast, brightness, colour saturation, and sharpness. Additional modifications, such as the insertion of black occlusion boxes, were also applied to simulate real-world occlusion scenarios. The dataset size plays a critical role in determining the overall performance of object detection models. Generally, an increase in dataset volume enhances model accuracy, as the network is exposed to a broader variety of feature representations. However, the scale of the dataset is often constrained by the available computational resources for model training. As such, striking an optimal balance between dataset size and hardware capacity is essential to ensure both effective learning and feasible processing times.

As illustrated in Fig. 6, the mAP graph demonstrates the performance comparison between the two datasets after 100 training epochs. The model trained on the 3K dataset achieved a mAP of 64% at an Intersection over Union (IoU) threshold of 0.5. In contrast, the model trained using the 15K dataset reached a significantly higher precision of 90% under the same IoU threshold. This substantial difference clearly indicates that the smaller 3K dataset lacks sufficient variability and volume to produce a robust and accurate model. Therefore, the 15K dataset, which provides improved generalizability and superior performance, was selected for all subsequent training, validation, and testing phases of this study.

In assessing the model's performance for the floating debris detection system on the river surface, several common evaluation metrics for object detection tasks were employed. The measurements include precision, recall, F1-score, mean average precision (mAP), and the Precision-Recall curve graph (PR curve). The results from the validation process highlight the model’s ability to generalize across different contexts and object categories, ensuring robustness and reliability in real-world applications.

The comparative analysis between YOLOv7 and YOLOv9 models trained on the same dataset reveals significant differences in performance metrics, including precision, recall, PR curve, and F1 score. The confusion matrices provide a granular view of each model's performance across different classes. YOLOv7’s confusion matrix shows fewer misclassifications, indicating a strong ability to correctly identify and classify objects. YOLOv9’s confusion matrix, while still performing well, indicates higher misclassifications in certain classes, suggesting areas for targeted improvements. YOLOv7, as expected, shows robust performance due to its established architecture and fine-tuning capabilities. The precision curve for YOLOv7 in Fig. 7 demonstrates high accuracy in predicting positive instances, with minimal false positives. This indicates its refined ability to discern between classes and accurately identify objects. Conversely, YOLOv9, being a more recent model, shows promising improvements but also exhibits areas that require further development. The precision curve for YOLOv9 in Fig. 8 shows a slight increase in false positives compared to YOLOv7.

However, it also indicates the potential for higher precision with additional training and optimization. Based on the precision evaluation presented in Table 4, YOLOv9 demonstrates a slight improvement in precision for most of these categories, namely Bottle, Can, Cup, Styrofoam, and Plastic Bag, compared to YOLOv7. This indicates that YOLOv9 is slightly better at correctly identifying these objects without producing false positives. Additionally, YOLOv9 shows a notable improvement in precision for branches, indicating a better ability to differentiate them from other objects. However, it has significantly lower precision for plastic containers compared to YOLOv7, suggesting a higher rate of false positives in this category. Meanwhile, for clustered trash, YOLOv7 outperforms YOLOv9 in this category, indicating that YOLOv7 is more effective in accurately identifying clustered trash.

The recall curves for YOLOv7 and YOLOv9 shown in Fig. 9 and Fig. 10, respectively, further elucidate the differences. YOLOv7 maintains a consistently high recall, suggesting it effectively captures most of the true positive instances. YOLOv9, while also high, demonstrates variability that suggests sensitivity to certain conditions or classes that YOLOv7 handles more uniformly. The PR curves highlight the trade-offs between precision and recall for both models. YOLOv7 shows a balanced curve, maintaining high precision and recall across various thresholds. YOLOv9, however, shows a more pronounced curve, indicating that while it can achieve high precision, it may require more careful threshold tuning to balance recall effectively. F1 score curves consolidate these observations. YOLOv7 presents a stable F1 score, confirming its balanced approach to precision and recall. YOLOv9, while having a competitive F1 score, shows variability that underscores the need for further training iterations and potentially more diverse datasets to stabilize its performance. Overall, while YOLOv7 demonstrates consistent and reliable performance across various metrics, YOLOv9 shows significant potential, with some areas requiring further refinement. The transition from YOLOv7 to YOLOv9 could yield enhanced performance with continued development and optimization, leveraging YOLOv9’s advanced features and potential for higher precision.

The deployed system successfully identified and classified various types of trash in real-time, demonstrating high accuracy and reliability. The NVIDIA Jetson's performance enabled seamless processing of video feeds, ensuring timely detection and alerting mechanisms. The deployment highlighted the system's potential to aid in flood monitoring and management by providing valuable data on trash accumulation in real time. This information can be used by local authorities and environmental agencies to implement more effective waste management and flood prevention strategies.

Based on the recall evaluation presented in Table 5, Bottle, Can, Plastic Bag, and Styrofoam: YOLOv7 has higher recall values for these categories compared to YOLOv9, indicating that it is better at detecting these objects and minimizing false negatives. Both models perform poorly in detecting branches, with YOLOv9 having a significantly lower recall than YOLOv7, suggesting that both models struggle to detect branches, but YOLOv9 struggles even more. For Cup and Clustered Trash, the recall values for these categories are relatively high for both models, with YOLOv7 having a slight edge, indicating effective detection of these objects. YOLOv9 has a notably higher recall compared to YOLOv7, suggesting it is better at detecting plastic containers without missing them, despite the lower precision noted earlier. Whereas, YOLOv7 performs better in terms of recall, indicating fewer missed detections compared to YOLOv9.

The mAP@0.50 metric evaluates a model's ability to locate objects with a moderate Intersection over Union (IoU) overlap of at least 0.50 (50%) with a ground truth object. Based on Table 6, the mAP@0.5 values indicate that YOLOv7 outperforms YOLOv9 across all classes. The high mAP values for YOLOv7 suggest better precision in detecting objects with at least 50% IoU overlap with ground truth objects. This indicates a more robust performance in object localization tasks at moderate precision levels.

The mAP@0.95 demands higher precision, requiring a minimum IoU overlap of 0.95 (95%) for a detection to be considered correct. It evaluates a model’s ability to precisely localize objects with high accuracy. When evaluating mAP@0.5:0.95, which demands higher precision for correct detections, YOLOv7 generally shows better performance compared to YOLOv9 as shown in Table 7. Notably, YOLOv7 excels in detecting branches, cans, and plastic bags with higher precision. The consistent performance of YOLOv7 across varying IoU thresholds highlights its superior adaptability and precision in object detection tasks, especially under stringent evaluation criteria.

Weather conditions can significantly impact the performance of object detection models. Adverse weather conditions in Malaysia, which mostly experiences rainfall, introduce additional challenges like reduced visibility and increased noise, which can complicate the detection process. This section compares the detection performance of YOLOv7 and YOLOv9 under various weather conditions, highlighting the strengths and weaknesses of each model. Under clear, sunny conditions, both YOLOv7 and YOLOv9 perform exceptionally well, maintaining high levels of accuracy. YOLOv7, with its well-established architecture, demonstrates slightly higher precision and recall, capturing fine details with minimal false positives. YOLOv9 also performs robustly, showcasing its advanced capabilities, although it exhibits a marginally higher rate of false positives compared to YOLOv7.

Rain introduces motion blur and water droplets, which can obscure parts of the objects and make detection challenging. In these conditions, YOLOv9 outperforms YOLOv7. The advanced features of YOLOv9, such as its enhanced ability to handle noise and partial occlusions, contribute to better accuracy and fewer misclassifications. While YOLOv7 maintains respectable performance, it struggles slightly more with false negatives and missing smaller or partially obscured objects. Figure 11 illustrates a comparison of detection outputs from YOLOv7 and YOLOv9 in a rainy environment, highlighting YOLOv9’s improved ability to localize and classify objects despite visual obstructions.