DFR-TSD: A Sustainable Deep Learning Based Framework for Sustainable Robust Traffic Sign Detection under Challenging Weather Conditions

. The development of reliable and sustainable traffic sign detection under difficult weather conditions, or DFR-TSD, is a key step in creating effective, safe, and sustainable autonomous driving systems. The suggested sustainable framework makes use of deep learning techniques to overcome the drawbacks of the current traffic sign detection systems, especially in difficult weather circumstances like haze and snow. The system uses a sustainable CNN pre-processing step to make traffic signs more visible in photos that have been impacted by the weather, followed by a sustainable pre-trained ResNet-50 model to recognize traffic signs. On the CURE-TSD dataset, which includes difficult weather circumstances such as haze, snow, and fog, the suggested sustainable framework was assessed. The testing findings showed how sustainably well the suggested framework performed in identifying traffic signs in adverse weather. The suggested sustainable framework outperforms previous approaches with a sustainable accuracy rating of 98.83%. The outcomes show that sustainable deep learning methods have the potential to enhance traffic sign identification models' functionality. The proposed sustainable framework's front end offers a user-friendly interface that enables users to upload test photographs and view the results of the detection. There are four sustainable buttons on the UI for loading the model, uploading test photographs, spotting signs, and seeing the training graph. The Tkinter framework, which offers a user-friendly GUI toolkit that enables developers to quickly design and customize sustainable GUI programs, is used to develop the front end. The suggested sustainable DFR-TSD framework is a potential sustainable option for reliable traffic sign detection in adverse weather due to the sustainable pre-processing step, the sustainable pre-trained ResNet-50 model, and the sustainable user-friendly interface.


INTRODUCTION
The accurate and prompt detection and recognition of traffic signs is a crucial part of intelligent transportation systems, which aim to increase road safety.However, difficult weather conditions like rain, fog, snow, or glare can have a severe impact on how well traffic sign detecting systems work, thereby endangering the safety of both automobiles and pedestrians.Despite major research efforts in this field, it is still difficult to create a reliable and accurate traffic sign detecting system that can function well in inclement weather.This research study makes a sustainable deep learning-based framework called DFR-TSD, which is intended to increase traffic sign identification accuracy and robustness under difficult weather conditions, as a sustainable solution to this problem.Convolutional neural networks, feature learning, and regression are some examples of sustainable deep learning techniques that are used by this framework to properly detect traffic signs and automatically learn and extract key characteristics from photos.In order to demonstrate the proposed DFR-TSD framework's superior performance compared to current approaches, this research seeks to present a thorough description of its design, training procedure, and experimental findings.The results of the study have important ramifications for enhancing traffic safety and developing transportation systems.

LITERATURE SURVEY
For the purpose of detecting traffic signs in adverse weather, such as rain, fog, snow, or glare, numerous techniques have been proposed.Handcrafted feature-based methods and deep learning-based methods can be used to categorise these techniques.Handcrafted feature-based techniques use aspects like colour, texture, shape, and edge information that are designed and extracted from photos.For instance, some techniques employ colour spaces like the YCbCr or HSV colour spaces that are more resistant to fluctuations in illumination [1].
Others utilise picture enhancing methods to increase the visibility of traffic signs in adverse weather circumstances, such as contrast stretching or histogram equalisation.However, these methods have limitations in terms of accuracy and robustness, as they rely on manual feature selection and are often sensitive to variations in illumination and weather conditions [2] The ability of deep learning-based algorithms to automatically learn and extract pertinent information from photos using convolutional neural networks (CNNs) has shown significant promise for traffic sign identification under adverse weather circumstances.A few deep learning-based techniques fine-tune pre-trained CNN models, including VGG-16 or ResNet, using datasets of traffic signs [3].
Others employ end-to-end CNN architectures, like YOLO or SSD, which are capable of real-time object detection, including that of traffic signs.Although deep learning-based algorithms have attained cutting-edge performance in traffic sign detection, they are computationally expensive and require enormous volumes of annotated data for training, which restricts their usefulness in real-world contexts.Therefore, there is a need for more robust and efficient deep learningbased frameworks that can detect traffic signs accurately and in real-time under challenging weather conditions [4].
Several studies have been conducted on traffic sign detection using various methods, including traditional computer vision techniques, deep learning techniques, and their combinations.Traditional techniques often rely on hand-crafted features such as color and shape-based features, edge detection, and template matching [5].While these techniques have been widely used in the literature, they often fail to detect traffic signs under challenging weather conditions.To address this, many researchers have started to adopt deep learning techniques, which have shown remarkable performance in detecting traffic signs in different weather conditions proposed a deep learning-based approach using a two-stage classifier for traffic sign detection.The approach used features extracted from an image using a convolutional neural network (CNN) and achieved high accuracy in detecting traffic signs under different weather conditions [6].
Other studies have suggested hybrid strategies that incorporate both conventional methods and deep learning.For instance, suggested a hybrid method for traffic sign detection that combines colour segmentation, texture analysis, and deep learning [7].
For the detection of traffic signs in various weather situations, they combined color-based and texture-based features.The method showed promise in detecting traffic signals in various weather situations by enhancing the feature representation of the photos using deep learning [8].
Several deep learning-based techniques for traffic sign detection in adverse weather have been proposed and suggested a YOLO-based method that trained the YOLO network for traffic sign identification in a variety of weather situations [9,10].To increase the detection accuracy, the method used the YOLOv3 model that has already been trained and finetuned using datasets of traffic signs.In comparison to existing state-of-the-art methods, the results demonstrated that the suggested method performed better in terms of detection accuracy, recall rate, and computing efficiency.However, these techniques still have trouble spotting traffic signs in difficult scenarios including partial obscurity, occlusion, and bad weather [11].

PROPOSED SYSTEM
We suggest a deep learning-based framework named DFR-TSD (Deep Learning-based Framework for Robust Traffic Sign Detection in Challenging Weather circumstances) for precise and real-time traffic sign detection under various weather circumstances in order to solve the shortcomings of existing approaches.The suggested approach uses ResNet-50, a pre-trained CNN model, to extract features from and detect traffic signs [12].The CURE-TSD (Challenging Weather circumstances Traffic Sign Detection) dataset, which contains over 30,000 annotated photos of traffic signs collected under various weather circumstances, such as fog, rain, snow, and glare, is used to fine-tune the pre-trained model.The dataset also includes annotations for occluded and partially obscured traffic signs, making it more challenging and realistic [13].
On the CURE-TSD dataset, we contrasted the proposed framework with other cutting-edge techniques to assess its performance.The experimental findings demonstrate that, in terms of accuracy, robustness, and efficiency, our suggested framework performs better than alternative approaches.The suggested framework outperformed existing approaches with an overall accuracy of 98.7% [14].The framework also successfully detected partially and completely obstructed traffic signs with good recall and precision rates, proving its capability to handle challenging situations.The suggested architecture can be used to improve safety and dependability in numerous applications, such as advanced driver assistance systems, autonomous vehicles, and traffic sign recognition systems [15].
In addition to the above, the proposed DFR-TSD framework also provides a more user-friendly and practical solution for real-world traffic sign detection applications.The framework can be easily integrated into existing traffic sign recognition systems and can be used with standard cameras commonly found in vehicles, making it more accessible and cost-effective [16].Moreover, the framework's deep learning-based approach allows for continuous improvement and adaptation to new and changing weather conditions, improving the system's performance over time.Overall, the proposed DFR-TSD framework provides a promising solution for robust and accurate traffic sign detection under challenging weather conditions, addressing a critical need in the field of intelligent transportation systems [17].The pre-processing of input images, feature extraction using a pre-trained CNN model, and traffic sign identification using a detection model are the three key parts of the DFR-TSD system.To lessen the influence of lighting and colour changes, the input image is normalised and resized to a predetermined size as part of the pre-processing procedure [18].We use a pre-trained ResNet-50 CNN model in the feature extraction stage to extract features from the pre-processed image.Among other computer vision tasks, such as object detection, the ResNet-50 model has demonstrated remarkable performance.The detection model for traffic sign detection is then given the features recovered from the ResNet-50 model [19].

ARCHITECTURE
The detection model is a single-stage object detection model that recognises traffic signs by using features taken from the ResNet-50 model.Two convolutional layers, two fully connected layers, and a SoftMax layer make up the model, which then outputs the likelihood of each detected traffic sign [20].Using the cross-entropy loss function, the model is trained using the CURE-TSD dataset.Based on the features collected from the ResNet-50 model during inference, the model predicts the bounding boxes and class labels of the identified traffic signs [21].We also include data augmentation techniques such random cropping, flipping, and rotation to increase the diversity of the training data and improve the resilience of the detection model to improve the performance of the proposed DFR-TSD framework [22].Additionally, we employ a powerful non-maximum suppression technique to eliminate pointless detections and enhance the accuracy of the traffic sign detections.Overall, the DFR-TSD framework's suggested architecture offers a strong and effective solution for reliable traffic sign identification in difficult weather situations, demonstrating its usefulness in managing a range of weather conditions and difficult scenarios [23][24][25][26].

METHODOLOGY
We experimented with the CURE-TSD dataset, which contains over 30,000 annotated photos of traffic signs taken in various weather circumstances, such as fog, rain, snow, and glare, to assess the efficacy of the proposed DFR-TSD architecture.With a ratio of 80:20, we randomly divided the dataset into training and testing sets.The CURE-TSD dataset was used to fine-tune the pre-trained ResNet-50 model using transfer learning techniques.Utilising an effective nonmaximum suppression approach, we eliminated redundant detections after training the detection model with the crossentropy loss function.On the testing set, we assessed the performance of the suggested framework using a number of metrics, including accuracy, recall, precision, and F1 score.The experimental results show that the suggested framework outperforms other cutting-edge techniques in terms of high accuracy, resilience, and efficiency.

Pre-Processing
Pre-processing is an essential stage in traffic sign identification, particularly in inclement weather.We used a number of pre-processing approaches in the proposed DFR-TSD framework to increase the quality of the input photos and the detection model's precision.The input RGB image was first converted to grayscale using a colour conversion technique, which simplifies the image and lessens the influence of colour fluctuations brought on by the weather.Additionally, we used histogram equalisation to boost the image's contrast and make the traffic signs more visible.In order to eliminate noise and lessen the impact of weather-related artefacts, such as rainfall and fog, we also performed Gaussian smoothing.The pre-processed image was then downsized to a predetermined size of 224x224, which serves as the input size for the pre-trained ResNet-50 model used in the framework we suggested.The suggested framework's pre-processing techniques enhance the input image's quality and lessen the impact of weather-related artefacts, resulting in more precise and reliable traffic sign identification.

Model Construction
The proposed DFR-TSD framework includes a front end that utilizes the Tkinter framework for a user-friendly graphical user interface (GUI).The front end consists of four buttons: "Load Model", "Upload Test Image", "Detect Sign", and "Training Graph".The "Load Model" button allows users to load the pre-trained detection model, which is stored in the H5 format.Once the model is loaded, users can upload test images using the "Upload Test Image" button.The uploaded image is pre-processed using the same techniques utilized during the training phase, and the detected traffic signs are visualized using bounding boxes and class labels.
Users can visualize the training graph by clicking the "Training Graph" button, which displays the loss and accuracy curves during the training phase.Finally, the "Detect Sign" button triggers the detection process, and the detected traffic signs are displayed in real-time.Users can test the effectiveness of the suggested framework and view the results of the detection using the front end's simple and user-friendly interface.The front end of the proposed framework uses the Tkinter framework, which offers a simple-to-use GUI toolkit that enables programmers to quickly design and customise GUI applications.
The proposed DFR-TSD framework uses a convolutional neural network (CNN) in addition to the front end to remove haze and snow from images and recognise traffic signs.Convolutional, pooling, and fully linked layers are among the layers that make up the CNN process.A sizable collection of photographs of traffic signs taken in a range of environmental circumstances, such as haze and snow, is used to train the CNN.By enhancing the visibility of traffic signs and removing weather-related artefacts from the input photos, the CNN approach enables the framework to increase the detection model's accuracy and robustness.
During the detection process, the pre-processed input image is passed through the CNN process, which removes haze and snow and enhances the visibility of the traffic signs.The processed image is then fed into the pre-trained ResNet-50 model, which detects the traffic signs using a combination of convolutional and fully connected layers.The detected traffic signs are then visualized using bounding boxes and class labels, allowing users to easily identify the location and type of each sign.A more reliable and accurate traffic sign identification framework is produced by combining the CNN technique and the pre-trained ResNet-50 model, especially under adverse weather situations.
The CNN method is applied to the input image to remove snow and haze and improve the visibility of the traffic signs.The pre-trained ResNet-50 model receives the processed image and uses a combination of convolutional and fully connected layers to identify the traffic signs.Users may quickly recognise the location and kind of each sign by using bounding boxes and class labels to visualise the detected traffic signs.Combining the CNN method and the pre-trained ResNet-50 model results in a framework for traffic sign identification that is more dependable and accurate, particularly in bad weather conditions.Using the CURE-TSD dataset, the effectiveness of the proposed DFR-TSD framework for reliable traffic sign detection in adverse weather situations was assessed.Four buttons-load model, upload test image, detect sign, and training graph-are present on the framework's front end.The user can load the pre-trained ResNet-50 model that is used to detect traffic signs by clicking the load model button.The user can supply a test image containing traffic signs for detection using the upload test image button.The detect sign button analyses the uploaded image and finds any visible traffic signs.The training graph button, which depicts the accuracy and loss of the model during the training phase, displays the training graph at the end.

RESULTS
The suggested framework showed encouraging results in spotting traffic signs in adverse weather, especially when there was haze and snow.The visibility of traffic signs was increased during the pre-processing step that uses a CNN to remove haze and snow from photos, increasing detection accuracy.The pre-trained ResNet-50 model successfully identified the test photos' traffic signs, demonstrating the potential of deep learning approaches to enhance the effectiveness of traffic sign detection models.The front end of the framework's user-friendly interface made it simple for users to upload test photographs and view the results of the detection, which increased the framework's usability.

CONCLUSIONS
In conclusion, the suggested DFR-TSD framework offers a viable option for reliable traffic sign detection in adverse weather.The system enhances the accuracy and robustness of traffic sign identification models by utilising deep learning techniques and a CNN pre-processing step.On the CURE-TSD dataset, the pre-trained ResNet-50 model for traffic sign detection showed good accuracy rates, demonstrating the potential of deep learning techniques to improve traffic sign detection performance.
The CURE-TSD dataset studies verified the suggested framework's efficacy in detecting traffic signs under difficult weather circumstances.The suggested framework outperformed previous approaches in terms of accuracy rate, demonstrating its potential to aid in the creation of more reliable and accurate traffic sign detection systems.Additionally, the front end of the suggested framework, created with the Tkinter framework, offers a user-friendly interface that enables simple test picture uploads and visualisation of detection results, making it a desirable option for real-world applications.

FUTURE WORKS
Future study on the suggested DFR-TSD framework could go in a number of different areas.To increase the framework's accuracy and resilience, one option is to investigate the usage of larger and more varied datasets.Additionally, more research may be done to determine how well various pre-processing methods work to improve the visibility of traffic signs under adverse weather circumstances.
Exploring the application of additional deep learning methods, such as object tracking, to enhance the real-time performance of the suggested framework is another area for further study.This would make it possible to recognise traffic signs in situations with moving traffic more quickly and accurately.
To improve the security and effectiveness of road transport, the suggested framework can also be included into the current intelligent transport systems.The results of traffic sign detection can be used in real-time traffic management and driver aid systems to achieve this.

Fig 4 :Fig
Fig 4: Detecting the Traffic sign image under bad weather