Development of a Novel Convolutional Neural Network Architecture Named RoadweatherNet for Trajectory-Level Weather Detection using SHRP2 Naturalistic Driving Data

Abstract

Driver performances could be significantly impaired in adverse weather because of poor visibility and slippery roadways. Therefore, providing drivers with accurate weather information in real time is vital for safe driving. The state-of-practice of collecting roadway weather information is based on weather stations, which are expensive and cannot provide trajectory-level weather information. Therefore, the primary objective of this study was to develop an affordable detection system capable of providing trajectory-level weather information at the road surface level in real-time. This study utilized the Strategic Highway Research Program 2 Naturalistic Driving Study video data combined with a promising machine learning technique, called convolutional neural network (CNN), to develop a weather detection model with seven weather categories: clear, light rain, heavy rain, light snow, heavy snow, distant fog, and near fog. A novel CNN architecture, named RoadweatherNet, was carefully crafted to achieve the weather detection task. The evaluation results based on a test dataset revealed that RoadweatherNet can provide excellent performance in detecting weather conditions with an overall accuracy of 93%. The performance of RoadweatherNet was also compared with six pre-trained CNN models, namely, AlexNet, ResNet18, ResNet50, GoogLeNet, ShuffleNet, and SqueezeNet, which showed that RoadweatherNet can provide nearly identical performance with a significant reduction in training time. The proposed weather detection model is cost-efficient and requires less computational power; therefore, it can be made widely available mainly owing to the recent thriving of smartphone cameras and can be used to expand and update the current weather-based variable speed limit systems.

Adverse weather events, such as snow, rain, and fog, can directly affect roadway safety by reducing the visibility and roadway surface friction, negatively affecting vehicle as well as driver performance, and increasing the required stopping sight distance. The data of the National Highway Traffic Safety Administration revealed that adverse weather is attributed to approximately 21% of vehicle crashes, 19% of crash injuries, and 16% of crash fatalities each year in the United States ( 1 ). Previous studies revealed that inclement weather conditions increased traffic injuries by 45% and related fatalities by 25% ( 2 , 3 ). The deployment of various systems, including the advanced driving assistance system (ADAS) and variable speed limit (VSL), can efficiently mitigate the impact of adverse weather and provide safe driving environments ( 4 ). However, accurate detection of weather conditions at the road surface level in real time is required to effectively operate these safety systems.

The existing practice of collecting weather information is mainly based on a road weather information system (RWIS) that can provide numerous weather data, including temperature, humidity, wind speed, visibility, and precipitation. However, the widespread deployment of RWIS is not financially viable. In addition, sensors on the weather stations are typically mounted at a higher elevation level, which might not essentially represent actual weather conditions at road surface level. Therefore, they cannot provide trajectory-level weather information. Therefore, in-vehicle sensors (e.g., video camera) could be more reliable and cost-effective for representative real-time weather detection at the road surface level.

Image-based weather detection has been comprehensively studied in the literature using different approaches. For example, Hautiére et al. ( 5 ) developed a method for visibility determination in fog weather conditions by calculating meteorological visibility distance using an onboard camera. They tested their system in three weather conditions (i.e., sunny, foggy, and dense foggy) and found a stable estimation of visibility distances under all weather conditions ( 5 ). Another study proposed a real-time snow detection system in which snowfall noises could be removed from images by applying a median filter ( 6 ). Similarly, Garg and Nayar ( 7 ) developed a method that can detect pixels affected by rain and remove rain pixels from images. A study conducted by Pomerleau ( 8 ) developed a visibility estimation system by measuring the decrease of contrasts between consistent road features (e.g., lane markings, road/shoulder boundaries, etc.) at several distances in front of the vehicle and tested the performance using simulated fog images. The study concluded that the proposed visibility estimation system could significantly detect all reduced visibility conditions because of inclement weather ( 8 ). In another study, Yan et al. ( 9 ) developed a weather detection system to detect sunny, cloudy, and rainy weather conditions based on images from an in-vehicle camera using the AdaBoost algorithm and found detection accuracy of more than 85% for all three weather conditions. A study by Roser and Moosmann ( 10 ) proposed a weather detection system to detect clear and rainy weather by extracting several features from single-color images, including local contrast, minimum brightness, sharpness, hue, and saturation. Mori et al. ( 11 ) proposed a method to detect light, moderate, and dense fog weather using in-vehicle camera images by evaluating visibility and distance information of a preceding vehicle and obtained an overall detection accuracy of 84%.

Most of the weather detection studies, as already described, require an arbitrary object in front of the vehicle. These detection methods are not reliable in everyday scenarios because the objects could be obstructed by other vehicles, especially in congested traffic conditions. Imaged-based weather detection systems using machine learning approaches may overcome this problem. However, very few studies have investigated the possibility of using machine learning for real-time weather detection. Considering this research gap, the primary objective of this study was to develop a reliable and cost-effective weather detection system capable of providing accurate weather in real time by using only a single in-vehicle video camera and by applying a promising machine learning technique, called convolutional neural network (CNN). The research objective was achieved by performing the following research tasks: (a) compiling an annotated image database consisting of seven weather categories, namely, clear, light rain, heavy rain, light snow, heavy snow, distant fog, and near fog, from the Strategic Highway Research Program 2 (SHRP2) Naturalistic Driving Study (NDS) video data; (b) developing a novel CNN architecture; (c) training, hyperparameter tuning, and validating weather detection model by applying the proposed architecture; and (d) testing the performance of the model and comparing it with other pre-trained models, namely, AlexNet, ResNet18, ResNet50, GoogLeNet, ShuffleNet, and SqueezeNet.

As this study utilized the trajectory-level SHRP2 NDS data and built on a novel CNN architecture, it has the potential to improve on the previous studies in many ways. The major contributions of this paper can be listed as follows:

Previous studies have used pre-trained CNN architecture to develop weather detection models. For instance, the study by Ibrahim et al. ( 12 ) developed a weather detection model based on ResNet50 architecture. However, pre-trained models, including ResNet50, are computationally expensive because of their complex structure. To effectively implement trajectory-level weather detection models in an emerging Connected Vehicle (CV) environment, it is extremely important to reduce the computational requirements. It is anticipated that mobile weather detection systems will be applied mainly on a smartphone/tablet platform in maintenance vehicles. Keeping in mind this research need, this study devised a novel CNN architecture from scratch, capable of training and running on a mini-computer with relatively less computational power, such as a smartphone, without compromising the detection accuracy.

Many previous studies have used fixed roadside cameras, sensors from the RWIS, and Google images to detect weather conditions, which might have some potential limitations, such as higher mounting elevation and unable to provide trajectory-level weather data at road surface level from the driver’s perspective. On contrary, this study used trajectory-level images at road surface level captured from inside a moving vehicle. For safety applications in a CV environment, trajectory-level weather conditions are more representative than images captured using roadside stationary cameras.

Very few studies have experimented with the levels of adverse weather conditions. To the extent of the authors’ knowledge, this study is one of the first attempts to detect seven levels of weather conditions, namely, clear, light rain, heavy rain, light snow, heavy snow, distant fog, and near fog. Moreover, it is worth mentioning that this study used one of the most valuable but also challenging naturalistic study datasets, the SHRP2 NDS data. Additional extensive data reduction steps were taken to identify and classify the various levels of adverse weather conditions to form a unique ground truth dataset that was never explored in previous studies.

This study also leveraged six promising pre-trained CNN architectures, namely, AlexNet, ResNet18, ResNet50, GoogLeNet, ShuffleNet, and SqueezeNet, to develop the best weather detection model for detection accuracy.

Data Preparation

The video data used in this study were collected from the SHRP2 NDS. Many previous studies have used this unique data to investigate driver behavior in an attempt to improve the safety of the roadways ( 13 – 17 ). The present NDS was conducted in six states in the United States from 2010 to 2013. All the vehicles that participated in the study were equipped with a data acquisition system (i.e., ADAS). Along with other data, the ADAS also collected videos of the roadways in the moving direction using a color camera under various roadway and weather conditions ( 18 – 20 ). NDS trips that occurred in adverse weather were collected from the Virginia Tech Transportation Institute in Blacksburg, Virginia, using two unique methods, which were developed by the research team exclusively for effective and accurate data collection in various weather conditions ( 21 – 23 ). In the first method, complementary weather data from the National Climate Data Center in Asheville, North Carolina, were used. The weather stations were considered as points of interest with an influence zone of 5 nmi around them to identify the potential locations of trips that occurred in adverse weather. This threshold for influence zone was adopted from a previous study ( 24 ). The second method used weather-related crash locations as points of interest and using the same threshold identified adverse weather trips.

Utilizing these methods, video data were collected for the trips that occurred in adverse weather and their respective matched trips in clear weather. To confirm particular weather conditions (i.e., heavy snow, light snow), all the collected videos were manually observed and verified. In total, 217 trips in clear, 172 trips in snow, 204 trips in rain, and 168 trips in foggy weather were selected and considered for further analysis. Consequently, images were extracted at a sampling rate of 12 frames per minute from the videos of the selected NDS trips to create a database of images consisting of more than 20,000 images. Afterward, all the images were cropped at the bottom to discard the dashboard resulting in an image size of 250 × 200 pixels.

Once the extraction of images from the videos was completed, all the images were manually annotated and grouped into seven weather categories, namely, clear, light rain, heavy rain, light snow, heavy snow, distant fog, and near fog. It is worth pointing out that manual annotation of images was a critical and time-intensive task. However, numerous criteria were fixed based on quantitative measures to define the weather categories to obtain precise annotation. In addition, the research team was provided with comprehensive training to remove any potential bias in the manual annotation process. Table 1 lists the criteria used during the annotation process and Figure 1 illustrates the sample images of weather conditions ( 25 ). After the image annotation, a balanced number of images of 2,500 per category were randomly selected for the development of the weather detection model.

Table 1.

Criteria for Image Annotation

Weather	Criteria
Clear	Clear visibility
	Road sign, marking and surroundings are visible
Light rain	Raindrops are visible
	Dry/slightly wet road surface
	Clear/moderate visibility
	Wiper at a low setting
	Road markings and information on road signs and vehicle ahead could be recognized
Heavy rain	Raindrops are visible
	Wet road surface
	Affected visibility
	Wiper at a high setting
	Road markings and information on road signs and vehicle ahead could not be recognized
Light snow	Snowflakes are visible
	Little/no snow on the road surface
	Clear/moderate visibility
	Road markings and information on road signs and vehicle ahead could be recognized
Heavy snow	Snowflakes are visible
	Surface covered with snow
	Affected visibility
	Road markings and information on road signs and vehicle ahead could not be recognized
Distant fog	Road markings and information on road signs could be easily recognized
	Roadside surroundings and traffic ahead are visible
	The horizon is undefinable
Near fog	Only a few road markings in front of the NDS vehicle could be observed
	Information on the road signs could not be read
	Roadside surroundings and traffic ahead could not be properly recognized
	The horizon is undefinable

Note: NDS = Naturalistic Driving Study.

Figure 1.

Sample images of weather conditions.

Methodology

As mentioned earlier, this study adopted a cutting-edge machine learning algorithm named CNN to train, validate, and test the weather detection algorithm. CNN is a deep learning algorithm that was developed to solve complex image classification tasks. Previous studies have revealed that CNN can provide excellent accuracy for an image classification problem, especially with Big Data ( 26 – 28 ). Therefore, CNN is evolving as a state-of-the-art method for image classification, object detection, and image-based pattern recognition. In this study, a novel CNN architecture was proposed and the research team named it RoadweatherNet as it was developed specifically to detect trajectory-level road weather conditions.

Development of RoadweatherNet

Similar to other deep learning models, the architecture of a CNN can be broadly categorized into three types of layers, namely, an input layer, hidden layers, and an output layer. The primary purpose of the input layer is to receive the annotated input images and pass them to the subsequent hidden layers. The input layer of RoadweatherNet was designed to receive seven weather categories with a square image size. It is worth mentioning that with the increase in image size, the accuracy usually improves; however, it requires more computational power which results in longer training time. Therefore, to select the optimum image size, a sensitivity analysis was performed, in which the accuracy of the models was tested using different image sizes with 20 × 20 pixels increment at every iteration, as can be seen in Table 2 and Figure 2a. To compare the performance of the models, all the parameters were kept constant and the default training options were used. The testing accuracy of the model with input image size of 20 × 20 pixels was around 83%, which improved gradually with the increase in image size and saturated at an accuracy of 92% for the model trained with 100 × 100 pixels. After that, no significant improvement in accuracy was observed. Although the computational time increased by 4.1 times for this model compared with the base model, the use of this image size was justified considering its significantly superior performance over the other models trained with smaller image sizes.

Table 2.

Model Performance under Different Input Image Size and Number of Convolutional Layers

Parameter	Value	Testing accuracy (%)	Validation accuracy (%)	Relative training time (min)
Input image size	20	82.5	83.9	1.0
	40	84.4	85.2	1.4
	60	87.6	87.5	1.5
	80	88.2	89.0	2.3
	100	90.9	91.1	4.1
	120	91.4	91.7	5.3
	140	91.5	91.7	5.9
	160	91.2	91.8	7.7
Number of convolutional layers	1	14.2	14.5	1.0
	2	83.5	84.1	1.1
	3	88.9	89.8	1.2
	4	90.4	90.7	1.3
	5	88.3	89.0	1.4
	6	86.7	88.0	1.6

Figure 2.

Selection of: (a) input image size and (b) number of convolutional layers.

After taking the input images, RoadweatherNet then passed the images to the subsequent hidden layers, where the majority of the computations occurred. Hidden layers can be grouped into three types of layers, namely, convolutional, rectified linear unit (ReLU), and pooling layer. The convolutional layer is the main building block of a CNN and consists of several filters. These filters are moved across the input image in such a way that all the pixels are covered at least once and the dot product between the filter and the input is generated at every special position of the image. The resulting outputs from all the filters are then piled along the depth dimension to get the output of the convolutional layer. The main purpose of the convolutional layer is to extract features from the input image. Whereas the initial convolutional layers extract more generic features, as the network gets deep, the subsequent convolutional layers extract more refined features ( 29 ).

A higher number of convolutional layers usually improves the performance of a CNN model; however, it makes the network complex and deeper, which increases the training time. In addition, very deep neural networks are often subjected to overfitting ( 30 ). Therefore, to select the optimum number of convolutional layers, a sensitivity analysis was performed using input image size of 100 × 100 pixels and keeping all the parameters constant. The results are listed in Table 2 and are illustrated in Figure 2b. It was found that the CNN model with only one convolutional layer could not learn at all and produced a very poor testing accuracy of only 14%. Adding an extra convolutional layer significantly improved the performance of the model with an overall testing accuracy of 84%. The performance of the models reached saturation after four convolutional layers, as can be seen in Figure 2b. Therefore, four convolutional layers were selected for the development of RoadweatherNet. The first convolutional layer took the images as input and applied 16 filters with a size of 3 × 3 pixels. The next three convolutional layers applied 32, 64, and 128 filters of the same size, respectively. The number of filters of the convolutional layers was chosen as powers of two to maximize the usage of the graphics processing unit (GPU). It is worth mentioning that the size of the filter was also selected based on sensitivity analysis.

After each convolutional layer, a ReLU layer was applied to perform a threshold operation on each element of the inputs to ensure fast and consistent training of RoadweatherNet. The ReLU layer applies a simple function that converts only the negative values to zero and keeps the positive values unchanged ( 31 , 32 ). Except for the last ReLU layer, all ReLU layers were followed by a pooling layer. This layer was applied to decrease the amount of information generated from the preceding convolutional layer to ensure the passing of only the most essential information to the next layers ( 33 ). The last ReLU layer was then linked to a fully connected layer to produce an output vector with seven dimensions based on the number of weather categories. The next layer of RoadweatherNet was a softmax layer that assigns decimal probabilities to each of the output classes. Finally, the last layer of RoadweatherNet was a classification layer, which provided the final weather condition based on the probabilities ( 33 ). The architecture of RoadweatherNet is shown in Figure 3 and the description of each layer along with learnable parameters are listed in Table 3. It is worth mentioning that any parameter that needs to be optimized at each iteration during training is considered a learnable parameter. For CNN models, weights and biases at each layer of the network are the learnable parameters. ( 34 ). Using transfer learning, other researchers could use and test the capability of the proposed RoadweatherNet model in detecting weather conditions using their image dataset.

Table 3.

Parameters of RoadweatherNet

Name	Description	Activation	Learnable parameter	Total learnable parameter
Input	100 × 100 × 3 images with “zero-centered” normalization	100 × 100 × 3	na	0
Conv-1	16 3 × 3 × 3 convolutions with stride [1 1] and padding [1 1 1 1]	100 × 100 × 16	W = 3 × 3 × 3 × 16	448
			B = 1 × 1 × 32
ReLU-1	ReLU	100 × 100 × 16	na	0
Pooling-1	2 × 2 max pooling with stride [1 1] and padding [1 1 1 1]	50 × 50 × 16	na	0
Conv-2	32 3 × 3 × 16 convolutions with stride [1 1] and padding [1 1 1 1]	50 × 50 × 32	W = 3 × 3 × 3 × 32	4,640
			B = 1 × 1 × 16
ReLU-2	ReLU	50 × 50 × 32	na	0
Pooling-2	2 × 2 max pooling with stride [1 1] and padding [1 1 1 1]	25 × 25 × 32	na	0
Conv-3	64 3 × 3 × 32 convolutions with stride [1 1] and padding [1 1 1 1]	25 × 25 × 64	W = 3 × 3 × 32 × 64	18,496
			B = 1 × 1 × 64
ReLU-3	ReLU	25 × 25 × 64	na	0
Pooling-3	2 × 2 max pooling with stride [1 1] and padding [1 1 1 1]	12 × 12 × 64	na	0
Conv-4	128 3 × 3 × 32 convolutions with stride [1 1] and padding [1 1 1 1]	12 × 12 × 128	W = 3 × 3 × 64 × 128	73,856
			B = 1 × 1 × 128
ReLU-4	ReLU	12 × 12 × 128	na	0
Fully connected	7 fully connected layers	1 × 1 × 7	W = 7 × 18432	129,031
			B = 7 × 1
Softmax	Softmax layer	11 × 1 × 7	na	0
Output	Classification output	na	na	0

Note: W = weights; B = bias; ReLU = rectified linear unit; Conv = convolutional; na = not applicable.

Figure 3.

Architecture of RoadweatherNet.

Parameter Tuning and Validation

After the crafting of the RoadweatherNet architecture, the default parameters and training options were carefully updated by observing the training progress and validation accuracy. It is worth mentioning that 80% of the images were used for training and validation and the remaining 20% were used to test the accuracy of the developed model. During validation, the cost of RoadweatherNet was minimized using two optimizers: stochastic gradient descent with momentum (SGDM) and root mean square propagation; however, SGDM produced the best optimization. It is worth mentioning that cost is an overall measure of performance of a CNN model and is measured by calculating the difference between predicted class and true class. The best performance of a CNN model is achieved only when the cost is properly optimized ( 33 ).

The hyperparameters and training options of the proposed model were carefully tuned utilizing one of the most commonly used methods called grid search. Grid search is an approach of tuning hyperparameters that searches and evaluates a model through a manually specified subset of hyperparameters ( 35 , 36 ). The subset used for grid search in this study was created by carefully observing the training progress and accuracy of the proposed model. It is worth mentioning that based on the initial observation, some of the parameters did not have a significant influence on the model performance, and, therefore, were not included in the grid search to reduce the tuning time and computation resources. Table 4 lists the updated parameters for the developed weather detection model.

Table 4.

Tuning of Hyperparameters of RoadweatherNet

Parameters	Initial/default value	Final value after parameter tuning
Optimizer	SGDM	SGDM
Number of convolutional layers	4	4
Initial learning rate	0.01	0.0001
Learning rate drop period	10	5
Max epochs	30	15
Batch size	128	50
Factor for L2 regularization	0.0001	0.004

Note: SGDM = stochastic gradient descent with momentum.

Figure 4 illustrates the increase in accuracy and decrease in loss over the training iteration during validation using the best set of parameters, which shows that the overall validation accuracy of RoadweatherNet was around 10% at the initial iteration that improved gradually until it reached a final overall validation accuracy of around 92.5% at the final iteration after 15 epochs of training. Similarly, the loss was also decreased until it reached a final value of around 0.1, as can be seen in Figure 4. The training and validation took about 41 min to complete using a computer with an Intel Core i7-7500U 2.70 Ghz processor, 12 GB RAM, and a NVIDIA GeForce 940MX GPU.

Figure 4.

Training progress of RoadweatherNet.

The proposed RoadweatherNet model was trained, validated, and tested using the Deep Learning Toolbox™ in MATLAB^® version 9.8 (R2020a). The architecture of RoadweatherNet was devised using deep network designer apps within this toolbox. The pre-trained models (i.e., AlexNet, ResNet18, ResNet50, GoogLeNet, ShuffleNet, and SqueezeNet) were also modified using the Deep Learning Toolbox™. The basic architecture of the pre-trained models is provided in MATLAB^®; however, to use the pre-trained networks, transfer learning techniques have been applied, where several layers of the pre-trained networks were modified and tuned to achieve the weather detection task ( 37 ).

Performance Evaluation of RoadweatherNet

The quality of trained RoadweatherNet was evaluated using the test dataset for several performance indices, including overall accuracy, precision, recall, specificity, false positive rate (FPR), and false negative rate (FNR). These indices have been widely used in the literature to evaluate the performance of machine learning models ( 38 – 40 ). Accuracy represents the overall ability of a model to correct classification and can be described using Equation 1.

Accuracy = \frac{TP + TN}{TP + TN + FP + FN}

(1)

where $TP$ , $TN$ , $FP$ , and $FN$ represent true positive, true negative, false positive, and false negative, respectively. $TP$ corresponds to the number of images that are correctly classified, whereas $TN$ corresponds to the number of negative images that are accurately classified. Similarly, $FP$ represents the number of negative images that are wrongly classified as positive images and $FN$ represents the number of positive images that are misclassified as negative images.

Sensitivity, also called recall, represents the ability of the model to precisely classify positive images and can be expressed using Equation 2; FNR is the counterpart of sensitivity and can be defined by Equation 3.

Sensitivity (Recall) = \frac{TP}{TP + FN}

(2)

FNR = 1 - Sensitivity

(3)

Specificity corresponds to the proportion of negative images that were correctly identified and can be defined using Equation 4. In addition, the counterpart of specificity is called FPR, as defined by Equation 5.

Specificity (TNR) = \frac{TN}{TN + FP}

(4)

FPR = 1 - Specificity

(5)

Precision corresponds to the degree of correctly identified positive images out of all the predicted positive images ( 38 ) and can be described using Equation 6.

Precision = \frac{TP}{TP + FP}

(6)

In classification problems, a model might have high recall with low precision value and vice versa. The best performance is achieved only when a model has balanced high recall and precision values, which could be measured using an index called the F1 score, as described in Equation 7. The F1 score represents the harmonic mean of precision and recall; a high F1 score indicates that the model is balanced with high recall and precision value.

F 1 score = 2 \times \frac{Precision \times Recall}{Preceision + Recall}

(7)

Results and Discussions

After training and validation, the performance of RoadweatherNet was evaluated using a test dataset, consisting of 20% of the original images. Performance indices were calculated for each class, as listed in Table 5, and visualized using a confusion matrix, as illustrated in Figure 5. RoadweatherNet provided an impressive overall detection accuracy of 92.5%, which is in accordance with the accuracy (91.9%) found during validation. The highest recall value was found for the heavy rain image group, in which out of 500 test images, 95.4% of the images were correctly classified. The heavy snow and near fog image group also had a high degree of recall with values of 95.2% and 94.8%, respectively. The highest precision value of 96% was also found for the heavy snow image group, which indicated that out of 496 predicted snowy images, 96% were actually snow. One of the interesting observations is that the precision, as well as recall values, were found to be superior for extreme adverse weather conditions, such as heavy rain, heavy snow, and near fog. Driver behavior and vehicle performance, as well as visibility and road surface frictions, are affected more in such extreme weather conditions; therefore, a high degree of recall and precision under such conditions is crucial for developing reliable safety countermeasures. The lowest performance for recall was found for the light rain image group with a value of 86.8%, in which out of 500 test light rain images, 53 were wrongly classified as other images, as can be seen from Figure 5.

Table 5.

Performance Measure of Trained RoadweatherNet

Image category	Recall (%)	Precision (%)	Specificity (%)	FPR (%)	FNR (%)	F1 score (%)	Overall accuracy (%)
Clear	93.2	95.5	99.3	0.7	6.8	94.3	92.57
Light rain	86.8	93.5	99.0	1.0	13.2	90.0
Heavy rain	95.4	88.3	97.3	2.7	4.6	91.7
Light snow	93.0	92.6	98.7	1.3	7.0	92.8
Heavy snow	95.2	96.0	99.3	0.7	4.8	95.6
Distant fog	89.6	88.7	98.1	1.9	10.4	89.2
Heavy fog	94.8	93.9	99.0	1.0	5.2	94.3

Note: FPR = false positive rate; FNR = false negative rate.

Figure 5.

Confusion matrix of trained RoadweatherNet.

The lowest FPR was found for the clear image group with a value of only 0.7%. Considering the safety-related practical applications, a high degree of FPR of the clear image group is particularly hazardous because it would increase the risk by exposing drivers to adverse weather without warnings. RoadweatherNet produced a negligible amount of such hazardous misclassification, especially for extreme adverse weather. More specifically, only one heavy rain image was classified as clear weather and other extreme adverse weather (e.g., heavy snow and near fog) did not have any such hazardous classification, as can be seen from Figure 5. Considering the FNR, the lowest value (4.8%) was found for heavy snow and the highest value (13.2 %) was found for light rain. The FNR of the clear image group was also reasonably low with a value of 6.8%. It is worth mentioning that a high FNR of clear weather will provide frequent false warnings, which might lead to disrespect for the warning systems and might decrease the compliance rate.

One of the major objectives of this study was to develop a weather detection model that is easy to implement and requires less computational power with a high degree of detection accuracy. To effectively implement trajectory-level weather detection models in an emerging CV environment, it is extremely important to reduce the computational requirements because such weather detection will be applied mainly on a smartphone/tablet platform. Keeping in mind this research need, this study devised a simple CNN architecture (RoadweatherNet) capable of training and running on a computer with relatively less computational power such as a smartphone. The performance of the proposed RoadweatherNet model was compared with some existing pre-trained CNN models, namely, AlexNet, ResNet18, Resnet50, GoogLeNet, ShuffleNet, and SqueezeNet, which revealed that the proposed RoadweatherNet model required significantly less time to train than the existing pre-trained CNN models, as shown in Table 6. It is worth mentioning that most of the pre-trained networks have lots of layers with complex structures and require a relatively large input image size ( 26 , 41 –44), whereas the proposed RoadweatherNet model has only 15 layers with an input image size of 100 × 100 pixels. Although the pre-trained models provided marginally better performance, the simple structure of RoadweatherNet significantly reduced the training time. Relative training time was also determined by dividing the training time of the pre-trained models with the training time of RoadweatherNet, which revealed that the training times of AlexNet, ResNet18, ResNet50, GoogLeNet, ShuffleNet, and SqueezeNet were, respectively, about 1.5, 4.9, 15.4, 6.8, 4.1, and 2.1 times higher than RoadweatherNet. It is worth mentioning that after the training, the proposed RoadweatherNet model can detect weather conditions instantaneously. Keeping in mind the practical aspects, such as applications in a CV environment, this study suggests the use of RoadweatherNet when the weather detection model needs to train and run on a smartphone platform. However, for other cases, when the weather detection model could be trained and applied off-road, such as in a traffic management center (TMC), the use of ResNet50 is suggested because of its relatively higher detection performance compared with the other models.

Table 6.

Comparison of RoadweatherNet with Other Pre-Trained CNN Models

Name	Number of layers	Input image size (pixels)	Validation accuracy (%)	Testing accuracy (%)	Training time (min)	Relative training time
RoadweatherNet (proposed CNN architecture)	15	100 × 100	91.9	92.5	41.2	1
AlexNet	25	227 × 227	92.7	93.1	61.7	1.5
ResNet18	71	224 × 224	93.1	94.1	201.3	4.9
ResNet50	50	224 × 224	94.3	94.6	632.5	15.3
GoogLeNet	144	224 × 224	93.2	93.9	278.4	6.8
ShuffleNet	172	224 × 224	92.4	93.1	168.9	4.1
SqueezeNet	68	227 × 227	92.5	92.8	85.5	2.1

Note: CNN = convolutional neural network.

The performance of RoadweatherNet was evaluated against other existing methods of weather detection, as shown in Table 7. The proposed RoadweatherNet model achieved an overall detection accuracy of about 93%, which is higher than most of the previous weather detection models. However, previous research from the same author group achieved marginally higher detection performance. For instance, a snow detection system based on texture-based image features combined with machine learning techniques was proposed by the research team, which achieved 96% accuracy in detecting two levels of snow ( 45 ). Another study of the research team leveraged various neural network methods to develop a fog detection system and achieved an impressive overall accuracy of 98% ( 33 ). The reason for getting a high degree of accuracy in these two previous studies is that they considered only two categories of adverse weather, whereas the proposed RoadweatherNet model is capable of detecting seven levels of weather conditions.

Table 7.

Evaluation of RoadweatherNet Against Weather Other Detection Methods

Study	Data source	Method	Weather categories	Overall accuracy (%)
Mori et al. ( 11 )	In-vehicle camera, radar	Extinction coefficient	Light fog, moderate fog, and dense fog	84
Roser and Moosmann ( 10 )	In-vehicle camera	SVM	Clear, light rain, heavy rain	85
Yan et al. ( 9 )	In-vehicle vision system	AdaBoost	Sunny, cloudy, rainy	85
Chu et al. ( 46 )	Image2weather dataset, EC1M dataset, Flickr, Google maps	RF	Sunny, cloudy, snowy, rainy, foggy	70
Qian et al.( 47 )	Roadside camera, in-vehicle camera	Nearest neighbor, Naive Bayes Boost, SVM, AdaBoost, DT	Dry, wet, iced, snow-covered, snow-packed	88
Lu et al. ( 48 )	Stationary camera	CNN	Sunny, cloudy	91
Guerra et al. ( 49 )	Flickr, Pixabay, Wikimedia, RFS dataset	Pre-trained CNN	Sunny, cloudy, snowy, rainy, foggy	80
Ozcan et al. ( 50 )	Roadside CCTV, mobile camera, NDS	VGG16	Clear, rain-wet, snow	77
Gbeminiyi Oluwafemi and Zenghui ( 51 )	Google, Flickr, Getty Images, Yahoo	Said Ensemble Method	Cloudy, sunshine, rainy, sunrise	86
Zhao et al. ( 52 )	Open source images from the internet	CNN-LSTM	Sunny, cloudy, foggy, rainy, snowy	91
Ibrahim et al. ( 12 )	Google image	ResNet50, WeatherNet	Clear, rain, snow	93
Khan and Ahmed ( 45 )	SHRP2 NDS trajectory-level video data	SVM, KNN, RF	Clear, light snow, heavy snow	96
Ali et al. ( 53 )	SHRP2 NDS trajectory-level video data	ANN, DT, RF, GBT	Clear, snowy	89
Khan and Ahmed ( 33 )	SHRP2 NDS trajectory-level video data	DNN, RNN, LSTM, CNN	Clear, distant fog, near fog	98
RoadweatherNet	SHRP2 NDS trajectory-level video data	CNN, pre-trained CNN	Clear, light rain, heavy rain, light snow, heavy snow, distant fog, and near fog	93

Note: SVM = support vector machine; RF = random forest; GBT = gradient boosted trees; DT = decision tree; CNN = convolutional neural network; RNN = recurrent neural network; DNN = deep neural network; ANN = artificial neural network; LSTM = long short-term memory; EC1M = European city 1 million; RFS = rain fog snow; NDS = naturalistic driving study.

Conclusions

The primary objective of this study was to develop an affordable weather detection system capable of providing trajectory-level weather information with reasonable accuracy at road surface level in real time using only a single video camera. The video data used in this study were collected from the SHRP2 NDS database. More than 20,000 images were extracted from the NDS videos in different weather and roadway conditions and were annotated manually and grouped into seven weather categories: clear, light rain, heavy rain, light snow, heavy snow, distant fog, and near fog. To train the weather detection model, a novel CNN architecture, named RoadweatherNet, was developed and subsequently trained and validated with proper hyperparameter tuning. Using a separate test dataset, the performance of the RoadweatherNet was evaluated for several performance measures, such as precision, recall, specificity, F1 score, FNR, and FPR. The evaluation results revealed that RoadweatherNet can provide a high degree of performance in detecting seven weather categories with an overall detection accuracy of around 93%. The performance of RoadweatherNet was also compared with several pre-trained CNN models, which showed that the proposed detection model requires significantly less time to train.

The proposed RoadweatherNet model has numerous safety implementations, especially in a CV environment. To effectively implement trajectory-level weather detection models in an emerging CV environment, it is extremely important to reduce computational requirements. It is anticipated that mobile weather detection systems will be applied mainly on a smartphone/tablet platform. With the rapid advancement in connectivity, processing power, and camera quality of smartphones, the proposed weather detection model can be trained and run on smartphones of regular road users and maintenance vehicles, thus making it an effective way for collecting real-time road weather information.

The current practice of collecting weather information is mainly based on RWIS, which is expensive and, therefore, its widespread deployment is not financially viable. It is worth mentioning that, based on the U.S. Department of Transportation, the average total cost of deploying RWIS is around USD 52,000 per unit ( 54 ). In contrast, the weather detection system proposed in this study is easily implementable, does not require a lot of computational power, and only needs a single video camera. Therefore, the proposed weather detection model has the potential to become an affordable way of collecting accurate trajectory-level weather information.

The proposed RoadweatherNet model can be implemented using the existing infrastructures and facilities. Through crowdsourcing, weather data from regular road users can be shared with the TMCs, where data from all the vehicles from a road network can be weighted to get more representative and accurate real-time weather ( 33 ). The TMCs can leverage this information to provide appropriate warnings back to the road users and to develop a more accurate and reliable weather-based VSL system, especially on roadways with no RWIS. The methodology provided in this study could be extended to detect work zones, pedestrian, lane changes, motor vehicle crashes, and road closures. In a CV environment, this information can easily be shared with other road users and TMCs. Subsequently, based on real-time road information, the TMCs can disseminate appropriate warnings to road users. However, in extremely harsh weather conditions, such as snowstorms and blizzards, there might not be enough regular vehicles on the roads, which makes the above-mentioned concept for collecting weather data not always feasible. In such extreme weather conditions, maintenance vehicles, such as snowplows, can be equipped with smart devices to collect geocoded weather data that could be easily classified via mobile apps using the proposed RoadweatherNet model.

Furthermore, the proposed RoadweatherNet model can be applied to enhance the existing trajectory-level data, such as SHRP2 NDS. Many studies have conducted in-depth investigations of the impact of weather on driver behavior and performance using this dataset ( 55 , 56 ); however, these studies adopted manual annotation to detect weather, which is time-consuming and often subjected to bias. RoadweatherNet can be applied to the SHRP2 NDS data to create accurate and refined weather variables, which will be highly beneficial for researchers and transportation practitioners for investigating the effect of weather and for deploying necessary countermeasures.

Footnotes

Acknowledgements

This work was conducted under the second Strategic Highway Research Program (SHRP2), which is administrated by the Transportation Research Board of the National Academies of Sciences, Engineering, and Medicine.

Author Contributions

The authors confirm the contribution to the paper as follows: study conception and design: Md Nasim Khan and Mohamed M. Ahmed; data collection: Md Nasim Khan, and Mohamed M. Ahmed; analysis and interpretation of results: Md Nasim Khan, and Mohamed M. Ahmed; draft manuscript preparation: Md Nasim Khan, and Mohamed M. Ahmed. All authors reviewed the results and approved the final version of the manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was sponsored by the Federal Highway Administration in cooperation with the American Association of State Highway and Transportation Officials (AASHTO) and the Wyoming Department of Transportation (WYDOT). Grant number: RS08217.

References

Federal Highway Administration. How Do Weather Events Impact Roads?FHWA Road Weather Management. http://www.ops.fhwa.dot.gov/weather/q1_roadimpact.htm. Accessed September 20, 2019.

Perry

A. H.

Symons

Highway Meteorology. Taylor & Francis Books, Inc, Boca Raton, FL, 2003.

Andrey

Mills

Leahy

Suggett

Weather as a Chronic Hazard for Road Transportation in Canadian Cities. Natural Hazards, Vol. 28, 2003, pp. 319–343.

Khan

M. N.

Ghasemzadeh

Ahmed

M. M.

Investigating the Impact of Fog on Freeway Speed Selection using the SHRP2 Naturalistic Driving Study Data. Transportation Research Record: Journal of the Transportation Research Board, 2018. 2672: 93–104. https://doi.org/10.1177/0361198118774748.

Hautiére

Tarel

J. P.

Lavenant

Aubert

Automatic Fog Detection and Estimation of Visibility Distance through Use of an Onboard Camera. Machine Vision and Applications, Vol. 17, No. 1, 2006, pp. 8–20. https://doi.org/10.1007/s00138-005-0011-1.

Hase

Miyake

Yoneda

Real-Time Snowfall Noise Elimination. Proc., 1999 International Conference on Image Processing (Cat. 99CH36348), Kobe, Japan, Vol. 2, No. M, IEEE, 1999, pp. 406–409. https://doi.org/10.1109/ICIP.1999.822927.

Garg

Nayar

S. K.

Detection and Removal of Rain from Videos. Proc., 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004., Washington, D.C., Vol. 1, IEEE, 2004, pp. 528–535.

Pomerleau

Visibility Estimation from a Moving Vehicle using the RALPH Vision System. Proc., IEEE Conference on Intelligent Transportation Systems, Boston, MA, IEEE, 1997, pp. 906–911.

Yan

Luo

Zheng

Weather Recognition Based on Images Captured by Vision System in Vehicle. In International Symposium on Neural Networks: Advances in Neural Networks ( Yu

Zhang

, eds.), Wuhan, China, 2009, Springer, Berlin, Heidelberg, pp. 390–398. https://doi.org/10.1007/978-3-642-01513-7_42.

10.

Roser

Moosmann

Classification of Weather Situations on Single Color Images. 2008 IEEE Intelligent Vehicles Symposium, Eindhoven, The Netherlands, IEEE, 2008, pp. 798–803. https://doi.org/10.1109/IVS.2008.4621205.

11.

Mori

Takahashi

Ide

Murase

Miyahara

Tamatsu

Fog Density Recognition by In-Vehicle Camera and Millimeter Wave Radar. International Journal of Innovative Computing, Information and Control, Vol. 3, No. 5, 2007, pp. 1173–1182.

12.

Ibrahim

M. R.

Haworth

Cheng

WeatherNet: Recognising Weather and Visual Conditions from Street-Level Images using Deep Residual Learning. International Journal of Geo-Information, Vol. 8, No. 12, 2019, p. 549.

13.

Das

Khan

M. N.

Ahmed

M. M.

Nonparametric Multivariate Adaptive Regression Splines Models for Investigating Lane-Changing Gap Acceptance Behavior Utilizing Strategic Highway Research Program 2 Naturalistic Driving Data. Transportation Research Record: Journal of the Transportation Research Board, 2020. 2674: 223–238.

14.

Das

Ahmed

M. M.

Exploring the Effect of Fog on Lane-Changing Characteristics Utilizing the SHRP2 Naturalistic Driving Study Data. Journal of Transportation Safety & Security, 2019, pp. 1–26. https://doi.org/10.1080/19439962.2019.1645777.

15.

Das

Ahmed

M. M.

Ghasemzadeh

Using Trajectory-Level SHRP2 Naturalistic Driving Data for Investigating Driver Lane-Keeping Ability in Fog: An Association Rules Mining Approach. Accident Analysis and Prevention, Vol. 129, 2019, pp. 250–262. https://doi.org/10.1016/j.aap.2019.05.024.

16.

Das

Ghasemzadeh

Ahmed

M. M.

Analyzing the Effect of Fog Weather Conditions on Driver Lane-Keeping Performance using the SHRP2 Naturalistic Driving Study Data. Journal of Safety Research, Vol. 68, 2019, pp. 71–80. https://doi.org/10.1016/j.jsr.2018.12.015.

17.

Das

Ahmed

M. M.

Prediction of Lane Change Maneuvers using the SHRP2 Naturalistic Driving Study Data: A Machine Learning Approach. Transportation Research Record: Journal of the Transportation Research Board, In Press, 2021.

18.

Hankey

J. M.

Perez

M. A.

McClafferty

J. A.

Description of the SHRP2 Naturalistic Database and the Crash, Near-Crash, and Baseline Data Sets. Virginia Tech Transportation Institute, 2016.

19.

Smadi

Hawkins

Hans

Bektas

B. A.

Knickerbocker

Nlenanya

Souleyrette

Hallmark

SHRP 2 Report: S2-S04A-RW-1: Naturalistic Driving Study: Development of the Roadway Information Database. Transportation Research Board of the National Academies, Washington, D.C., 2015.

20.

Campbell

K. L.

The SHRP 2 Naturalistic Driving Study: Addressing Driver Performance and Behavior in Traffic Safety. TR News, No. 282, 2012, pp. 30–35.

21.

Ahmed

M. M.

Ghasemzadeh

Hammit

Khan

M. N.

Das

Ali

Young

R. K.

Eldeeb

Driver Performance and Behavior in Adverse Weather Conditions: An Investigation using the SHRP2 Naturalistic Driving Study Data-Phase 2. Final Report WY-18/05F. Wyoming Department of Transportation, 2018.

22.

Ghasemzadeh

Hammit

B. E.

Ahmed

M. M.

Eldeeb

Complementary Methodologies to Identify Weather Conditions in Naturalistic Driving Study Trips: Lessons Learned from the SHRP2 Naturalistic Driving Study & Roadway Information Database. Safety Science, Vol. 119, 2019, pp. 21–28. https://doi.org/10.1016/j.ssci.2019.01.006.

23.

Ahmed

M. M.

Young

R. K.

Ghasemzadeh

Hammit

Ali

Khan

Das

Eldeeb

Implementation of SHRP2 Results within the Wyoming Connected Vehicle Variable Speed Limit System: Phase 2 Early Findings Report and Phase 3 Proposal. Laramie, Wyoming, 2017.

24.

Ahmed

M. M.

Abdel-Aty

Lee

Real-Time Assessment of Fog-Related Crashes using Airport Weather Data : A Feasibility Analysis. Accident Analysis and Prevention, Vol. 72, 2014, pp. 309–317. https://doi.org/10.1016/j.aap.2014.07.004.

25.

Khan

M. N.

Das

Ahmed

M. M.

Non-Parametric Association Rules Mining and Parametric Ordinal Logistic Regression for an In-Depth Investigation of Driver Speed Selection Behavior in Adverse Weather using SHRP2 Naturalistic Driving Study Data. Transportation Research Record: Journal of the Transportation Research Board, 2020. 2674: 101–119.

26.

Krizhevsky

Sutskever

Hinton

G. E.

ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems, Vol. 25, 2012, pp. 1097–1105.

27.

Simonyan

Zisserman

Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv preprint arXiv:1409.1556, 2015.

28.

Sermanet

Eigen

Zhang

Mathieu

Fergus

Lecun

OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks. In Proc., 2nd International Conference on Learning Representations, ICLR 2014, Banff, Canada, 2014.

29.

Liu

Bennamoun

Sohel

Boussaid

Computer Vision for Human-Machine Interaction. In Computer Vision For Assistive Healthcare ( Leo

Farinella

G. M.

, eds.), Elsevier, Amsterdam, The Netherlands, 2018, pp. 127–145.

30.

Yeoh

Y. J.

Morie

Tamukoh

A Hardware-Oriented Dropout Algorithm for Efficient FPGA Implementation. Neural Information Processing—ICONIP 2017, Vol. 10639 LNCS, 2017, pp. 821–829. https://doi.org/10.1007/978-3-319-70136-3_87.

31.

Goodfellow

Bengio

Courville

Deep Learning. MIT Press, Cambridge, MA, 2016.

32.

Nair

Hinton

G. E.

Rectified Linear Units Improve Restricted Boltzmann Machines. Proc., 27th International Conference on Machine Learning, Haifa, Israel, 2010.

33.

Khan

M. N.

Ahmed

M. M.

Trajectory-Level Fog Detection Based on In-Vehicle Video Camera with TensorFlow Deep Learning Utilizing SHRP2 Naturalistic Driving Data. Accident Analysis and Prevention, Vol. 142, 2020, p. 105521. https://doi.org/10.1016/j.aap.2020.105521.

34.

Deeplizard. Deep Learning & Deep Learning Fundamentals. https://deeplizard.com/learn/video/gZmobeGL0Yg. Accessed November 30, 2020.

35.

Chicco

Ten Quick Tips for Machine Learning in Computational Biology. BioData Mining, Vol. 10, No. 1, 2017, pp. 1–17. https://doi.org/10.1186/s13040-017-0155-3.

36.

Das

Khan

M. N.

Ahmed

M. M.

Detecting Lane Change Maneuvers using SHRP2 Naturalistic Driving Data: A Comparative Study Machine Learning Techniques. Accident Analysis and Prevention, Vol. 142, 2020, p. 105578. https://doi.org/10.1016/j.aap.2020.105578.

37.

MathWorks. Convolutional Neural Network. https://www.mathworks.com/discovery/convolutional-neural-network-matlab.html. Accessed November 25, 2020.

38.

Maeda-Gutiérrez

Galván-Tejada

C. E.

Zanella-Calzada

L. A.

Celaya-Padilla

J. M.

Galván-Tejada

J. I.

Gamboa-Rosales

Luna-García

Magallanes-Quintanar

Guerrero Méndez

C. A.

Olvera-Olvera

C. A.

Comparison of Convolutional Neural Network Architectures for Classification of Tomato Plant Diseases. Applied Sciences, Vol. 10, No. 4, 2020, p. 1245. https://doi.org/10.3390/app10041245.

39.

Ardakani

A. A.

Kanafi

A. R.

Acharya

U. R.

Khadem

Mohammadi

Application of Deep Learning Technique to Manage COVID-19 in Routine Clinical Practice using CT Images: Results of 10 Convolutional Neural Networks. Computers in Biology and Medicine, Vol. 121, 2020, p. 103795. https://doi.org/10.1016/j.compbiomed.2020.103795.

40.

Osman

O. A.

Hajij

Bakhit

P. R.

Ishak

Prediction of Near-Crashes from Observed Vehicle Kinematics using Machine Learning. Transportation Research Record: Journal of the Transportation Research Board, 2019. 2673: 463–473.

41.

Zhang

Ren

Sun

Deep Residual Learning for Image Recognition. Proc., IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, Vol. 2016, IEEE, 2016, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90.

42.

Szegedy

Liu

Jia

Sermanet

Reed

Anguelov

Erhan

Vanhoucke

Rabinovich

Going Deeper with Convolutions. Proc., IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Boston, MA, IEEE, 2015, pp. 1–9. https://doi.org/10.1109/CVPR.2015.7298594.

43.

Zhang

Zhou

Lin

Sun

ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. Proc., IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, IEEE, 2018, pp. 6848–6856. https://doi.org/10.1109/CVPR.2018.00716.

44.

Iandola

F. N.

Han

Moskewicz

M. W.

Ashraf

Dally

W. J.

Keutzer

SqueezeNet: AlexNet-Level Accuracy with 50x Fewer Parameters and <0.5MB Model Size. arXiv:1602.07360v4, 2016.

45.

Khan

M. N.

Ahmed

M. M.

Snow Detection using In-Vehicle Video Camera with Texture-Based Image Features Utilizing K-Nearest Neighbor, Support Vector Machine, and Random Forest. Transportation Research Record: Journal of the Transportation Research Board, 2019. 2673: 221–232.

46.

Chu

W. T.

Zheng

X. Y.

Ding

D. S.

Image2weather: A Large-Scale Image Dataset for Weather Property Estimation. IEEE 2nd International Conference on Multimedia Big Data, Taipei, Taiwan, IEEE, 2016, pp. 137–144. https://doi.org/10.1109/BigMM.2016.9.

47.

Qian

Almazan

E. J.

Elder

J. H.

Evaluating Features and Classifiers for Road Weather Condition Analysis. Proc., International Conference on Image Processing, ICIP, Phoenix, AZ, IEEE, 2016, pp. 4403–4407. https://doi.org/10.1109/ICIP.2016.7533192.

48.

Lin

Jia

Tang

C. K.

Two-Class Weather Classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 12, 2017, pp. 2510–2524. https://doi.org/10.1109/TPAMI.2016.2640295.

49.

Guerra

J. C. V.

Khanam

Ehsan

Stolkin

McDonald-Maier

Weather Classification: A New Multi-Class Dataset, Data Augmentation Approach and Comprehensive Evaluations of Convolutional Neural Networks. Proc., 2018 NASA/ESA Conference on Adaptive Hardware and Systems (AHS), Edinburgh, IEEE, 2018, pp. 305–310.

50.

Ozcan

Sharma

Knickerbocker

Merickel

Hawkins

Rizzo

Road Weather Condition Estimation using Fixed and Mobile Based Cameras. Advances in Computer Vision. CVC 2019. Advances in Intelligent Systems and Computing, Vol. 943, 2020, pp. 192–204.

51.

Gbeminiyi Oluwafemi

Zenghui

Multi-Class Weather Classification from Still Image using Said Ensemble Method. Proc., 2019 Southern African Universities Power Engineering Conference/Robotics and Mechatronics/Pattern Recognition Association of South Africa, Bloemfontein, South Africa, IEEE, 2019, pp. 135–140. https://doi.org/10.1109/RoboMech.2019.8704783.

52.

Zhao

Wang

A CNN–RNN Architecture for Multi-Label Weather Recognition. Neurocomputing, Vol. 322, 2018, pp. 47–57. https://doi.org/10.1016/j.neucom.2018.09.048.

53.

Ali

E. M.

Khan

M. N.

Ahmed

Mohamed

. Real-Time Snowy Weather Based on Machine Vision and Vehicle Kinematics: A Non-Parametric Data Fusion Technique. Presented at 99th Annual Meeting of the Transportation Research Board, 2020.

54.

U.S. Department of Transportation. RITA | ITS | Costs: Road Weather Information System (RWIS) Environmental Sensor Station (ESS).

55.

Ghasemzadeh

Ahmed

M. M.

Exploring Factors Contributing to Injury Severity at Work Zones Considering Adverse Weather Conditions. IATSS Research, Vol. 43, No. 3, 2019, pp. 131–138. https://doi.org/10.1016/j.iatssr.2018.11.002.

56.

Ghasemzadeh

Ahmed

M. M.

Drivers’ Lane-Keeping Ability in Heavy Rain: Preliminary Investigation using SHRP 2 Naturalistic Driving Study Data. Transportation Research Record: Journal of the Transportation Research Board, 2017. 2663: 99–108.