Effectiveness of Data Imbalance Treatment in Weather-Related Crash Severity Analysis

Abstract

Accurate predictive modeling is often hindered by the prevalent issue of class imbalance within weather-related crash datasets. To address this critical challenge, this study introduces a novel and tailored synthetic data generation technique aimed at effectively handling nominal predictors specific to weather-related crash cases in North Carolina. Data treatment techniques such as the synthetic minority over-sampling technique-nominal (SMOTE-N) and adaptive synthetic-nominal (ADASYN-N) were investigated in this study. A comprehensive comparison of these data treatment techniques is conducted using two prominent machine learning models: the bagging algorithm (random forest [RF]) and the boosting algorithm (extreme gradient boosting [XGBoost]). The findings indicate that the effectiveness of data treatment varies with the severity level and the algorithm used. The ADASYN-N technique was observed to be highly effective for severe and moderate injury crash prediction using both the RF and XGBoost models, while the control dataset and SMOTE-N demonstrated notable performance in property damage only crash prediction using both the RF and XGBoost models. The findings from evaluating the performance of these models with data treatment methods serve as a benchmark for practitioners in selecting appropriate synthetic sample generation techniques, consequently facilitating the development of more accurate crash severity prediction models and contributing to enhanced traffic safety strategies.

Keywords

data and data science machine learning (artificial intelligence)statistical methods data analysis safety modeling and forecasting

Weather-related vehicle crashes are particularly hazardous because of their unpredictable nature. Not only can such crashes cause serious physical harm, but they can also have a substantial economic impact. Research indicates that weather-related crashes in the U.S.A. in 2014 cost approximately US$46 billion ( 1 ). Furthermore, weather-related crashes are more likely to be fatal than other types of crashes ( 2 ), with statistics from the same year showing that they accounted for 21% of all fatalities ( 1 ). The psychological repercussions of such crashes can be long-lasting and have a substantial economic impact in addition to reducing the quality of life of those affected ( 3 ).

To mitigate the impact of weather-related crashes, practitioners must be able to accurately predict and identify the factors associated with their severity levels. However, because of the highly imbalanced nature of crash data, traditional machine learning models are not robust enough to effectively classify crash severity ( 4 , 5 ). Thus, it is crucial to address the problem of class imbalance before predicting severity classes on weather-related crash datasets.

Common approaches, such as using only one classifier or the majority class to predict all classes, often lead to inaccurate results ( 6 ). Two of the most popular approaches are undersampling and oversampling techniques. In undersampling, the number of instances from the majority weather-related crash severity class are reduced to match the frequency of the minority class ( 4 , 5 ). However, there is a risk of losing vital information about the majority class. On the other hand, oversampling involves replicating weather-related crash events from the minority class to increase their representation in the dataset. In the context of crash events, the rare occurrence of fatal/severe crashes gives room for overfitting in the case of the oversampling technique ( 4 ).

Because of the weakness of undersampling and oversampling, the synthetic minority oversampling technique (SMOTE) ( 7 ) and adaptive synthetic (ADASYN) ( 8 ) methods are used to balance the data, thus avoiding the risk of overfitting or data loss ( 4 , 6 ). The SMOTE works by generating new events that are interpolations of existing minority class events, thereby improving the diversity of instances without simply duplicating data. The ADASYN technique, however, takes this a step further by adaptively generating minority data points based on their density distribution with more weight on the most difficult-to-learn data instances. However, a significant limitation arises when dealing with a dataset that is entirely composed of nominal predictors. These methods struggle because they rely on the concept of distance or interpolation between points.

Modern data sources such as weather-related crash data have resulted in complex datasets that are cumbersome to model with classic statistical methods ( 9 – 12 ). On the other hand, machine learning models are robust enough to deal with complex datasets with high-dimensional features. While these models are becoming popular in predicting traffic crashes, the presence of data imbalance in weather-related crash severity datasets often introduces unintended biases to resultant machine learning models, making it essential to address these imbalances before beginning the model training process ( 13 , 14 ). This problem becomes even more complex with a dataset containing predominantly nominal predictors.

Therefore, this study aims to assess the efficacy of different data treatment techniques and their influence on machine learning models, particularly focusing on handling nominal predictors. Specifically, data treatment techniques involving two synthetic approaches, namely the synthetic minority over-sampling technique-nominal (SMOTE-N) and adaptive synthetic-nominal (ADASYN-N) technique, are investigated for nominal predictors. The study employed two crash severity classification models, the bagging algorithm (random forest [RF]) and the boosting algorithm (extreme gradient boosting [XGBoost]). Evaluating the performance of these models with different data treatment methods provides insights into how such techniques can enhance the accuracy of machine learning models when applied to weather-related traffic crash data.

Background and Motivation

Weather-Related Crashes

Research in crash severity prediction, focusing specifically on weather-related traffic crashes, is vital for understanding how various factors influence the severity of such incidents. Accurate crash severity prediction is crucial for emergency response planning, resource allocation, and the development of effective countermeasures to reduce the impact of these crashes. Numerous studies have been conducted to investigate the factors affecting the crash severity of weather-related traffic crashes ( 15 – 19 ). These factors can be broadly categorized into three main groups, driver-related, vehicle-related, and environmental-related factors, with emphasis on weather conditions ( 19 – 21 ). Driver-related factors include driver age, gender, impairment (e.g., alcohol or drug use), distraction, and fatigue ( 22 ). Vehicle-related factors encompass vehicle type, size, and safety features. Environmental-related factors consist of road conditions, weather conditions, lighting, and traffic characteristics. Understanding the impact of these factors is crucial for developing effective crash severity prediction models. The selection of predictor variables significantly affects the accuracy and interpretability of crash severity prediction models ( 23 ). Previous research has identified a wide range of potential predictor variables, including emphasizing weather conditions and their interactions with driver characteristics (e.g., age and gender), roadway attributes (e.g., speed limit and road type), and other environmental conditions (e.g., lighting), and crash-specific variables (e.g., crash type and time of day) ( 24 – 27 ). Das et al. ( 9 , 10 ) identified other contributing factors, such as locality and road-specific features. Feature selection techniques, such as stepwise regression, principal component analysis, and recursive feature elimination, have been employed to identify the most influential variables for crash severity prediction.

Evaluation metrics play a crucial role in assessing the performance of crash severity prediction models. Commonly used metrics include accuracy, precision, recall, F1 score, and the area under the curve–receiver operating characteristic curve (AUC-ROC) ( 26 ). In addition, confusion matrix analysis provides insights into model performance across different severity levels. Studies have compared the performance of different models and techniques, highlighting the strengths and limitations of each approach. Higher accuracy and AUC-ROC values indicate better model performance ( 28 ).

Accurate crash severity prediction models have practical implications for road safety management. For instance, emergency response systems can use predicted severity levels to dispatch appropriate medical personnel and resources during adverse weather conditions. Transportation agencies can prioritize weather-specific road safety improvements and allocate funding based on predicted crash severity hotspots ( 29 ). Furthermore, crash severity prediction models can aid in the development of intelligent transportation systems and advanced driver assistance systems to prevent or mitigate risks associated with weather-related crashes.

Approaches for Addressing the Imbalance Problem in Crash Data

The phenomenon of class imbalance has been identified as a significant challenge within the realm of statistical modeling and machine learning methodologies. The disproportionate representation of classes often results in the dominance of the majority class, thereby undermining the reliability of the predictions pertaining to the minority class. This accentuates the pivotal role that data plays in these modeling frameworks, given their data-dependent nature. This aspect is particularly salient within the context of traffic crash data analysis. Infrequent crashes, while less common in the dataset, are typically associated with higher severity and concomitant socioeconomic costs ( 14 ). Consequently, the accurate prediction of these minority class crash events assumes critical importance ( 23 ).

Iranitalab and Khattak ( 30 ) undertook a comparative analysis of four distinct statistical and machine learning methodologies, namely the multinomial logit (MNL), nearest neighbor classification (NNC), support vector machine (SVM), and RF, in the context of traffic crash severity prediction. The findings revealed that machine learning algorithms, encompassing the NNC, SVM, and RF, generally outperformed the traditional MNL model with respect to prediction accuracy. This result was further strengthened by a recent report by Ogungbire et al. ( 31 ) comparing deep learning, machine learning, and statistical models for weather-related crash severity prediction. Nonetheless, a common challenge encountered by both studies pertained to the classification of infrequent severe injury crashes, such as those resulting in disabling or fatal injuries. Therefore, there is a pressing need to implement data treatment techniques to mitigate the issues arising from the imbalance in crash data and investigate their effectiveness on different machine learning algorithms.

To address the prevalent issue of class imbalance in crash severity analysis, minority oversampling techniques such as the SMOTE and the ADASYN technique have been developed and explored in the past ( 7 , 8 ). The SMOTE generates synthetic instances of the minority class by employing a bootstrapping approach in combination with the k-nearest neighbors’ algorithm, which has been widely used in road safety analysis. Similarly, the ADASYN technique adopts a density distribution-based measure to determine the required number of samples from the minority class, in contrast to the SMOTE’s uniform weight assignment. Besides oversampling the minority class, undersampling of the majority class has also been incorporated in crash analysis. For instance, Fiorentini and Losa ( 32 ) utilized a random undersampling of the majority class strategy to develop models predicting crash type, specifically focusing on two levels of collision severity: property damage only (PDO) and fatal + injury crashes. Their findings demonstrated that the random undersampling of the majority class significantly improved the prediction performance for the minority class compared to the model trained on imbalanced data. Table 1 provides a summary of selected weather-related crash severity studies ( 33 – 38 ) that have explored machine learning techniques in the past.

Table 1.

Summary of Previous Weather-Related Crash Severity Studies Using Machine Learning

Authors	Independent variables	Model used	Findings
Mondal et al. ( 33 )	Manner of crash, weather condition, route class, hour of the day, type of the intersection, light condition, road surface condition, work zone related, day of the week, and school bus	RF and Bayesian additive regression trees (BART)	Performance of the RF model was found to be very good, with a higher skill score of 0.73 compared to 0.61 for BART
Zhang et al. (37)	Crash time, vehicle type, and roadway feature	Spatial multinomial logit model	Observed the importance of the spatial terms in weather-related crashes, therefore suggesting the existence of spatial correlation.
Zeng et al. ( 34 )	Driver type, vehicle type, vehicle registered province, crash time, crash type, response time of emergency medical service, and horizontal curvature and vertical grade of the crash location	Bayesian spatial generalized ordered logit model	An increase in precipitation is associated with a decrease in the probability of light and severe injury crashes, and an increase in the probability of medium crashes
Call et al. ( 35 )	Elevation, slope, location, time, and severity as well as over 25 other fields related to the crash, such as single-vehicle, weather-related, and driving under the influence	Logistic regression	Adverse weather-related crashes were most common in the winter season and correlated with snowfall; excessive speed was more likely in these crashes, but they were generally less severe; an increase in roadway slope increases the likelihood of crashes
Ghasemzadeh and Ahmed ( 36 )	Speed, age, land use, crash type, driving under the influence, lighting condition, weather, road type, traffic control devices, vehicle age, vehicle type, construction type, location, and surface condition	Probit–classification tree	Presence of traffic control device and lighting conditions are significant interacting variables in the developed complementary crash severity model for work zone weather-related crashes
Ashifur Rahman et al. ( 38 )	Lightning condition, annual average daily traffic, roadway type, functional class, area type, roadway alignment, shoulder width, and posted speed limit	Cluster correspondence analysis	Variety of attributes linked to speed limit, lighting condition, alignment, area type, manner of collision, restraint usage, and alcohol/drug can have associative impacts on fatal/severe injury and moderate injury crashes in the State of Louisiana

Note: RF = random forest.

In summary, class imbalance is a major issue in statistical and machine learning methods, which can be addressed by using data treatment methods such as the SMOTE-N and ADASYN-N technique. However, there is a challenge when the dataset has predominantly nominal predictors with few or no numeric predictors. This study aims to address a critical gap in the existing literature by introducing a novel technique for synthetic data generation specifically tailored for nominal predictors in the context of crash severity analysis. It is hypothesized that these data treatment techniques will improve the prediction accuracy when trained on machine learning models compared to the control or raw dataset. The proposed technique is expected to improve the accuracy of weather-related crash severity classifications. The findings from this study serve as valuable insights into the impact of these techniques on the accuracy and robustness of machine learning models when applied to crash data, contributing to more reliable and effective crash severity analysis methodologies.

Methodology

Data Used and Processing

The data used in this study was obtained from the Highway Safety Information System (HSIS). The HSIS is a comprehensive, multi-state database encompassing a wide array of information. It includes detailed crash data, extensive roadway inventory details, traffic volume statistics, and specialized inventory data that covers aspects such as intersections, interchanges, and roadside hardware elements. Crashes that occurred in North Carolina between January 1, 2015 and December 31, 2017 were extracted. Crashes are reported using case numbers and observations with the same number indicating that the vehicles involved are part of the same crash incident. To gain a thorough understanding of the crash occurrence process, Washington and Haque ( 39 ) argued that crashes with different causes should be modeled separately. Therefore, only weather-related crashes were extracted, while crashes occurring under clear weather conditions were excluded in this study.

The final dataset only included crashes that happened under non-clear weather conditions (i.e., cloudy, rain, fog/smog, sleet/hail/freezing rain/drizzle, severe crosswinds, or blowing sand conditions, as described in the crash reports). Crash severity is defined in the HSIS database as five different levels (i.e., fatal crashes, injury type A, injury type B, injury type C, and no injury/PDO). In this study, the crash severity was re-categorized into three levels, that is, severe injury (fatal and injury type A), moderate injury (injury type B and injury type C) and PDO/no injury. The re-grouping is done to simplify the analysis by reducing complexity while ensuring that each category is meaningful. It is also intended to capture similar outcomes where closely related interventions are required. Overall, a total of 238,252 weather-related crashes were recorded within the study period with 2952 severe injury crashes, 71,688 moderate injury crashes, and 163,612 PDO crashes.

The summary statistics gives information about the counts and percentages of different types of weather conditions for severe injury, moderate injury, and PDO injury crashes. Cloudy weather is the most common weather condition in all three categories, with 63.4% of severe injury crashes, 57.6% of moderate injury crashes, and 56.6% of no injury crashes occurring under cloudy conditions. Rain is the second most common weather condition, followed by snow, fog, smog, and smoke. Sleet, hail, and freezing rain/drizzle are less common, with blowing sand and dirt being the least common weather condition for all three severity types.

The dataset presented in Table 2 underwent initial preprocessing to facilitate model selection and evaluation. Initially, the comprehensive dataset was partitioned into two distinct subsets: a training set and a testing set. The formation of the test dataset was accomplished through the methodology of random sampling, ensuring a distribution proportional to the size of each constituent class. Consequently, 20% of the data, uniformly distributed across classes, was extracted to constitute the testing set. A random seed was set to ensure that when the data is resampled, it does not select part of the samples that was used in training the dataset. The residual 80% of the data served as the control training set, subject to diverse treatment methodologies to generate corresponding treated training datasets.

Table 2.

Descriptive Statistics of Variables for Analysis

Category	Description	Severe injury (SI)		Moderate injury (MI)		PDO
Category	Description	Count	%	Count	%	Count	%
Weather condition (weather)
1	Cloudy	1871	63.4	41,293	57.6	92,663	56.6
2	Rain	893	30.3	26,828	37.4	60,374	36.9
3	Snow	41	1.4	1085	1.5	4296	2.6
4	Fog, smog, smoke	104	3.5	1247	1.7	2786	1.7
5	Sleet, hall, freezing rain/drizzle	39	1.3	1196	1.7	3324	2
6	Severe crosswinds	3	0.1	31	0.04	132	0.08
7	Blowing sand, dirt	1	0.03	8	0.01	37	0.02
Contributing factor of the crash (contfac)
1	No contributing factors	1191	40.3	32,095	44.8	75,111	45.9
2	Disregarding signs or signals	98	3.3	2124	3.0	2492	1.5
3	Exceeded safe speed/speed limit or fail to reduce speed	548	18.6	17,302	24.1	41,651	25.5
4	Improper turn or right turn on red	17	0.6	730	1.0	1979	1.2
5	Crossed centerline, improper lane change, or use of an improper lane	222	7.5	2135	3.0	6102	3.7
6	Overcorrected, oversteered, improper passing, or improper backing	77	2.6	1503	2.1	3019	1.8
7	Failing to yield to the right-of-way or driver inattention	267	9.0	9147	12.8	18,870	11.5
8	Operating too closely, aggressive driving, or alcohol use	429	14.5	3768	5.3	6578	4.0
9	Visibility obstruction or defective equipment	16	0.5	471	0.7	1275	0.8
10	Other/unable to determine	87	2.9	2413	3.4	6535	4.0
Road surface condition (roadsurf)
1	Dry	1290	43.7	27,329	38.1	60,413	36.9
2	Wet, presence of water (standing/moving)	1577	53.4	41,617	58.1	93,882	57.4
3	Ice, snow, slush	81	2.7	2708	3.8	9248	5.7
4	Sand, mud, dirt, gravel, fuel, or oil	4	0.1	34	0.05	69	0.04
Functional class of road (fclass)
1	Principal arterial—interstate, freeways, and expressways	402	13.6	12,617	17.6	35,002	21.4
2	Principal arterial—other	682	23.1	22,955	32.0	50,092	30.6
3	Minor arterial	693	23.5	18,303	25.5	39,919	24.4
4	Major collector	789	26.7	11,795	16.5	24,314	14.9
5	Local	386	13.1	6018	8.4	14,285	8.7
Location type (intersecti)
0	Non-intersection	2516	85.2	57,486	80.2	138,677	84.8
1	Intersection	436	14.8	14,202	19.8	24,935	15.2
Light condition (lightnew)
1	Daylight	1727	58.5	51,558	71.9	118,084	72.2
2	Dusk and dawn	163	5.5	3582	5.0	7799	4.8
3	Dark lighted roadway/unknown lighting	244	8.3	7146	10.0	15,206	9.3
4	Roadway not lighted	818	27.7	9402	13.1	22,523	13.8
Road characteristic (roadchar)
1	Straight-leveled road	1675	56.7	50,437	70.4	118,417	72.4
2	Straight-grade/hillcrest/bottom	485	16.4	12,319	17.2	27,794	17.0
3	Curve-leveled/grade/hillcrest	788	26.7	8876	12.4	17,221	10.5
4	Not stated/unknown	4	0.1	56	0.1	180	0.1
Driver gender (drv_sex)
1	Male	2006	68.0	38,058	53.1	92,305	56.4
2	Female	946	32.0	33,630	46.9	71,307	43.6
Driver age (ageis)
1	15–19 years	262	8.9	7105	9.9	16,690	10.2
2	20–69 years	2505	84.9	60,725	84.7	138,762	84.8
3	≥70 years	185	6.3	3858	5.4	8160	5.0
Speed limit class (slgrp)
1	< 20 mph	5	0.2	415	0.6	1,482	0.9
2	> 20 but ≤ 30 mph* (30 mph included)	17	0.6	1064	1.5	3042	1.9
3	> 30 but ≤ 40 mph	302	10.2	15,354	21.4	35,896	21.9
4	> 40 but ≤ 50 mph	860	29.1	28,690	40.0	62,077	37.9
5	> 50 but ≤ 60 mph	1495	50.6	20,053	28.0	42,518	26.0
6	> 60 mph	273	9.2	6112	8.5	18,597	11.4
Crash type (crashtyp)
1	Ran off-road	113	3.8	2524	3.5	5590	3.4
2	Jackknife, overturn/rollover	124	4.2	1191	1.7	1276	0.8
3	Pedestrian/pedal cyclist	186	6.3	484	0.7	53	0.0
4	Animal or movable object	26	0.9	727	1.0	9287	5.7
5	Parked vehicle or fixed object	664	22.5	9296	13.0	22,011	13.5
6	Rear-end collision	395	13.4	29,527	41.2	67,554	41.3
7	Left-/right-turn crashes	340	11.5	9243	12.9	15,346	9.4
8	Head-on collision	416	14.1	1534	2.1	763	0.5
9	Sideswipe or angle collision	599	20.3	15,879	22.2	37,251	2.8
10	Other	89	3.0	1283	1.8	4481	2.7
Work zone area (workzone)
0	No	2880	97.6	69,872	97.5	159,385	97.4
1	Yes	72	2.4	1816	2.5	4227	2.6
Vehicle type (vehicle)
1	Passenger car/taxi	1331	45.1	40,185	56.1	90,650	55.4
2	Pickup, light truck, sports utility, or van	1228	41.6	28,305	39.5	65,899	40.3
3	Commercial bus, school bus, activity bus, other bus	13	0.4	255	0.4	593	0.4
4	Single unit truck, truck/trailer, truck/tractor, tractor doubles, semitrailer, farm equipment, or other heavy trucks	192	6.5	1847	2.6	5557	3.4
5	Motor scooter, moped, pedal cycle, or motorcycle	179	6.1	876	1.2	185	0.1
6	Other	9	0.3	220	0.3	728	0.4
Seasonal factors (season)
1	Spring	623	21.1	18,168	25.3	44,375	27.1
2	Summer	762	25.8	17,669	24.6	37,989	23.2
3	Autumn	717	24.3	15,408	21.5	33,642	20.6
4	Winter	850	28.8	20,443	28.5	47,606	29.1
Road terrain (terrain)
1	Flat	748	25.3	13,405	18.7	29,761	18.2
2	Rolling	1975	66.9	53,124	74.1	120,807	73.8
3	Mountainous	229	7.8	5159	7.2	1304	8.0
Time of day (TOD)
1	Midnight–03:00 a.m.	193	6.5	2,047	2.9	3,992	2.4
2	03:00–06:00 a.m.	177	6.0	1880	2.6	4381	2.7
3	06:00–09:00 a.m.	412	14.0	11,886	16.6	29,002	17.7
4	09:00 a.m.–midday	366	12.4	9488	13.2	22,060	13.5
5	Midday–03:00 p.m.	445	15.1	13,421	18.7	29,605	18.1
6	03:00–06:00 p.m.	552	18.7	18,351	25.6	41,449	25.3
7	06:00–09:00 p.m.	499	16.9	10,164	14.2	23,404	14.3
8	09:00 p.m.–midnight	308	10.4	4451	6.2	9719	5.9
Day of the week (dow)
1	Sunday	412	14.0	6389	8.9	13,217	8.1
2	Monday	485	16.4	12,518	17.5	29,077	17.8
3	Tuesday	410	13.9	12,078	16.8	28,917	17.7
4	Wednesday	465	15.8	10,533	14.7	24,472	15.0
5	Thursday	335	11.3	9869	13.8	22,882	14.0
6	Friday	449	15.2	12,493	17.4	28,711	17.5
7	Saturday	396	13.4	7808	10.9	16,336	10.0
Locality (locality)
1	Agricultural	1573	53.3	21,185	29.6	47,922	29.3
2	Residential	597	20.2	13,674	19.1	27,615	16.9
3	Commercial	760	25.7	35,720	49.8	85,696	52.4
4	Institutional	8	0.3	625	0.9	1379	0.8
5	Industrial	14	0.5	484	0.7	1000	0.6

Note: PDO = property damage only.

Treatments for Imbalanced Data

In the domain of machine learning, there are a variety of approaches that can be employed to tackle imbalanced crash data. The consequence of neglecting this discrepancy can result in a biased model, particularly for the minority class. In this study, two data imbalance treatment techniques were employed to assess how they influence the prediction of crash severity related to weather conditions.

Synthetic Minority Oversampling Technique-Nominal

The SMOTE, introduced by Chawla et al. ( 7 ), addresses class imbalance by oversampling minority instances through the creation of synthetic data points. The SMOTE-N, as presented in Algorithm 1, is an extension designed for nominal datasets, using a modified version of the value difference metric (MVDM) to measure the distance between categorical feature values. The MVDM considers the occurrences of values and their response classes. The distance between feature vectors is calculated using a weighted Euclidean or Manhattan distance. In the case of the SMOTE-N, weights are often disregarded, as it primarily aims to balance data distribution between classes rather than direct classification.

Algorithm 1: Synthetic Minority Oversampling Technique-Nominal (SMOTE-N) Data Generation
	Input: D = ${x_{1}, y_{1}}$ , $y_{s} \in Y = {1, \dots, C}$ // training dataset D with n samples, and p-dimensional feature space
	Output: Synthetic data
1:	Calculate the distance metric for each sample using the MVDM
2:	$δ (V_{1}, V_{2}) = \sum_{i = 1}^{h} \| \frac{C_{1 i}}{C_{1}} - \frac{C_{2 i}}{C_{2}} \|$
3:	For each $x_{i}$ in minority class
4:	for i = 1 to $m_{s}$ :
5:		for j = i+ 1 to $m_{s}$ :
6:			$δ (V_{1}, V_{2}) = 0$
7:			for f = len(p):
8:				if instance (X [0, f], nominal):
9:					Calculate the MVMD distance for categorical features
10:					$C_{1} = \sum_{k} [X [y = m_{s}] [k, f] = [X [i, f]$ indicator function for C₁
11:					$C_{2} = \sum_{k} [X [y = m_{s}] [k, f] = [X [i, f]$ indicator function for C₂
12:					$δ (V_{1}, V_{2}) = \sum_{i = 1}^{h} \| \frac{C_{1 i}}{C_{1}} - \frac{C_{2 i}}{C_{2}} \|$
13:				else:
14:				Calculate the Euclidean distance for numerical features
15:			end for
16:		end for
17:	end for
18:	Compute the distance between two feature vectors
19:	$Δ (X, Y) = w_{x} w_{y} \sum_{b = 1}^{p} δ (x_{b}, y_{b})^{r}$
20:	return X, Y
21:	end function

Adaptive Synthetic-Nominal Technique

The ADASYN-N technique is designed to address class imbalance with nominal predictors as an extension of the ADASYN technique ( 8 ) initially intended for numeric predictors. In the ADASYN-N technique, as presented in Algorithm 2, the aim is to generate synthetic data for the minority class in a way that prioritizes events that are challenging to learn. The process has a training dataset containing n samples, where x_i represents data in a p-dimensional feature space X and y_i is the class label. The number of synthetic data points to be generated (G) is determined based on the desired level of balance, controlled by the parameter β (ranging from 0 to 1). For each x_i in the minority class, the algorithm calculates k-nearest neighbors in the p-dimensional space and computes r_i, which represents the ratio of majority events among these nearest neighbors. Higher r_i values indicate events that are more challenging to learn. These r_i values are normalized, ensuring that the sum of normalized r_i values equals 1.

Algorithm 2: Adaptive Synthetic-Nominal (ADASYN-N) Data Generation
	Input: D = ${x_{1}, y_{1}}$ , $y_{s} \in Y = {1, \dots C}$ // training dataset D with n samples and p-dimensional feature space
	Output: Synthetic data
1:	Calculate the number of synthetic data to be generated
2:	$G = (m_{l} - m_{s}) \times β$
3:	Separate dataset into minority and majority classes
4:		$m_{s}$ = number of instances in minority class
5:		$m_{l}$ = number of instances in majority class
6:	For each $x_{i}$ in minority class
7:	for i = 1 to $m_{s}$ :
8:		Calculate the number of majority instances in nearest neighbor
9:		$H_{i}$ = number of majority instances in nearest neighbor
10:		Calculate $r_{i}$ as ratio of majority domination in k-nearest neighbors
11:		$r_{i} = \frac{H_{i}}{k}$
12:		Normalize $r_{i}$
13:		$\hat{r_{i}} = \frac{r_{i}}{\sum_{i = 1}^{m_{s}} r_{i}}$
14:		Calculate number of synthetic data to be generated
15:		$g_{i} = \hat{r_{i}} x G$
16:		Generate data by replicating $x_{i}$
17:		GenerateData ( $x_{i}, g_{i}$ )
18:	end for
19:	Generate Data function
20:	function GenerateData ( $x_{i}, g_{i}$ )
21:		Generate data by replicating $x_{i}$
22:		for i = 1 to $g_{i}$ :
23:			syntheticData = replicate $x_{i}$
24:		end for
25:	end function

Feature Selection

A variable section technique called permutation feature importance (PFI) was employed as presented in Algorithm 3. The idea behind this technique, as presented in Figure 1, is to destroy a feature of interest x_j $\in X$ by perturbing it such that it becomes uninformative. For example, observations in x_j are randomly permuted where the marginal distribution P(x_j) stays the same.

Algorithm 3: Permutation Feature Importance
	Input: trained model f; feature matrix X, target vector y, loss function L (y, f)
	Output: Sorted list of features by descending PFI
1:	Calculate the original model error $e_{orig}$
2:	$e_{orig} = L (y, f (X))$
3:	Initialize an empty list to store feature importance values: PFI_values
4:	For each feature j from 1 to the number of feature p:
5:	for j = 1 to $p + 1$ :
6:		Create a permutated feature matrix $x_{perm}$ by randomly shuffling the value of feature j in the data X
7:		Estimate the permuted error $e_{perm}$ using the error measured:
8:		$e_{perm} = L (y, f (x_{perm}))$
9:	end for
10:	Calculate the PFI $PF I_{j}$ for each feature j using:
11:		PFI as the difference between original error and permuted error:
12:		$PF I_{j} = e_{perm} - e_{orig}$
13:	Append the calculated $PF I_{j}$ value to the PFI_value list
14:	Sort the feature in descending order of their PFI values
15:	end function

Figure 1.

Permutation feature importance.

The errors without permuting the features and with permuted feature values are measured. Repetition of the feature permutation was done 500 times and the average of the differences of both errors was computed. Figure 2 shows how the result of the PFI is interpreted. For example, if crash type is the most important variable, it implies that destroying information about crash type by permuting it increases the error of the model the most. This interpretation of the PFI helps one to select features of the model that are most sensitive and guides model refinement.

Figure 2.

Permutation feature importance and its interpretation.

Machine Learning Techniques

Extreme Gradient Boosting

An XGBoost model is an ensemble of decision trees, where each subsequent tree tries to correct the errors made by the previous ones. It is an iterative process that aims to minimize a loss function. Mathematically, the prediction $\hat{y}$ of an XGBoost model with K trees for an input x can be represented using Equation 1 :

\hat{y} = \sum_{k = 1}^{K} f_{k} (x)

(1)

where f_k(x) is the prediction of the kth tree.

The objective function that XGBoost tries to minimize is represented in Equation 2 :

obj (θ) = L (θ) + Ω (θ)

(2)

where θ represents the parameters of the model, L(θ) is the training loss function, and Ω(θ) is a regularization term that controls the complexity of the model.

In the case of a classification problem such as traffic crash severity prediction, L(θ) is typically the log loss for binary classification, or the softmax loss for multiclass classification. The “gradient boosting” part of XGBoost is because it trains each new tree to predict the negative gradient (or “residual”) of the loss function with respect to the current predictions. This is why it is called “gradient boosting,” as it uses gradient information to boost the performance of the ensemble. The regularization term Ω(θ) in the objective function is what distinguishes XGBoost from regular gradient boosting. In XGBoost, Ω(θ) is expressed as Equation 3 :

Ω (θ) = γ T + \frac{1}{2} λ \sum_{j = 1}^{T} {w_{j}}^{2}

(3)

where T is the number of leaves in the tree, w_j represents the scores on the leaves, γ controls the complexity of the model (the number of leaves in the trees), and λ controls the L2 regularization on the leaf scores. This regularization term helps to prevent overfitting by penalizing complex models.

Random Forest Model

The RF model is an ensemble learning method that operates by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees. Suppose a RF model consists of N decision trees ( 40 ). Each tree gives a classification, and the tree “votes” for that class. The forest chooses the classification having the most votes (over all the trees in the forest). Each tree is grown as follows.

If the number of cases in the training set is N, then N cases are sampled at random—but with replacement from the original data. This sample is the training set for growing the tree.

If there are M input variables, a number m is specified such that at each node, m variables are selected at random out of the M and the best split on these m is used to split the node. The value of m is held constant during the forest growing.

Each tree is grown to the largest extent possible and there is no pruning.

For a given test record, each tree in the forest gives a classification. The forest chooses the classification having the most votes (over all the trees in the forest) and takes the average of outputs by different trees in the case of regression. Mathematically, the prediction of a RF model for an input x can be defined as shown in Equation 4 :

\hat{y} = \frac{1}{N} \sum_{i = 1}^{N} f_{i} (x)

(4)

where f_i(x) is the prediction of the ith decision tree. The model would take input features of a traffic incident (such as speed, weather condition, time of day, etc.) and output a crash severity class (severe injury, moderate injury, or PDO). The model would be trained on a labeled dataset, and the aim would be to minimize the discrepancy between the predicted and actual labels. The RF’s ability to combine multiple decision trees helps it to avoid overfitting and generally results in a robust prediction performance.

Results

The results of data imbalance treatment using the SMOTE-N and ADASYN-N technique are presented in this section. The dataset from both treatment methods is applied to the RF and XGBoost. The dependent variable has three ordered levels: “1” is severe injury crashes, “2” is moderate injury crashes, and “3” is PDO crashes.

Feature Extraction

Figure 3 represents the PFI as determined by the RF model. The description of variables on the y-axis are provided in Table 2. The PFI is a technique for estimating the contributions of individual features to the predictive power of a model by observing the effect on model performance by randomly permuting the values of each feature, one at a time ( 31 ). In this study, the RF model was trained on a subset of the dataset. A representative dataset was used for this process because this technique is computationally demanding. In the plot, each bar corresponds to a specific feature in the dataset, and the length of the bar corresponds to the importance of that feature. Positive values indicate that the performance of the model decreases when the feature is shuffled, suggesting that the model relies on the feature to make accurate predictions. In other words, when the feature is perturbed or randomized (shuffled), the model’s predictability suffers because it loses access to the valuable information contained within that feature. Therefore, a decrease in performance on shuffling implies that the feature plays a crucial role in the model’s overall accuracy. This highlights the importance of the feature in the predictive process, as it significantly contributes to the model’s ability to make accurate predictions. Conversely, negative values indicate that the performance of the model improves when the feature is shuffled, suggesting that the model might be overfitting to noise in the feature.

Figure 3.

Permutation feature importance plot.

The top three features obtained from this analysis are crash type, vehicle type, and locality, suggesting that these features are the most important for predicting the severity of a crash according to the trained model. However, it is important to note a few caveats. Firstly, while PFI provides a useful way to rank the importance of features, it does not provide any information about the nature of the relationship between each feature and the target variable ( 28 ). Secondly, it is possible that important features might appear unimportant if they are highly correlated with other features ( 28 , 33 ). Finally, the results might differ if the analysis were performed on the full dataset or a different sample because this analysis was performed on a sample of the original dataset.

Comparing Treatment Methods and Machine Learning Techniques

The model performance metrics are summarized in Table 3. The confusion matrix for the RF and XGBoost are summarized in Table 4.

Table 3.

Model Performance Metrics

Model	Level of crash severity	Train			Test
Model	Level of crash severity	Precision, %	Recall, %	F1 score, %	Precision, %	Recall, %	F1 score, %
Control
RF	Severe injury	99.95	89.69	94.54	69.44	6.76	12.32
	Moderate injury	99.53	82.32	90.11	63.51	18.38	28.51
	PDO	92.65	99.84	96.11	72.32	95.93	82.47
XGBoost	Severe injury	99.97	93.54	96.65	9.48	10.32	9.88
	Moderate injury	99.68	88.48	93.75	31.64	36.53	33.91
	PDO	94.62	99.92	97.20	49.98	58.19	53.77
SMOTE-N
RF	Severe injury	99.60	99.9	99.75	22.65	10.41	14.26
	Moderate injury	94.70	97.40	96.03	42.77	39.17	40.89
	PDO	97.51	94.40	95.93	74.08	77.53	75.77
XGBoost	Severe injury	99.16	99.72	99.44	17.68	18.78	18.21
	Moderate injury	79.51	81.49	80.49	40.78	47.46	43.87
	PDO	81.21	78.74	79.96	75.31	69.64	72.36
ADASYN-N
RF	Severe injury	98.85	99.88	99.36	13.79	19.46	16.14
	Moderate injury	92.99	97.79	95.33	40.89	51.07	45.42
	PDO	98.19	91.89	94.94	76.16	67.32	71.47
XGBoost	Severe injury	96.22	99.36	97.76	10.97	25.27	15.30
	Moderate injury	75.31	80.49	77.81	38.86	53.33	44.96
	PDO	79.71	71.41	75.33	76.13	61.96	68.32

Note: RF = random forest; XGBoost = extreme gradient boosting; SMOTE-N = synthetic minority over-sampling technique-nominal; ADASYN-N = adaptive synthetic-nominal; PDO = property damage only.

Table 4.

Confusion Matrix for the Random Forest (RF) and Extreme Gradient Boosting (XGBoost)

Predicted	Reference
	RF						XGBoost
	Train			Test			Train			Test
	Severe injury	Moderate injury	PDO	Severe injury	Moderate injury	PDO	Severe injury	Moderate injury	PDO	Severe injury	Moderate injury	PDO
Control
Severe injury	1984	1	0	50	15	7	1136	12	9	76	22	12
Moderate injury	19	44,293	192	230	3286	1658	223	15,138	2505	345	6530	27,049
PDO	209	9,517	122,474	460	14,576	39,281	853	38,661	120,152	319	11,325	13,885
SMOTE-N
Severe injury	122,523	300	236	77	109	154	122,319	528	511	139	297	350
Moderate injury	58	119,521	6662	321	7002	9047	184	99,960	25,572	322	8485	11,999
PDO	85	2845	115,768	342	10,766	31,745	163	22,178	96,583	279	9095	28,597
ADASYN-N
Severe injury	122,054	781	634	144	389	511	121,423	2828	1946	187	629	889
Moderate injury	103	124,799	9309	328	9129	12,871	554	102,720	33,130	314	9533	14,685
PDO	49	2034	112,723	268	8359	27,564	229	22,066	87,590	239	7715	25,372

Note: SMOTE-N = synthetic minority over-sampling technique-nominal; ADASYN-N = adaptive synthetic-nominal; PDO = property damage only.

Best Treatment Method

The focus is to assess the effect of the selected methods on the classification of crash severity into three categories, severe injury (SI), moderate injury (MI), and PDO, as seen in Figure 4. The primary evaluation metric used was the F1 score, which considers both precision and recall, providing a balanced performance assessment. For the RF model, the best-performing data imbalance treatment method was the ADASYN-N technique, achieving a test F1 score of 45.42. The ADASYN-N technique effectively addressed the class imbalance in the case of moderate injury crash, resulting in improved predictions for this category. However, the control dataset proved most effective for handling the class imbalance specific to PDO, which has a test F1 score of 82.47. The ADASYN-N technique was also identified as the best method for addressing the imbalance in the case of severe injury crashes, achieving a test F1 score of 16.14. The ADASYN-N technique significantly improved the model’s ability to predict severe injury crashes.

Figure 4.

Performance metrics on the test dataset.

For the XGBoost model, the data imbalance treatment method that yielded the highest test F1 score of 44.96 was the ADASYN-N technique for predicting moderate injury crashes. This data treatment method effectively handled the imbalance in the moderate injury crash category, leading to improved predictions for this severity level compared to the control dataset. The SMOTE-N emerged as the best-performing method for PDO crashes, with a test F1 score of 72.36. The SMOTE-N successfully mitigated the class imbalance issue, resulting in more accurate predictions for PDO crashes. The SMOTE-N was identified as the most effective data imbalance treatment method for severe injury crashes, achieving a test F1 score of 18.21. The SMOTE-N significantly improved the predictive performance for this severity category.

A notable consideration is that the choice of evaluation metric has a significant effect on the assessment of data imbalance treatment methods. While the F1 score was used as the primary metric, other metrics prioritizing different aspects of classification performance may lead to different rankings of the methods. For example, if recall is prioritized over precision, the best-performing data imbalance treatment method might change.

In machine learning, the difference in performance between the training and test datasets can provide valuable insights into how well the model is learning from the data. In this study, the F1 score was used to evaluate this, which balances both precision and recall. A large difference between the training and test F1 scores could indicate overfitting, while a small or negative difference might suggest underfitting or a good fit, depending on the absolute scores.

Model Fit on Datasets

Overfitting is a common issue in machine learning where the model learns the training data so well that it includes noise or random fluctuations. This results in a model that performs extremely well on the training data but poorly on unseen test data (Figure 4). Potential overfitting was observed in some scenarios.

The XGBoost model on the control dataset for severe injury crashes demonstrated the most pronounced case of overfitting, with an F1 score difference of 86.77.

The RF model with the SMOTE-N method for severe injury crashes and the RF model with the ADASYN-N method for severe injury crashes also showed large F1 score differences, indicating potential overfitting.

In these cases, the models achieved very high F1 scores on the training data, suggesting they were able to capture the nuances of the training data very well. However, their performance dropped significantly on the test data (Figure 4), suggesting that they may have overfit to the noise or specific patterns in the training data that do not generalize to the unseen data. On the other hand, a small or negative difference in F1 scores between the training and test datasets could suggest that the model is underfitting or fitting the data well. Underfitting happens when a model is too simple to capture the complexity of the data, performing poorly on both the training and test data. If the model fits the data well, it will have good performance on both datasets. From our analysis, the following scenarios demonstrate good fitting:

the XGBoost model with the ADASYN-N method for PDO crashes has an F1 difference of 7.01;

the XGBoost model with the SMOTE-N method for PDO crashes has an F1 difference of 7.60;

the RF model on the control dataset for PDO crashes has an F1 difference of 13.64.

In these cases, the models achieved reasonable F1 scores on both the training and test data, suggesting they were able to generalize well the unseen data. These observations highlight the importance of evaluating the performance of a model not only on the training data but also on unseen test data. If a model is overfitting, strategies such as simplifying the model, collecting more data, or using regularization or cross-validation might help to improve its generalization. If a model is underfitting, making the model more complex or engineering better features could help it capture the underlying patterns in the data.

Discussion

The results delve into the effectiveness of data imbalance treatment methods when used on different machine learning algorithms in the context of weather-related crash severity analysis. The findings demonstrate that the choice of method significantly influences the model’s ability to handle imbalanced data, and the effect varies depending on the severity category being predicted and the specific machine learning algorithm being used. One noteworthy observation is the consistent effectiveness of the ADASYN-N method for addressing moderate and severe injury crashes in both the RF and XGBoost models. The ADASYN-N method appears to be a robust approach for generating synthetic samples of the minority class, effectively balancing the class distribution and enhancing the model’s performance in predicting severe and moderate injury crashes. The consistent success of the ADASYN-N method across different algorithms indicates its potential as a reliable data imbalance treatment method for these crash severity levels. Fountas et al. ( 41 ) argued that the generation of data for weather-related crashes involves underlying latent processes that should be considered during analysis. Recognizing these processes is essential for enhancing crash severity prediction accuracy and for the proposed various statistical methods to address this issue effectively ( 41 – 43 ). The ADASYN-N method appears to address this limitation and is more resilient to the heterogeneity introduced in the dataset, consistently performing well across all the algorithms tested.

The ADASYN-N method’s effectiveness in predicting moderate and severe injuries can be attributed to the specific strengths of this approach. Unlike standard oversampling techniques, the ADASYN-N method adaptively adjusts the synthetic sample generation based on the learning difficulty of minority classes. This feature is beneficial for severe and moderate injury crashes, which might have more complex or less consistent patterns than PDO crashes. By focusing more on the harder-to-learn examples, the ADASYN-N method helps create a more balanced and representative training set for these types of crashes.

The notable performance of the ADASYN-N method in predicting moderate injury crashes suggests that this category, while not as rare as severe injury crashes, still benefits significantly from a more balanced representation in the training data. The method’s data generation process helps the model better understand the variations in moderate injury crashes, improving its predictability in this class. While both the ADASYN-N technique and SMOTE-N improve the model’s ability to predict severe injury crashes, the improvement is marginal. In addition, the model may become too attuned to the nuances of the training data, that is, there is a potential risk of overfitting, reducing its generalizability to real-world situations. In contrast, the prediction of moderate injury crashes, supported by a larger dataset, tends to be more robust and generalizable.

The marginal improvement in severe crash prediction achieved in this study, while seemingly small, is nonetheless significant, especially considering the high stakes involved in severe crash scenarios and the challenges presented by the highly imbalanced dataset. Even a slight enhancement in predicting severe crashes can be crucial. These crashes, although less frequent, often have the most devastating consequences, including serious fatalities or injuries ( 44 ). Therefore, any improvement in accurately identifying potential severe crashes, no matter how marginal, is valuable as it could contribute to life-saving interventions, more effective emergency response planning, and targeted safety measures ( 45 ). In the scenario of weather-related crashes, a marginal improvement indicates that the model is overcoming some of the inherent biases and learning meaningful patterns related to severe crashes.

High model performance on training data is often achievable with complex models. However, these models may not generalize well to unseen data, which is a phenomenon known as overfitting ( 46 ). Conversely, a model that is too simple may not perform well even on the training data, leading to underfitting, and therefore also lacking generalization ( 46 , 47 ). A pronounced case of overfitting was observed in some cases, notably the XGBoost model on the control dataset for severe injury crashes and the RF model using both the SMOTE-N and ADASYN-N method for the same crash category. This is evidenced by high F1 scores on training data but a substantial drop in performance on test data, indicating these models captured training data nuances, including noise, which did not generalize well to unseen/uncaptured data. This result shows that while these methods can improve model learning for underrepresented classes, they also introduce the risk of overfitting, especially when the synthetic samples do not perfectly represent the real-world data distribution ( 32 ). In such cases, the importance of interpretability becomes paramount; understanding the model’s decision-making process allows practitioners to discern whether predictions are based on genuine patterns or artifacts of the synthetic data ( 48 ). Therefore, one must weigh the benefits of improved predictions against the potential loss of transparency when settling for a machine learning model, ensuring the chosen model maintains a level of interpretability that supports reliable and actionable insights in practical applications.

Furthermore, the study highlights the superior performance of the SMOTE-N for the XGBoost model in predicting PDO crashes. This observation suggests that the SMOTE-N effectively addresses the class imbalance present in the PDO crashes category for the XGBoost algorithm. This finding emphasizes the significance of selecting data imbalance treatment methods tailored to the characteristics of the machine learning algorithm and crash severity type to achieve optimal performance.

Conclusions

This study successfully showcases the importance of choosing appropriate data treatment techniques, specifically the SMOTE-N and ADASYN-N technique, for handling nominal predictors in machine learning models applied to weather-related traffic crash severity prediction. The effectiveness of these methods varies depending on the crash severity level being predicted and the machine learning algorithm used. The ADASYN-N technique proved particularly effective in balancing class distribution and enhancing the accuracy of both the RF and XGBoost models for severe and moderate injury crash predictions. In contrast, the control dataset showed notable efficiency with the RF model for predicting PDO crashes, while the SMOTE-N significantly improved the XGBoost model’s performance in the same category. This indicates that the XGBoost model benefited from a more balanced dataset provided by SMOTE-N, which likely offered more examples of the minority class to learn from. The high prediction accuracy of PDO crashes in the RF model is not unusual as the model shows its bias to predicting the majority class.

The study establishes a clear benchmark for practitioners in selecting suitable techniques to generate synthetic samples for addressing underrepresented crash categories, such as severe and moderate injury crashes. By demonstrating the varying effectiveness of the SMOTE-N and ADASYN-N technique across different machine learning models and crash severity levels, this study provides valuable insights for optimizing data treatment in crash severity prediction. The insights gained from this study are instrumental in guiding the development of more accurate and reliable crash severity prediction models, ultimately aiding in better informed and more effective traffic safety measures and policy decisions.

Despite these promising findings, further research is warranted to explore the applicability of the introduced synthetic data generation technique to other weather-related datasets and enhance its performance in real-world scenarios. Assessing the technique’s scalability and efficiency in handling large-scale datasets would be crucial, particularly for real-time applications or big data scenarios. In addition, investigating the technique’s effectiveness on more complex and diverse datasets would contribute to a deeper understanding of its potential in various crash severity analysis contexts.

This study demonstrates that the SMOTE-N and ADASYN-N technique are effective data imbalance treatment methods for improving weather-related crash severity prediction in RF and XGBoost models, respectively. The effectiveness varies by crash severity category, with the ADASYN-N technique excelling in severe and moderate injury crash predictions and the SMOTE-N in PDO crash predictions. This study provides valuable guidance for researchers in choosing suitable techniques to create synthetic samples, particularly for underrepresented categories in weather-related crash severity prediction. The study paves the way for transportation agencies to develop more accurate crash severity prediction models. Such models are essential in enhancing road safety measures and building safer as well as more resilient roadways, especially in adverse weather conditions, ultimately protecting road users.

Footnotes

Author Contributions

The authors confirm contribution to the paper as follows: study conception and design: S.S. Pulugurtha, A. Ogungbire; data collection: A. Ogungbire; analysis and interpretation of results: A. Ogungbire; draft manuscript preparation: A. Ogungbire, S.S. Pulugurtha. All authors reviewed the results and approved the final version of the manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This paper is partially prepared based on information collected for a research project funded by the United States Department of Transportation – Office of the Assistant Secretary for Research and Technology (USDOT/OST-R) University Transportation Centers Program (Grant # 69A3551747127).

ORCID iDs

Abimbola Ogungbire

Srinivas S. Pulugurtha

This paper is disseminated in the interest of information exchange. The views, opinions, findings, and conclusions reflected in this paper are the responsibility of the authors only and do not represent the official policy or position of the USDOT/OST-R, or any other State, or The University of North Carolina at Charlotte or other entity. The authors are responsible for the facts and the accuracy of the data presented here. This paper does not constitute a standard, specification, or regulation.

References

FHWA. 21st Century Operations Using 21st Century Technologies. How Do Weather Events Impact Roads. https://ops.fhwa.dot.gov/weather/q1_roadimpact.htm. Accessed April 28, 2023.

Saha

Schramm

Nolan

Hess

Adverse Weather Conditions and Fatal Motor Vehicle Crashes in the United States, 1994-2012. Environmental Health, Vol. 15, No. 1, 2016, pp. 1–9.

Grounds

M. A.

Joslyn

S. L.

Communicating Weather Forecast Uncertainty: Do Individual Differences Matter?

Journal of Experimental Psychology: Applied, Vol. 24, No. 1, 2018, p. 18.

Gao

Ren

A Deep Learning Approach for Imbalanced Crash Data in Predicting Highway-Rail Grade Crossings Accidents. Reliability Engineering & System Safety, Vol. 216, 2021, p. 108019.

Kim

Lym

Kim

Developing Crash Severity Model Handling Class Imbalance and Implementing Ordered Nature: Focusing on Elderly Drivers. International Journal of Environmental Research and Public Health, Vol. 18, No. 4, 2021, p. 1966.

Khuat

T. T.

M. H.

Evaluation of Sampling-Based Ensembles of Classifiers on Imbalanced Data for Software Defect Prediction Problems. SN Computer Science, Vol. 1, No. 2, 2020, p. 108.

Chawla

N. V.

Bowyer

K. W.

Hall

L. O.

Kegelmeyer

W. P.

SMOTE: Synthetic Minority Over-Sampling Technique. Journal of Artificial Intelligence Research, Vol. 16, 2002, pp. 321–357. https://arxiv.org/pdf/1106.1813.pdf.

Bai

Garcia

E. A.

ADASYN: Adaptive Synthetic Sampling Approach for Imbalanced Learning. Proc., 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), Hong Kong, IEEE, New York, 2008. https://doi.org/10.1109/IJCNN.2008.4633969.

Das

Hossain

Rahman

M. A.

Sheykhfard

Kutela

Case Study on the Traffic Collision Patterns of E-Scooter Riders. Transportation Research Record: Journal of the Transportation Research Board, 2023: 03611981231185770. https://doi.org/10.1177/03611981231185770.

10.

Das

Vierkant

Gonzalez

J. C.

Kutela

Sheykhfard

Bayesian Network for Motorcycle Crash Severity Analysis. Transportation Research Record: Journal of the Transportation Research Board, 2023. 2677: 51–63.

11.

Sheykhfard

Haghighi

Das

Fountas

Evasive Actions to Prevent Pedestrian Collisions in Varying Space/Time Contexts in Diverse Urban and Non-Urban Areas. Accident Analysis & Prevention, Vol. 192, 2023, p. 107270.

12.

Khanpour

King

Sheykhfard

Haghighi

Drivers’ Reported Crash History, Sensitivity to Reward and Punishment, Personality, and Demographics: A Case Study in Iran. Transportation Research Interdisciplinary Perspectives, Vol. 21, 2023, p. 100902.

13.

Liu

A Genetic Programming Approach for Real-Time Crash Prediction to Solve Trade-Off Between Interpretability and Accuracy. Journal of Transportation Safety & Security, Vol. 15, No. 4, 2023, pp. 421–443.

14.

Morris

Yang

J. J.

Effectiveness of Resampling Methods in Coping with Imbalanced Crash Data: Crash Type Analysis and Predictive Modeling. Accident Analysis & Prevention, Vol. 159, 2021, p. 106240.

15.

Al-Mistarehi

B. W.

Alomari

A. H.

Imam

Mashaqba

Using Machine Learning Models to Forecast Severity Level of Traffic Crashes by R Studio and ArcGIS. Frontiers in Built Environment, Vol. 8, 2022, p. 860805.

16.

Zhao

Wang

Liu

Jackson

Investigating the Effects of Monthly Weather Variations on Connecticut Freeway Crashes from 2011 to 2015. Journal of Safety Research, Vol. 71, 2019, pp. 153–162.

17.

Wei

Zhang

Das

Applying Explainable Machine Learning Techniques in Daily Crash Occurrence and Severity Modeling for Rural Interstates. Transportation Research Record: Journal of the Transportation Research Board, 2023. 2677: 611–628.

18.

Das

Dutta

Sun

Patterns of Rainy Weather Crashes: Applying Rules Mining. Journal of Transportation Safety & Security, Vol. 12, No. 9, 2020, pp. 1083–1105.

19.

Yang

Wang

Yuan

Liu

Predicting Freeway Traffic Crash Severity Using XGBoost-Bayesian Network Model with Consideration of Features Interaction. Journal of Advanced Transportation, Vol. 2022, 2022. https://doi.org/10.1155/2022/4257865.

20.

Robbins

C. J.

Fotios

Motorcycle Safety After-Dark: The Factors Associated with Greater Risk of Road-Traffic Collisions. Accident Analysis & Prevention, Vol. 146, 2020, p. 105731.

21.

Hou

Huo

Leng

Mannering

A Note on Out-of-Sample Prediction, Marginal Effects Computations, and Temporal Testing with Random Parameters Crash-Injury Severity Models. Analytic Methods in Accident Research, Vol. 33, 2022, p. 100191.

22.

Dingus

T. A.

Guo

Lee

Antin

J. F.

Perez

King

M. B.

Hankey

Driver Crash Risk Factors and Prevalence Evaluation Using Naturalistic Driving Data. Proceedings of the National Academy of Sciences, Vol. 113, No. 10, 2016, pp. 2636–2641.

23.

Sattar

Oughali

F. C.

Assi

Ratrout

Jamal

Rahman

S. M.

Transparent Deep Machine Learning Framework for Predicting Traffic Crash Severity. Neural Computing and Applications, Vol. 35, No. 2, 2023, pp. 1535–1547.

24.

Duddu

V. R.

Kukkapalli

V. M.

Pulugurtha

S. S.

Crash Risk Factors Associated with Injury Severity of Teen Drivers. IATSS Research, Vol. 43, No. 1, 2019, pp. 37–43.

25.

Shi

Wong

Y. D.

M. Z.

Palanisamy

Chai

A Feature Learning Approach Based on XGBoost for Driving Assessment and Risk Prediction. Accident Analysis & Prevention, Vol. 129, 2019, pp. 170–179.

26.

Yuan

Abdel-Aty

Gong

Cai

Real-Time Crash Risk Prediction Using Long Short-Term Memory Recurrent Neural Network. Transportation Research Record: Journal of the Transportation Research Board, 2019. 2673: 314–326.

27.

Islam

Mannering

A Temporal Analysis of Driver-Injury Severities in Crashes Involving Aggressive and Non-Aggressive Driving. Analytic Methods in Accident Research, Vol. 27, 2020, p. 100128.

28.

Meng

Finley

Wang

Chen

Liu

T. Y.

Lightgbm: A Highly Efficient Gradient Boosting Decision Tree. Advances in Neural Information Processing Systems, Vol. 30, 2017. https://papers.nips.cc/paper_files/paper/2017/file/6449f44a102fde848669bdd9eb6b76fa-Paper.pdf.

29.

Jamal

Zahid

Rahman

M. T.

Al-Ahmadi

H. M.

Almoshaogeh

Farooq

Ahmad

Injury Severity Prediction of Traffic Crashes with Ensemble Machine Learning Techniques: A Comparative Study. International Journal of Injury Control and Safety Promotion, Vol. 28, No. 4, 2021, pp. 408–427.

30.

Iranitalab

Khattak

Comparison of Four Statistical and Machine Learning Methods for Crash Severity Prediction. Accident Analysis & Prevention, Vol. 108, 2017, pp. 27–36.

31.

Ogungbire

Kalambay

Gajera

Pulugurtha

S. S.

Deep Learning, Machine Learning, or Statistical Models for Weather-Related Crash Severity Prediction. Mineta Transportation Institute, San Jose, CA, 2023.

32.

Fiorentini

Losa

Handling Imbalanced Data in Road Crash Severity Prediction by Machine Learning Algorithms. Infrastructures, Vol. 5, No. 7, 2020, p. 61.

33.

Mondal

A. R.

Bhuiyan

M. A. E.

Yang

Advancement of Weather-Related Crash Prediction Model Using Nonparametric Machine Learning Algorithms. SN Applied Sciences, Vol. 2, 2020, pp. 1–11.

34.

Zeng

Hao

Lee

Chen

Investigating the Impacts of Real-Time Weather Conditions on Freeway Crash Severity: A Bayesian Spatial Analysis. International Journal of Environmental Research and Public Health, Vol. 17, No. 8, 2020, p. 2768.

35.

Call

D. A.

Medina

R. M.

Black

A. W.

Causes of Weather-Related Crashes in Salt Lake County, Utah. The Professional Geographer, Vol. 71, No. 2, 2019, pp. 253–264.

36.

Ghasemzadeh

Ahmed

M. M.

Complementary Parametric Probit Regression and Nonparametric Classification Tree Modeling Approaches to Analyze Factors Affecting Severity of Work Zone Weather-Related Crashes. Journal of Modern Transportation, Vol. 27, 2019, pp. 129–140.

37.

Zhang

Wen

Yamamoto

Zeng

Investigating Hazardous Factors Affecting Freeway Crash Injury Severity Incorporating Real-Time Weather Data: Using a Bayesian Multinomial Logit Model with Conditional Autoregressive Priors. Journal of Safety Research, Vol. 76, 2021, pp. 248–255.

38.

Ashifur Rahman

Das

Sun

Using Cluster Correspondence Analysis to Explore Rainy Weather Crashes in Louisiana. Transportation Research Record: Journal of the Transportation Research Board, 2022. 2676: 159–173.

39.

Washington

Haque

M. D.

On the Commonly Accepted Assumptions Regarding Observed Motor Vehicle Crash Counts at Transport System Locations. In Transportation Research Board (TRB) 92nd Annual Meeting Compendium of Papers, Transportation Research Board (TRB), National Academy of Sciences, Washington, D.C., 2013.

40.

Biau

Analysis of a Random Forests Model. The Journal of Machine Learning Research, Vol. 13, 2012, pp. 1063–1095.

41.

Fountas

Fonzone

Gharavi

Rye

The Joint Effect of Weather and Lighting Conditions on Injury Severities of Single-Vehicle Accidents. Analytic Methods in Accident Research, Vol. 27, 2020, p. 100124.

42.

Fountas

Fonzone

Olowosegun

McTigue

Addressing Unobserved Heterogeneity in the Analysis of Bicycle Crash Injuries in Scotland: A Correlated Random Parameters Ordered Probit Approach with Heterogeneity in Means. Analytic Methods in Accident Research, Vol. 32, 2021, p. 100181.

43.

Cai

Abdel-Aty

Lee

Wang

Developing a Grouped Random Parameters Multivariate Spatial Model to Explore Zonal Effects for Segment and Intersection Crash Modeling. Analytic Methods in Accident Research, Vol. 19, 2018, pp. 1–15.

44.

Martins

R. S.

Saqib

S. U.

Raja

M. H. R.

Gillani

Zafar

Collision Versus Loss-of-Control Motorcycle Accidents: Comparing Injuries and Outcomes. Traffic Injury Prevention, Vol. 23, No. 5, 2022, pp. 255–259.

45.

Hansson

S. O.

Zero Visions and Other Safety Principles. In The Vision Zero Handbook: Theory, Technology and Management for a Zero Casualty Policy ( Björnberg

K. E.

Hansson

S. O.

Belin

M. Å.

Tingvall

, eds.), Springer International Publishing, Cham, 2022, pp. 1–75.

46.

Salman

Liu

Overfitting Mechanism and Avoidance in Deep Neural Networks. arXiv Preprint arXiv:1901.06566, 2019.

47.

Chung

Haas

P. J.

Upfal

Kraska

Unknown Examples & Machine Learning Model Generalization. arXiv Preprint arXiv:1808.08294, 2018.

48.

Antoniadi

A. M.

Guendouz

Wei

Mazo

Becker

B. A.

Mooney

Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review. Applied Sciences, Vol. 11, No. 11, 2021, p. 5088.