Abstract
Existing measures of road safety were primarily designed to evaluate motor vehicle crashes. Consequently, they are not well-suited for alternate or emerging modes of micromobile transportation, particularly the e-scooter, whose popularity has surged without a corresponding body of research on their safety. Effective safety analysis depends on complete, high-quality data capable of accounting for the relevant mode-specific dangers. The established criteria for measuring consequences and hazard exposure in risk metrics used for motor vehicles do not apply. Most road safety data sources and schemas have a similar motor vehicle-centric bias. This framing presents challenges when it comes to selecting and interpreting data about alternate modes of transportation like micromobility. This paper discusses a basic theory of risk metric selection and the purpose of transportation safety measures. It applies these ideas to the emerging mode of micromobility transportation and recommends appropriate criteria and limitations for each component of a metric. This paper also evaluates existing data sources and schemas to provide examples of bias and estimate the relative size of each issue. These considerations may serve as useful guidelines for further research in the area and help inform the requirements of data collection necessary to better answer questions of safety.
Keywords
Motor vehicle crashes are a leading cause of death in the United States for people under the age of 54 ( 1 ). Reducing motor vehicle crashes is therefore a major public safety issue. In the last century, as transportation research has evolved, improvements to infrastructure, policy, and vehicles have dramatically increased safety (2, 3). However, negative issues associated with heavy automobile usage, like traffic congestion and air pollution, have also increased public concerns ( 4 ). The historically high levels of automobile dependence in the United States (1, 5) may not be sustainable or desirable in the future ( 6 ). This raises questions about the future of urban transportation in the United States.
One proposed mechanism to improve the future of urban transportation is micromobility (7–11), a term which refers to small, lightweight, low-speed vehicles including bicycles, e-scooters, e-bikes, hoverboards, and many more. E-scooters have recently become a front-runner in this new generation of micromobility. They are becoming widely available as rentals in urban areas, and could play a role in a large-scale shift to more efficient transportation, made more attractive or feasible with new technology that allows for electric motors and phone-based rentals. These options offer an exciting alternative for short-distance trips that are too far to walk and have long been dominated by private automobile trips ( 8 ).
Early research suggests many possible benefits: increased access to transportation ( 9 ), the possibility of faster, cheaper travel ( 9 ), and decreased vehicle congestion ( 10 ). Other long-term benefits have also been suggested, including decreasing car ownership ( 9 ), decreasing greenhouse gas emissions (7, 8), and reducing the land-use requirements for parking ( 9 ).
However, both the real and perceived safety risks of these transportation modes are a serious deterrent to their adoption (11, 12). The high center of gravity of e-scooters makes them less stable than bicycles, their motors enable faster speeds with less work, and their wheels are smaller, which may make them tip more easily on rough terrain or potholes ( 13 ). Users may weave between the sidewalk and the shoulder of the road, exposing them to conflict with both pedestrians and motor vehicles ( 14 ). Lastly, e-scooter users, like motorcyclists, pedestrians, and cyclists, have no built-in protection (e.g., a crumple-zone like that built into modern automobiles) to absorb the collision, and are vulnerable in the case of a crash. All these issues pose potential safety risks.
Since 2018, when e-scooter popularity exploded, many studies have been conducted on e-scooter injuries. However, reconciling new information on e-scooters with existing data on private automobiles, motorcycles, bicycles, and other forms of personal conveyance is not easily done. For instance, the media may report an injury risk of “20 per 100,000” trips ( 15 ) and a rate of “16 fatalities per 100,000 licensed drivers” ( 16 ) but interpreting this may not be intuitive. How should these two numerical statements that are similar on the surface, but entirely different in meaning, be understood and compared? How should we think about risk and exposure to risk? Are e-scooters 25% more dangerous than driving? Is it safer to take an e-scooter trip than to be a licensed driver? These kinds of comparisons are often published as evidence of safety without fully considering the source or context—let alone such factors as the population, event, evidence, or units in use. Techniques for measurement and standardization have not yet been formalized across the field. Thus, the question of how dangerous these devices prove to be in practice, particularly in comparison with the modes they replace, remains unanswered.
The purpose of this paper is to survey common issues and best practices in transportation safety analysis and evaluate how well they apply in the nascent field of micromobility safety research. The metrics and methodologies developed for motor vehicles do not necessarily transfer well to micromobility, as many of the influential factors (e.g., patterns of use) are different. Comparisons between different modes and metrics are also fraught with complications that can be difficult to qualify. This review investigates the scope of potential issues in metric selection, data collection, and analysis. It explores the challenges that make comparisons between (or even sometimes within) transportation modes so difficult and makes recommendations for future research in micromobility that could foster standardization.
Metrics of Transportation Risk
Risk assessment commonly provides answers to three questions ( 17 ):
(1) What adverse events can happen?
(2) How likely are these events?
(3) How adverse are the consequences?
A risk metric attempts to provide quantitative answers to these questions to facilitate communication and informed decision-making. It is generally represented as a rate of consequences per some amount of hazard exposure ( 18 ), such as the number of collisions per vehicle-miles traveled.
There are many metrics used to discuss risk in transportation, with varying suitability based on purpose ( 19 ). As there is an array of options describing the numerators, or possible consequences in play (collisions, property damage, injuries, and fatalities being the most common) and the different denominators, representing the amount of hazardous exposure (including trips, distance traveled, population, time spent, users, and vehicles), numerous metrics result from the different combinations.
This issue is further complicated by the additional degrees of freedom available to researchers to select their metrics, including exclusion criteria, sources, and included populations. These all provide additional ways to make a metric more specific for a given use case, but less comparable with other metrics.
Bias adds an additional challenge to an already complicated measurement landscape. In a paper discussing best practices for aviation risk metrics, Villarreal ( 19 ) suggests two main types of bias in transportation metrics: (1) leveling distinctions (inappropriately combining data for a summary level statistic), and (2) omissions (failing to include relevant pieces of information). The influence of these factors, alongside additional metric evaluation criteria described below, provide a framework for considering the advantages and limitations of different metrics.
Metric Criteria
Johansen and Rausand discuss 11 different criteria in the selection of risk metrics ( 18 ). A subsection of these is of particular relevance in the choice between different metrics of transportation risk. In Table 1, we introduce the relevant concepts that we will refer to in this section as we evaluate the relative strengths and weaknesses of different metrics. We also expand on these selected criteria by providing an interpretation that focuses on their application to transportation safety, and in the development of standards for a new mode of transport.
Applied Vocabulary of Metric Criteria
Consequences (Numerators)
Transportation consequences describe the acute and immediate failures (e.g., a crash or collision) or their human and property outcomes. For example, the National Highway Traffic Safety Administration (NHTSA) describes its road safety mission as “save lives, prevent injuries, and reduce economic costs due to road traffic crashes” ( 20 ). In the context of micromobile vehicles, which are relatively inexpensive and slow-moving compared with conventional motor vehicles, property damage is not a major concern in traffic crashes. This leaves three major ways of measuring consequences: fatalities, injuries, and incidents. This section will discuss each in turn, and how the metric should be applied to micromobility risk analysis.
At the highest level, the choice of consequence can shift the focus of the metric from crash avoidance (i.e., fewer crashes or collisions) to crash mitigation (i.e., lower human or property outcomes) ( 21 ). It is easy to imagine an example where an airline might prioritize reducing the number of crashes that occur, whereas a regulator might focus on reducing fatalities by mandating pre-flight safety briefings. Thus determining the consequence included in a metric is generally dependent on the transportation mode, availability of data, and the priorities of the decision-maker.
Fatalities
Fatalities are one of the most widely visible and easily defined metrics for transportation risk. The severity of a fatality is objective (making contextual factors less important), binary (making it possible to count precise instances), and universally understood as deeply undesirable (making it a valid and rational measure of harm). Therefore, the most important concern for fatality-based metrics is generally reliability, which can change the scope or interpretability of the data.
One component of this is the maximum length of time after the crash for a death to be attributed to the wounds sustained in it. This threshold is different for different transportation modes. For example, aviation uses a 7-day threshold ( 19 ), whereas NHTSA uses a 30-day threshold in their Fatality Analysis Reporting System (FARS). There are also nuances in the definitions used between different modes: the Bureau of Transportation Statistics includes incident-related (e.g., non-collision work accidents) fatalities for rail and transit, but not for aviation or highway deaths ( 22 ). Such nuances make comparisons between modes much less straightforward.
Evidence from NHTSA FARS suggests that vulnerable road users killed in collisions are less likely to die at the scene in comparison with motor vehicle occupants. According to their data, cyclists, pedestrians, and other non-motorists are almost half as likely to die at the scene of the crash as the occupant of a motor vehicle (28%, 34%, and 22% of their fatalities vs. 44%, respectively; see Table 2). This could potentially lead to underreporting of fatalities in groups that are more likely to have longer survival times after the crash or are less likely to be tracked and reported on correctly. Although it is difficult to assess the distribution of survival times past 30 days in micromobility users because of the limited number of fatalities available to analyze, it is worth noting that in the second bike-sharing death in New York City’s bike-sharing program, the victim of a crash in April did not die of the injuries sustained until June ( 23 ).
Fatalities by Person Type and Death at Scene of Crash, According to National Highway Traffic Safety Administration Fatality Analysis Reporting System Data 2005–2019
Although NHTSA FARS is considered to be the most complete source of road transportation fatalities in the United States, it omits roughly 1,000 motor vehicle-related deaths a year that occur outside of the 30-day crash window, and 1,000–2,000 non-traffic deaths that occur off public roads ( 23 ). Because NHTSA FARS data is motor vehicle centric, fatal collisions involving vulnerable road users are disproportionately underrepresented. Some evidence on the potential scale of this issue comes from the National Safety Council (NSC), which tracks all motor vehicle-related fatalities up to a year after the crash, even if they are non-traffic incidents (see Table 3).
Comparison of National Highway Traffic Safety Administration (NHTSA) and National Safety Council (NSC) Motor Vehicle Fatalities in 2019
NSC reported 7,668 total pedestrian fatalities in 2019, whereas NHTSA reported 6,205. Of the 3,011 total traffic fatalities in 2019 counted by NSC but not counted by NHTSA, nearly half, 1,463, were pedestrians. Using NHTSA’s fatality criteria in place of the NSC includes only 73% of pedalcycle and 77% of pedestrian fatalities, but roughly 96% of the motor vehicle fatalities that did not involve a pedestrian or a pedalcyclist.
Micromobility fatalities may face similar or more severe gaps in reporting. In Austin, TX, many injured riders surveyed at the emergency room said they had been injured off public roads: 33% on the sidewalk, 5% on car-free paths, and 3% in parking areas ( 15 ). These locations could put roughly 40% of crashes out of the scope of traditional road safety reporting. Legislation dictating where e-scooters and other micromobility devices are legal to ride (e.g., on sidewalk) could also have a major influence on where crashes occur.
The reporting criteria for motor vehicle fatality data have been developed based on the common characteristics of these fatalities, such as survival time and crash location. It is difficult to know whether these same criteria can be applied to micromobility fatalities, as their characteristics are not well understood. With the current state of knowledge, it is more important to prioritize completeness and broad criteria. This may eventually lead to the development of exclusionary criteria that could be implemented to facilitate speed or ease of reporting. Aside from these challenges, fatalities meet all the criteria for a strong risk metric.
Injuries
An alternate method to report the human toll of crashes is the injuries sustained. Like fatalities, injuries are a valid and rational measure of the risk involved in transportation, because they reflect important human preferences (avoiding injury). However, injuries are significantly more complex to measure: there are many different types of injury (low precision); injuries can be more or less severe, and that severity may change (for worse or better) over time (posing problems of reliability); the circumstances of the injury can have a dramatic influence on how or whether it is reported (sensitivity to contextuality).
In many initial studies on e-scooters, injuries are considered as discrete counts based on the number of patients whose injuries meet given criteria. These may be descriptive criteria like “severe” ( 15 ), “serious” ( 26 ), or “incapacitating” ( 16 ) which add precision to the metric, but can make the data less comparable with other sources. Standardized systems such as the Abbreviated Injury Scale exist partially for this purpose and provide a structured, more reliable way of tracking severity. Other sources of e-scooter injury data have tracked intervention-based measures, such as “sought medical treatment” ( 27 ), “required ambulance” ( 28 ), “went to emergency room,”“required inpatient admission,”“required ICU” ( 29 ), “required surgery” ( 30 ), and so on. Interventions decided by the medical professional rather than by the injured party are more objective and replicable; they should be preferred in most circumstances. However, intervention-based measures are still sensitive to contextual issues, as other factors play a role in whether they are pursued or reported. Studies have found that many factors have a significant effect on the likelihood of a person choosing to visit the emergency room or take an ambulance, including demographics (like race, gender, and age), financial concerns, and insurance status, making these metrics particularly prone to selection bias (31, 32). The severity or type of the injury itself can affect its visibility as well; for example, low-severity injury crashes are significantly more likely to go unreported to the police ( 33 ).
A second concern with injury data is how sensitive it is to the contextual influences of the reporting. Small changes in injury criteria can change the instances by an order of magnitude. Although most motor vehicle crashes that go unreported to the police are property damage only, NHTSA research in 2015 found that 18% of those unreported crashes did involve an injury, with the majority of the injured having sought medical treatment ( 27 ). This is in line with the findings of a 2003 study which found that NHTSA’s estimated injuries were roughly 20% short of equivalent hospital data on motor vehicle injuries ( 34 ). This figure changes enormously if the scope is narrowed to crashes where the respondent was injured as a pedestrian, in which case almost half (49%) of the crashes were unreported ( 27 ). Furthermore, if no motor vehicle was involved, the discrepancy becomes larger still. In 2015, Centers for Disease Control and Prevention data based on their Web-based Injury Statistics Query and Reporting System counted 467,000 total bicycle-related injuries ( 35 ). NHTSA’s estimate for cyclist injuries for the same year (which excludes non-motor vehicle or private property crashes) was 45,000 ( 16 ), less than one-tenth of all bicycle-related injuries.
Another complication in comparing injuries between modes is the variety of injury type and severity. Vulnerable road users face different kinds of injuries than motor vehicle occupants ( 36 ), both because they lack the protection of a motor vehicle, and because their mechanisms of injury in the event of an incident are different. For example, standing micromobility options like e-scooters share a common mechanism of injury with pedestrians and hikers, denoted as FOOSH: Fall On Out-Stretched Hand, which results in damage to the shoulder, wrist, and forearm ( 37 ). Early studies based on emergency room presentations have found that e-scooter injuries are dominated by fractures and abrasions, mostly occurring on the upper and lower limbs, and involve falling from the scooter and coming into contact with the ground (15, 29). In contrast, injuries to motor vehicle occupants involve contact at high speeds with the interior parts of the car, such as the air bags, seat belt buckle, steering wheel, flying glass, and other components ( 38 ). These, and the abrupt change in speed, can result in serious injury to the head, chest, and spine ( 38 ).
Micromobility users may not only be more exposed to superficial injuries, but also to injuries that may not be tracked and reported by established standards. For example, cycling crashes can often cause maxillofacial (face or jaw) damage ( 39 ), which may be presented at a dentist’s office rather than at an emergency room. Dental trauma may also be delayed in presentation and vulnerable to complications ( 40 ), so that the damage is not immediately obvious. Even if dental damage were noted by the police at the scene, it is not listed as a criterion for injury severity in the NHTSA FARS Crash Report Sampling System (CRSS) coding manual ( 41 ). This is an example of injury data that goes largely unreported, despite being potentially painful, undesirable, and expensive.
Micromobility injuries are substantially more common than fatalities, which means there is more data available. However, the data that exists often suffers from underreporting or is missing important context. Special care should be taken when trying to compare micromobility injuries with other modes of transportation so that comparisons are not made inappropriately between injuries of different types and severity. Different sources of injury data must also be accounted for clearly and consistently.
Incidents
Incidents have different definitions and implications depending on the mode and source of the data. Here we refer to the general concept of a single, negative event or point of failure. Different modes have different criteria for what constitutes a reportable incident, based on the relevant risks for that mode ( 22 ); for example, rail and transit authorities include incidents that involve a fatality even if there was no moving transport involved, such as in a fall or a fire. Other modes only count fatalities from collisions (distinct from incidents in that they must include some moving transport).
This framing puts an emphasis on the likelihood of the failure rather than its severity. Counting collisions can equate the same weight to a minor fender-bender as to a three-car crash that results in many fatalities. This may be desirable in some circumstances (e.g., tracking if collisions have decreased in an intersection) but misleading in others (e.g., if a difference in severity may be important).
Different kinds of incidents may see different patterns and face different issues in reporting. Minor crashes or property-damage-only crashes provide less incentive for reporting and are particularly likely to go unreported (27, 33). However, property damage provides an incentive for drivers to report their incident to the police and their insurance company. The same incentive does not exist for cyclists or micromobility users, which may make incidents in which a rider crashes without injury unlikely to be reported. NHTSA estimates that only about half of property-damage-only crashes get reported to police ( 33 ), but minor, single-vehicle incidents of this nature are virtually never reported for micromobility.
The heterogeneous nature of “incidents” makes them difficult to compare, almost by definition. Tracking collisions alone conflates minor and major collisions, unless they are tracked separately as different incidents. Even within subcategories, there may still be a considerable range of severity, depending on criteria. This loss of precision may also pose a challenge in conducting comparisons across modes because of the changes in scale and context: a terrible crash that kills all passengers is a different level of catastrophe for a passenger jet, an automobile, or an e-scooter. Small e-scooter incidents, like hitting a pothole, lack an equivalent across modes but may still be noteworthy to e-scooter users.
From a user perspective, incident importance is related to consequences: fatalities, injuries, and property damage. The e-scooter operator Bird, for example, only counts incidents that resulted in “any kind of injury” ( 42 ). In such situations, the criterion for counting an incident begins to weigh more heavily in the meaning of the metric than the count of the incident itself. Because of this, except for situations where it really is only the incident likelihood that matters to the metric, incident counts alone are unlikely to be the most appropriate or relevant measurement. Incidents are too far removed from the human consequences.
Hazard Exposure (Denominators)
Transportation risk is usually measured through denominators representing hazard exposure, including vehicles, users, population, time, trips, and distance. For instance, one way to describe risk is using the number of collisions per the number of vehicles present. This provides a way of standardizing the data and providing context. Each type of denominator emphasizes a unique perspective on risk and answers a different type of question. For example, per vehicle captures the number of operable vehicles in use, but not how much they are being used, or by whom (see Table 4).
Measures of Hazard Exposure
The choice of denominator depends largely on the level of decision-maker, and what the scale of their interest is. The appropriate level of aggregation is especially dependent on the scope of the decision. For instance, if a stakeholder is making policy based on the interests of a population, they might be interested in a denominator to put the risk in relation to the population or the total number of vehicles. In contrast, an individual making individual decisions might be more interested in a denominator expressing the risk in a more local context, such as a per trip, per mile, or per user. More local contexts for a denominator are also possible, such as traffic counts passing through a given intersection. If the denominator reflects a manageable source of exposure to an undesirable consequence, aligning with the concepts outlined in Table 1, it can be suitable in safety analysis.
Risk for certain incidents, such as the failure to park or tripping over a misplaced e-scooter, should be described in general terms because the incidents are not necessarily linked to an individual’s usage or extent of travel. To consider the travel-specific risk for micromobility, measures linked to usage, such as trips, time, or distance, are required. Within those three measures, using trips is least desirable for comparisons between modes, because the characteristics of micromobility trips can be very different from motor vehicle trips. However, when comparability is not a priority, the lack of available data on distance or duration can make trips the best available measure of travel-related exposure.
Time spent traveling is the measure of exposure most closely linked to risk. In micromobility risk because certain incidents, like falling off an e-scooter, pose a risk the entire time the e-scooter is in use, every minute could be considered exposure to a falling hazard. For recreational use, where the goal is to spend an hour in a vehicle for fun or exercise, time may be a rational metric for the risk of different activities. However, transportation objectives cover distance, not time (people have distances between destinations they wish to travel, not an allotment of time to spend traveling in the safest way). The risk associated with the varying amounts of time spent traveling the same distance by different modes will be mostly captured by a distance-based metric. This recommends against using time except as a comparison with recreational activities.
This leaves distance as the preferred measure for exposure, but it is not without concerns. In single-person modes of transportation such as the e-scooter, vehicle-miles and person-miles should be equivalent. However, because the replacement value of a multiple-occupant mode would require an uplift in miles to a factor of n occupants to account for a modal shift (three people going one vehicle-mile in a car would take three vehicle-miles in a single-person vehicular mode), it is much more straightforward for the purposes of comparison to use person-miles rather than vehicle-miles. At the same time, not all miles of travel are equally risky. According to the Federal Highway Administration (FHWA), which publishes tables comparing fatalities by estimated vehicle-miles traveled by state and functional road class, miles of travel on a principal arterial road through a rural area is about five times as risky as the same miles of travel on an interstate or freeway travel through an urban area ( 43 ). This raises contextual and issues when it comes to using miles of travel as a measure of exposure: researchers and reporting systems should be careful to track, report, and describe the nature of the miles of travel in question.
Another issue with measuring both time and distance is the lack of reliable data on both metrics, particularly for micromobility. The primary source for information on time and distance traveled by bicycle are survey studies such as the National Household Travel Survey (NHTS) conducted by FHWA. These surveys require respondents to recall their bicycle trips and estimate their length and duration ( 44 ), which may not be accurate. E-scooters, being a newer mode of transportation, do not appear on the NHTS at all. The private companies that own and operate fleets of rental e-scooters or bicycles sometimes collect GPS data on trip length, route, and duration, which provides a much more accurate picture of usage. However, each company only collects data for its own vehicles, and this does not capture trips taken using personally owned micromobility vehicles. In contrast, vehicle-miles are much easier to quantify from automobile odometers and thus data on travel distance is much more accurate and widely available for motor vehicles.
Metric Interpretation
An underlying assumption in many of these metrics is that the hazardous exposure has a fixed, defined relationship with the consequence; for example, that the risk can be safely described as one fatality per 100 million vehicle-miles traveled, and that the expected fatalities for 300 million vehicle-miles would be a proportional three fatalities. This property is called scale invariance, but it may not hold with all modes of travel, or micromobility in particular.
Cyclists have a well-documented “safety-in-numbers” effect, where increased ridership sees a decreasing marginal effect on the increase in collisions. A meta-analysis of these studies found that the elasticity of the increase in collisions proportional to travel volume tends to be about 0.5, or the square root of the travel volume (
45
). Roughly speaking, collisions are a function of the square root of the volume. Thus, doubling the number of cyclists may be expected to only increase collisions by roughly 40%, whereas halving cyclists would only decrease collisions by 30%, as
Sources of Data
Existing reporting systems, especially large-scale programs, were created to focus on motor-vehicle travel and motor-vehicle risk. This means that while NHTSA’s reporting systems like FARS and CRSS collect, estimate, and report data on cyclists killed and injured in collisions, these data are limited only to crashes where a motor vehicle was also involved. This leaves out incidents between vulnerable road users when a motor vehicle was not present, so it does not support analyses that include the intrinsic risk of the mode, and the risk that the mode may pose to other vulnerable road users. Particularly in emerging modes of micromobility, the reporting structure and standards may not yet exist to generate reliable data. In such limited samples, it is easy for an artifact of the reporting to appear as a real effect in the data. For example, in the U.K., their pedal cycle safety factsheet depicts an increased likelihood of motor vehicle involvement when the injuries are less serious ( 26 ).
During analyses, it is important to ensure an “apples-to-apples” approach, matching evidence and event type across studies. Injuries reported at the scene of a crash, injuries self-reported to an insurance agency or micromobility operating company, injuries presenting to emergency rooms, injuries leading to hospital admissions, and fatal injuries should all be considered as related, but highly distinct, variable categories. The limits of the visibility of each event should be considered carefully in relation to the perspective of the data source: how did they come by the information, and what external factors affected it?
Accessibility of the many data sources ranges from publicly available resources such as the NHTSA FARS database to proprietary, to partially available data owned by groups such as the IIHS or original equipment manufacturers like Bird and Lime. Data on motor vehicles is more complete and widely available than data on micromobility vehicles like bicycles and e-scooters. The data sources that are available can also be difficult to compare because of differences in populations and methodology.
Information to distinguish between categories is not always available, nor are all crashes straightforward. In Las Vegas, a car passenger leaned out the passenger-side window to push over a cyclist and killed both himself and the cyclist ( 46 ). The event is difficult to understand even in narrative form—different coding standards, agencies, and individuals might all see the event and record it as a vehicular homicide or a traffic collision. These ambiguous scenarios highlight the need for researchers and analysts to have robust coding schemas and know how to use them.
Data sources have a powerful effect on what gets counted in the data. Different data sources on micromobility can lead to studies with similar goals but opposite results. For example, one study in the United Kingdom reported that approximately 90% of serious cyclist injuries between 2011 and 2016 were caused by collisions with motor vehicles ( 26 ), but a 2012 study by Stranges et al. found in the United States that about 90% of cyclist injuries in 2010–2011 involved no motor vehicle ( 26 ). The United Kingdom report was based on government data, collected through police reports, and Stranges et al. used emergency room injuries. Data can also be influenced by context such as when it is collected, the method of data collection, and the strength of coding schemas.
Reporting Based on the Healthcare System
Hospital-reported incidents only include those who came to the hospital owing to transportation-related injuries. This excludes anyone who died at the scene, and anyone who did not seek medical care at a hospital. However, those who do not present at a hospital are not necessarily uninjured: injured persons may be unwilling to seek medical care for financial reasons or the nature of their injuries, or they may visit a medical care provider other than a hospital (e.g., an urgent care center, private physician, or specialist). As previously discussed, emergency rooms ( 31 ) and ambulance transportation ( 32 ) may be especially sensitive to this selection bias because of their expense and the availability of alternatives. The size of these effects and the bias they produce may be subject to regional, cultural, and economic differences.
As many early studies on e-scooter injuries (15, 30, 37, 47, 48) have based their conclusions on emergency room data, selection bias is a prominent concern. For example, many of these e-scooter injury studies have reported on the rates of suspected alcohol involvement; 29% (n = 125) of Austin, TX respondents reported consuming alcohol before the accident ( 15 ); 27% of cases in an Auckland, NZ study found that the treating clinician suspected alcohol as a contributing factor ( 30 ); 16% of e-scooter patients in a Utah Emergency Department study reported having been intoxicated ( 47 ); and 18% of e-scooter accident victims reported alcohol consumption in a Dallas, TX study. This is likely the impetus for H.R.424, “The Safe Scooters Act,” which suggests studying the risk of intoxicated e-scooter riders as a major component of safe e-scooter use ( 49 ). However, although these results might indicate a high rate of inebriation, it is not necessarily related to the e-scooters themselves, as people presenting at emergency rooms often have elevated rates of inebriation ( 50 ), as do motor vehicle fatalities ( 51 ).
Mistakes can also occur in interpretation and recording of this information. For example, the World Health Organization publishes an International Classification of Diseases (now on its eleventh edition, ICD-11) that facilitates the collection and analysis of mortality data ( 52 ). Although regular updates increase diagnostic power and specificity, they may also lead to error. One study on the reliability of coding during the transition from ICD-9 to ICD-10 found that whereas the classification of cycling injuries had high agreement between the emergency department and an independent coder, the pedestrian injuries were misclassified more than half the time ( 53 ). Even as new coding schemas are developed to include codes for e-scooters and other like forms of transportation, it is far from certain that these codes will be quickly and accurately adopted.
Another limitation with these data is that cause-of-death data information is limited to fatalities. Much like hospital data, codes are determined by the best understanding of an uninvolved party. Crash reports are often cryptic and difficult for coroners to interpret, leading to vague catch-all codes and indecisive terminology ( 54 ). This lack of detail in cause-of-death data has real-world impacts on researchers. A 1992 study found that the association between death from head injury and the motorcyclist’s failure to wear a helmet was only statistically significant if data from the coroner’s diagnosis was used, but was not significant using the less detailed, less complete data from death certificates alone ( 55 ).
When healthcare data is used in micromobility research, consideration should go into the reliability and consistency of the reporting. If e-scooter injury reports are gathered from emergency rooms, those data must be compared with motor vehicle injuries reported by emergency rooms, as the factors that determine whether someone seeks treatment in the emergency room (injury severity and financial means) should affect both groups equally. If there is reason to believe that one group is more likely to visit the emergency room, then this comparison becomes less reliable.
Reporting Based on Police Reports
In 1981, NHTSA estimated, based on a telephone survey, that about 47% of crashes were unreported to police; an updated version of the research in 2008 estimates that number is now down to 30% ( 27 ). Some were not eligible for police reporting, depending on the state ( 27 ), whereas others involved property damage or injuries deemed not serious enough to warrant involving the police.
Crash reports and their coding schemas have many possible points of failure. Police may fail to fill out reports or may fill them out incorrectly ( 56 ). The forms police use to record data may not have appropriate options or detail, particularly for reporting evolving modes like micromobility. For instance, although the California Highway Patrol is planning to add an entry to their 555 Crash Reporting Form to indicate whether a micromobility device was involved in a collision, it still will not distinguish between micromobility device type (e.g., e-scooter, e-bike) ( 57 ).
Police crash reports can be especially unreliable when it comes to documenting the injuries of victims; the officer recording information may not closely or accurately inspect them, for reasons such as lack of opportunity or medical training ( 33 ). Throughout the United States, police crash reports typically categorize injuries according to KABCO (K = killed; A = incapacitating injury; B = non-incapacitating injury; C = possible injury; O = no injury apparent) ( 33 ), broad categories that are open to interpretation. Police officer accuracy assessing the severity of injuries present at the scene of the crash may also be an issue ( 58 ).
Research shows that cyclist and motorcyclist crashes are consistently under-reported, especially single-vehicle crashes ( 59 ), partly because collisions resolved without police are not included in most collision datasets. As previously noted, the police reporting-based NHTSA estimates of motor vehicle-related injuries have been found to be 20% lower than corresponding estimates based on hospital-reported data ( 34 ). This reporting bias against minor collisions is true across modes, but is especially prominent in crashes that do not involve motor vehicles, where police data is particularly limited. For example, bystanders may call the police if they observe a minor collision between an automobile and a cyclist, even if the cyclist is largely uninjured. In contrast, a cyclist who sprains their wrist after hitting a pothole might go to the emergency department but has no reason to call the police. A New Zealand cyclist study used a prospective methodology to follow cyclists over a 5-year period ( 60 ); by cross-referencing cyclist self-reported crashes with New Zealand Accident Compensation Claims (ACC) data, hospital discharge data, and police reports, they found that although 29% of all crashes involved an ACC claim, only 6.5% were reported to police.
Crash reports from police are commonly used in motor vehicle risk analysis, but are not recommended as a source for incident or injury counts from micromobility because of low reporting rates. Only very serious micromobility crashes are likely to appear in police reports, so those data may be suitable if the numerator of interest is severe or fatal crashes.
Reporting From Other Stakeholders
Although police and healthcare-based reporting are the primary sources of data in many major reporting systems, there are other stakeholders who may also report on micromobility incidents.
Media Reports
In particularly unusual incidents, there may be no coding schemas to describe them. In these cases, narrative information is the only way to gain additional details, and for this, media reporting can be a rich source. For example, there are cases of pedestrians killed by cyclists ( 61 ), cyclists killed by e-scooters ( 62 ), and fatal cyclist collisions on a bike path ( 63 ). Although these incidents are uncommon, the knowledge that they exist can be useful. Evidence from the general class of problem including risk assessment and danger to bystanders is of interest. At the same time, media reporting is almost invariably constrained to cover unusual or exciting incidents. News reports are unlikely to cover basic details that are not of interest to their audience, and they have systematic reporting differences when compared with a standard database like FARS, particularly a failure to report negative information ( 64 ); for example, cases that do not involve drunk driving are more likely to omit any mention of drunk driving, as opposed to reporting on the confirmed absence of alcohol involvement.
Self-Reporting
In theory, participants in the incident should be one of the most knowledgeable sources of information available. However, human limitations and incentives mean that this is not often true. The parties involved must be aware of and willing to comply with the self-reporting requirements, which can be challenging when reporting is cumbersome or difficult to do, or the potential reporter does not have any incentive to report the incident. It has been documented that in motor vehicle collisions, drivers are more likely to self-report an incident when they were not at fault ( 65 ) and are less likely to self-report incidents that were caused by behaviors like distracted driving ( 56 ). Different studies on the level of agreement between self-reported and state-reported collisions have found moderate to substantial levels of agreement (66, 67). It is unknown whether micromobility users would have similar levels of responsiveness and honesty in surveys about crashes or injuries.
Insurance Agencies
NHTSA’s research on under-reported collisions suggests that about 80% of motor vehicle crashes are reported to insurance agencies, more than are reported to the police. The severity of the crash is predictive of whether it was reported, with crashes that are more severe being more likely to be reported ( 27 ). Because of their financial stake in the system, insurance agencies have long been excellent collectors, keepers, and analyzers of collision data. However, neither auto insurance nor home/renters insurance covers liability for damages caused on an e-scooter, and even private scooter insurance may not cover damages that occur on rented e-scooters ( 68 ). With responsibility for damages distributed to the riders, there is no central mechanism for tracking. In the future, there may be a need for micromobility operators or other third parties to act as insurance providers to fill the existing coverage gap.
Micromobility Providers
Providers of rental micromobility vehicles collect trip data from their users, including locations, routes, and trip duration ( 69 ). Many cities and municipalities require that these data be made available for city planning purposes, and some cities like Austin, TX publish anonymized and aggregated versions of these data for researchers to use ( 70 ). However, the companies have no mechanism to automatically collect injury reports or crashes the way they record location data. The two largest scooter providers, Bird ( 42 ) and Lime ( 71 ), allow customers to report injuries through their apps or customer support, and both user agreements stipulate doing so in the event of a crash (72, 73). However, this is not enforced or otherwise incentivized. This makes the completeness of their incident and injury data difficult to ascertain, especially as these datasets are not available for public use.
Discussion
As the micromobility domain is still nascent, the techniques and strategies governing data practices are still forming. Micromobility refers to a diverse set of devices and products, and has many key differences compared with motor vehicles. Taking a “one size fits all” approach to data and analysis may miss key insights or bias results. Researchers must be conscientious about the unique needs of their subject matter, and carefully weigh the compromises they make.
Findings
New modes of transportation like micromobility introduce change into the transportation system. This change offers new advantages that are balanced by new risks. For e-scooters, the safety risks are under particular scrutiny. To fully understand these risks and compare them with those of motor vehicles, it is necessary to have comparable measures and methodologies. Unfortunately, the current approaches used for motor vehicles do not necessarily transfer well to other forms of transportation.
In this paper, we describe the potential for harm or damage in fractional terms, with consequences as the numerator and hazard exposure as the denominator. The consequences refer to harm or damage that occurs, and include such events as collisions and crashes, injuries and fatalities, and property damage. Hazard exposure refers to the opportunity for the consequence to occur and includes such variables as vehicles, users, population, time, trips, and distance.
Collisions, crashes, and incidents are common consequences of transportation use. These terms refer to the general concept of a single negative incident or point of failure, putting emphasis on the likelihood of failure rather than the severity, which can be misleading. Injuries and fatalities are also used to quantify transportation hazards, but the severity of injuries can vary and reporting is often subjective. Fatalities are a more concrete measure, but can still go unreported. Measuring risk exposure such as time or distance is similarly complicated; the choice of denominator needs to reflect the goal of the research and account for data availability, which is particularly limited for micromobility.
An additional data issue comes from the reporters and recorders themselves, who are limited by their knowledge and biases. Many data sources are automobile-centric, with incidents only being reported if an automobile was involved, limiting their usefulness to micromobility research. Healthcare data is preferable but needs to be kept consistent by source. Police reports are limited by the information provided to the police and their understanding of the situation. Self-reported data are similarly flawed, and people are not necessarily honest about their behaviors after an incident has occurred. Media reports are limited by the understanding of the writers, who are often not well-versed in transportation or statistical methods. Coding schemas limit the information about incidents and can lead to error. Overly limited coding categories on forms can exclude important information, whereas complex schemas can lead to human error in coding. Correct coding is also limited by the coder’s understanding of both the situation that occurred, and the schema into which they are inputting the data.
Recommendations
Quantifying the risk of any transportation mode is challenging, and micromobility adds unique complexities because of such factors as its recent emergence and difficulty accessing data. We recommend the following considerations when considering micromobility risk:
Choose risk metrics based on the specific questions the analysis is attempting to answer, using multiple metrics if necessary
Understand the strengths and shortcomings of the selected metric (see Table 1)
Fatalities and injuries should generally be considered optimal for measuring consequences, and within these metrics ensure that fatality criteria and injury severity coding systems are consistent, although other measures may be preferable depending on the goals of the specific analysis
Person-miles of travel should generally be considered an optimal risk-exposure measure for comparison of micromobility with other modes of transportation, although other measures may be preferable depending on the goals of the specific analysis
Trips are a suitable risk-exposure measure only between modes with comparable trip characteristics
Consider the effects of underreporting, particularly for low-visibility consequences such as minor injuries/incidents
Healthcare data should be considered carefully as they are typically not complete and may rely on inconsistent coding schemata
Law enforcement data should be considered carefully as it is particularly vulnerable to underreporting for micromobility
Maintain data context: document and retain information in relation to collection circumstances
Future research should focus on methods for collecting, maintaining, and standardizing micromobility data. For instance, collection and coding practices require criteria and coding schemas that cover the relevant risks of the mode. Data maintenance practices such as methods for retaining the original incident contexts are important, and procedures for data chain of custody should be examined. Standardized terminology will be an important tool for consistency and communication, and methods of standardizing datasets will allow for more robust comparisons.
Conclusion
To misquote a famous aphorism, all data are wrong—some data are useful. There is also a (sometimes exaggerated) illustration of survivorship bias from World War II, from the father of sequential analysis, Abraham Wald ( 74 ). When the United States military analyzed the damage to surviving airplanes, they concluded that the most heavily hit areas of the plane would require the most armor. It was Wald who saw a story in the absence of that information. He only had data on the airplanes that had survived their hits and been recovered. The damage that no surviving planes had received was the damage that no planes could survive. It was not the most heavily damaged areas, but the least damaged areas, that needed the most armor. In the same way, understanding the context and limitations of the data can make up for what would otherwise be misleading shortcomings. As micromobility matures and its safety data grows completer and more complex, it will still never be a perfect representation—but it will become more and more useful to researchers. Our understanding of the source and extent of the risks micromobility poses will improve with quality data and analysis.
Footnotes
Acknowledgements
The authors would like to thank Dr. Valerie J. Gawron for her guidance, and Dr. Bridget A. Lewis and Claudette H. Bishop for their assistance in revisions.
Author Contributions
The authors confirm contribution to the paper as follows: study conception and design: Elizabeth Karpinski; data collection: Eleanor Bayles; analysis and interpretation of results: Elizabeth Karpinski; Eleanor Bayles; draft manuscript preparation: Elizabeth Karpinski; Eleanor Bayles; Tracy Sanders. All authors reviewed the results and approved the final version of the manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the MITRE corporation.
