Abstract
The lack of alignment between drivers’ and cyclists’ road-sharing knowledge results in unsafe interactions. To address this issue, educational countermeasures must clearly present and apply evidence-based practices to increase the likelihood that drivers will learn how to share the road safely with cyclists. In this study, we redesigned an existing Virginia road-sharing safety educational handbook to support a series of experiments. The redesign was based on established principles of instructional, organizational, and visual design. Virginia drivers completed a comprehension test after reviewing road-sharing educational material online. Results showed that reviewing the redesigned brochure did not improve global comprehension, law-based knowledge, and procedural knowledge about sharing the road with cyclists. However, the improved design of the educational material enhanced drivers’ declarative knowledge of road-sharing laws and safety. Further research is needed to determine the effectiveness of transferring these design choices to other transportation domains, so policymakers and instructors can effectively prioritize approaches for improving road safety.
Keywords
Introduction
Eight hundred fifty-seven cyclists were killed in 2018 in traffic accidents in the US, the highest number since 1990. 96% of the cyclists killed in 2018 were involved in a collision with a single motorized vehicle (National Center for Statistics and Analysis; NCSA, 2019). Crash fatality rates provide evidence of the scale of safety issues arising from cyclists and drivers sharing the road. Understanding the underlying predictors of these crashes is critical to establish how best to address and mitigate them. Multiple environmental, engineering, technology, and human factors play a role.
Road-sharing Knowledge Misalignment
The current research focuses on drivers’ knowledge of road rules related to cyclists in Virginia, where 560 reported bicycle crashes resulted in eight fatalities in 2020 (Virginia Department of Motor Vehicles; VA DMV, 2020). Drivers need more awareness and knowledge of Virginia’s road-sharing laws and education materials. Misalignment of expectations and understanding between motorists and cyclists can lead to frustration and crashes (Still & Still, 2019). For example, cyclists perceive more road hazards than drivers; this difference could result in behavior that seems unpredictable from a motorist perspective but is expected from a cyclist perspective (Still & Still, 2019). A lack of knowledge of cycling-related concepts, laws, and safe practices can also contribute to drivers’ frustration with sharing the road with cyclists, leading to unsafe interactions.
Road Safety Educational Countermeasures
Transportation practitioners may take an engineering, enforcement, or education approach to safety when developing countermeasures intended to mitigate a specific crash predictor or cause (Brookshire et al., 2016). Educational approaches provide information about specific laws and safe practices or target attitudinal changes within demographic groups through training and awareness outreach. Many educational approaches have been implemented in the U.S. to create a safer environment for cyclists, but they often do not target drivers directly.
State driver manuals and licensing exams address sharing the road with cyclists (Ayres, 2006), but many states do not emphasize the cyclists’ presence, relevant state laws, or road-sharing best practices (Venkatraman et al., 2021). Educational standards, learning objectives, and associated assessments around road sharing are not implemented consistently across the country. Initial instruction for drivers primarily focuses on vehicle operation and skill acquisition (Kuiken & Twisk, 2001) and may not highlight essential facts and general information for drivers to establish declarative knowledge about safe road-sharing during the early cognitive phase of skill acquisition (Anderson, 1982; Newell, 1994).
Goals
In this study, we aimed to investigate the impact of text-based perspective and incremental design changes to existing educational material on drivers’ learning outcomes about safe road-sharing. The topic of perspective is beyond the scope of this paper, but see Proaps (2022) for a detailed discussion. We used the Northern Virginia Regional Commission’s (NVRC) Sharing Virginia Roads handbook to conduct this research. First, we created a revision of the existing handbook by reducing content, removing pedestrian references, and increasing terminology consistency. Second, we created a redesigned brochure by incorporating instructional, visual, and organizational design principles into the revised handbook. We predicted the redesigned brochure would increase comprehension more than the revised handbook.
Method
Participants
One hundred forty-four university students in Eastern Virginia participated in the study. Students ranged from 18 to 53 years old and self-reported having a Virginia driver’s license or permit. Participants were eligible to receive course research credit.
Design
The study presented here is part of a larger program of research. We employed a 2 (educational material: revised handbook, redesigned brochure) x 2 (perspective: first-person, third-person) between-subjects design for the current study. The independent variable of interest in this paper, educational material, included two levels: (1) a revised handbook, and (2) redesigned brochure each of which were developed by applying evidence-based design elements. The independent variable, driver perspective, included two levels: (1) First-person (i.e., I), and (2) third-person (i.e., the driver). This perspective manipulation is not the focus of the current paper, but see Proaps (2022) for a more in-depth discussion of perspective’s impact on road-sharing safety knowledge. Drivers’ knowledge of road-sharing is the primary dependent variable (DV) discussed in this paper. Other DVs beyond the scope of this paper included judgments of knowledge about road-sharing, attitudes toward cyclists, and intentions to safely share the road (Proaps, 2022).
Stimuli
We used the 6th edition of the Share Virginia Roads handbook published in 2018 to carry out the current research (Figure 1). The handbook provides a review of safety procedures and traffic regulations for all road users in Virginia. We developed two versions of the existing handbook: (1) a revised version of the existing handbook, and (2) redesigned brochure.
Example of Original Share VA Roads Handbook.
Revised handbook
To create a revised handbook (Figure 2), we made specific incremental updates to the existing handbook. The revision served as a valid and fair comparison before updating content for the perspective manipulations (see Proaps, 2022).
Example of Revised Handbook.
Pedestrian references
To “weed” out extraneous details irrelevant to the instructional goals of road-sharing between cyclists and drivers (i.e., coherence principle; Mayer & Moreno, 2003), we removed 57 references to pedestrians.
Terminology consistency
To maintain consistent terminology across all stimuli, “motorists” was replaced with “drivers,” “bikes” was replaced with “bicycles,” and second-person perspective (“You”) references were replaced with the corresponding perspective for the text-based perspective independent variable.
Reduced length
The existing 55-page version of the handbook was reduced to 14 pages to mitigate potential participant fatigue and to ensure that only the most relevant information was included across conditions.
Images
Finally, image format was the focus of a follow-up study so all images were removed from the stimuli for future research exploring learning outcomes with countermeasures employing imagery alongside written text (Proaps, 2022).
Learning objectives
The educational material included a combination of three categories of knowledge aligning with the comprehension test: procedural, declarative, and legal information. Procedural knowledge included information related to driver or cyclist behavior involving a sequence of steps (e.g., a driver allowing space for a cyclist to merge into a lane) and general declarative knowledge including definitions of important road-sharing concepts (e.g., sharrows). A mix of procedural and general declarative knowledge were used to create a third category focusing on laws, legal requirements, and specific references to items in VA’s vehicle code (i.e., legal passing distance).
Redesigned brochure
We applied additional human-centered design (HCD) or user-centered design (UCD) principles to improve the revised handbook’s visual, instructional, and organizational design described above. Multiple design principles were carefully balanced to ensure that information in the redesigned brochure was visually appealing, understandable, and usable (Figure 3).
Example of Redesigned Brochure.
Color, contrast, and white space
To enhance the brochure’s learning effectiveness, we increased the overall color contrast, applied a consistent color scheme, and increased the white space throughout the brochure. These visual design elements are crucial instructional design principles that can help reduce distractions, improve retention, and enable learners to focus on the learning activity (Poulin, 2018). Using color and contrast can enhance visual perception while incorporating white space can improve comprehension (Kosslyn et al., 2006; Ware & Mikaelian, 1987).
Signaling and cueing
According to Cognitive Load Theory (CLT; Chandler & Sweller, 1991), poorly designed instructional materials can impose additional cognitive demands on learners (Cierniak et al., 2009). To address this concern, signaling and cueing strategies were employed in the redesigned brochure to draw learners’ attention to important parts of the instruction (Schneider et al., 2018) and improve retention (Mayer, 2014; Paas & Sweller, 2012). For instance, color blocking was utilized to group related information and indicate high priority, law-based information.
Typography
Text information presented legibly with clear typography can influence reading speed, comprehension, and retention (Lin & Liu, 2011). Sans-serif font was selected for the text because it is a best practice in digital design to facilitate reading on a screen (Nielsen Norman Group, 2006; Poppins font). A variety of font styles were employed for additional reading cues, such as italics to highlight law-based information and bold to indicate major topics.
Information organization
Related, categorized information is better remembered than unrelated facts or uncategorized items (Bousfield, 1953). The organization and categorization of information into meaningful chunks play a role in the perception and memorability of that information (e.g., memory consolidation) (Sweller et al., 2011). The content in the existing Share VA Roads handbook is organized into three sections based on the road user: Cyclists, drivers, and pedestrians. Major headings were reorganized to align with the location of road-sharing activities. For example, we condensed 22 sidewalk references across all road accesses into a single section. Such changes eliminated redundancies, presented information consistently, and ensured all road users had a shared understanding of the laws and best practices.
Measures
Driving and cycling experience
The aim of the proposed education countermeasure was to educate drivers about Virginia traffic laws and regulations. Participants were required to confirm they held a Virginia learner’s permit or driver’s license in a screener. The Virginia cycling and driving experience questionnaire by Still and Still (2019) was used to gather more detailed information about participants’ cycling and driving experience (e.g., frequency, years, and miles driven or ridden). Participants were also asked other questions beyond the scope of this paper.
Comprehension test
The comprehension test contained 20 questions about how cyclists should use the road and what road rules drivers should follow when encountering cyclists. Each section of the revised handbook and redesigned brochure were represented in the test. Some true or false items were adapted from the knowledge assessment Still and Still (2019) administered to drivers and cyclists to assess road-sharing laws and responsibilities. Other items were revised from the July 2020 VA DMV Driver Manual Knowledge Practice Exam. Questions were either true/false or contained four multiple-choice items. Internal consistency was moderately high (α = .710).
Questions aligned with the three categories of knowledge: Ten declarative knowledge items, ten procedural knowledge items. The law category was assessed through a mix of four declarative and four procedural items. A composite score of correct items within these three categories was created for further exploratory analyses.
Attention check. Four attention check questions were included to improve data quality and reduce inattentive or random responding (Oppenheimer et al., 2009).
Procedure
All study materials were administered through an unmoderated Qualtrics survey. Participants who qualified through the screener reviewed and signed the informed consent, received instructions, and completed demographics questionnaires. Participants were randomly assigned to one of the four conditions and reviewed the material before taking the comprehension test. Participants were given a maximum of 20 minutes to review the material, and another 20 minutes to complete the comprehension test. This timeframe was based on pilot testing to balance multiple methodological concerns such as ecological validity, participant fatigue, and unproctored cheating (Cluskey et al., 2011). The final questionnaires and debrief information were presented before exiting the study.
Results
Measures underwent pilot testing and quality assurance. Data were assessed for quality, insufficient effort responding, extreme response durations, nonresponses, missing values, and outliers. The results reported in this paper reflect an analysis of comprehension test scores analysis. Other results are discussed elsewhere (Proaps, 2022).
Participants
We removed 24 (8 female) participants who self-reported cycling more than 64 miles per year based on Hoaglin & Iglewicz’s (1987) recommended outlier labeling rule. We removed these extreme values to control for any effects of expertise on outcome measures and ensures the experimental design and educational material were sensitive to the countermeasure’s target audience of inexperienced cyclists. Of the 200 participants who started the study, we retained 144 (111 female) who ranged from 18 to 53 years old (M = 21.73, SD = 5.842), reported riding a bicycle on the road an average of 3.17 years (Min = 0; Max = 20; SD = 5.34).
Comprehension
A factorial multivariate analysis of variance (MANOVA) was used to address the hypotheses. Participants answered an average of 13.5 out of 20 questions correctly (i.e., an average proportion of correct responses = .675). Overall scores ranged from 5 to 20, and the aggregated average for nine items surpassed 70%.
We predicted that participants reviewing the redesigned brochure would have higher comprehension scores than participants reviewing the revised handbook across text perspective. There was no between-subjects main effect of educational material (p = .849) on global comprehension, which did not support our prediction. Participants in the redesigned brochure condition (M = .676, SD = .171) did not score significantly higher on the global comprehension test than participants in the revised handbook condition (M = .673, SD = .167).
Three exploratory 2 x 2 factorial ANOVAs were conducted to compare the effects of text perspective and educational material on the declarative knowledge, procedural knowledge, and law-based knowledge factor scores. Educational material had no significant effect on procedural knowledge (p = .640) or law-based knowledge (p = .573) factors. Educational material had a significant main effect on the declarative knowledge factor, F(1, 4.955), p = .027, η2 p = .031 (Figure 4). Mean comparisons indicated that participants who reviewed the redesigned brochure answered more declarative knowledge questions (M = 8.47) correctly than participants who reviewed the revised handbook (M = 7.73) across text perspectives. The redesigned material’s effect on declarative knowledge supports our prediction.
Main Effect of Educational Material on Declarative Knowledge Scores Across Perspectives.
Discussion
Various countermeasures are implemented to tackle the problem of unsafe road-sharing behaviors, such as law enforcement, road signage and lane markings, and education programs. Unfortunately, drivers and cyclists lack an alignment of road-sharing knowledge and most driver education does not target cyclist-driver interactions. The current study is a starting point for investigating potential theory-driven education countermeasure targeting safe road-sharing behaviors between cyclists and drivers.
In this current research, we sought to redesign educational material to foster Virginia drivers’ knowledge of road rules described in a Virginia road-sharing handbook. Reviewing the redesigned brochure did not improve global comprehension or law-based and procedural knowledge scores. Participants who read the redesigned brochure scored higher on declarative knowledge questions than participants who read the revised handbook. The participants in this study (and the intended target audience) were drivers with limited cycling experience. In the context of sharing the road with cyclists, the participants were engaged in the earliest phases of learning by remembering new facts and concepts (Krathwohl, 2002). These results suggest small, evidence-based design changes to countermeasures may help facilitate early cognitive learning phase in which learners are developing declarative knowledge (Bloom et al., 1956).
Limitations
While not comprehensive, we wish to address two limitations of the current work. For example, participants in this study were exposed to the material in a single setting. In reality, the target audience would have an opportunity to review similar educational material multiple times at their own pace before taking a licensing exam. Second, we did not employ the pretest and posttest method, commonly used to assess intervention outcomes (e.g., Lewis et al., 2008). There are reported knowledge elicitation benefits of pretesting (e.g., Little & Bjork, 2011), but we were concerned about a pretest serving as a learning event (i.e., testing effect; Brown et al., 2014). It would have also been difficult to delineate any direct effects of our experimental manipulations on learning if participants were cued to pay attention to those elements before they encountered the experimental stimuli.
Future research
We recommend two avenues of future research. First, research could continue to explore the benefits of removing any information not directly related to the goals and objectives of road-sharing instruction, such as references to pedestrians, as Clark and Mayer (2016) suggest. Second, the Share VA Roads handbook is currently available as a downloadable PDF and a website with different layouts and interaction design. The redesigned brochure in this study was developed for online viewing with an endless scroll pagination. Future work should examine the efficacy of web-based versions with varying interaction styles across multiple devices.
Conclusion
Sleet et al. (2007, 2011) have called on experts in the road safety domain to adopt an interdisciplinary perspective to solve critical safety problems. The current work offers a practical, small step toward road safety by incorporating established instructional, organizational, and visual design principles to Virginia road-sharing educational material. Further research is needed to determine the effectiveness of these instructional design principles in learning outcomes in this road-sharing context. Researchers must also explore if these design choices translate to other transportation domains, so policymakers and instructors can prioritize approaches for improving road safety (Brookshire et al., 2016). Our shared objective must be to apply these evidence-based practices to increase the likelihood that drivers will learn how to share the road safely with cyclists, and, ultimately, save lives.
Footnotes
Acknowledgements
The authors thank Nicholas Zakhar for his invaluable contribution in designing the revised brochure for this research. We credit Font Awesome for the use of their open-source icon library in the brochure. Finally, we recognize the Northern Virginia Regional Commission (NVRC) ’s vital work in educating the public about road-sharing safety.
Author Note
This research represents a portion of a Dissertation that served as partial fulfillment of a Human Factors Psychology Doctoral degree.