Abstract
Background. Noise-induced hearing loss is a significant occupational injury for firefighters exposed to intermittent noise on the job. It is important to educate firefighters about using hearing protection devices whenever they are exposed to loud noise. Computer technology is a relatively new health education approach and can be useful for tailoring specific aspects of behavioral change training. The purpose of this study is to present the development process of an Internet-based tailored intervention program and to assess its efficacy. Method. The intervention programs were implemented for 372 firefighters (mean age = 44 years, Caucasian = 82%, male = 95%) in three states (California, Illinois, and Indiana). The efficacy was assessed from firefighters’ feedback through an Internet-based survey. Results. A multimedia Internet-based training program was developed through (a) determining program content and writing scripts, (b) developing decision-making algorithms for tailoring, (c) graphic design and audio and video productions, (d) creating computer software and a database, and (e) postproduction quality control and pilot testing. Participant feedback regarding the training has been very positive. Participants reported that they liked completing the training via computer (83%) and also that the Internet-based training program was well organized (97%), easy to use (97%), and effective (98%) and held their interest (79%). Almost all (95%) would recommend this Internet training program to other firefighters. Conclusion. Interactive multimedia computer technology using the Internet was a feasible mode of delivery for a hearing protection intervention among firefighters. Participants’ favorable feedback strongly supports the continued utilization of this approach for designing and developing interventions to promote healthy behaviors.
Introduction
Good hearing is important for many facets of life, including job safety and performance. Repeated exposure to loud noise, whether on or off the job, causes noise-induced hearing loss (NIHL), an irreversible but preventable condition for which there is currently no effective medical treatment (National Institutes of Health, 1990). Although occupational noise exposure is the most common cause of NIHL (National Institutes of Health, 1990), it is also one of the most preventable. Firefighters and other emergency medical service personnel are routinely exposed to loud noise on the job from vehicles, sirens, alarms, and tools (Ewigman, Kivlahan, Hosokawa, & Horman, 1990; Morse, Owen, & Becker, 1992; Tubbs, 1994, 1995; U.S. Department of Health & Human Services, 1990). They are also exposed to other hazardous and ototraumatic agents (e.g., vibration, organic solvents, carbon monoxide, and certain metals) that may have an additive damaging effect on hearing (Morata et al., 2002; Sliwinska-Kowalska et al., 2004).
Thus, it is important to focus efforts on education for firefighters to keep them fully informed about the hazards of noise and to ensure that they assume responsibility for protecting their hearing. Promoting the use of effective hearing protection devices (HPDs) consistently and correctly through effective behavioral interventions is essential to preventing NIHL in firefighter who are exposed to hazardous noise. To develop the most effective ways to assist firefighters in adopting desired hearing protection behavior, it is important to understand what factors influence their hearing protection behavior.
Computers have been found to be a useful method for delivering health education messages regarding hearing protection to a variety of populations, including construction workers (Collins, Murphy, & Bierman, 2004; Eakin, Brady, & Lusk, 2001; Hong, Ronis, Lusk, & Kee, 2006) and factory workers (Lusk et al., 2003). Because of firefighters’ unique and demanding work, it is important to provide specialized training programs for them regarding noise, NIHL, and HPD use. Targeted strategies are necessary to assist them in protecting their hearing while working in stressful and hazardous conditions (Hong, Samo, Hulea, & Eakin, 2008). Web-based computer programs are a useful and economical solution for the specialized training needs of specific populations such as firefighters. Web-based programs are readily available, provide opportunities for confidential and time-efficient instruction, and are flexible, allowing individuals to access the program at their own pace and at a time that fits their schedule (Bull, McFarlane, & King, 2001). Web-based programs can also effectively combine interactive, personalized information with proven behavioral theory techniques, including skills training in HPD use. In addition, web-based programs have the advantage of being able to tailor behavior change messages through adaptive or feedback-based methods, which have been shown to be more effective than content delivered in a face-to-face group format (Collins et al., 2004; Newman et al., 2002).
The purposes of this article are (a) to describe the creation of an Internet-based, tailored hearing protection education program for firefighters that specifically addresses the unique needs of their profession and (b) to assess efficacy of the program based on firefighters’ feedback on using the program.
Method
Development of the project was conducted in three phases: Phase 1—convening focus groups to gain insight regarding program content, Phase 2—program design and pilot testing, and Phase 3—testing the efficacy of the web-based program. All study protocols were approved by the Committee of Human Research of the University of California San Francisco and the Institutional Review Board of NorthShore University Health System in Evanston, Illinois.
Phase 1: Focus Groups
Twenty-four firefighters from two fire stations in the San Francisco Bay area in California participated in focus group sessions (Hong, Vogel, Eakin, Chin, & Jayaraman, 2010). The purpose of these sessions was to elicit information regarding firefighters’ perceptions about major occupational health and safety issues, their views on the importance of hearing in firefighting, their level of awareness about NIHL, their use of hearing protection, their noise exposure on and off the job, and the content of hearing conservation training programs that they have attended. Each focus group included a fire chief or lieutenant as well as regular full-time, part-time, and/or volunteer firefighting staff.
Focus group results indicated that good hearing is crucial for firefighters because most information on the job is communicated verbally. All participants indicated that they knew how to use HPDs correctly and that they were aware of the dangers of loud noise; however, they did not use HPDs regularly. Many misperceptions about noise and noise exposure were uncovered during the focus group sessions, such as the perceptions that NIHL was an unavoidable part of their job, that exposure to noise of short duration regardless of intensity was less damaging to hearing than other types of noise exposure, and that HPD use would diminish their level of hearing and consequently their ability to do their jobs effectively.
Phase 2: Program Design and Pilot Testing
Program design
The program designed for this study was called S.I.R.E.N.—Safety Instruction to Reduce Exposure to Noise and hearing loss. S.I.R.E.N was theoretically based, using a model derived from the Health Promotion Model (Pender, Murdaugh, & Parsons, 2011), Social Cognitive Theory (Bandura, 1997), and the Health Belief Model (Glanz, Rimer, & Lewis, 2002). According to a recent review of behavioral science theory, change theory and explanatory theory are complementary and the strongest interventions may be built from multiple theories (Glanz & Bishop, 2010). As predictors for use of HPDs (dependent variable), the theoretical framework includes several cognitive–perceptual factors (perceived benefits, barriers, self-efficacy of HPD use; susceptibility to and severity of hearing loss) and modifying factors (demographic characteristics; interpersonal influences, e.g., social models and interpersonal support for using HPDs; and situational factors, e.g., choice of HPDs and organizational support for health). The theoretical framework used for this study has been tested, and it demonstrated utility as a causal model for predicting firefighters’ hearing protection behaviors (Hong, Chin, & Ronis, 2011). The content of the intervention was based on strong predictors (self-efficacy, benefits, barriers, and susceptibility) of firefighters’ hearing protection behaviors and prior hearing protection programs designed for factory workers (Lusk et al., 2003) and operating engineers in construction (Hong et al., 2006). In addition, results of the focus group sessions with firefighters (Hong et al., 2008) were incorporated into the intervention. The survey questionnaire for this study was also developed to measure all factors of the theoretical framework. More detailed information on the framework can be found elsewhere (Hong et al., 2011).
A web-based computer application was chosen for ease of availability and autonomous application. The program was designed to be a self-directed, self-study course within a fixed time frame. Throughout the S.I.R.E.N. program design process, the study team focused on making content specific to the high-risk culture of firefighting, appropriate to firefighters’ needs, and reflective of their knowledge and cultural references (Isaacs, 1993; Koch, Andrew, Salamonson, Everett, & Davidson, 2010; Sternberger & Meyer, 2001). Since the program was designed to be presented entirely by computer, it needed to be self-explanatory for participants with a wide range of educational levels. Therefore, information was presented to the users in both audio and visual formats simultaneously (Bracken, Jeffres, & Neuendorf, 2004).
To make the program as accessible as possible, the length was limited to 35 minutes. An online preintervention questionnaire (approximately 17 minutes) and a postintervention feedback form (approximately 6 minutes) were additional. The program was accessed via the web, requiring only standard mouse and keyboard use for the pre- and postintervention questionnaires, as well as the intervention and control programs. All programs for this study were created with Microsoft .NET technology.
Instructional strategies
Two interventions were created for this study: a tailored hearing protection training program for the experimental group and a cardiovascular disease prevention, nutrition, and exercise program for the control group. Both programs contained audio and graphics files; however, the control program was not interactive or tailored to specific user needs. Storyboards, including text, screen graphics, and branching algorithms, were created to plot the flow, content, and feedback loops of the program.
To keep the study unbiased, a randomization algorithm was developed to put people in either the control or experimental group. The algorithm also made sure there were enough participants distributed to both groups. Additional auditing details such as total running time, time spent by each participant, and number of visits were also recorded. The system was developed using Microsoft .NET technologies and deployed as an Internet-based application, so that people from anywhere in the world can access it, without the need to install the program individually on each participant’s computer. A Microsoft SQL Server database was used to save all information captured within the program. The system was also developed keeping touch screens in mind for people who would want to host it on a kiosk at a particular location. Survey questions and answers were programmed into the database and could be easily changed in future, without actually changing any code. We were able to produce the results as a comma-separated file for use with external data-processing systems. The system also has the capability to continue from where it left off, in case there is loss of connectivity or the user decides to pause and continue at a later time.
Drawing on lessons learned from the creation of previous computer-based instruction programs for hearing protection (Eakin et al., 2001; Hong et al., 2006), all learning events were designed in a modular fashion so they could be easily maintained and updated as needed. The learning events were presented in sequential order, building on information from prior events. In addition, the program was tailored to each individual participant by taking into account his or her responses to the preintervention survey questionnaires and to the program learning events. This approach is unique to web-based health communication programs and was designed to provide the most pertinent content to participants while bypassing information not reflective of the user’s individual needs (Bensley et al., 2004).
Based on focus group data and expert consultant input, the study team determined that program information needed to be presented in a fast-paced, entertaining, and relevant format. Interactive, asynchronous learning methods were used to reinforce information in a fast-paced format to increase participants’ attention. Arcade-style games and extended, user-driven role-play scenarios with branching story lines were created in order to allow participants to experience the outcomes of effective and ineffective choices regarding HPD use. Frequent positive reinforcement was included to promote learning and retention (Dihoff, Brosvic, Epstein, & Cook, 2004). These methods were selected to provide confidentiality for participants and to allow maximum opportunities for participants to acquire and to practice specific skills.
Informed consent for this study was included in the web-based program and was obtained from all participants in the pilot test and in the test of program efficacy before completing the pretest survey. The computer program randomly assigned participants to either the hearing protection intervention program or the cardiovascular disease prevention control program. Questionnaire data, as well as key data points in the intervention program, were collected electronically and stored in a secure electronic database that was easily generated and maintained via computer. Information on program randomization, total running time, and time spent on each individual learning event and the number of return visits was recorded.
Pilot testing
Process evaluations of program content and function were conducted multiple times during the design phase. After a model of the program was created, members of the research team tested the program and reevaluated the logic algorithms. Next the program was evaluated by consultants and fire chiefs in target locations to assess content and appropriateness to the learner population. The program was then tested by about 20 firefighters from a fire department in the San Francisco Bay area, which was not a part of targeted fire departments for the larger evaluation. Their comments and suggestions were used to improve the program. The program tested in this stage was not a beta test of the program but rather a working copy used to assess the overall function and program design. For this reason, firefighter members of the targeted fire departments for the larger study were not included in this pilot testing phase.
Phase 3: Assessment of Program Efficacy
Between January 2010 and January 2012, a total of 495 firefighters from 35 fire departments in three states (California, Illinois, and Indiana) participated in the S.I.R.E.N. program. About 47% completed the hearing protection intervention program and 53% completed the cardiovascular disease prevention control program. By the randomization, there were no statistically significant differences in the baseline characteristics between the intervention and control groups. Of those who completed the programs, more than 75% (372/495) completed feedback survey. This article included those (N = 372) who completed the feedback survey in the analysis.
Although it is beyond the focus of the present study, it should be noted that the efficacy of the intervention in increasing firefighters’ use of hearing protection will be assessed when postintervention surveys are completed. Currently the project is conducting postintervention follow-up data to assess the participants’ behavioral changes in the use of HPDs. Both short-term and long-term effects measured at 3 months and 12 months after the intervention, respectively, will be determined, and the results will be published in the near future.
Results
Characteristic of Participants
Demographic characteristics of the participants are summarized in Table 1. Participants ranged in age from 23 to 61, with a mean age of 44 years. The majority of participants were Caucasians (82%) and males (95%). On average, they had been working in fire services for 17 years, with 85% reporting daily or weekly exposure to loud noise while working as a firefighter. The majority of participants (85%) perceived their hearing as good or better. Although all firefighters should use hearing protection 100% of the time that they are exposed to loud noise, participants in this study reported only 34% actual use and 55% intention to use in the next 3 months at preintervention. The majority of firefighters in this study (90%) rated their computer/Internet skills as average or above average.
Demographic Characteristics of the Study Participants (N = 372)
NOTE: HPD = hearing protection device.
Intention to use of HPDs in future: What percentage of the time will you use hearing protection in the next 3 months when you are exposed to loud noise?
Overall Feedback
Participant response and feedback regarding the Internet-based training was very positive (see Table 2). The majority of participants said the hearing protection intervention program was well organized (97%), easy to use (97%), and effective (98%) and held their interest (79%). More than 95% of participants indicated that logging on to the training program was easy for them, that they had no problem using the computer for this program (94%), and that they liked receiving training by computer (83%). Because of what they learned, 88% of participants in the hearing protection intervention group reported they would do more to protect their hearing, whereas 69% in the cardiovascular disease prevention control group reported they would do more to protect their cardiovascular health. Almost all participants (95%) said they would recommend this Internet-based training program to other firefighters.
Participants’ Satisfaction With the Internet-Based Training Program (N = 372)
Best and Worst Features of the Program and Lessons Learned
Of those who completed the feedback survey, about 79% (295/372) provided written comments to questions about best features. About 45% (167/372) and 22% (81/372) provided their thoughts on the worst features and suggestions for improving the intervention programs. The majority of comments for both the best and worst features were primarily related to the use of computer technology and web-based format. Participants liked the computerized online program; they thought it was convenient, quick, and easy to use; and they liked having information presented in an interactive multimedia format. Participants also thought that the information presented was useful, and they liked the logical progression of the program.
On the other hand, participants reported that the worst features of the program were related to computer problems, long download times, Internet or web browser problems, and lack of human interaction and instant feedback from the instructor or trainer. Several participants, particularly firefighter paramedics in the cardiovascular disease prevention control group, also reported that the content was too basic for their level of training and could have been more sophisticated overall. Additionally, several participants from a Midwestern fire department, the largest participating department with more than 4,500 uniformed members, reported that the content of the survey and the hearing protection intervention was not applicable to them since HPDs are not provided in their work environments. Education components, scripts, and learning events were all written with this in mind. Participants indicated that they liked the convenience of the program, the ability to access it at any time from any computer, the audiovisual style, and the interactive and tailored components. However, problems with variability in users’ Internet providers, computer processing speeds, and the ability of a computer to download and/or stream audio and video components had a significant impact on the speed of the program and consequently on an individual’s experience with the program. In fact, the majority of comments regarding the worst features of the program included web-based or computer problems such as slow page loading speed, problems with synchronization of audio and graphic files, and having to click each time when changing pages.
Because the program included interactive components, it was not possible to program it solely with standard web page HTML language. To provide a truly interactive experience, a Microsoft .NET format was selected to allow for embedded video, audio, and Flash components. Since not all computers were able to process these components in the same manner, some users encountered a less than desirable interface with the program. One solution may be to create a version of the program that could be downloaded from the web and run directly from a user’s computer, thus bypassing potential problems with the Internet.
Another aspect of the program that participants liked was the ability to exit the program and reaccess it at a later date at the point that they left off. This feature was especially important for firefighters because of the unpredictable nature of their work hours and environment. Unfortunately, this feature did not work as anticipated for all users, allowing them to reenter the program only at the beginning, not where they left off. This was problematic because the program was not designed to allow users to skip ahead or to navigate nonsequentially through the program. A solution to this problem would be to create navigation tabs or menu options to allow users to select where they want to go in the program. To prevent users from skipping critical content, navigation could be designed to route them to key components of the program and to require sequential presentation of learning events within those specific components.
Finally, many users commented on the content of the program. Users of both the hearing protection intervention program and the cardiovascular health control program observed that they liked the content and learned from it, whereas others commented that there was either too much or too little information, that the program was too simplistic, or that it was irrelevant to their work/life situation. As stated earlier, content for this program was theoretically based and drew on prior experience with computer-based hearing protection programs as well as the results of focus group sessions and content experts. Participant suggestions for improving the content included more tailoring, more interactive components, the option to navigate nonsequentially through the program, more testimonials from firefighters, more examples of typical and atypical noise exposures, more information on choosing the best type of hearing protection, and inclusion of a menu or program syllabus. Participants also suggested that to improve the training program, it should be mandatory for all firefighters. As with any training program, it would not be possible to create a learning experience that was completely satisfactory for every user. However, many of the content suggestions are useful in creating a more personal, tailored, and interactive learning experience, something that is not typical of most health education programs available on the web today.
Conclusion
This study demonstrated that interactive multimedia computer technology using the Internet is a feasible mode of delivery for a hearing protection intervention among firefighters. This is consistent with previous research suggesting that computers are a useful method for delivering hearing protection education to a variety of populations exposed to NIHL, including construction workers (Collins et al., 2004; Eakin et al., 2001; Hong et al., 2006) and factory workers (Lusk et al., 2003). Internet-based programs are advantageous because of their cost-effectiveness and their ability to be widely disseminated to a large number of individuals and populations (Bond, 2006). Internet-based programs are easily accessible, time efficient, and flexible, allowing individuals to access them according to their own schedule (Bull et al., 2001). Flexibility and convenience are particularly important for firefighters, who often face unpredictable work hours and environments. Furthermore, Internet-based programs allow for tailored content based on an individual participant’s needs.
Future Internet-based programs should focus on developing more sophisticated content, including more real-life examples and testimonials, and maximizing individual tailoring and participant interaction. Future programs should also take steps to improve the user interface by including a menu or program syllabus, facilitating better navigation through the program, and devising alternative methods to deliver the program that are not affected by individuals’ personal computers or Internet connections. However, despite these recommendations for improvements, participants’ favorable feedback in this study strongly supported the continued use of a web-based computerized approach for designing and developing interventions to promote healthy behaviors among firefighters.
Footnotes
Authors’ Note
This work was supported by the U.S. Department of Homeland Security, Federal Emergency Management Administration Assistance to Firefighters Grant (Grant No. EMW-2007-FP-00785; PI: O. Hong). The contents are solely the responsibility of the authors and do not necessarily represent the official views of the U.S. Department of Homeland Security, Federal Emergnecy Management Administration. The authors would like to acknowledge firefighter union leaders and fire chiefs for their collaborations and all of the participants from fire departments in California, Illinois, and Indiana in the United States.
