Regional Collaboration Among Urban Area Security Initiative Regions: Results of the Johns Hopkins Urban Area Survey

Abstract

Regional collaboration has been identified as a potential facilitator of public health preparedness efforts. The Urban Area Security Initiative (UASI) grant program, administered by the Federal Emergency Management Agency (FEMA) since 2003, has provided 64 high-risk metropolitan areas funding to enhance their regional preparedness capabilities. This study describes informal and formal regional collaboration infrastructure, as well as regional collaboration–related activities and assessment methods, in FFY2010 UASI regions. A cross-sectional online survey was administered via Survey Monkey from September through December 2013. Points of contact from FFY2010 funded UASI metropolitan areas completed the survey, with a response rate of 77.8% (n=49). Summary statistics were calculated to describe the current informal and formal regional collaboration infrastructure. Additionally, the cross-sectional survey collected rates of agreement with 8 collaborative preparedness statements at 3 time points. The survey found that UASI regions are engaging in collaborative activities and investments to build capabilities, with most collaboration occurring in the prevention, protection, and response mission areas. Collaborative relationships in preparedness among emergency managers and municipal chief executive officers improved during the FFY2010 UASI performance period compared to the pre-UASI award period, with lasting effects. The majority of UASI regions reported conducting independent assessments of capabilities and their measurement at the UASI region level. Urban areas that received a FFY2010 UASI grant award are engaging in collaborative activities and have established interjurisdictional relationships in preparedness. The use of grant funds to encourage collaboration in preparedness has the potential to leverage limited resources and promote informed investments.

Collaboration among local, state, and federal public health agencies, as well as with other homeland security–related federal agencies, is necessary for enhanced public health preparedness,¹ and efficient emergency management has been associated with intergovernmental coordination of planning efforts at all levels of government.² While all disasters have a local component, local governments may lack sufficient resources to handle disasters on their own. Moreover, disasters do not recognize geopolitical boundaries and often affect more than one jurisdiction at a given time. In fields other than preparedness (eg, public administration), regionalism^* has been shown to be cost-effective, for example, by promoting resource sharing and reducing duplicative efforts.³ In public health, regionalization has been identified as a critical dimension of transformation of the public health system to execute increased demands of preparedness.⁴

Since 2003, the Federal Emergency Management Agency (FEMA) has administered the Urban Area Security Initiative (UASI) grant program. The program requires regional governance and collaboration to promote public health and safety as a condition of funding, but it does not require or collect information on collaboration during specific preparedness-related activities. It is the highest funded grant (in dollars allocated) under FEMA's Homeland Security Grant Program umbrella, administering approximately $8 billion in grant funding from federal fiscal year (FFY) 2003 until FFY2014.^5-8 Grant recipients are major metropolitan areas in the United States that have been determined to have the highest risk for terrorism. In FFY2010, the UASI program funded the most metropolitan areas (n=64) in its history, representing 32 US states, the District of Columbia, and Puerto Rico.⁵

In 2004, the Government Accountability Office (GAO) reported that regional collaboration could be enhanced by the development of a comprehensive strategic plan with measurable goals and objectives; the presence of a regional organization with interdisciplinary representation; and flexibility in membership requirements, collaboration processes, and areas that have traditionally engaged in collaborations (ie, other than emergency management).⁹ The report proposed that the federal government could support these efforts through grant programs and associated requirements. In 2007, the Department of Homeland Security (DHS) identified “Increasing Regional Collaboration” as the first of its 10 homeland security priorities.¹⁰ Despite this increased attention and funding, in 2009 the GAO criticized FEMA for failure to track the effectiveness of the UASI grant program's goal of regional collaboration.¹¹ In 2011, Congress called on the National Academy of Public Administration (NAPA) to aid FEMA in helping to investigate, develop, and implement quantifiable homeland security grant performance measures, specifically related to the UASI program.¹² Despite the fact that regional collaboration is a goal of the UASI grant program,^5,11 NAPA did not develop any performance measures for regional collaboration and instead recommended that FEMA develop a multidisciplinary team to study this area in the future.¹²

In the interim, the National Association of County and City Health Officials (NACCHO) established 4 approaches to regionalizing public health services in emergency preparedness: coordinating, standardizing, centralizing, and networking; these are described in detail elsewhere.¹³ In 2013, NACCHO surveyed 2,532 local health departments across 48 states, with an overall response rate of 79%. Local health departments were randomly assigned to receive 3 question subsets (core questions; core questions plus Module 1; core questions plus Module 2, which included questions on preparedness), and reported statistics were weighted to account for sampling/dissimilar nonresponse. The study found that 52% of local health departments were engaging in cross-jurisdictional sharing of resources. Of all programmatic areas and organizational functions reported, the highest percent of local health departments reported cross-jurisdictional sharing of resources among departments in emergency preparedness (35%).¹⁴ Adopting a regional approach to planning has been identified as a successful preparedness collaboration technique,¹⁵ and regional planning models and recovery frameworks have already been developed.^16,17

Formal and informal infrastructure, as well as activities and accomplishments, in preparedness-related regional collaboration have been preliminarily explored in a small number of case studies.^18-20 A comparative look across case studies found regionalization to be preceded by a variety and combination of impetuses, including enhancing local public health capacity, a perceived need for a coordinated response, or to more efficiently coordinate federal preparedness funding. Activities also varied; some cases identified a focus on developing more formal regional organizations, while others focused on building more informal social networks.²¹ For example, regionalization among Massachusetts's 351 autonomous public health boards was incentivized by the need to efficiently allocate and use federal funds.^18,19,22 This resulted in increased efficiency, coordination, standardized operations, capacity, perceived value of public health as a partner among other responding agencies, mutual aid agreements, capability development, and regional social network capacities, as well as facilitated regional training sessions, enhanced response to real-time challenges, and the development of a forum for increased communication with other emergency response agencies.¹⁸ Barriers to regionalization included funding, multiple mandates, cultural differences among public health and more operationally focused disciplines, and overlapping regional systems in different disciplines.¹⁸

In one particular preparedness region in Massachusetts, regionalization work was described as bridging the gap between state and local response capacities. Accomplishments in this region included development of local and regional plans; upgraded emergency response equipment and supplies; strengthened relationships with other first responders; developing, conducting, and participating in training, exercises, and drills to test local and regional capacity; and developing regional services and capacities.¹⁹ In the national capital region surrounding Washington, DC, most collaborative efforts focused on coordination, and some new activities led to regional capacity-building, although with little focus on standardization. The UASI program's requirement of a regional approach was perceived to be beneficial in that it forced the region to come together. However, some stakeholders believed that the grant's focus on equipment did not support the heavy human resource burden of preparedness in public health.²⁰

Anecdotally, local health departments have demonstrated active participation in the collaborative activities of UASI regions. The 2014 National Homeland Security Conference included a separate track entitled “The Emergency Medical Response Community” for sharing of best practices among the UASI public health and medical community, including those on regional health and emergency medical planning, hospital preparedness, and mental health.²³ In the Baltimore UASI, for example, local health departments across funded municipalities actively participate in the health and medical committee, the primary group responsible for all core capabilities that are health and medical related, including Environmental Response/Health and Safety, Fatality Management Services, and Public Health and Medical Services. Moreover, they are instrumental in the region's gap analysis and capability assessment that inform the UASI's strategic policies and priorities.

Although these case studies examined regionalization efforts in public health preparedness in particular geographic areas, little empirical evidence exists about regional preparedness activities or incentives occurring on a national scale in metropolitan areas with high risk or that are encouraged by a federal grant. Moreover, it remains unclear whether homeland security funding is a driver behind successful development of formal and informal regional collaboration infrastructure, and what constitutes such infrastructure, especially in major urban areas at high risk and whose health departments serve a large proportion of the population. To contribute to the evidence base, this study explores if and how financial incentives, in the form of federal preparedness grants, can build or enhance regional collaboration in preparedness, as well as how UASI regions formally and informally engage in and measure regional collaboration throughout the disaster cycle. In the context of this study, regional collaboration is defined as collaboration among independent municipalities funded by the same UASI grant, encompassing a wide range of activities (eg, joint planning, training, operations) aimed at coordinating the capabilities and resources of various entities (eg, agencies, organizations, and individuals from many tiers of public and private sectors) for the common purpose of preventing, protecting against, responding to, and recovering from intentional as well as natural threats to people or property.

Methods

A brief (12-question) internet-based survey, the Johns Hopkins Urban Area Survey Tool (JHUAST), was developed in coordination with practice-based partners familiar with the UASI program. The survey incorporated questions about structure and funding of regional collaboration among UASI recipient jurisdictions. Respondents were asked about regional collaboration measurement methods and use of regional collaboration strategies during exercises or real-world events.

The survey was administered via SurveyMonkey (SurveyMonkey.com, Portland, OR) and distributed by email to the point(s) of contact for each UASI in September 2013 (n=63). If more than 1 contact per UASI region existed, 1 email was sent to all points of contact for that region and they were asked to coordinate to complete a single survey on behalf of the UASI. All FFY2010 UASI recipient regions were recruited to participate; however, the FFY2010 Miami and Fort Lauderdale UASI areas were combined for the purposes of this investigation secondary to the later consolidation of their individual UASI programs.

Recipients were asked to respond within 6 business weeks. Three reminder emails were sent to UASI contacts during this period. Incorrect contact information was corrected on an ongoing basis. At the conclusion of the 6-week period, nonrespondents were sent an email allowing an additional 2 weeks to respond. If after a week they had still not completed the survey, they were sent an additional reminder email. At the expiration of this extension deadline, the study team worked with practice-based partners at the Baltimore City Mayor's Office of Emergency Management to encourage additional responses. The Baltimore UASI chairman contacted by phone all remaining nonresponding UASI regions for which contact information was available to secure recruitment. The survey was officially closed in December 2013.

Summary statistics were calculated from the survey responses to describe the current informal and formal regional collaboration infrastructure. Additionally, the cross-sectional survey collected rates of agreement with 8 collaborative preparedness statements at 3 time points: before receipt of any UASI grant award (ie, before the region's first UASI award, varying from 2003-2010 across UASI regions); during the FFY2010 UASI award Performance Period (ie, 36 months from 2010 until 2013); and after the conclusion of the FFY2010 UASI Performance Period (ie, from the grant's conclusion in 2013 until the time of survey response) for any metropolitan regions for which UASI funding was discontinued after FFY2010 (UASI regions that continued to receive funding after FFY2010 were asked to abstain from this question, n=22). The precise start and end dates of the grant's performance periods may have differed across states due to differences in grant administration, and the survey was administered within months of the conclusion of the FY2010 UASI award's performance period. Because UASI grant performance periods are up to 36 months, the survey requested that respondents report on collaborative preparedness statements before receipt of any UASI award to avoid confusion about overlapping performance periods with prior awards. Collaborative preparedness statements were developed in coordination with practice-based partners and were designed to address leadership engagement in key collaboration-building activities across phases of the disaster cycle and that survey respondents would be aware of.

Levels of agreement were collected on a Likert scale (with 1 indicating strong disagreement, 2 disagreement, 3 neither agreement nor disagreement, 4 agreement, and 5 strong agreement) with a “don't know” option. Mean scores and mean difference in scores across the 3 time points were calculated for matched pairs in the cross-sectional sample.

For each of the collaborative preparedness statements, a Skillings Mack test was performed to determine if there were differences among any of the 3 time-specific self-reported scores. While the Skillings Mack test determines significant differences between scores for each collaborative preparedness statement, it does not indicate between which scores the differences exist. Thus, following a significant p-value (≤.05) from the Skillings Mack test, 3 Wilcoxon Signed Rank tests were performed to determine significant differences among pairwise comparisons (ie, before–during; during–after; before–after). The experiment-wise error rate of 0.05 was adjusted by the number of comparisons being made, and significance at the pairwise comparison level was determined at an alpha of ≤.0167 (0.05/3=0.0167). “Don't know” respondents were dropped prior to calculating overall mean scores for each collaborative preparedness statement, as well as during calculation of the difference in means (ie, any individual who responded “don't know” to a statement at either time point in a particular comparison was excluded from analyses for that comparison). “Don't know” scores were dropped only following a sensitivity screening where analyses were repeated with (1) “don't know” values excluded and (2) “don't know” values replaced with “neither disagree nor agree.” No substantial difference in the magnitude or direction of the means or mean difference was detected across these 2 analyses. Therefore, it was determined that dropping the “don't know” responses did not have a substantial impact on the overall findings. The software used for these analyses included STATA version 10 (STATACorp LP, College Station, TX; 2013) and Microsoft Excel (Microsoft, Redmond, WA; 2011).

The Johns Hopkins Bloomberg School of Public Health Institutional Review Board reviewed this study and determined it to be not human subject research.

Results

Of 63 UASI areas, 49 (77.8%) responded to the JHUAST. Twenty-two responding urban areas (44.9%) reported loss of UASI funding after FFY2010. Results represent data from all respondent UASI areas surveyed, except where otherwise indicated.

Regional Collaboration Activities by Mission Area

Urban areas were asked to report on which non–grant-required activities they engaged in with their regional UASI partners during the 3-year FFY2010 UASI Grant Performance Period. Table 1 describes the respondents in each mission area (prevention, protection, mitigation, response, and recovery) that indicated FFY2010 UASI grant–funded municipalities within the metropolitan region worked together during the grant's performance period for each activity.

Table 1.

Percent UASI Regions Reporting Collaborative Activity Participation by Mission Area (n=49)

Activity	Multi-municipal Plan Development	UASI-Sponsored Exercises ^a	State-Sponsored Exercises ^b	Federally Sponsored Exercises ^c	Information/Intelligence Sharing	Operational Coordination	UASI-Sponsored Multi-municipal Public Information Campaigns
Mission Area	% Yes	%Yes	% Yes	%Yes	% Yes	%Yes	% Yes
Prevention	81.63	55.10	40.82	22.45	87.76	69.39	71.43
Protection	81.63	63.27	40.82	24.49	85.71	71.43	63.27
Mitigation	67.35	48.98	34.69	20.41	59.18	57.14	53.06
Response	87.76	77.55	61.22	32.65	81.63	83.67	59.18
Recovery	61.22	42.86	30.61	18.37	44.90	63.27	57.14

Most of the funding/organization done by the UASI.

Most of the funding/organization done by the state.

Exercises administered through the National Exercise Program.

Across mission areas, UASI regions reported working together the most during multi-municipal plan development (61.22-87.76%), for operational coordination (57.14-83.67%), and on UASI-sponsored multi-municipal plan development (53.06-71.43%) during all mission areas. Conversely, UASI regions reported working together the least during federally sponsored exercises, corresponding to generally lower rates of collaboration during state and UASI-sponsored exercises across all 5 mission areas. Fewer respondents reported working together during the recovery and mitigation mission areas. More respondents indicated collaborative activity during the protection and response mission areas.

Funds Spent on Regional Capability Enhancement

Respondents were asked to report what percentage of their FFY2010 UASI award was spent on regional capability enhancement, described as “equipment, resources or personnel considered the joint property and/or for the joint use of all jurisdictions within the UASI; not intended for the primary use of any individual municipality.” Forty-nine percent of respondents indicated spending 75% to 100% on such investments, and 74% of respondents indicated spending the majority of their FFY2010 UASI award (>50%) on regional capability enhancement. Only 2% of respondents indicated that no funds were spent on regional capability enhancement, and 10% of respondents indicated that less than one-quarter of funds were spent on this purpose.

Changes in Relationships

Respondents were asked to rate their level of agreement with a series of statements representing collaborative relationships and preparedness activities for both local emergency managers and municipal chief executive officers in UASI regions before any UASI grant award, during the FFY2010 UASI grant award performance period, and after the FFY2010 UASI grant award performance period if FFY2010 was the last year the grant was received. Higher Likert scores (level of agreement) with collaborative preparedness statements was indicative of greater perceived collaboration.

Table 2 reports the numerical mean Likert scores and the mean difference between Likert scores for matched responses for agreement with each of the collaborative preparedness factors before receipt of any UASI grant award and during the FFY2010 UASI award performance period. For all collaborative preparedness statements, respondents indicated significant increases in agreement with each of the collaborative preparedness statements (mean difference=0.809-1.213, p-value ≤0.0167) during the FFY2010 UASI grant performance period compared to before any UASI grant award.

Table 2.

Comparisons in Self-Reported Levels of Agreement (5-point Likert Scale Scores) with Collaborative Preparedness Statements Before and During UASI Award

Collaborative Preparedness Statement	Mean Likert Scale Score During	Standard Deviation Likert Scale Score During	Mean Difference in Likert Scale Scores (During–Before)	Standard Deviation in Likert Scale Scores (During–Before)	P-Value (Wilcoxon Signed Rank Test)
The emergency managers within my UASI knew each other's names and how to contact one another. (n=47)	4.830	0.524	1.021	1.189	<0.001
The emergency managers within my UASI worked with each other on preparedness activities prior to an emergency. (n=47)	4.702	0.779	1.213	1.267	<0.001
The emergency managers within my UASI worked with each other during an emergency that affected only one municipality within the UASI metropolitan region. (n=46)	4.413	0.979	0.913	1.244	<0.001
The emergency managers within my UASI worked with each other during an emergency that affected the majority of the UASI metropolitan region. (n=47)	4.426	0.972	0.809	1.191	<0.001
The municipal chief executive officers within my UASI knew each other's names and how to contact one another. (n=46)	4.565	0.834	0.848	1.173	<0.001
The municipal chief executive officers within my UASI worked with each other on preparedness activities prior to an emergency. (n=45)	4.200	1.120	1.133	1.179	<0.001
The municipal chief executive officers within my UASI worked with each other during an emergency that affected only one municipality within the UASI metropolitan region. (n=43)	4.163	1.090	0.909	1.042	<0.001
The municipal chief executive officers within my UASI worked with each other during an emergency that affected the majority of the UASI metropolitan region. (n=44)	4.250	1.059	0.841	1.098	<0.001

Table 3 reports the numerical mean Likert scores and the mean difference between Likert scores for matched responses for agreement with each of the collaborative preparedness factors before receipt of any UASI grant award and after the FFY2010 UASI award performance period if UASI funding was discontinued after FFY2010. For all but 2 collaborative preparedness statements—“The emergency managers within my UASI worked with each other during an emergency that affected the majority of the UASI metropolitan region” and “The municipal chief executive officers within my UASI knew each other's names and how to contact one another”—respondents indicated significant increases in agreement with each of the collaborative preparedness statements (mean difference=0.722-0.900, p-value≤0.0167) after the conclusion of the FFY2010 UASI performance period if UASI funding was discontinued after FFY2010 compared to before any UASI grant award. In addition, if UASI funding was discontinued after FFY2010 (range, n=20-22), there was no significant decrease in levels of agreement with any of the collaborative preparedness statements compared to during the FFY2010 UASI performance period (data not shown).

Table 3.

Comparisons in Self-Reported Levels of Agreement (5-point Likert Scale Scores) with Collaborative Preparedness Statements Before and After UASI Award

Collaborative Preparedness Statement	Mean Likert Scale Score After	Standard Deviation Likert Scale Score After	Mean Difference in Likert Scale Scores (After–Before)	Standard Deviation in Likert Scale Scores (After–Before)	P-value (Wilcoxin Signed Rank Test)
The emergency managers within my UASI knew each other's names and how to contact one another. (n=20)	4.650	0.933	0.900	1.293	0.009
The emergency managers within my UASI worked with each other on preparedness activities prior to an emergency. (n=20)	4.150	1.226	0.900	1.294	0.006
The emergency managers within my UASI worked with each other during an emergency that affected only one municipality within the UASI metropolitan region. (n=20)	4.150	1.182	0.800	1.056	0.002
The emergency managers within my UASI worked with each other during an emergency that affected the majority of the UASI metropolitan region. (n=20)	4.050	1.234	0.550	0.999	0.028
The municipal chief executive officers within my UASI knew each other's names and how to contact one another. (n=19)	4.263	1.147	0.895	1.449	0.023
The municipal chief executive officers within my UASI worked with each other on preparedness activities prior to an emergency. (n=18)	3.611	1.378	0.833	1.249	0.009
The municipal chief executive officers within my UASI worked with each other during an emergency that affected only one municipality within the UASI metropolitan region. (n=18)	4.000	1.237	0.833	1.043	0.003
The municipal chief executive officers within my UASI worked with each other during an emergency that affected the majority of the UASI metropolitan region. (n=18)	3.889	1.278	0.722	1.074	0.009

Moreover, most UASI regions (88%) reported that they have sufficiently operationalized regional collaboration such that they would work or have worked together as a region during a real-world event. In fact, 92% of respondents reported the ability to share resources among FFY2010 UASI-funded municipalities in their region during a real-world event, without state involvement.

Measuring Regional Collaboration

Figure 1 shows the percent of respondents engaging in different types of measurement methods for real-world events and exercises. Most respondents reported that an assessment of regional collaboration was included in after-action reports of one or more individual municipality(ies) in the UASI during real-world events (61.2%) and exercises (69.4%). More respondents reported development of UASI-wide after-action reports following exercises (51.0%) compared to real-world events (30.6%). However, the same proportion of respondents reported conducting a UASI-wide hotwash (ie, debriefing) (44.9%) during real-world events, as well as exercises. Only 22% and 26.5% of respondents reported measuring regional collaboration through contribution to a state-wide after-action report for a real-world event or exercise, respectively. Only 16.3% and 12.2% of respondents reported that their UASI has not measured regional collaboration.

Figure 1.

Regional Collaboration Measurement Methods During Exercises and Real-World Events (n=49)

Over half (55%) of UASIs reported conducting an independent assessment of efforts to enhance emergency management capabilities (excluding a FEMA-sponsored assessment). Thirty-three percent indicated that they had not conducted such an assessment, and 12% did not know. Over three-quarters of respondents indicated that they measure preparedness capabilities at the UASI level, 49% reported measuring these capabilities at the state level, and 69.2% reported measuring them at the municipality level (Figure 2).

Figure 2.

Level at Which UASIs Measure Capabilities

Discussion

An evidence-supported understanding of the relationship of regional collaboration and preparedness can guide practice and policy decisions, including levels of federal, state, and local public health investments in preparedness.¹ Research into the topic of regional collaboration and national preparedness can inform investments made by the US government. This analysis provides a unique opportunity to understand how federal government grant funds are being used to support the advancement of regional collaboration. In an era of reduced resources, grant programs aimed at regional collaboration may allow for a cost-effective solution to sustainability and streamlining of resources, decreasing geographical redundancies in capabilities, and promoting the development of relationships that will be necessary in the event of a large-scale disaster.

Between FFY2003 and FFY2009, over 500 million UASI dollars were spent on planning,⁵ but the types of planning activities this investment supported remained unclear. Results of this investigation show that UASI regions are working to develop multijurisdictional plans (ie, regional plans) among their member jurisdictions across all 5 mission areas (Table 1). Moreover, UASI regions report translating these plans into action; most UASI regions reported working with their UASI-funded jurisdictions to operationally coordinate and 92% of UASI regions reported capacity for resource sharing among the municipalities funded by the FFY2010 UASI award during a real-world incident or event (ie, without state involvement). This demonstrates evidence of multijurisdictional planning efforts and translation to implementation during real-world disasters. Structural variability in the nation's more than 3,000 local public health agencies has been identified as an impediment to coordinating with other emergency response disciplines, especially during disasters that cross geopolitical borders.⁴ These results indicate that the UASI program may be able to encourage effective collaboration and promote efficiency and timeliness in the wake of disasters by eliminating the need for state involvement for resource-sharing purposes.

Although most UASI regions are spending funds on regional versus jurisdiction-specific capability enhancement, 10% of UASI regions still reported spending less than 25% of their FFY2010 UASI award on regional capability enhancement, and 4% reported that they don't know how funds are being spent with respect to jurisdictional or regional capability enhancement. This may be indicative of a need for increased oversight to ensure grantees are, in fact, investing in regional resources and regional capability enhancement and/or additional clarification or education on the grant's goals and purposes. While it is interesting to note that most UASI regions invested most of their FFY2010 UASI grant in regional capabilities, it is not yet known if and how this investment is associated with preparedness outcomes. Additional research exploring the association between the amount of funds spent on regional capability enhancement and preparedness outcomes is necessary.

Additional research to assess and quantify the development of social capital through regionalization has already been proposed,¹² and development of social capital has been posited as the strongest potential of regionalization.²⁴ A report on ongoing collaboration among local health departments found 67% of them collaborating, 12% cooperating, 12% coordinating, and 7% networking in the emergency preparedness programmatic area. The largest percentage of local health departments reported collaborating in emergency preparedness compared to all other programmatic areas reported. Notably, 2% of local health departments indicated that they were not involved in partnerships or collaborations or had no program in the emergency preparedness programmatic area, the lowest level compared to all other programmatic areas reported.¹¹ Results from JHUAST suggest that UASI regions may have experienced increased perceptions of collaborative preparedness after receipt of the UASI award, with results lasting beyond the performance period of their award (Tables 2 and 3). These findings lend themselves to the hypothesis that grant programs focused on regional collaboration in public health and public safety may have the potential to increase collaborative preparedness, with lasting effects. In addition, collaboration of this type may be a sign of improved preparedness. Additional research comparing UASI regions to non-UASI regions while controlling for demographic differences and other potential modifiers should be conducted to determine if the observed increase in agreement with collaborative preparedness statements is, in fact, associated with the UASI grant award and with improved preparedness. Additional perspectives on changes in collaboration (eg, among different disciplines and/or ranks of emergency responders) may also be sought to further assess the scope of perceived changes in collaborative preparedness activities. Additionally, since this survey was conducted only a few months after the conclusion of the FFY2010 UASI award performance period, additional queries to determine long-term changes in collaborative preparedness activities over time should be performed.

While this study did not aim to identify or develop metrics to measure regional collaboration, we did find that most UASI regions (55%) have conducted independent assessments of efforts to enhance emergency management capabilities (excluding a FEMA-sponsored assessment). Moreover, 75.5% of UASI regions reported measuring their emergency management capabilities at the UASI region level. While no standardized or “official” regional collaboration measurements exist, it is clear that some measurements are occurring in practice. These should be explored and relevant ones used to inform ongoing efforts to improve the science of preparedness measurement and metric development. Moreover, as most UASI regions reported assessment of regional collaboration in municipality-level after-action reports following real-world events as well as exercises, these documents should be systematically reviewed to identify variables that are consistently collated and used to convey successful or flawed regional collaboration or capability enhancement.

Strengths and Limitations

Strengths of this study include its use of a population-level survey and a high response rate. Additionally, the research team capitalized on practice-based partnerships to ensure practice-based research relevance, well-designed survey questions, and recruitment of participants. Although the research team attempted to reduce limitations through study design, some remain.

First, contact information was not complete or up to date for all UASI regions, in part because of personnel turnover. Additionally, the contact on file for the UASI program may not have been the most appropriate person to complete the survey or comment specifically on collaboration (eg, a new hire, an individual whose job responsibilities exclusively focused on grant management, or who did not interact or was not aware of the relationships among municipal chief executive officers). Contact information was corrected on an ongoing basis by the study team.

Second, because most FFY2010 UASI recipients were receiving multiple federal homeland security/emergency management grants, and some were in the midst of multiple UASI grant award performance periods, it is not possible to infer whether the regional collaboration activities and perceptions identified by the JHUAST were a result of the FFY2010 UASI award, or any UASI award. Notably, this study sought only to describe activities of UASI regions, not to imply causation by or association with receipt of a UASI grant or other federal resources. Future research should collect and analyze data from UASI regions and non-UASI regions to determine if engagement in collaborative activities is modified by receipt of a UASI grant award.

Third, the collaborative preparedness statements attempted to capture agreement at 3 distinct time points (ie, before, during, and after) in a single round of data collection. However, the cross-sectional study design may have resulted in recall bias among respondents. Moreover, because the study relies on self-reported data, factors such as participant understanding of the survey questions and interpretation and use of rating scales may have affected responses and the associated validity of the findings. Finally, although survey questions focused on activities described in previous research to have public health participation, public health–specific capabilities were not explicitly addressed.

Conclusion

Metropolitan regions funded by a FFY2010 UASI award engaged in a variety of collaborative public health preparedness and emergency management activities across the disaster cycle. They reported the development of cross-jurisdictional collaborative relationships in preparedness, ability to operationally coordinate and share resources during a disaster, and assessments of their regional efforts. Additional research should be conducted to better understand the relationship of regional collaboration and overall national preparedness, as well as methods to assess and incentivize it. An evidence-informed understanding of regional collaboration and national preparedness can promote efficient government spending and enhanced public health and safety.

Footnotes

Acknowledgments

The authors wish to recognize the following individuals for their invaluable support in the development and execution of this study: Carol Thompson, MS, MBA, Biostatistics Center, Johns Hopkins Bloomberg School of Public Health, and Steve Davis, All Hands Consulting. This work was supported by the Federal Emergency Management Agency's National Exercise Division through the Department of Homeland Security's Science and Technology Directorate.

References

Frist

. Public health and national security: the critical role of increased federal support. Health Aff, 2002; 21(6):117-130.

National Emergency Management Association website. Updated 2014. www.nemaweb.org. Accessed April 23, 2014.

Caruson

, MacManus

, Kohen

, Watson

. Homeland security preparedness: the rebirth of regionalism. Publius J Federalism, 2005; 33(1):143-168.

Salinsky

, Gursky

. The case for transforming governmental public health. Health Aff, 2006; 25(4):1017-1028.

National Urban Area Security Initiative Association. A Report on the Effectiveness of the Urban Area Security Initiative Program. August 2011. http://www.portlandoregon.gov/rdpo/article/455154. Accessed April 23, 2014.

FY 2012 Homeland Security Grant Program. Federal Emergency Management Agency website. Updated March 18, 2014. http://www.fema.gov/fy-2012-homeland-security-grant-program#2. Accessed April 23, 2014.

FY 2013 Homeland Security Grant Program (HSGP). Federal Emergency Management Agency website. Updated March 18, 2014. http://www.fema.gov/fy-2013-homeland-security-grant-program-hsgp-0. Accessed April 23, 2014.

FY 2014 Homeland Security Grant Program (HSGP). Federal Emergency Management Agency website. Updated March 24, 2014. https://www.fema.gov/fy-2014-homeland-security-grant-program-hsgp. Accessed April 23, 2014.

US Government Accountability Office. Homeland Security: Effective Regional Coordination Can Enhance Emergency Preparedness. GAO-04-1009. September 2004. http://www.gao.gov/assets/250/244172.pdf. Accessed April 23, 2014.

10.

US Department of Homeland Security. National Preparedness Guidelines. September 2007. http://www.fema.gov/pdf/emergency/nrf/National_Preparedness_Guidelines.pdf. Accessed April 23, 2014.

11.

US Government Accountability Office. Urban Area Security Initiative: FEMA Lacks Measures to Assess How Regional Collaboration Efforts Build Preparedness Capabilities. GAO-09-651. July 2009. http://www.gao.gov/assets/300/292039.pdf. Accessed April 23, 2014.

12.

National Academy of Public Administration. Improving the National Preparedness System: Developing More Meaningful Grant Performance Measures. October 2011. http://www.napawash.org/wp-content/uploads/2012/06/11-07.pdf. Accessed April 23, 2014.

13.

National Association of County and City Health Officials. Planning Beyond Borders: Using Project Public Health Ready as Regional Planning Guidance for Local Public Health. February 2007. http://www.naccho.org/topics/emergency/PPHR/upload/PPHR_Regional-Guidance_FINAL.pdf. Accessed April 23, 2014.

14.

National Association of County and City Health Departments. 2013 National Profile of Local Health Departments. January 2014. http://www.naccho.org/topics/infrastructure/profile/upload/2013-National-Profile-of-Local-Health-Departments-report.pdf. Accessed April 23, 2014.

15.

Bashir

, Lafronza

, Fraser

, Brown

, Cope

. Local and state collaboration for effective preparedness planning. J Public Health Manag Pract, 2003; 9(5):344-351.

16.

Myers

, Myers

, Grant

. The creation of regional partnerships for regional emergency planning. J Bus Contin Emer Plan, 2010; 4(4):338-351.

17.

Lesperance

, Olson

, Stein

, et al. Developing a regional recovery framework. Biosecur Bioterror, 2011; 9(3):280-287.

18.

Koh

, Shei

, Judge

, et al. Emergency preparedness as a catalyst for regionalizing local public health: the Massachusetts Case Study. Public Health Rep, 2008; 123(4). http://www.publichealthreports.org/issueopen.cfm?articleID=2086. Accessed October 22, 2014.

19.

Grieb

, Clark

. Regional public health emergency preparedness: the experience of Massachusetts Region 4b. Public Health Rep, 2008; 123(4):450-460.

20.

Stoto

, Morse

. Regionalization in local public health systems: public health preparedness in the Washington metropolitan area. Public Health Rep, 2008; 123(4):461-473.

21.

Stoto

. Regionalization in local public health systems: variation in rationale, implementation, and impact on public health preparedness. Public Health Rep, 2008; 123(4):441-449.

22.

Stoto

. Regionalization in local public health systems. California Health Policy Forum. Issue Brief. September 2007. http://www.cahpf.org/GoDocUserFiles/412.RegionalizationIssueBriefFinal.pdf. Accessed April 23, 2014.

23.

National Homeland Security Conference. 2014 National Homeland Security Conference program: Securing our liberty while protecting our homeland. May 2014. http://nationaluasi.com/dru/program/NHSA_Conf_Program_2014.pdf. Accessed September 1, 2014.

24.

Koh

, Elqura

, Judge

, Stoto

. Regionalization of local public health systems in the era of preparedness. Annu Rev Public Health, 2005; 29:205-218.