Enhancing Machine Learning Explainability of Disaster Preparedness Models from the FEMA National Household Survey to Inform Tailored Population Health Interventions

Abstract

Devastating mortality, morbidity, economic, and quality of life impacts have resulted from disasters in the United States. This study aimed to validate a preexisting machine learning (ML) model of household disaster preparedness. Data from 2021 to 23 Federal Emergency Management Agency’s National Household Surveys (n = 21,294) were harmonized. Importance features from the preexisting random forest ML model were transferred and tested in multiple linear and logistic regression models with updated datasets. Multiple regression models explained 42%–53% of the variance in household disaster preparedness. Features that improved the odds of overall disaster preparedness included detailed evacuation plans (odds ratios [OR] = 3.5–5.5), detailed shelter plans (OR = 4.3–11.0), having flood insurance (OR = 1.5–2.0), and higher educational attainment (OR = 1.1). Having no specified source of disaster information lowered preparedness odds (OR = 0.11–0.53). When stratified further by older adults with Black racial identities (n = 350), television as a main source of disaster-related information demonstrated associations with increased preparedness odds (OR = 2.2). These results validate the importance of detailed evacuation and shelter planning and the need to consider flood insurance subsidies in population health management to prepare for disasters. Tailored preparedness education for older adults with low educational attainment and targeted television media for subpopulation disaster-related information are indicated. By demonstrating a feasible use case to import ML model findings for regression testing in new datasets, this process promises to enhance population management health equity for those in sites that do not yet utilize local ML.

Introduction

The frequency and intensity of severe weather disasters across the United States are predicted to continue to increase in the context of climate change.^1,2 Disaster events disproportionately impact low income and minoritized communities; stemming from historical and structural segregation with lasting impacts in under-resourced areas.^3,4 This reality was underscored by Hurricane Helene, with over half a million North Carolina residents under disaster declaration following the deadly storm characterized by a high social vulnerability to disasters.⁵ Natural hazards more often become disasters when political and socioeconomic challenges are exacerbated among marginalized groups.³ A 2021 study found a higher risk of flooding in historically redlined communities compared to non-redlined communities, where people of color remain more likely to reside.⁶ In addition, Estien et al. found that historically redlined neighborhoods in California are associated with a disproportionate burden of environmental exposures, such as pollution and extreme heat.⁷ Early recovery and data collection efforts following Hurricanes Helene and Milton are ongoing at the time of this writing, but it is likely low income communities and communities of color may bear the brunt of the impact of these deadly storms.⁸

A substantial health equity gap remains in the efficacy of interventions for household disaster preparedness, including by place of residence, age, race, social capital, and socioeconomic status.⁹ In addition, tailoring interventions can help optimize patient outcomes and build trust.¹⁰ Machine learning (ML) and artificial intelligence (AI) tools may be able to help identify tailored interventions to address household disaster preparedness disparities if presented in a meaningful way for clinical and population health translation decision makers.¹¹ For many key clinical and population health leaders, the explainability and interpretability of AI/ML models can be too limited for adequate human governance, organizational and practice adoption, or human-machine teamwork development.¹² ML/AI literacy is a major challenge for public health and health care leaders in the face of rapid ML/AI innovation. Further, algorithms may worsen health disparities and amplify bias for marginalized groups.¹³ Different types of ML interpretability and explainability may need to vary by different audiences.¹⁴ For example, clinicians may be more interested in the feature importance of prioritizing care or discovering new features, generalizing or generating knowledge.¹⁴ Patients pay more attention to similarity classifications to understand how care was tailored to their unique identities and needs.¹⁴ Currently, however, ML interpretability methods or AI explainability techniques mainly serve as an algorithmic audit tool for the developers, auditors, and regulators, rather than the professional end users and subjects of AI.¹⁵ When working with communities to design and tailor public health and health care interventions, having a more interpretable or explainable and transparent model that replicates findings across traditional statistics and novel AI/ML models may promote trust in the findings across leadership decision makers, communities, clinicians, and end users.¹⁵

Previous research using the Federal Emergency Management Agency’s (FEMA) National Household Preparedness data identified the top importance features of household disaster preparedness from ML models among four groups: 1) the total adult population of all races and ethnicities, 2) older adults all races and ethnicities, 3) adults with Black/African American racial identities, and 4) older adults with Black/African American racial identities.¹¹ Targets for tailored disaster preparedness interventions for Black/African American older adults included those aimed at addressing economic and transportation disparities, financial literacy, and accessible and trustworthy resources for disaster planning.¹¹ Given the importance of explainability of AI/ML models in building trust when working with health disparity communities, hypothesis testing is warranted to further elucidate the consistency of these relationships across analytic models and generate meaningful results for data-to-decisions in population health and clinical practice.

The long-term goal of this line of research is to inform culturally tailored interventions by subpopulations with the vision of eliminating disaster health disparities. This current report aims to validate the previously developed AI/ML model of disaster preparedness in a new dataset using analytic methods to enhance explainability to stakeholders in settings where AI/ML models and experts may not be accessible or feasible.

Objective

The purpose here was to test the associations of household disaster preparedness and key importance features identified by previous ML models in a new aggregate national sample and disaggregated groups by age and Black/African American identity.

Materials and Methods

Study design

This is a data science use case demonstration of extending the federated transfer model for data-to-decision insights for settings without AI/ML model access or feasibility. A standard federated data transfer model is used in settings where private information should not be shared or breached to a central AI/ML training, testing, and prediction modeling platform. Rather, data is stored locally, and the key features of the model are imported for local training, testing, and prediction development. Findings are then shared back to the central site in an honest broker fashion, without the sharing of individual, private data. This study extends this model with consideration for sites that must retain private health care and public health data, but do not have the expertise or resources for local AI/ML model activity. Further, many population health and clinician decision makers may not yet trust the AI/ML findings without intuiting or supplementing the findings with the linear magnitude and direction of the relationships between the independent variables and the outcome. The current study begins with the top 20 important features from the previously developed AI/ML model,¹¹ as candidate explanatory variables to generate a linear model using a different, updated dataset to demonstrate how using AI/ML models developed elsewhere may continue to inform local analytics and practice decisions. Findings from this study are reported according to the Strengthening the Reporting of Observational Studies in Epidemiology guidelines.¹⁶

Setting

The original survey design is described by FEMA.¹⁷ The survey took place by telephone in the US and its territories using a nationally representative sampling frame with intentional oversampling among geographies with hazard-specific risks.

Participants

As a publicly available and de-identified dataset analysis, this study did not meet the definition of human subjects research according to the revised common rule (45 Code of Federal Regulations (CFR) 46). Analysis of publicly available data enhances transparency, replicability, and additional explainability in this first use case.

Variables

The outcome variable was a readiness score with a possible range of 0–1. The mean score of non-missing items was used to calculate the outcome score, composed of 12 yes/no response options to the question “What have you done to prepare for a disaster or emergency in the last year?”

Make a plan.

Save for a rainy day.

Practice emergency drills or habits.

Test family communication plan.

Safeguard documents.

Plan with neighbors.

Assemble or update supplies.

Get involved in your community.

Make your home safer.

Know evacuation routes.

Document and insure property.

In addition, a binary cutoff score for the preparedness outcome was generated to align with FEMA’s annual executive summaries which describe the percent of the population who had completed three or more preparedness actions. In alignment, the binary score for the outcome was based on a cutoff score of ≥ 0.25.

The remaining predictor variables were selected as the 20 top importance features generated from previously published ML/AI random forest model,¹¹ for the aggregated and each disaggregated model using 2018–2020 data. The annual assessment data evolved substantially each year in the administered survey, and for this study, only the variables that were able to be harmonized between the 2018–2020 and 2021–2023 datasets were entered.

Demographic variables to describe the sample were sex or gender, age, racial identity, ethnicity, language (Spanish or English) the survey was administered, educational attainment, household income, and disability status.

Data sources/measurement

As a transparent demonstration, the 2021, 2022, and 2023 FEMA National Household Survey datasets which were harmonized and combined were used.¹⁷ Data were downloaded for this study on September 7, 2022, for the 2020 set and updated on June 11, 2024, with the last two years of data.

Bias

The COVID-19 pandemic context must be considered in the interpretation of these findings. In addition, the important features generated from a previous study using data from 2018 to 2020 may not be generalizable to 2021–2023, especially for the variables that were not harmonized. As described in an original study,¹¹ the intent was to mitigate data racial bias using methods to aggregate and disaggregate models for insights, rather than enter racial identity as an explanatory variable for the outcome. The methods here are utilized to inform the potential for tailored intervention solutions for communities most at risk for poor outcomes. This study addresses insights for tailored interventions based on older adult age and Black/African American racial identity.

Statistical methods

Here, the data using multiple linear regression and logistic regression models were analyzed to enhance explainability with coefficient and OD to convey the magnitude and direction of associations. Differences in data-to-decision interpretations across the various analytic models were also explored. In a 2024 study, a random forest model was used to find out which features are important for predicting household disaster preparedness.¹¹ The results elucidated importance features, or which features mattered the most, but did not reveal exactly how the importance features (independent variables) were associated with the outcome (i.e., direction and magnitude of association).¹⁸ In the present analyses, ordinal variables were tested first using dummy coding for each level of the outcome and reported as continuous variables for parsimony if there was no substantive change in the interpretation of results between how these variables were entered into the model.

Results

Participants

Table 1 provides the descriptive results of the demographic data of survey participants. While the sampling frame used by FEMA was intended to generate a population representative dataset with over-representation among those in hazard-specific risk areas, these results describe the survey-takers and do not apply population weights. Supplementary Table S1 provides these descriptive data by each subpopulation of all older adults, participants with Black or African American racial identities, and Black older adults.

Table 1.

Demographic Variables for the 2021, 2022, and 2023 National Household Surveys, Including Participant Data from Hazard Oversamples

Demographic characteristics	n	%
Sex or gender
Female	11,705	53.34
Male	10,058	45.83
Other	183	0.83
Age category, years
18–29	4440	20.23
30–39	4477	20.40
40–49	3426	15.61
50–59	3391	15.45
60–69	3413	15.55
70–79	2203	10.04
80+	596	2.72
Self-reported racial identity
Asian	919	4.19
American Indian/Alaska Native	562	2.56
Black or African American	2687	12.24
Native Hawaiian/Pacific Islander	186	0.85
White	15,775	71.88
Two or more races	873	3.98
Other	944	4.30
Ethnicity: Hispanic, Latino, or of Spanish origin
Yes	4312	19.65
No	17,634	80.35
Language in which the survey was completed
English	20,586	93.80
Spanish	1360	6.20
Education
Less than a high school diploma	871	3.97
High school degree or diploma	5171	23.56
Some colleges, no degree	4976	22.67
Associate’s degree	2582	11.77
Bachelor’s degree	4806	21.90
Postgraduate degree	3540	16.13
Annual household income, USD
Under $10,000	1903	8.67
$10,000 to $49,000	8541	38.92
$50,000 to $99,999	6577	29.97
$100,000 to $149,999	2884	13.14
More than $150,000	2041	9.30
Disability status
Yes	5699	25.97
No	16,247	74.03

USD, United states dollars.

Outcome

A mean of 0.29 (standard deviation [SD] = 0.24) was observed for the whole sample preparedness score outcome (n = 21,294) as well as for those over the ages 60+ (n = 6084) and disaggregated by Black/African American racial identity (n = 2589). Disaggregated by older adults (ages 60+) with Black/African American racial identity, the mean outcome was 0.30 (SD = 0.26, n = 350).

Main results

Figure 1 provides a visualization of the logistic regression results for the whole sample. For presentations to data-to-decision makers with a wide range of statistical competencies, the figures can be explained in Table 2. Each subpopulation tested is included in a separate figure. Figure 2 depicts disaster preparedness OD for the older adults subpopulation. Supplementary Figures S1 and Figure S2 relay the OD results for the subpopulation who identified as Black or African American and Black older adults, respectively.

FIG. 1.

Odds ratios for whole sample household disaster preparedness (n = 21,294).

FIG. 2.

Odds ratio for older adults household disaster preparedness (n = 5919).

Table 2.

Explanation of Figures 1 and 2, and Supplementary Figures S1 and Figure S2

1. Odds ratio is listed on the horizontal (bottom) axis.

2. Each row shows the variable’s estimated odds ratio and confidence interval. If you picture these data visualizations as drones, the drone cockpit is the odds ratio, and the wingspan is the confidence interval.

3. If the confidence interval, or wingspan flies over the “1” line, the relationship is no better than chance. In general, confidence intervals crossing an odds ratio of 1 means it is no better than chance or there are equal odds that the factor increases or decreases household disaster preparedness. On this graph, this “no better than chance” line is a thicker vertical red line at the “1.”

4. If the estimates and interval, or drone cockpit and wingspan, fall below the “1” line, then the factor is associated with LOWER odds of household disaster preparedness.

5. If the estimates and interval, or drone cockpit and wingspan, all fall above the ‘1” line, then the factor is associated with HIGHER odds of household disaster preparedness.

6. The further the estimate, or drone cockpit, is to the right, the stronger the positive association.

7. The wider the confidence interval, or wingspan, the less precise this estimate is. A narrower confidence interval, or wingspan, indicates a more precise estimate.

The multiple linear regression models explained between 42% and 53% of the variance in household disaster preparedness scores, demonstrating a strong model fit for psychosocial and human behavior model testing (Supplementary Table S2). Across all models tested, having a detailed evaluation and sheltering plan demonstrated the strongest and most consistent associations with overall disaster preparedness scores. The length of time for which participants were prepared to shelter in their homes without power or running water was less consistently associated with overall preparedness scores. This length of time variable was associated with overall disaster preparedness for the whole sample but was not significantly associated with overall preparedness for the much smaller subpopulations of Black older adults. Because the time participants were prepared to shelter in their homes without power or running water was not a top importance feature for all with Black/African American Racial identities or all older adults in previous ML models, it was not tested in the currently presented validation, and explainability linear regression models.

Having no disaster information source also demonstrated a consistent negative association with disaster preparedness scores across all groups tested. Specific to older adults with Black/African American identities, television as a source of disaster information was associated with higher scores of preparedness. For the older adult subpopulation, using the information received to inform action or take no action was associated with overall preparedness scores.

The length of time participants sustained preparation along with the intention to prepare for the future demonstrated a positive association with overall scores across all models. Affective level of confidence in the ability to prepare was also consistently positively associated with the outcome in all samples except the much smaller sample of older adults with Black/African American identities.

Being a homeowner, as opposed to a renter, was not associated with overall preparedness in any model, but having flood insurance was consistently associated with overall preparedness. Demographic factors such as household income, sex or gender, ethnicity, and language were only consistently associated with overall preparedness for the whole sample and not the subpopulations tested.

The remaining results are depicted in Supplementary Table S2 by population tested for the multiple linear regression results. Further, Supplementary Table S3 relays a color-coded comparison of model consistency and inconsistencies among the previously tested ML/AI models, presents multiple linear regression, and presents logistic regression results.

Discussion

For executive leaders and practitioners in population health and health care, this study demonstrates a use case for those hesitant to trust ML/AI findings or who do not have the resources and expertise for ML analytics in their organization. In the past, many AI/ML models were presented as a “black box” with inadequate explainability needed for human governance and well-informed decision making.¹⁹ This study uniquely adds to the body of published literature by transparently demonstrating an extension of the federated transfer model. This use case was accomplished by taking the top importance features from a previously developed AI/ML model to inform the currently reported regression model testing validation in a separate dataset. This procedure may be useful for data-to-decision insights in settings where AI/ML models aren’t feasible due to limited capacity, expertise, or other resources. As AI/ML-powered technology proliferates in public health and clinical practice, it presents a risk of widening the digital divide and health disparity gap between low and well-resourced settings and communities.²⁰ This procedure may be one way to mitigate disparities so that settings without AI/ML might still benefit from the insights and advancements generated by AI/ML at other locations.

Here, the content focused on a needs assessment for tailored interventions for household disaster preparedness. Using annually collected, publicly available data from FEMA, this study demonstrated a feasible use case to import ML model findings for regression testing in new/updated datasets. This process promises to enhance population management health equity for those in sites that do not yet utilize local ML, as well as provide a needs assessment for population health management implications. Consistently, findings revealed that those with 1) a plan to leave the community for an evacuation, and 2) a plan that included information about where to shelter in the event of a disaster had at least 3.5–11.0 odds of overall household disaster preparedness across all groups (Figs. 1 and 2; Supplemental Figs. 1 and 2). Features unique to specific groups in this analysis were validated as well, which is discussed in further detail below.

Detailed disaster plans

Given the consistency of associations across models for evacuation and shelter plans, practitioners, leaders, and local (community) governments may wish to consider testing assessments with follow-up health education plans in their organizations using the following key items from the FEMA National Household Survey:

“Does your plan include information about how to leave your community for an evacuation?”

“Does your plan include information about where to shelter or a safe place you can stay in the event of a disaster?”

The goals of evacuation or sheltering in place include protecting the health of the public, avoiding fatalities, or minimizing population risk/exposure.²¹ Recommendations to shelter in place vs. evacuate depend on factors such as disaster type, population distributions, timing considerations, and community resources and individual resources relative to the disaster.²¹ Given the uniqueness of each community, population health leaders are well positioned to impact preparedness by (1) ensuring individuals are aware of community warning systems and what they mean, (2) supporting community members in planning for different scenarios, (3) testing emergency alerts and plans, and (4) making needed updates to infrastructure or alert systems following testing.^22,23 This process will also assist local government and public health leaders in understanding the needs of community members, allowing more effective planning.²⁴ Specific to older adults, these findings resonate with previous research about the expressed need for more evacuation assistance.²⁵

Disaster information sources

Findings from this study demonstrated lower preparedness scores among those without specific sources of disaster information. This association was consistent for older adults and those with Black or African American racial identities, and even stronger among Black older adults. Disaster information sources include television, radio, social media, internet publications, printed publications, emails, word of mouth, and push notifications from alert systems.¹⁷ With current technology and the prevalence of smartphones and tablets, emails, text messages, and push notifications to mobile devices have become more popular. Interventions to improve access to real time disaster communication, such as through the FEMA app, improve location-based notifications of impending and unfolding disaster events. However, populations with limited technological access and competency may require other, tailored information sources with special consideration for the elderly, those in low- or no-income households, and non-English speaking individuals who may not understand the local language.^26,27 Broadband/internet connectivity is essential to receiving official, timely notifications.²³ The digital divide, the gap between those with access to the internet and technological literacy and those without, compounds disaster risk and negative impacts for those in marginalized and low income groups and/or areas.²⁰

Further efforts to improve the availability of disaster information sources to these underrepresented populations are warranted. Television is the most commonly utilized disaster information source for older adults.²⁸ These findings indicate the importance of television as a source of disaster information for African American older adults to improve preparedness and the need for further research testing of televised disaster information as a tailored subpopulation intervention. The Federal Communications Commission requires that certain emergency information be provided in both audio and visual formats for persons who are deaf or hard of hearing and for the blind or visually impaired (i.e., closed caption, crawls, or scrolls that appear on the screen), which may enhance literacy and reach for those without disabilities as well.²⁹ These findings corroborate previous research on the association between emergency preparedness and emergency news media exposure.³⁰

Reliance on public transportation

Populations that depend on public transit are more vulnerable when services are disrupted during disasters.³¹ Unsurprisingly, our results demonstrate decreased preparedness scores among those who rely on public transportation from all respondents. Subpopulation disparities were not consistently observed across the tested models, which may have been due to differences in power among the models tested. Addressing transportation disparities requires a comprehensive approach that includes improving the resilience of public transportation infrastructure, ensuring equitable access to emergency information, and developing targeted evacuation plans that consider the unique needs of public transit-dependent populations. Additional research demonstrates the shift to for-hire vehicles and taxis when public transportation is compromised during weather-related emergencies.³² Yet, these are often costly and inherently may not be realistic for all who rely on public transportation. Population health managers, especially those in government agencies, have a duty to plan for and provide resources in disasters, including infrastructure and resources to evacuate, safely shelter in place, and receive disaster warnings.^20,22 Enhancing preparedness plans with transportation assistance outreach and poverty mitigation, particularly in areas with high public transit use, may help address this disparity. This enhanced planning may also involve identifying safe exit routes, shelter locations near transit hubs, and improving inter-agency coordination for efficient evacuations.

Insurance preparedness

As more volatile and damaging flood patterns and events occur, adaptation and housing insecurity for at-risk populations are linked to flood insurance costs, availability, changes, and post-disaster coverage.³³ Having flood insurance mitigates stress and improves resilience in a disaster’s aftermath.³⁴ Those without flood insurance or who must manage post-disaster insurance disputes demonstrate higher odds of post-disaster stress and depression.^35,36 Populations who identify as Black or Hispanic have higher rates of flood insurance compared to others.³⁷ By indicating flood insurance is a consistent factor associated with overall household preparedness among the whole population, older adults, Black or African American people, and older Black or African American people, the associations found in our models corroborate this previous research. Findings from this study provide a needs assessment to support momentum for flood insurance as an important intervention target to build on the existing strengths among the groups studied, including Black or African American subpopulations. Paradoxically, research has demonstrated that insurance purchasing behavior increases immediately after a disaster with an increase that sustains for only about three years after disaster.³⁸ Population health managers can assess household flood risk using interactive and publicly available mapping tools such as National Oceanic and Atmospheric Administration (NOAA’s) Sea Level Rise Viewer.³⁹ Effective social service assistance for low income individuals to access funding support for flood insurance coverage may be warranted.

Others

Regardless of the specific type of disaster, there is wide agreement that income level and educational attainment are positively associated with preparedness resources, actions, and behaviors.^40,41 Previous experience with disasters has been associated with perceived risk and preparedness for future disasters, especially for those who have experienced material losses and or personal harm during prior disasters.^42
–44 Moreover, older adults who have had negative experiences with disasters and those with higher incomes are more likely to be prepared for a disaster.⁴⁵

Statistical heterogeneity

This study explored the consistency and inconsistency of findings across two different datasets using three analytic models: random forest, multiple linear regression, and logistic regression. Statistical heterogeneity occurs when different analytic techniques show different results when testing the same variables in a way that changes the interpretation or meaning of the findings. See Table 3 for definitions of terms to clarify. Consistency across models, enhancing confidence in the findings, and practical implications are considered. Our study identified areas where findings were consistent across these three analytic methods. This consistency can boost the trustworthiness of AI/ML models and improve the interpretability of findings for key stakeholders and decision makers. Consistent findings from different models can build confidence in the results. However, this confidence might lead to oversimplified interpretations, particularly for rare factors or nonlinear relationships that could significantly impact the outcomes. For practical implications, this approach allows for the transfer or repurposing of findings from a centralized AI/ML model. By identifying the most important features in one organization or initiative, other data-to-decision makers can more comfortably analyze local data or understand the magnitude and direction of linear relationships.

Table 3.

Definitions of Technical Terms

Random forest model	A machine learning algorithm that combines multiple decision “trees” to make predictions. Each tree is created from a random subset of data, and these trees collectively create a “Forest.”
Importance features	A specific variable in a dataset that significantly contributes to the outcome, impacting the model’s prediction.
Statistical heterogeneity	Different analytic techniques show different results when testing the same variables in a way that changes the interpretation or meaning of the findings.

Future research

Several methodological advancements are indicated for longitudinal designs, ML/AI interpretation methods, and FEMA data informatics. First, longitudinal research is needed on household disaster preparedness to link to post-disaster outcomes and strengthen causal inferences. The findings in our study were from analysis of annual waves of national, cross-sectional surveys, and longitudinal data collection is needed to strengthen causal inference. Aligned with the findings of a recently published systematic review and meta-analysis on interventions to enhance household disaster preparedness, further intervention refinement and intervention efficacy testing are needed specific to decreasing health equity gaps.⁴⁶ Second, and like a previous recent publication,¹¹ future studies will apply ML models to this updated data and explore if the linear regression and other ML models such as random forest agree or differ on the important features or predictors. Global and local model-agnostic or interpretation methods, such as the SHapley Additive exPlanations (SHAP) Summary Plot, can also be applied to inform the interpretability of the black-box ML models.¹⁵ Fairness metrics can also be explored to minimize AI bias to address household disaster preparedness disparities.⁴⁷ Last, the process of data cleaning, ingestion, and harmonization to generate large datasets across different years of data collection in the FEMA National Household Survey can be time and resource intensive. Likewise, data management and sharing informatics resources are considerable to manage, store, and share large datasets across waves of data collection. FEMA has not yet achieved the consistency of other federal agency’s publicly available data for households and individuals across cycles of data. Enhanced harmonization and informatics efficiency prior to public data release aligned with the Center for Disease Control and Prevention’s National Health and Nutrition Examination Survey standards for variable naming and harmonization conventions across cycles of data is recommended. Alternately, federal resources to support research community engagement in these preprocessing requirements are needed.

This study focused on stratified results by Black or African American identities and further research is warranted among other ethnic and racial identity populations. The strengths of television media to reach older adult Black or African American subpopulations warrant additional research, particularly at the intersection of disability and health literacy outreach. These research findings point to the affordability and access to household flood insurance as an unexpected factor associated with overall household disaster preparedness across groups. While this study investigated subpopulations by age and Black racial identity, additional research is warranted on disparities in home flood insurance patterns for those with disabilities.⁴⁸ In the context of natural disasters, such as flooding and hurricanes, flood insurance coverage patterns are a key component of affordable housing planning and housing security for low income communities.³³ Additional research on disparities from population migration patterns linked to flood insurance provides a novel research direction. Further, migration pattern research is warranted associated with the cost and availability of flood insurance as climate change predictions indicate increases in flooding’s volatility.^1,2

Limitations

This study utilized a national sample of cross-sectional survey data, which means the analysis tested for association but not causation. Future work in longitudinal study designs is needed to strengthen causal inference from this work. Further, ML models based on correlational and cross-sectional data warrant additional, more rigorous testing with applied theoretical updated frameworks and longitudinal data to advance an understanding of how the features led to the outcome.^49,50 Not all variables from the previously developed AI/ML models were available in the updated dataset.

Conclusions

These results validate the importance of a specified evacuation and disaster shelter plan for all populations. The findings also support the need to consider flood insurance subsidies and tailored disaster preparedness education for older adults with low educational attainment. Further investigation and efforts are warranted to improve household disaster preparedness for those who report no specific source of disaster information. Specific to older adults with Black or African American racial identities, these results support the continued use of television media for tailored subpopulation disaster-related information. By demonstrating a feasible use case to import ML model findings for regression testing in new datasets, this process promises to enhance health equity for those in sites that do not yet utilize local ML.

Use of Artificial Intelligence in published research

Microsoft 365 Co-pilot was originally used on February 17, 2025, to create copy editing suggestions in the revision process for grammar, word choice, and flow of one paragraph in response to one reviewer's comment using a prompt to enhance the clarity and readability of one paragraph of the previously submitted article. The AI suggestion was reviewed, and the author edited the article directly. AI had no substantive role in drafting or revising the article outside of copyediting recommendations. The authors accept responsibility for the accuracy of the text.

Footnotes

Acknowledgment

The authors gratefully acknowledge Kevin Ndayishimiye, Castner Incorporated Data Science Intern and Undergraduate student at Howard University for software programming.

Authors’ Contributions

T.A.: Conceptualization, validation, project administration, writing draft, and editing. W.Z.: Conceptualization, methodology, formal analysis, writing draft and editing; interpretation of data for future research. S.E.S.: Formal analysis of data for practice translation; writing draft and editing. W.S.: Formal analysis and interpretation of data for practice and health disparity translation; writing draft and editing. J.C.: Conceptualization, methodology, software, formal analysis, visualization, supervision, funding acquisition, and writing draft and editing.

Author Disclosure Statement

The authors W.Z., S.E.S., and W.S. have no conflicts of interest, competing interests, personal financial interests, funding, employment, or other competing interests to disclose. T.A. was an independent contractor for Castner Incorporated. J.C. is the President and Principal of Castner Incorporated.

Funding Information

Research reported in this publication was supported in part by the National Institute of Environmental Health Sciences of the National Institutes of Health under Award Number R25ES033452 (PI: J.C.) and the National Institute of Minority Health and Health Disparities under Award Number R43MD017188 (PI: J.C.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Research reported in this presentation was supported in part by the National Institute for Occupational Safety and Health (NIOSH) under Federal Training Grant T42OH008433 (T.A.). The content is solely the responsibility of the authors and does not necessarily represent the official views of NIOSH.

Supplementary Material

Supplementary Figure S1

Supplementary Figure S2

Supplementary Table S1

Supplementary Table S2

Supplementary Table S3

References

National Centers for Environmental Information. Assessing the Global Climate in January 2025. National Oceanic and Atmospheric Administration. February 12, 2025. Available from: https://www.ncei.noaa.gov/news/global-climate-202501 [Accessed February 16, 2025].

The Intergovernmental Panel on Climate Change. Reports. The Intergovernmental Panel on Climate Change. 2025. Available from: https://www.ipcc.ch/reports/ [Accessed February 16, 2025].

Davis

, Berke

, Holloman

, et al. Support Strategies for Socially Marginalized Neighborhoods Likely Impacted by Natural Hazards.; 2021. Available from: https://coastalresiliencecenter.unc.edu/wp-content/uploads/sites/845/2021/07/Support-Strategies-for-Socially-Marginalized-Neighborhoods.pdf [Accessed November 28, 2024].

Egede

, Walker

, Campbell

, Linde

, Hawks

, Burgess

. Modern day consequences of historic redlining: Finding a path forward. J Gen Intern Med, 2023; 38(6):1534–1537; doi: 10.1007/s11606-023-08051-4

Marshall

, Sawyer

, King

, DeSalvo

, Peacock

. More than half a million north carolinians under disaster declaration after hurricane helene were at high social vulnerability to disasters. October 7, 2024. Available from: https://www.census.gov/library/stories/2024/10/hurricane-helene.html [Accessed November 28, 2024].

Katz

. A Racist Past, a Flooded Future: Formerly Redlined Areas Have $107 Billion Worth of Homes Facing High Flood Risk—25% More Than Non-Redlined Areas. March 14, 2021. Available from: https://www.redfin.com/news/redlining-flood-risk/ [Accessed November 28, 2024].

Estien

, Wilkinson

, Morello-Frosch

, Schell

. Historical redlining is associated with disparities in environmental quality across California. Environ Sci Technol Lett, 2024; 11(2):54–59; doi: 10.1021/acs.estlett.3c00870

Howell

, Elliott

. Damages done: The longitudinal impacts of natural hazards on wealth inequality in the United States. Soc Probl, 2019; 66(3):448–467; doi: 10.1093/socpro/spy016

Amberson

, Heagele

, Wyte-Lake

, et al. Social support, educational and behavioral modification interventions for improving household disaster preparedness in the general community-dwelling population: A systematic review and meta-analysis. Front Public Health. Published Online, 2023; doi: 10.3389/fpubh.2023.1257714

10.

Monsen

, Peterson

, Mathiason

, Kim

, Votava

, Pieczkiewicz

. Discovering public health nurse–specific family home visiting intervention patterns using visualization techniques. West J Nurs Res, 2017; 39(1):127–146; doi: 10.1177/0193945916679663

11.

Shukla

, Amberson

, Heagele

, et al. Tailoring household disaster preparedness interventions to reduce health disparities: Nursing implications from machine learning importance features from the 2018–2020 FEMA national household survey. Int J Environ Res Public Health, 2024; 21(5):521; doi: 10.3390/ijerph21050521

12.

Adler-Milstein

, Aggarwal

, Ahmed

, et al. Meeting the moment: Addressing barriers and facilitating clinical adoption of artificial intelligence in medical diagnosis. NAM Perspect, 2022; 2022(9); doi: 10.3147/8/202209c

13.

National Academies of Sciences E and M. Examining the History, Consequences, and Effects of Race-Based Clinical Algorithms on Health Equity. ( Alper

, Taffe

, Martinez

., eds.). National Academies Press; 2023. doi:10.1722/6/27301

14.

van der Shaar

, Crabbe’

. Interpretable Machine Learning. van der Shaar Lab. Available from: https://www.vanderschaar-lab.com/interpretable-machine-learning/ [Accessed November 28, 2024].

15.

Molnar

. Interpretable Machine Learning: A Guide For Making Black Box Models Explainable. 2nd ed.. Leanpub; 2022.

16.

Vandenbroucke

, von Elm

, Altman

, et al.; STROBE Initiative. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and elaboration. PLoS Med, 2007; 4(10):e297; doi: 10.1371/journal.pmed.0040297

17.

Federal Emergency Management Agency. National Household Survey. 2023. Available from: https://www.fema.gov/about/openfema/data-sets/national-household-survey [Accessed December 17, 2023].

18.

Ashraf

. Correlation in machine learning—All you need to know. Medium.com. September 21, 2023. Available from: https://medium.com/@abdallahashraf90x/all-you-need-to-know-about-correlation-for-machine-learning-e249fec292e9 [Accessed November 28, 2024].

19.

Pfaff

, Girvin

, Crosskey

, et al.; N3C and RECOVER Consortia. De-black-boxing health AI: Demonstrating reproducible machine learning computable phenotypes using the N3C-RECOVER Long COVID model in the All of Us data repository. J Am Med Inform Assoc, 2023; 30(7):1305–1312; doi: 10.1093/jamia/ocad077

20.

Terry

. Why Digital Inclusion is Key for Surviving Climate Disasters. October 7, 2024. Available from: https://www.human-i-t.org/digital-inclusion-climate-change/#:∼:text=Poor%20internet%20access%20intertwines%20with,perfect%20storm%20of%20disaster%20vulnerability [Accessed November 28, 2024].

21.

McLennan

, Bearman

, Ryan

. Routledge Handbook of Environmental Hazards and Society. In: ( McGee

, Penning-Rowsell

., eds). Routledge Handbook of Environmental Hazards and Society. 1st ed. Routledge; 2022:335–347. doi:10.4324/9780367854584

22.

Federal Emergency Management Institute. A Citizen’s Guide to Disaster Assistance: Unit 5;. 2003. Available from: https://training.fema.gov/emiweb/downloads/is7complete.pdf [Accessed November 28, 2024].

23.

U.S. Department of Homeland Security. Emergency Alerts. Ready.gov. October 29, 2024. Available from: https://www.ready.gov/alerts [Accessed November 28, 2024].

24.

Kako

, Steenkamp

, Ryan

, Arbon

, Takada

. Best practice for evacuation centres accommodating vulnerable populations: A literature review. Inter J Disaster Risk Reduction, 2020; 46:101497; doi: 10.1016/j.ijdrr.2020.101497

25.

Heagele

. A qualitative survey of household emergency preparedness for the elderly and the medically frail living in coastal urban communities. J Emergen Manag, 2021; 19(2):143–163; doi: 10.5055/jem.0493

26.

Wilson

, Tate

, Emrich

. Flood recovery outcomes and disaster assistance barriers for vulnerable populations. Front Water, 2021; 3; doi: 10.3389/frwa.2021.752307

27.

Flores

, Sullivan

, Yu

, Friedrich

. Health disparities in the aftermath of flood events: A review of physical and mental health outcomes with methodological considerations in the USA. Curr Environ Health Rep, 2024; 11(2):238–254; doi: 10.1007/s40572-024-00446-7

28.

Bell

, Singer

, Solway

, Kirch

, Kullgren

, Malani

. Predictors of Emergency Preparedness Among Older Adults in the United States. Disaster Med Public Health Prep, 2021; 15(5):624–630; doi: 10.1017/dmp.2020.80

29.

Federal Communications Commision. Accessibility to Emergency Information on Television.; 2022. Available from: https://www.fcc.gov/sites/default/files/accessibility_to_emergency_information_on_television.pdf [Accessed November 28, 2024].

30.

Paek

, Hilyard

, Freimuth

, Barge

, Mindlin

. Theory-Based Approaches to understanding public emergency preparedness: Implications for effective health and risk communication. J Health Commun, 2010; 15(4):428–444; doi: 10.1080/10810731003753083

31.

Daniels

, Macarthur

. Transportation System Resiliency and Disaster Response and Recovery: A Review on U.S. Metropolitan Long-Range Transportation Plans; 2022. Available from: https://ppms.trec.pdx.edu/media/project_files/TRB_TRBAM-22-00866_Paper.pdf [Accessed November 28, 2024].

32.

Qiao

, Haraguchi

, Lall

. Enhancing urban resilience to extreme weather: The roles of human transition paths among multiple transportation modes. Inter J Geographical Inform Sci, 2024; 28:1–20; doi: 10.1080/13658816.2024.2418040

33.

, Spidalieri

. Home is where the safer ground is: The need to promote affordable housing laws and policies in receiving communities. J Environ Stud Sci, 2021; 11(4):682–695; doi: 10.1007/s13412-021-00702-4

34.

Platt

, Mahdavian

, Carpenter

, Wiens

, Schultmann

. Were the floods in the UK 2007 and Germany 2013 game-changers? Philos Trans A Math Phys Eng Sci, 2020; 378(2168):20190372; doi: 10.1098/rsta.2019.0372

35.

Mulchandani

, Smith

, Armstrong

, Beck

, Oliver

, English National Study of Flooding and Health Study Group . Effect of insurance-related factors on the association between flooding and mental health outcomes. Int J Environ Res Public Health, 2019; 16(7):1174; doi: 10.3390/ijerph16071174

36.

McKenzie

, Longman

, Bailie

, et al. Insurance issues as secondary stressors following flooding in Rural Australia—A mixed methods study. Int J Environ Res Public Health, 2022; 19(11):6383; doi: 10.3390/ijerph19116383

37.

Friedman

, Fussell

, Nakatsuka

, Yucel

. Hispanic disaster preparedness in the United States, 2017: Examining the association with residential characteristics. Cityscape, 2021; 23(3):205–239.

38.

Kousky

. Disasters as learning experiences or disasters as policy opportunities? examining flood insurance purchases after hurricanes. Risk Anal, 2017; 37(3):517–530; doi: 10.1111/risa.12646

39.

NOAA Office for Coastal Management. Sea Level Rise Viewer. November 21, 2024. Available from: https://coast.noaa.gov/digitalcoast/tools/slr.html [Accessed November 28, 2024].

40.

Al-Rousan

, Rubenstein

, Wallace

. Preparedness for natural disasters among older us adults: A nationwide survey. Am J Public Health, 2015; 105(Suppl 4):S621–S626; doi: 10.2105/AJPH.2013.301559r

41.

Chae

, Choudhury

, Franco-Castano

, Adepoju

PMOE

. Self-perceived disaster preparedness in minority older adults: A cross-sectional study. Am J Disaster Med, 2022; 17(2):117–125; doi: 10.5055/ajdm.2022.0425

42.

Becker

, Paton

, Johnston

, Ronan

, McClure

. The role of prior experience in informing and motivating earthquake preparedness. Inter J Disaster Risk Reduction, 2017; 22:179–193; doi: 10.1016/j.ijdrr.2017.03.006

43.

Lawrence

, Quade

, Becker

. Integrating the effects of flood experience on risk perception with responses to changing climate risk. Nat Hazards, 2014; 74(3):1773–1794; doi: 10.1007/s11069-014-1288-z

44.

Kirby-Straker

R “Rowie”

, Straker

. “The Effect of Experiencing Disaster Losses on Risk Perceptions and Preparedness Behaviors.; 2023. Available from: https://hazards.colorado.edu/weather-ready-research/the-effect-of-experiencing-disaster-losses-on-risk-perceptions-and-preparedness-behaviors [Accessed November 28, 2024].

45.

Adepoju

, Herrera

, Chae

, Han

. Optimizing disaster preparedness planning for minority older adults: One size does not fit all. Int J Environ Res Public Health, 2022; 20(1):401; doi: 10.3390/ijerph20010401

46.

Amberson

, Ndayishimiye

, Cloud

, Castner

. Cognitive interview validation of a novel household hazard vulnerability assessment instrument. West J Nurs Res, 2024; 46(2):90–103; doi: 10.1177/01939459231217935

47.

Forbes Technology Council. AI & Fairness Metrics: Understanding and Eliminating Bias. July 14, 2023. Available from: https://councils.forbes.com/blog/ai-and-fairness-metrics [Accessed November 28, 2024].

48.

Mendes-Da-Silva

, Lucas

, Carvalho

JVdF

. Flood insurance: The propensity and attitudes of informed people with disabilities towards risk. J Environ Manage, 2021; 294:113032; doi: 10.1016/j.jenvman.2021.113032

49.

Obermeyer

, Emanuel

. Predicting the future—Big data, machine learning, and clinical medicine. N Engl J Med, 2016; 375(13):1216–1219; doi: 10.1056/NEJMp1606181

50.

Chen

, Asch

. Machine learning and prediction in medicine—Beyond the peak of inflated expectations. N Engl J Med, 2017; 376(26):2507–2509; doi: 10.1056/NEJMp1702071

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.10 MB

0.02 MB