Reviewing Health Security Capacities in Nigeria Using the Updated WHO Joint External Evaluation and WHO Benchmarks Tool: Experience from a Country-Led Self-Assessment Exercise

Abstract

Across the world, the level of pandemic preparedness varies and no country is fully prepared to respond to all public health events. The International Health Regulations 2005 require state parties to develop core capacities to prevent, detect, and respond to public health events of international concern. In addition to annual self-assessment, these capacities are peer reviewed once every 5 years through the voluntary Joint External Evaluation (JEE). In this article, we share Nigeria's experience of conducting a country-led midterm self-assessment using a slightly modified application of the second edition of the World Health Organization (WHO) JEE and the new WHO benchmarks tool. Despite more stringent scoring criteria in the revised JEE tool, average scoring showed modest capacity improvements in 2019 compared with 2017. Of the 19 technical areas assessed, 11 improved, 5 did not change, and 3 had lower scores. No technical area attained the highest-level scoring of 5. Understanding the level of, and gaps in, pandemic preparedness enables state parties to develop plans to improve health security; the outcome of the assessment included the development of a 12-month operational plan. Countries need to intentionally invest in preparedness by using existing frameworks (eg, JEE) to better understand the status of their preparedness. This will ensure ownership of developed plans with shared responsibilities by all key stakeholders across all levels of government.

Introduction

Infectious disease outbreaks do not respect borders.¹ The speed at which disease outbreaks spread has drastically increased with globalization and ease of travel, as demonstrated by the international spread of diseases including Ebola,^1-3 Middle East respiratory syndrome,⁴ severe acute respiratory syndrome,⁵ and COVID-19.^6-9 The impact of these outbreaks and epidemics on health systems and populations is substantial, slowing and sometimes reversing socioeconomic gains.^10,11 Several disease outbreaks have been recorded in Nigeria in the last 10 years with imported cases of Ebola in 2014,¹² annual outbreaks of yellow fever,^13,14 Lassa fever,^15-17 cerebrospinal meningitis,¹⁸ cholera,^19,20 measles,²¹ and most recently, COVID-19.²² Infectious disease outbreaks have exacerbated the humanitarian crises continuing in northeast Nigeria, reflecting the interaction between humans, animals, and the environment.^23-25 Consequently, there is a continuous emphasis on strengthening pandemic preparedness while prioritizing response efforts to protect the population from disease threats and ultimately improve health security.^25-27

In response to lessons learned from the severe acute respiratory syndrome outbreak in 2003, the World Health Organization (WHO) accelerated the revision of the International Health Regulations (IHR), which were adopted in 2005²⁸ and came into force in 2007. The IHR are essential for addressing global health security²⁹ and its implementation is underpinned by a comprehensive IHR monitoring and evaluation framework.^30-32 The framework, with its 4 components—State Party Self-Assessment Annual Reporting tool, voluntary Joint External Evaluation (JEE), after-action reviews, and simulation exercises—aims to provide a comprehensive, accurate, country-level overview of the implementation requirements under the IHR. The need for a monitoring and evaluation framework for IHR compliance was highlighted by the inadequacies of the annual self-reporting previously used as the sole indicator of emergency preparedness and IHR compliance.³³ The JEE is an important structure on which to base health security strengthening efforts, even if it does not predict country performance during a pandemic. The most critical aspect to countries, more than the JEE scores, is how countries respond.³⁴

Using the JEE to Strengthen Health Security in Nigeria

Nigeria is a state party to the IHR and mandated to report annually on its compliance status at the World Health Assembly. In June 2017, Nigeria conducted a JEE,^35,36 which provided an opportunity for IHR stakeholders across government ministries, departments, and agencies (MDAs) to jointly identify gaps in the country's health security capacity.

Following the JEE,³¹ countries³⁷ use the findings to develop a costed, multiyear National Action Plan for Health Security (NAPHS)³⁸ aimed at addressing identified health security gaps. In Nigeria, the NAPHS was developed with a 5-year timeline, costed, and launched,^36,39,40 which enabled country stakeholders to implement a plan to strengthen health security capacities using a One Health approach. In Nigeria, developing the NAPHS facilitated access to a credit facility from the World Bank Regional Disease Surveillance Systems Enhancement project.⁴¹ As the national IHR Focal Point, the Nigeria Centre for Disease Control worked with other MDAs to begin the implementation of activities within the NAPHS,³⁶ with support from the Regional Disease Surveillance Systems Enhancement project⁴¹ and other partners.

Given the long interval of 4 to 5 years before repeating the JEE,³⁰ Nigeria developed a framework for a midterm JEE to review its IHR core capacities in health security. Many systems improvements had been made since the initial JEE was conducted in 2017 and stakeholders needed to align with a multisectoral plan to address key priorities. The Nigeria Centre for Disease Control sought guidance from WHO and from partner organizations working on IHR implementation that served as national external experts, including US Centers for Disease Control and Prevention, Public Health England, and Resolve to Save Lives.

The country-led, midterm JEE was conducted from November 18 to 22, 2019.⁴² The objectives were to: (1) conduct a self-assessment of IHR implementation using the updated second edition JEE tool, (2) review progress and identify challenges in the implementation of the NAPHS, (3) compare scores from the self-assessment with the WHO benchmarks tool to identify core activities for health security implementation and develop annual operational plans specific to the JEE technical areas, and (4) map prioritized health security activities to government and partner resources.

The JEE tool used in 2017 was updated by WHO in 2018.⁴³ The revised JEE tool and the WHO benchmarks tool⁴⁴ now include a minimum set of prescriptions that state parties should aspire to for each technical area to demonstrate sufficient core capacity. Scoring in this updated tool is dependent on multisectoral coordination and accomplished through a One Health approach.^29,43

By sharing Nigeria's experience, we intend to reemphasize the need for countries to invest in similar efforts that ensure collective contribution to strengthening health security capacities. We envisage that implementing the lessons from this process will stimulate a renewed drive among countries for a self-sustaining model that embodies the tenets of the IHR monitoring and evaluation framework, through self-appraisal. These self-appraisals will potentially lead to the development and execution of feasible national and subnational action plans that support IHR compliance and efforts to strengthen health security and health systems using a multisectoral approach, with all relevant stakeholders.

Country-Led Process for the Midterm JEE

The midterm JEE in Nigeria was a hybrid process that incorporated an internal assessment and review by a team of evaluators from development partners and public health institutions working in health security implementation.

The internal assessment commenced with a preworkshop where technical leads: (1) were updated on the revised second edition JEE and the WHO benchmarks tool, (2) conducted a review of self-assessed JEE scores with feedback provided by a team of subject matter experts on the draft report documenting progress made since the 2017 JEE including the rationale for scores, and (3) prioritized 2 to 3 benchmark actions for implementation in 2020 using the WHO benchmarks tool, based on the implementation challenges observed since the NAPHS was launched in 2018 and on systems performance as measured by real-world events, including after-action reviews of MDAs.⁴⁴ Before the midterm assessment, technical area leads reviewed results of previously conducted after-action reviews and simulation exercises to prioritize selected benchmarks, demonstrate progress, and spotlight challenges with implementing recommendations from the first JEE. The benchmark tool, which was used as a planning rather than an assessment tool, helped identify actions to move to the next level. Participation during the midterm JEE included state epidemiologists, given their critical roles in coordinating response to public health events at the subnational level.⁴⁵ The evaluation included discussions (at plenary) around the progress and challenges since the development of the NAPHS, and reaching consensus on capacity scores and priority actions for implementation. We used mean scores to determine changes in Nigeria's JEE scores by technical area and indicator between 2017 and 2019 (Table 1).

Table 1.

Changes in Nigeria's JEE Scores by Technical Area and Indicator, 2017 to 2019

Subsequently, a 1-year operational plan for 2020 was developed, and responsibilities were assigned to respective MDAs with a proposed implementation timeline and tracker. Feedback was provided to the country team at a high-level stakeholders' meeting after a team of external evaluators from development partner agencies and host-country experts had obtained full agreement on all aspects of the report findings and recommendations. Core to this process was the wide engagement of key stakeholders across relevant MDAs with a role to play in IHR implementation.

Status of Health Security Capacity Following the Midterm JEE

Of the 19 technical areas assessed, 11 recorded increases in the mean score of their composite indicators compared with the 2017 JEE (Table 1), 5 showed no change in scoring, and 3 showed lower scores (Table 2). No indicator attained the highest-level scoring of 5. Key national priorities for action (Table 3) were identified following the exercise. JEE scores indicate the level of health security capacity on a scale from 1 to 5: 1 = no capacity, 2 = limited capacity, 3, developed capacity, 4 = demonstrated capacity, 5 = sustainable capacity.⁴³

Table 2.

Overall Change in Nigeria's JEE Scores by Technical Area, 2017 to 2019

Change^a	Technical Area	Number of Technical Areas
Increased^b	National legislation, policy, and financing Biosafety and biosecurity National laboratory system Reporting Emergency preparedness Emergency response operations Linking public health and security agencies Medical countermeasures Risk communication Point of entry Zoonotic disease	11
No change	IHR coordination, communication, and advocacy Antimicrobial resistance Surveillance Chemical events Radiation emergencies	5
Decreased	Food safety Immunization Human resources	3

Average score per technical area.

Technical areas increased but have not reached the highest scoring level of 5 (sustainable capacity).

Abbreviation: IHR, International Health Regulations.

Table 3.

Key National Priorities in Nigeria Following the Midterm JEE

	National Priorities	Description
1	Budgetary allocation	Increasing domestic budget allocation, release, and tracking for health security Harnessing resources in the Regional Disease Surveillance Systems Enhancement project across government ministries, departments, and agencies
2	Improved coordination	Implementing activities in technical areas, which requires leadership from the highest level Improving coordination across government ministries, departments, and agencies and working closely with the national IHR Focal Point (Nigeria Centre for Disease Control)—legally defined after the JEE in 2017 Training and ensuring availability of dedicated staff to prioritize activities identified in the NAPHS for government ministries, departments, and agencies
3	Implementation and monitoring	Provision of oversight by the honourable ministers in relevant government ministries, departments, and agencies (health, agriculture, and environment) through request for IHR/NAPHS implementation updates on a frequent basis

Abbreviations: IHR, International Health Regulations; JEE, Joint External Evaluation; NAPHS, National Action Plan for Health Security.

Prevent

Scores for national legislation, policy, and financing; zoonotic diseases; and biosafety and biosecurity increased under the Prevent core area between 2017 and 2019. The score for national legislation and policy increased from 1 to 2. Two new indicators were added to the financing component of this technical area and served as new baseline scoring. Challenges identified included an incomplete review of the Public Health Act Bill (2013)⁴⁶ and the National Health Act (2014)⁴⁷ by the legislative arm of government. It was evident that there was an overwhelming need for training of dedicated technical officers across IHR implementing MDAs on the relevance of IHR and awareness of existing guiding policies.

In the zoonotic diseases technical area, scores increased for 2 indicators as follows: indicator P4.1 (coordinated surveillance systems in place) increased from 2 to 3, and indicator P4.2 (mechanisms for responding to diseases) increased from 1 to 3. A key challenge was the discordance between prioritized NAPHS activities and findings from the 2017 JEE. Therefore, there was no mutual accountability for the implementation of these activities by the responsible MDAs. The score for 1 indicator, P6.1 (whole-of-government biosafety and biosecurity) under the biosafety and biosecurity technical area, increased from 1 to 2. Institutionalizing training programs for biosafety laboratory staff was identified as key to addressing the limited numbers of skilled personnel across the laboratories.

Scores for indicators in antimicrobial resistance, food safety, and immunization decreased. A score of 4 was attained for new indicator P3.1 (effective multisectoral coordination on antimicrobial resistance). For indicators P3.3 (infection prevention and control) and P3.4 (optimize use of antimicrobial medicines in human and animal health and agriculture), scoring decreased compared with 2017, from 2 to 1. A major gap was the need to explicitly capture activities that focus on the environment within the One Health operational plan, and publishing documents addressing appropriate use, availability, and quality of antimicrobials to optimize the use of antimicrobial medicines in human and animal health. Key challenges experienced in this technical area included a delay in presidential assent to an amended Animal Disease Control Act, and nonexistent specific regulations for antimicrobial stewardship for both human and animal health.

Scoring for food safety technical area decreased from 2 to 1 but reflected a more stringent and rigorous core capacity requirement. Despite the existence of a framework to respond to and manage food safety emergencies, there was no evidence of the functionality of such frameworks. Furthermore, multisectoral efforts on response to reported food-related outbreaks have been ad hoc, while recommendations in the approved National Policy on Food Safety and Implementation Strategy⁴⁸ have been partially implemented. In the immunization technical area, the score for measles immunization coverage decreased from 3 to 2, based on more recent coverage survey data rather than the use of administrative data, which was used in the original JEE. Immunization-related challenges are being addressed through the implementation of a comprehensive approach including: (1) nationwide and targeted measles vaccination campaigns (with the goal of 84.5% national coverage); (2) using community health influencers, promoters, and services; (3) leveraging the newly established Basic Healthcare Provision Fund to improve access to routine immunization via primary healthcare services; (4) coordinating the equitable improvement of immunization coverage through a National Emergency Routine Immunization Coordination Centre; and (5) the approval of a cold chain equipment optimization plan to be implemented by the government of Nigeria with funding support from Gavi, the Vaccine Alliance.

Detect

Most indicators in the Detect core area recorded an increase in scores. Nigeria's highest scores were 4 for D1.1 (laboratory testing for detection of priority diseases) and D3.1 (systems for efficient reporting externally); however, Nigeria's previous high score was 4 for D4.4 (field epidemiology training program or other applied epidemiology training program is in place), which decreased to 3 in the midterm JEE. One new indicator, D4.3 (in-service trainings are available), was added to the human resources technical area.

Laboratory technical area scores increased for 3 of the 4 indicators: D1.1 (laboratory testing for detection of priority diseases) increased from 3 to 4, D1.2 (specimen referral and transport system) increased from 1 to 2, and D1.3 (effective national diagnostic network) increased from 2 to 3. However, there was no evidence that quality improvement guidance documents were followed or available. Sample transportation between levels was poorly coordinated and ad hoc, laboratory information management systems are not available, and overall, there was a poor governance structure in the laboratory system. In the surveillance technical area, scores increased from 2 to 3 for indicator D2.2 (use of electronic tools) and decreased from 3 to 2 for D2.1 (surveillance systems). The decrease in D2.1 was ascribed to the requirement for indicator-based surveillance and event-based surveillance at the subnational level. Challenges in this technical area included minimal involvement of private health facilities in reporting surveillance data and a nonoperational interoperability system at national and subnational levels. Indicator D4.3 (in-service trainings are available) attained a score of 3 while there was a decrease in scoring from 3 to 2 for indicator D4.4 (field epidemiology training program or other applied epidemiology training program is in place). Key challenges included the inability to continue the Nigeria Field Epidemiology and Laboratory Training Program basic level training, having sufficient public health workforce at subnational level, and the creation of an incentivized career path for already trained personnel. These challenges are in addition to difficulties experienced coordinating the cadre of human and animal healthcare counterparts to establish unified human resources competencies.

Respond

Indicators in this core area recorded an increase in scores between the 2 assessments. Scores increased either from 1 to 2, or 2 to 3. Indicators R2.3 (emergency exercise management program) and R5.5 (addressing perceptions, risky behaviors, and misinformation) increased from 3 to 4. In the emergency preparedness technical area, the score increased from 2 to 3 for indicator R1.1 (strategy emergency risk assessments conducted and emergency resources identified and mapped) and remained at 1 for R1.2 (national multisectoral, multihazard emergency preparedness measures, including emergency response plans, are developed, implemented, and tested). The scores of all 3 indicators in the emergency response operations technical area increased between the assessments, including from 3 to 4 for indicator R2.3 (emergency exercise management program). Scores increased from 2 to 3 for the remaining 2 indicators: R2.1 (emergency response coordination) and R2.2 (emergency operation center capacities, procedures, and plans). Improvement in scoring was a result of the implementation of key recommendations from the 2017 JEE, especially around conducting joint simulation exercises between public health and security agencies. In addition, establishing public health emergency operation centers across several states in Nigeria provided a platform for improved emergency response coordination to other disease outbreaks. Challenges in this technical area include limited staff and operational funding at the subnational level for the emergency operations centers. The coordination mechanisms also need to be strengthened as there was no evidence these emergency operation centers were activated within the required 120 minutes of notification of a public health event.

Scoring for indicator R3.1 (public health and security authorities are linked during a suspect or confirmed biological event) increased from 1 to 2. A major challenge in this technical area was the inability to conduct capacity-strengthening simulation exercises for staff across MDAs despite identifying and establishing points of contact across relevant civil MDAs and security formations for the proposed engagements. In the medical countermeasures and personnel deployment technical area, only 1 of 3 indicators increased in scoring: R4.1 (system in place for activating and coordinating medical countermeasures during a public health emergency) increased from 1 to 2.

Of the 5 indicators in the risk communications technical area, 3 indicators showed increased scores. Indicator R5.5 (addressing perceptions, risky behaviors, and misinformation) increased from 3 to 4, reflecting the use of the connect center to collect information on rumors and misinformation. Data from these platforms supported the development of targeted messaging and risk communication approaches to managing rumors during outbreaks. Scores increased from 1 to 2 for the remaining 2 indicators: R5.1 (risk communication systems for unusual/unexpected events and emergencies) and R5.3 (public communication for emergencies). Challenges in this technical area included limited funding for risk communications activities, lack of a monitoring and evaluation framework, and lack of a coordination structure for risk communications at the subnational levels.

Other IHR-Related Hazards and Points of Entry

Scores remained the same for all indicators except POE.1 (routine capacities established at points of entry), which increased from 1 to 3. Challenges identified in this technical area included inadequate national legislative backing for port health services at points of entry, limited cross-border collaboration at ground crossings, and suboptimal all-hazards coordination at points of entry—chemical and radiation components in existing plans and protocols were not all-inclusive or robust enough.

Discussion

Countries should continuously conduct self-appraisal exercises to determine their level of pandemic preparedness and develop realistic plans to strengthen health security core capacities, especially during periods between events. These self-appraisals include system assessments (eg, JEE and State Party Self-Assessment Annual Reporting) and real-world events including after-action reviews. Nigeria has conducted after-action reviews for previous outbreaks,^49,50 which helped identify major gaps in preparedness and response to outbreaks. Both assessments in 2017 and 2019 suggest that more work is required^27,51 to strengthen health security capacities in Nigeria.

The country-led assessment provided an opportunity to reenergize high-level cross-government political commitment to strengthening Nigeria's health security capacities. Furthermore, the assessment enabled the integration of the lessons from previous outbreaks into the larger operational planning process, prioritizing recent and relevant risks, and institutionalizing the recommendations from such exercises. Through these reviews, activities such as the establishment of state public health emergency operation centers were introduced and have proven to be of value in Nigeria's COVID-19 response coordination. Specifically, the public health emergency operation centers have continued to serve as a useful framework for driving the subnational level response to the COVID-19 pandemic and as a common rallying point for a multisectoral response by stakeholders at the strategic, tactical, and operational levels. The country-led assessment reflected a commitment to collective action, self-improvement, learning, and transparency. The country was able to leverage the momentum garnered across key IHR stakeholders to review the status of the activities prioritized in the NAPHS and to self-appraise its performance. This was done while providing data representative of the implementation status of the World Bank-funded Regional Disease Surveillance Systems Enhancement project, as part of midpoint evaluations for the country and the World Bank.

Implementing the IHR is not the sole responsibility of the Nigeria Ministry of Health³³ or its implementing agencies. Although a nascent national public health institute, the Nigeria Centre for Disease Control organized and galvanized support for strengthening health security by bringing key stakeholders together from different sectors like the experience during the 2017 JEE. Consequently, MDAs—especially in health, agriculture, and environment—have continued to work in a continuum irrespective of the mandate of the supervising parent ministry. Sustaining this work will be important especially through the newly formed IHR interministerial technical working group tasked with ensuring ongoing coordination of IHR activities across the MDAs.

The midterm assessment was anchored on transparency, peer-to-peer reviews, and trust—tenets that are core to the JEE. The JEE second edition tool,⁴³ now more specific (eg, emphasizing subnational core capacities and spotlighting the animal health sector), enabled the collection of new in-depth information on the status of key technical areas while retaining the integrity of the original tool. Updated and more relevant baseline scores were obtained as a result. Due to the integration with operational planning, the exercise provided clear direction in identifying high-priority activities to be implemented in 2020. The use of the WHO benchmarks tool by the team of evaluators, subject matter experts, and in-country technical leads was of immense value in helping to reach consensus on priorities for implementation for each technical area based on the country's assessed JEE scores. The benchmarks are a useful supplement to the existing IHR monitoring and evaluation framework; they are not an assessment tool but enabled easy reference to prioritize actions to attain the next capacity level of the JEE. In a few technical areas, where the implementation of activities had just commenced or were still ongoing since 2017, scoring did not change substantially because all the prescriptions per the benchmark tool have not been attained. These ongoing activities are noted as comments in the final report.⁵² With the revised, more stringent tool, performing a midterm assessment may make it more challenging for countries to demonstrate progress after the initial JEE; the new scores might not reflect the amount of progress made in a single sector or technical area, but may reflect some progress that is more conservatively measured and robust. This challenge was limited to only a few indicators, however, and such nuances should be well communicated and documented. Using this approach, the scores were more reflective of actual progress and more objective.

The JEE, and its approach, is one of several efforts to spotlight the need for better preparedness and ensure compliance with IHR by countries signatory to it. Although the JEE was not developed to compare pandemic preparedness across countries, with the COVID-19 pandemic, there has been more focus on the JEE's utility to better understand response indicators such as time-to-first-case detection.⁵³ The JEE was designed for epidemic-prone diseases and not necessarily pandemics. A major distinction between both public health events is that epidemics are expected to be primarily managed by the health sector, whereas pandemics require a whole-of-society approach including cutting through bureaucratic red tape across different levels of society. It is also evident that the JEE does not measure critical indicators on pandemic preparedness, such as health system resilience.⁵⁴ However, recognizing this limitation has helped guide and support advocacy efforts for strengthening health security.⁵⁵ The JEE has highlighted that health security capacity may not necessarily translate to having the capability to respond to public health events. Such capabilities can only be observed during real-world events—a gap that after-action reviews^56,57 have been designed to fill in addition to other IHR monitoring and evaluation framework components. Regardless, the JEE remains critical to help structure national action planning for health security, providing a common language and framework for countries to work toward. In Nigeria, the midterm JEE was carried out because the country had made substantial progress in the 2 years since the first JEE. The midterm review was a useful process to assess this progress, define operational priorities, and integrate with country planning cycles, which was not explicitly part of the first JEE process. As such, countries can decide how and when similar assessments can be conducted, once or twice a year, as part of a continuous improvement process. The 2019 midterm JEE assessment in Nigeria highlighted where progress had been made since 2017 and enabled reprioritization of systems strengthening efforts. An important point to also consider going forward is that generalization of the findings from the initial JEE—by extrapolating preparedness scores to all administrative entities—could be misleading, especially in countries with autonomous subnational governing structures like in Nigeria.

Nigeria is governed through a federal system. The constitution provides for a substantial degree of autonomy across the 3 tiers of government—federal, state, and local. There are 36 states each with their own budget, priorities, and constitutional authority for health sector interventions. Participation by state epidemiologists and their feedback provided for a balanced view on potential IHR implementation challenges at the subnational level, from the states and local government areas with an acknowledgment of these limitations in the 1999 Constitution of the Federal Republic of Nigeria (as amended).⁵⁸ Within Nigeria's constitution, there are limitations around the development and review of legislation. Specifically, only the federal government can legislate and establish laws on items on the exclusive list, while federal and state governments can establish laws on items on the concurrent list. Where there is an existing federal law, however, states must defer to the former. Items on the residual list are solely the responsibility of state governments. Public health (eg, disease detection, surveillance, reporting, coordination) is on the residual list.⁵⁹ The power to establish legislation requires improved involvement of the subnational level on assessments such as the JEE and subsequent planning for investments in health security. With Nigeria's federated nature of governance, geographic and bureaucratic distance from the frontlines to the center of governance,⁶⁰ the role of the state epidemiologists, and contribution during the midterm JEE, as well as other personnel at the subnational level⁴⁵ are critical to strengthening health security. This was evident with scoring in the surveillance technical area, for instance, which required commensurate development of similar capacities at the subnational level. This type of limitation has spurred conversations on the importance of assessments of health security capacities at the subnational level. The need for this type of granular assessment is even more evident with the varied response to the COVID-19 pandemic in Nigeria and role of key personnel—especially at the subnational level. As such, subsequent JEEs or health security assessments need to capture the nuances on preparedness within these entities versus having a national representative scoring. This may be misleading, giving a false sense of preparedness. We recommend that countries endeavor to have representation from stakeholders at the subnational level during such discourse, to provide insights from their unique experience working at that level. Furthermore, countries can domesticate the JEE tool to better capture more qualitative and formative information that existing health information systems cannot unearth. This can help to identify specific areas that require additional efforts for strengthening, within the confines of the constitution or other existing laws.

Assessing health security capacities at the subnational level can be expensive and time-consuming. However, several granular assessments have been proposed using data that already exist in health information and disease surveillance systems. Examples include the timeliness milestones and metrics.⁶¹ Standardized outbreak milestones enable countries to define and calculate relevant timeliness metrics, designed around milestones, to address their own needs. Milestones can include the dates when a country predicts, detects, verifies, and responds to an outbreak; notifies authorities; and launches a multisectoral investigation, conducts laboratory tests, implements control measures, and informs the public. A timeliness metric is then measured as the time interval between 2 relevant outbreak milestones—and importantly, can be calculated at the subnational level. WHO has used a combination of these and other indicators, such as the State Party Self-Assessment Annual Reporting tool indicators for preparedness capacity, and immunization indicators to capture real-world event data in its “triple billion” methods and make a distinction between measures of capacity and capability.⁶² Other measures include a new global target of 7-1-7, whereby every suspected outbreak is identified within 7 days of emergence, reported to public health authorities with initiation of investigation and response efforts within 1 day, and effectively responded to—as defined by objective benchmarks—also within 7 days.⁶¹ Each of these can be applied at both country and subnational levels to entrench a transparent framework for assessing outbreak response capabilities and identify gaps in health security capacities within the framework of health systems strengthening.

One of the objectives of the midterm JEE was to inform the review of the country's NAPHS. With the review, it was evident that ownership and accountability were weak despite the availability of performance management tools—a monitoring and evaluation framework and an implementation tracker. Other issues identified were: (1) the NAPHS monitoring and evaluation framework was not sufficiently robust to track output and outcomes and was laden with numerous activities that needed to be pruned³⁶; (2) electronic tools (eg, NAPHS tracker) developed to track the status of implementation were difficult to implement especially where the legislation, coordination, communication, review, and provision of feedback across various IHR implementing MDAs was weak and suboptimal; (3) NAPHS and related plans need to be created for shorter periods (eg, annually or every 2 years); (4) subsequent planning needs to depend on the outcome of the evaluation of activities previously implemented³⁶; and (5) strategic realignment of priority activities can help maximize already existing funding sources (eg, the Regional Disease Surveillance Systems Enhancement project) to support strengthening health security capacities. In addition, the NAPHS has too many, often ambitious, prioritized annual activities. These activities are not entirely included in the annual work plans of the MDAs and therefore often have no budget. Hence implementation was severely hampered. Of the 634 planned activities for fiscal year 2019, only 4% were completed by responsible MDAs. Implementation status of many other planned activities was either unknown, past due, or not updated. IHR participating or implementing MDAs also had limited understanding of their roles as it relates to IHR. This further compounded the implementation and follow-through of prioritized activities—even when funding was available.

Poorly executed NAPHS activities by countries may have also contributed to the weak response to COVID-19 across countries, as gaps identified after the 2017 JEE were not addressed. Coordination among relevant MDAs with a role in health security can help improve the implementation of activities that address health security gaps. In Nigeria, following the midterm JEE, the Nigeria Centre for Disease Control led and advocated for the establishment of an interministerial technical working group for IHR. This was, with equitable representation, from relevant MDAs to drive and sustain the implementation of health security activities, and champion advocacy efforts as required. A key requirement for membership in the technical working group was for members to be capable of making decisions or influencing decision makers. Although the Nigeria Centre for Disease Control is the national IHR Focal Point, within the federated structure in Nigeria, the various MDAs are independent and thus need to be mutually accountable with a governance model that enables this alignment. As such, members from the MDAs developed terms of reference for the national technical working group to describe their roles and responsibilities, including reporting processes. The NAPHS tracker was developed to support the reporting process across the MDAs and foster the desire of the technical working group to institutionalize its mutual accountability. Challenges with deployment of the NAPHS tracker have been discussed here and elsewhere.³⁶

Conclusion

Although evidence suggests that the IHR monitoring and evaluation framework, and particularly the JEE indicators, do not provide a complete assessment of pandemic preparedness,⁵³ we found that the framework provides readily available tools that can be deployed to help identify gaps in preparedness. Assessments such as this are also equally important and needed by state parties to guide strategic development of operational plans (not just the NAPHS) for prioritizing the most immediate health security needs. Based on these assessments, state parties can develop holistic and realistic plans to further strengthen health security and build requisite core capacities as stipulated in IHR. Despite the challenges with the JEE and its representation of preparedness before the COVID-19 pandemic, it has helped to improve understanding of the status of pandemic preparedness across countries.²⁷ State parties need to consistently and intentionally invest in pandemic preparedness, and strengthening health security should be seen as a collaboration beyond the ministries of health alone.⁶³

It is now more critical that preparedness activities are led by countries, based on the nuances and peculiarities of governance and governing structures. This will help foster sustainability and ownership of developed plans, which need to be backed by consistent funding and anchored on multisectoral engagement across various subnational entities and relevant MDAs—led by a national public health institute—for a safer world.

Acknowledgments

The authors would like to acknowledge the active participation of national experts in preparing for the midterm JEE mission. We also recognize the invaluable partnerships with governments including the governments of the United Kingdom and the United States for providing technical experts for this country-led review process; with other intergovernmental organizations, particularly FAO, OIE, West Africa Health Organization, Resolve to Save Lives, World Bank, Georgetown University, Public Health England, US Centers for Disease Control and Prevention, Pro-Health International, African Field Epidemiology Network, Volte Health, and University of Maryland, Baltimore for their contribution of expertise. We appreciate the financial support provided by Resolve to Save Lives and Public Health England for this activity and the WHO Nigeria Country Office for technical, planning, and financial support. The authors would like to acknowledge the continuing support and commitment of all of these to the implementation and principles of the IHR.

References

de Bengy Puyvallée

, Kittelsen

. “Disease knows no borders”: pandemics and the politics of global health security. In: Bjørkdahl

, Carlsen

, eds. Pandemics, Publics, and Politics: Staging Responses to Public Health Crises. Singapore: Palgrave Pivot; 2019:59-73.

Ericsson

, Steffen

, Isaäcson

Viral hemorrhagic fever hazards for travelers in Africa. Clin Infect Dis. 33(10):1707-1712.

Okunromade

, Lokossou

, Anya

, et al. Performance of the public health system during a full-scale yellow fever simulation exercise in Lagos State, Nigeria, in 2018: how prepared are we for the next outbreak. Health Secur. 2019; 17(6):485-494.

Zumla

, Dar

, Kock

, et al. Taking forward a ‘One Health’ approach for turning the tide against the Middle East respiratory syndrome coronavirus and other zoonotic pathogens with epidemic potential. Int J Infect Dis. 2016; 47:5-9.

Ruan

, Wang

, Levin

. The effect of global travel on the spread of SARS. Math Biosci Eng. 2006; 3(1):205-218.

Yang

, Shi

, Lu

, et al. Clinical and CT features of early stage patients with COVID-19: a retrospective analysis of imported cases in Shanghai, China. Eur Respir J. 2020; 55(4):2000407.

, Liu

, Zhao

, et al. Clinical characteristics of imported cases of coronavirus disease 2019 (COVID-19) in Jiangsu Province: a multicenter descriptive study. Clin Infect Dis. 2020; 71(15):706-712.

Hâncean

, Perc

, Lerner

. Early spread of COVID-19 in Romania: imported cases from Italy and human-to-human transmission networks. R Soc Open Sci. 2020; 7(7):200780.

Craig

, Heywood

, Hall

. Epidemiology and infection risk of COVID-19 importation to the Pacific islands through global air travel. Epidemiol Infect. 2020; 148:e71.

10.

Huber

, Finelli

, Stevens

. The economic and social burden of the 2014 Ebola outbreak in West Africa. J Infect Dis. 2018; 218(suppl 5):S698-S704.

11.

World Bank. The Economic Impact of the 2014 Ebola Epidemic: Short- and Medium-Term Estimates for West Africa. Washington, DC: World Bank; 2014.

12.

Shuaib

, Gunnala

, Musa

, et al. Ebola virus disease outbreak - Nigeria, July-September 2014. MMWR Morb Mortal Wkly Rep. 2014; 63(39):867-872.

13.

Nwachukwu

, Yusuff

, Nwangwu

, et al. The response to re-emergence of yellow fever in Nigeria, 2017. Int J Infect Dis. 2020; 92:189-196.

14.

Monath

, Vasconcelos

PFC

. Yellow fever. J Clin Virol. 2015; 64:160-173.

15.

Akpede

, Asogun

, Okogbenin

, Okokhere

. Lassa fever outbreaks in Nigeria. Expert Rev Anti Infect Ther. 2018; 16(9):663-666.

16.

Roberts

Nigeria hit by unprecedented Lassa fever outbreak. Science. 2018; 359(6381):1201-1202.

17.

Maxmen

Deadly Lassa-fever outbreak tests Nigeria's revamped health agency. Nature. 2018; 555(7697):421-422.

18.

Nnadi

, Oladejo

, Yennan

, et al. Large outbreak of neisseria meningitidis Serogroup C — Nigeria, December 2016–June 2017. MMWR Morb Mortal Wkly Rep. 2017; 66(49):1352-1356.

19.

Dalhat

, Isa

, Nguku

, et al. Descriptive characterization of the 2010 cholera outbreak in Nigeria. BMC Public Health. 2014; 14:1167.

20.

Elimian

, Musah

, Mezue

, et al. Descriptive epidemiology of cholera outbreak in Nigeria, January-November, 2018: implications for the global roadmap strategy. BMC Public Health. 2019; 19(1):1264.

21.

Hofstetter

, DuRivage

, Vargas

, et al. Text message reminders for timely routine MMR vaccination: a randomized controlled trial. Vaccine. 2015; 33(43):5741-5746.

22.

Adegboye

, Adekunle

, Gayawan

Early transmission dynamics of novel coronavirus (COVID-19) in Nigeria. Int J Environ Res Public Health. 2020:17(9):3054.

23.

Marcantonio

, Rizzoli

, Metz

, et al. Identifying the environmental conditions favouring West Nile Virus outbreaks in Europe. PLoS One. 2015; 10(3):e0121158.

24.

Bezirtzoglou

, Dekas

, Charvalos

. Climate changes, environment and infection: facts, scenarios and growing awareness from the public health community within Europe. Anaerobe. 2011; 17(6):337-340.

25.

Lau

, Smythe

, Craig

, Weinstein

. Climate change, flooding, urbanisation and leptospirosis: fuelling the fire?. Trans R Soc Trop Med Hyg. 2010; 104:631-638.

26.

Poletto, Gomes

, Pastore y Piontti

, et al. Assessing the impact of travel restrictions on international spread of the 2014 west African Ebola epidemic. Euro Surveill. 2014; 19(42):20936.

27.

Shahpar

, Lee

, Wilkason

, Buissonnière

, McClelland

, Frieden

. Protecting the world from infectious disease threats: now or never. BMJ Glob Health. 2019; 4(4):e001885.

28.

World Health Organization. World Health Assembly adopts new International Health Regulations. Published May 23, 2005. Accessed October 29, 2021. https://apps.who.int/mediacentre/news/releases/2005/pr_wha03/en/index.html

29.

Talisuna

, Yahaya

, Rajatonirina

, et al. Joint external evaluation of the International Health Regulation (2005) capacities: current status and lessons learnt in the WHO African region. BMJ Glob Health. 2019; 4(6):e001312.

30.

World Health Organization (WHO). International Health Regulations (2005): IHR Monitoring and evaluation Framework. Geneva: WHO; 2018. Accessed December 16, 2021. https://www.who.int/publications/i/item/international-health-regulations-(-2005)-ihr-monitoring-and-evaluation-framework

31.

Bell

, Tappero

, Ijaz

, et al. Joint external evaluation—development and scale-up of global multisectoral health capacity evaluation process. Emerg Infect Dis. 2017; 23(13):S33-S39.

32.

Lee

, Katz

, Eaneff

, Mahar

, Ojo

. Action-based costing for national action plans for health security: accelerating progress toward the International Health Regulations (2005). Health Secur. 2020; 18(1_suppl):S53-S63.

33.

World Health Organization (WHO). Implementation of the International Health Regulations (2005): report of the review committee on second extensions for establishing national public health capacities and on IHR implementation. Sixty-Eighth World Health Assembly Provisional Item 15.3. Published March 27, 2015. Accessed December 16, 2021. https://apps.who.int/gb/ebwha/pdf_files/WHA68/A68_22Add1-en.pdf

34.

Lee

, Frieden

Why even well-prepared countries failed the pandemic test. Foreign Affairs. March 29, 2021. Accessed September 5, 2021. https://www.foreignaffairs.com/articles/united-states/2021-03-29/why-even-well-prepared-countries-failed-pandemic-test?utm_medium=promo_email&utm_source=lo_flows&utm_campaign=registered_user_welcome&utm_term=email_1&utm_content=20210905

35.

World Health Organization (WHO). Joint External Evaluation of IHR Core Capacities of the Federal Republic of Nigeria. Geneva: WHO; 2017. Accessed September 19, 2020. https://www.who.int/publications/i/item/WHO-WHE-CPI-REP-2017.46

36.

Ojo

, Dalhat

, Garfield

, et al. Nigeria's Joint External Evaluation and National Action Plan for Health Security. Health Secur. 2020; 18(1):16-20.

37.

World Health Organization. Strategic Partnership for Health Security and Emergency Preparedness (SPH) Portal. Accessed December 16, 2021. https://extranet.who.int/sph/home

38.

Mghamba

, Talisuna

, Suryantoro

, et al. Developing a multisectoral National Action Plan for Health Security (NAPHS) to implement the International Health Regulations (IHR 2005) in Tanzania. BMJ Glob Health. 2018; 3(2):e000600.

39.

Nigeria Centre for Disease Control. Heads of 19 agencies affirm support for the new National Action Plan for Health Security. Published December 17, 2018. Accessed December 16, 2021. https://ncdc.gov.ng/news/159/heads-of-19-agencies-affirm-support-for-the-new-national-action-plan-for-health-security

40.

Nigeria Centre for Disease Control (NCDC). National Action Plan for Health Security Federal Republic of Nigeria (2018-2022). Abuja: NCDC; 2018. Accessed December 2021 , 2021. https://extranet.who.int/sph/sites/default/files/document-library/document/Nigeria%20National%20Action%20Plan%20for%20Health%20Security.pdf

41.

The World Bank. Regional Disease Surveillance Systems Enhancement (REDISSE). Accessed December 16, 2021. https://projects.worldbank.org/en/projects-operations/project-detail/P154807

42.

Nigeria Centre for Disease Control. Press release – Nigeria holds mid-term Joint External Evaluation of International Health Regulations. Published November 22, 2019. Accessed December 16, 2021. https://ncdc.gov.ng/news/204/press-release---nigeria-holds-mid-term-joint-external-evaluation-of-international-health-regulations

43.

World Health Organization (WHO). Joint External Evaluation Tool, Second Edition. International Health Regulations (2005). Geneva: WHO; 2019. Accessed September 19, 2020. https://www.who.int/publications/i/item/9789241550222

44.

World Health Organization (WHO). WHO Benchmarks for International Health Regulations (IHR) Capacities. Geneva: WHO; 2019. Accessed September 19, 2020. https://www.who.int/publications/i/item/9789241515429

45.

Atanda

, Agogo

, Fasominu

, et al. After-action reviews as a best practice tool for evaluating the response to urban disease outbreaks in Nigeria. In: Katz

, Boyce

, eds. Inoculating Cities: Case Studies of Urban Pandemic Preparedness. Cambridge, MA: Academic Press; 2021:61-77.

46.

Federal Republic of Nigeria. Nigeria Public Health Act bill, 2013. Accessed January 5, 2022. https://drive.google.com/file/d/1jlZ8yX0KpYe1imeZgijoKzU7V9sLJnbF/view?usp=sharing

47.

Federal Republic of Nigeria. National Health Act, 2014. Federal Republic of Nigeria Official Gazette. 101(145):A139-A172. Accessed January 5, 2022. https://nigeriahealthwatch.com/wp-content/uploads/bsk-pdf-manager/2018/07/01_-Official-Gazette-of-the-National-Health-Act-FGN.pdf

48.

Federal Ministry of Health (FMOH). National Policy on Food Safety and Its Implementation Strategy. Abuja: FMOH; 2014. Accessed January 5, 2022. https://drive.google.com/file/d/0B1DAmtM1BcbMczJtRDBnczlxc00/view?resourcekey=0-Gfd5O5dfyUMEWxyMYg4qvg

49.

Eteng

, Mandra

, Doty

, et al. Notes from the field: responding to an outbreak of monkeypox using the One Health approach - Nigeria, 2017-2018. MMWR Morb Mortal Wkly Rep. 2018; 67(37):1040-1041.

50.

World Health Organization. After Action Review (AAR) – Nigeria. Accessed April 18, 2021. https://extranet.who.int/sph/after-action-review?region=All&country=232

51.

Prevent Epidemics. Nigeria. Accessed September 19, 2020. https://preventepidemics.org/countries/nga/

52.

Nigeria Centre for Disease Control (NCDC). Country-Led Midterm Joint External Evaluation of IHR Core Capacities. Abuja, Nigeria: NCDC; 2020. Accessed September 19, 2020. https://ncdc.gov.ng/themes/common/docs/protocols/119_1581414518.pdf

53.

Haider

, Yavlinsky

, Chang

, et al. The Global Health Security Index and Joint External Evaluation score for health preparedness are not correlated with countries' COVID-19 detection response time and mortality outcome. Epidemiol Infect. 2020; 148:e210.

54.

Razavi

, Erondu

, Okereke

. The Global Health Security Index: what value does it add?. BMJ Glob Health. 2020; 5(4):e002477.

55.

Kandel

, Sreedharan

, Chungong

, et al. Joint external evaluation process: bringing multiple sectors together for global health security. Lancet Glob Health. 2017; 5(9):e857-e858.

56.

Mayigane

, de Vázquez

, Vente

, et al. The necessity for intra-action reviews during the COVID-19 pandemic. Lancet Glob Health. 2020; 8(12):e1451-e1452.

57.

Stoto

, Nelson

, Piltch-Loeb

, Mayigane

, Copper

, Chungong

. Getting the most from after action reviews to improve global health security. Global Health. 2019; 15(1):58.

58.

Federal Republic of Nigeria. Constitution of the Federal Republic of Nigeria 1999. Abuja: Federal Republic of Nigeria; 1999. Accessed December 16, 2021. https://publicofficialsfinancialdisclosure.worldbank.org/sites/fdl/files/assets/law-library-files/Nigeria_Constitution_1999_en.pdf

59.

Onyemelukwe

IHR Implementation in Nigerian Law: Mapping of Legal Authorities and Analysis of Legislation at Federal Level. Abuja: Nigeria Centre for Disease Control; 2020. Accessed August 10, 2021. https://ncdc.gov.ng/themes/common/docs/protocols/116_1580654680.pdf

60.

Okotoni

Problems and prospects of Nigerian bureaucracy. J Soc Sci. 2017; 7(3):223-229.

61.

Frieden

, Lee

, Bochner

, Buissonnière

, McClelland

. 7-1-7: an organising principle, target, and accountability metric to make the world safer from pandemics. Lancet. 2021; 398(10300):638-640.

62.

World Health Organization (WHO). Thirteenth General Programme of Work (GPW13): Methods for Impact Measurement. Geneva: WHO; 2020. Accessed December 16, 2021. https://www.who.int/publications/m/item/thirteenth-general-programme-of-work-(gpw13)-methods-for-impact-measurement

63.

Raile

, Raile

, Post

. Analysis and action: the political will and public will approach. Action Res. 2021; 19(2):237-254.