Abstract
The World Health Organization (WHO) has implemented one of the recommendations of the International Health Regulations (IHR) review committee on the second extension by developing the IHR monitoring and evaluation framework, and one of the components was the external evaluation.1-3 WHO developed a joint external evaluation (JEE) tool and published it in February 2016. 4 The tool has 19 technical areas divided into the following sections: prevent, detect, respond, and other IHR-related hazards and points of entry. The core elements are: (1) it is essential to prevent and reduce the likelihood of outbreaks and other public health hazards and events as defined by the IHR; (2) detecting threats early can save lives; and (3) rapid and effective response requires multisectoral, national, and international coordination and communication. 4
Since the launch of the JEE process, 90 countries have undergone joint external evaluation as of November 2018. This entire process has not only evaluated capacities required for health security but succeeded in bringing together relevant stakeholders of the country and from outside, such as government and nongovernment institutes, international organizations, donors, and partners. This is a paradigm shift for the success of implementation of the IHR capacities for health security. 5 Countries have benefited from the JEE in establishing stakeholder networks, which are mostly functional and support the development and implementation of the national action plan for health security. 5
When the first edition of the JEE tool was launched, some of the member states raised concerns about not having enough consultation while the tool was being developed, and they brought this to the attention of WHO in writing, meetings, and forums. To address this concern, the JEE secretariat has developed a feedback form and circulated it, in October 2016, to experts, including those who have participated in the JEE missions, and technical experts from all over the world representing member states, institutions, partners, agencies, and WHO offices. In addition, there were several lessons learned from the use of the first edition, and they were reflected in the feedback of the experts who participated in the JEE missions.
The secretariat had received more than 350 individual pieces of written feedback, which were collated into an annotated version of the JEE tool. In addition, the secretariat organized a technical review meeting in April 2017, with more than 90 technical specialists nominated by various governments, institutions, agencies, partners, and WHO technical focal points participating. 6 The participants were divided into working groups, discussed all the feedback that the secretariat had received, and recommended changes. Since then the secretariat has incorporated these changes into a second edition of the JEE tool, published on February 2, 2018. 6 The entire process of revision has taken almost 16 months, and the tool has undergone many formal and informal reviews by the relevant experts for each technical area within and outside of WHO. The entire revision process has ensured the integrity of the first edition of the tool is carried over to the second edition.
Summary of Changes
In total, the new tool has 49 indicators as compared to the first edition, which had 48. The names of 2 technical areas have been changed: Real-Time Surveillance is now Surveillance, and Workforce Development is now Human Resources. 7 In the first edition, there were issues of interpretation of various indicators, attributes, and questionnaires; thus, the new edition is updated with more than 400 footnotes.4,7 In the new tool, when animal and human health scores are given, instead of taking an average, it is advised to use the lower score of those 2 in order to help the sector with a lower score to get more attention for intervention. 7
A few of the technical areas have only minor changes, often related to semantics—like IHR coordination, biosafety and biosecurity, immunization, national laboratory system, reporting, preparedness, medical countermeasures and personnel deployment, linking public health and security, risk communication, points of entry, chemical events, and radiation emergencies—for the purposes of clarity and interpretation. 7 A few of the technical areas have changes in indicators, either by adding, combining, splitting, or moving from one technical area to another.
Two indicators of national legislation have been combined, as both were measuring aspects of the assessment, adjustment, and implementation of the legislation, policies, and administrative arrangements for the IHR. As the technical area is about financing, along with legislation and policy, 2 additional indicators for finance have been added to the area of national legislation, policy, and finance that were not in the first edition.
Two indicators on antimicrobial resistance (AMR) are combined, and a new indicator of effective coordination is added in order to align with the global action plan for AMR. 8 In zoonosis, an indicator on the workforce is incorporated in the human resources technical area, and the rest of the indicators are updated to better reflect output and outcome. The food safety indicator is split into 2, with a view to reflecting detection and response capacities.
Surveillance now has 3 indicators: indicators for the event, indicator, and syndromic surveillance are combined as a surveillance system. The rest of the indicators of surveillance remain similar, with a few changes that reflect output and outcome of the system.
Human resources now have 4 indicators with the addition of a new indicator on inservice training capacities. The addition of this indicator has the following 2 objectives: (1) many countries are strengthening their human resource capacity through these kinds of training, which should be captured in the evaluation; and (2) “what gets measured gets done”—that is, the indicator can ensure that countries will consider more inservice training as part of their human resource strategy.
An indicator on the veterinary workforce from zoonosis is incorporated in the indicator that linked to the multisectoral workforce as required for IHR implementation. The emergency response operations now have 3 indicators; 1 of the indicators on case management has been moved to medical countermeasures and personnel deployment.
Two previous indicators on the capacity to activate and on operational procedures for emergency operations are combined as an emergency operation center, and an additional indicator on emergency coordination has been added.
Some practical concerns were identified on a rollout of the new tool, and the WHO secretariat has proposed the following solutions.
The experts raised a concern on the interpretation of the scores between the tools. The secretariat has added a section called “summary of changes between JEE tool first and second editions” explaining how to interpret the new tool. The summary provides details about the changes in indicators, highlights the major changes in the indicators, and provides guidance on how to interpret these changes to compare the scores for each technical area. There are only a few technical areas where indicators are added or combined or moved to another section that require further interpretation, but the rest of the technical areas do not need any additional interpretation. Finally, the countries that have already undergone JEE using the first edition can monitor their level of improvement using the new JEE tool without any difficulties, using the same methodology for interpretation. 7
There was concern about how to manage the countries that have already started their self-evaluation process using the first edition. The WHO secretariat has recommended that for countries that have already begun self-evaluation using the first tool, the external evaluation will use the same tool, too. Countries that started self-evaluation after the launch of the revised tool would use the new tool for self-evaluation and external evaluation. To date, 84 countries have used the first edition and 6 have used the second edition of the JEE.
Footnotes
Acknowledgments
The authors acknowledge all the technical focal points of technical areas of WHO and WHO regional offices, as well as the feedback received from all JEE experts from member states, institutions, partners, agencies, and donors and all meeting participants in the JEE review meeting. We would like to particularly acknowledge financial support from the US Centers for Disease Control and Prevention, the governments of Germany and Finland, and the Bill and Melinda Gates Foundation, and technical support from the various member states, the US Department of Health and Human Services, the US Department of Agriculture, FAO, OIE, WB, and IAEA. Finally, we thank all countries who have volunteered for a JEE and partners and organizations whose support have been instrumental to the success of the external evaluations.