The Value of Joint External Evaluation as a Tool for Country-level Health Security Monitoring

We commend Tsai and Katz ¹ for their article comparing the results of national self-assessments of International Health Regulations (IHR) ² core capacity achievement using the WHO-produced IHR Monitoring Questionnaire (IHRMQ) ³ with those of external evaluators using the Joint External Evaluation (JEE) approach and tool.^4,5 With global health security largely dependent on individual states' ability to detect and respond to public health threats, having reliable (and trusted) measures of country-level capabilities is essential. Further, the authors' contribution is timely, given the growing demand for JEEs and ongoing global debate about how to best measure countries' ability to detect and respond to public health events of international concern.

The authors' analysis focuses on the difference between the scores generated by countries' self-assessments of core capabilities compared with external assessors' judgments, finding that global health security is markedly worse when evaluated by external assessors. ¹ Reasons the authors suggested for the difference included measurement biases introduced by different perceptions of and expertise among assessors and differences in the interpretation of indicators presented in the monitoring tools used. ¹

In 2017, as part of annual IHR monitoring activities, the Solomon Islands Ministry of Health and Medical Services informally trialed use of the JEE tool alongside the IHRMQ. ⁶ While not a research project per se, similar results to those presented in the Tsai and Katz article were found. We add our observations that the JEE tool was more conducive to stimulating critical debate among stakeholders, was more engaging, and resulted in the production of better insight into health security strengths and weaknesses in the country. From a practical perspective, we found the addition of a 5-point scoring scale (compared to binary response options of the IHRMQ), the comparative simplicity, and the emphasis on the provision of evidence to support claims of capacity achievement as features of the JEE tool that allowed more refined (and often more conservative) judgments of capacity attainment to be made.

Building on Tsai and Katz's work, further research is required to understand what enables and limits the accurate monitoring of country-level progress toward achieving the capacities required to prevent, detect, and respond to public health emergencies, and what influence evaluation-derived information has on capacity development programming. Qualitative research that explores such factors will provide knowledge to supplement Tsai and Katz's findings.