Analyzing Targeted Violence Against Medical Workers and EVD Incidence in the 2018-19 Democratic Republic of the Congo Outbreak Using Vector Autoregression and Granger Causality

Abstract

The purpose of this study was to evaluate the relationship between violent events targeted against aid workers and the incidence of Ebola virus disease (EVD) during the 2018-19 Democratic Republic of the Congo (DRC) outbreak in the North Kivu and Ituri provinces. Time series models using vector autoregression were constructed using violent event data confined to the outbreak region from the Armed Conflict Location and Event Dataset (ACLED), in combination with EVD incidence reporting from the World Health Organization and DRC Ministry of Health, to examine intervariable temporal relationships, paired with Granger causality testing to assess both uni- and multidirectional associations. Violent events against aid workers and Granger-causing EVD incidence were found to be significant across 2 principal lag-ranges, 8 to 14 days and 22 to 29 days, both suggesting plausible causal associations. The multivariate model for violent events and violence-related fatalities Granger-causing EVD incidence was also found to be significant at lags greater than 9, reinforcing the plausible causal association. Findings from the study suggested that the relationship between targeted violence and EVD incidence may be explained etiologically, as significant lag-ranges corresponded to plausible patient presentation timeframes, based on the incubation period for EVD. Additionally, the findings may also be explained through impact on operations, when events targeting facilities, supply lines, and personnel affect treatment capability.

This study evaluated the relationship between violent events targeted against aid workers and the incidence of Ebola virus disease (EVD) during the 2018-19 DRC outbreak. Time series models using vector autoregression were constructed using violent event data confined to the outbreak region from the Armed Conflict Location and Event Dataset (ACLED), in combination with EVD incidence reporting from the WHO and DRC Ministry of Health, to examine intervariable temporal relationships, paired with Granger causality testing to assess both uni- and multidirectional associations.

On August 1, 2018, the government of the Democratic Republic of the Congo (DRC) announced that a cluster of Zaire Ebolavirus cases, also known as Ebola virus disease (EVD), had been identified in the North Kivu province near the city of Beni.¹ The outbreak came only a week after the DRC Ministry of Health declared an end to another unrelated Ebola outbreak in the western Equateur province, approximately 2,500 km from where the new cases were identified.¹ Only 5 weeks into the outbreak, it rapidly became apparent that regional insecurity and conflict would have a negative impact on the international community's ability to respond effectively to the EVD outbreak.²

As the outbreak progressed, World Health Organization (WHO) reporting, international media outlets, and nongovernmental organizations (NGOs) operating in the region all emphasized the difficulty in responding to the outbreak amid regional violent conflict and directed attacks on aid workers and treatment facilities, resulting in forced suspension of response activities and having a significant impact on continuity of operations.^3-11 Despite detailed reporting and the seemingly clear effects that violence in the region has had on response efforts, questions have arisen regarding the relationship between violence in the region and its impact on the epidemiology of the outbreak.

The purpose of this research study was to assess the relationship between violent events targeted against aid workers and EVD during the 2018-19 DRC outbreak in the North Kivu and Ituri provinces. In addition to identifying the presence of statistically significant temporal relationships, the study also sought to determine whether delayed case incidence could be attributed to the magnitude or severity of a given violent event, measured through violent event fatalities. In measuring both frequency and magnitude of violent events, discerning acute versus delayed impact to case presentation enabled differentiation of temporal relationships manifesting as short-term compromised operations, as opposed to being directly related to increased case incidence following the incubation period.

Methods

Data on EVD incidence for the 2018-19 EVD outbreak in North Kivu and Ituri provinces in the DRC were obtained from the DRC Ministry of Health and summarized in WHO situation reporting, outlining confirmed and probable case counts and deaths, by day, for a 270-day period between August 3, 2018, and April 30, 2019.^12,13 For purposes of this study, only confirmed cases were included in the incidence count over the study period; these were defined as suspected cases with laboratory confirmation via positive IgM antibody, positive PCR, or viral isolation, as defined in the WHO outbreak situation reports.^12,13 Probable cases were defined through symptom identification—such as fever; blood in stool; bleeding from gums; bleeding into skin, eyes, or urine—or through positive point-of-care rapid diagnostic testing.¹⁴ Probable cases changed to confirmed cases following laboratory confirmation; suspected cases (ie, probable cases without lab confirmation) were excluded.¹²

Variables included in this study to measure violent events were derived from the Armed Conflict Location and Event Data (ACLED) project, a database that collects dates, actors, types of violence, locations, and fatalities of all reported sociopolitical violence and protest events across Africa, South and Southeast Asia, the Middle East, Europe, and Latin America.¹⁵ In order to quantify violent events targeting aid workers or facilities in the outbreak region, a series of database queries were executed using keyword searches, generating a count of targeted violent events, a running count of keyword references, and total fatalities across all violent events for each outbreak day under study. Keywords used for identifying events targeting aid workers or associated facilities were selected based on past WHO reporting on violence against aid workers in the region and categorized into: victims (or actors), facility types, known organizations operating in the region, and disease characteristics. Table 1 summarizes keywords used for event inclusion, along with the number of occurrences, number of unique records, and percentage of records, all over the 270-day study period.

Table 1.

Selection Criteria for ACLED Events Targeted Against Aid Workers

ACLED Keyword	Number of Occurrences	Number of Unique Records	Percentage of Records (Out of 270 Days) (%)
“Ebola”	25	18	6.667
“Medical”	2	2	0.741
“Health”	16	16	5.926
“Doctor”	4	4	1.481
“Nurse”	2	2	0.741
“Infect^*”	1	1	0.370
“NGO”	5	5	1.852
“Isolat^*”	1	1	0.370
“UN”	4	4	1.481
“MONUSCO”	23	22	8.148
“Red Cross”	2	2	0.741
“Humanitarian	2	2	0.741
“Hospital”	4	4	1.481
“Clinic”	6	6	2.222
“Treatment”	5	5	1.852

Represents Boolean wildcard operator used during keyword searches. Specifies leading characters and captures any trailing character combinations (eg, infect^* = infection, infected, infecting, infect).

Data Analysis

A time series analysis was conducted to assess the relationship between violent events and outbreak epidemiology. More specifically, vector autoregression (VAR) modeling was paired with Granger causality testing to assess not only correlation, but also the plausibility of temporally forecasting univariate and multivariate relationships.^16-18 Granger causality testing is a method commonly used in economic forecasting; it aims to measure the feasibility of predicting future values of a time series using prior values of another time series. In context, the definition of “cause” under these circumstances relies entirely on the predictability of series (X_t), where if another series (Y_t) contains information in past terms that can be used to aid in predicting X_t, then Y_t is said to “cause” X_t, according to Granger (1969).¹⁹ R Statistics v3.6.0 was used for all statistical procedures, including the “zoo,” “forecast,” “vars,” and “tseries” packages.^17,20-22 Prior to modeling, assumptions testing was performed on both the univariate and multivariate time series datasets, including taking the first difference in the EVD incidence variable to satisfy the assumption for stationarity, or the necessity that the time series' unconditional joint probability distributions remain consistent when shifted in time (ADF = −9.3079, p < 0.01). Model parameters included specification of a lag, chosen either manually or by optimizing based on an information criterion. Rather than testing multiple VAR models at static lags, the procedure was modified to iterate through a range of lags—1 to 30 days—collecting information on each model to evaluate performance for any given lag. Choice of lag-range, 30 days, was directly related to the incubation period for EVD, which can be anywhere from 2 to 21 days, with 4 to 10 days being average.^23,24 Following model fitting, impulse response functions were evaluated for lags found to be significant, which is a method commonly used in econometrics to better understand intervariable reactions to a shock in one or more of the model variables.¹⁸ By tracing the transmission of a single shock through an otherwise noisy system, impulse response testing allows for enhanced interpretation of intervariable relationships over time, vice globally.²⁵

Results

Figure 1 provides an overview of the total number of targeted violent events, EVD cases, and EVD fatalities, spanning 270 days from August 3, 2018, to April 30, 2019. EVD incidence, in the form of confirmed case counts per day, ranged from 0 to 27, with an average of 5.41 cases per day (σ = 4.65). Most days (83.7%, n = 226) had at least 1 new EVD case. There was a gradual increase in cases per day over time, consistent with the concept of increased transmission as the outbreak progressed. EVD fatalities ranged from 0 to 37, averaging 4.97 fatalities per day (σ = 6.17). Most days (67.4%, n = 182) had at least 1 fatality. The highest event-fatality day occurred on December 6, 2018, resulting from several isolated attacks by the Allied Democratic Forces (ADF) and other militia forces against civilians, DRC Armed Forces (FARDC), and a convoy of Union for Progress and Reform (UPC) politicians. Violent events targeting aid workers ranged from 0 to 13 per day, averaging 2.47 per day (σ = 2.34).

Figure 1.

Multivariate Time Series Plot with Difference = 0

Adjusted R-squared values, measuring the proportion of outcome variance that can be explained by exposure or predictor variables, was chosen as the primary metric to assess model performance over the 30 iterative lags. The model using violence-related fatalities and targeted violent events against aid workers to explain case incidence ranged from 0.091 in a lag of 1 day (r = 0.091, p < 0.001) to 0.348 in day-30 lag (r = 0.348, p < 0.001), where overall, the top performing lags ranged from approximately 16 to 30 days. Additionally, a significant increase in explained variance in the violent events against aid workers model between lags of 8 (r = 0.225, p < 0.001) and 9 days (r = 0.290, p < 0.001) suggested that case incidence and violence-related fatalities best explain the variance in violent incidents against aid workers approximately 9 or more days following high-incidence and/or high-fatality days. The other 2 multivariate models, using violent event counts and fatalities as dependent variables, outperformed the case incidence model, with max adjusted R-squared values of 0.702 (r = 0.702, p < 0.001) and 0.368 (r = 0.368, p < 0.001), respectively, which was to be expected since event counts and violence-related fatalities are interdependent variables (Figure 2).

Figure 2.

Overview of Vector Autoregression Models Over 30 Lag-Days

Moving from VAR model performance metrics, which measured shared variance between variables included in the multivariate time series models, the plot in Figure 3 assesses both univariate and multivariate plausible causal relationships between case incidence, violence-related fatalities, and violent events targeting aid workers. Granger causality testing was performed on each univariate and multivariate model combination across 30 lags, where the plot illustrates significance (p) values for each Granger-test iteration. To account for multiple testing, both adjusted and unadjusted significance values were calculated using the Benjamini-Hochberg false discovery rate correction procedure.²⁶ Justification for reporting both sets of values in Figure 3 stems from wide disagreement in statistics regarding correction methods for multiple testing with significant temporal dependencies, where both when and how to correct for multiple testing in time series is largely unsettled.^26,27 Evaluated at α = 0.05, 1 of the 6 univariate models, in addition to all 3 multivariate models, were found to establish Granger causality at any tested lag. Specifically, targeted events against aid workers Granger-causing EVD case incidence was found to be significant at lags of 8 to 14 days (F = 2.185 to 1.791, p ≤ 0.05) as well as lags between 22 and 29 days (F = 1.691 to 1.580, p < 0.05). Expanding on the relationship, impulse response testing, also known as shock analysis, was performed based on a bootstrapped 95% confidence interval, finding that the shock from a targeted violent event would initially result in an increased case count, with the most significant impact occurring between lags of 9 and 12 days.

Figure 3.

Univariate and Multivariate Relationships by Variable Combination Over 30 Lags

All 3 multivariate models were statistically significant at various lags for establishing Granger causality. The model assessing Granger causality for EVD case incidence was significant at lags between 9 and 30 days (F = 2.102 to 1.632, p < 0.05), the model assessing Granger causality for violence-related fatalities was significant at lags of 1 to 6 days (F = 17.741 to 2.081, p < 0.01), and the model assessing cause for events was significant at isolated lags between 11 (F = 1.565, p = 0.048) and 24 days (F = 1.404, p = 0.042). EVD case incidence Granger-causing targeted events against aid workers was not found to be significant at any lag, further supporting the unidirectional relationship between EVD incidence and violent events.

Discussion

Past qualitative research has found that infectious disease epidemiology is influenced by ecological and environmental factors in addition to human demographics and behavior.²⁸ To date, public health literature on humanitarian crises mainly relies on interviews, focus groups, commentaries, and editorials. Public health also often uses media and/or government reporting, professional newsletters, and organizational campaigns as dissemination platforms as opposed to peer-reviewed studies.²⁹ Through those sources and methods, in combination with using past infectious disease outbreaks as a basis for establishing precedent, general themes have emerged that provide insight into the contributing factors that proliferate disease spread in conflict-affected and complex security environments. Factors that affect all infectious disease outbreaks, regardless of the security environment, such as infection prevention measures, adequate surveillance, and health systems in place to handle surge capacity, all become increasingly complex and strained in conflict-affected areas because of issues related to logistics and supply management, staffing, poor governance and organization, and improvised treatment facilities.^28,29 Issues more specific to complex security environments include attacks on individuals, such as doctors, nurses, security, and support staff; targeting of facilities directly affiliated with medical relief efforts (hospitals and clinics); obstructions that impede delivery of supplies and transport of resources; pressured discrimination by local groups on medical staff; and criminalization of healthcare workers.^28,29 Local culture and practices can also significantly contribute to outbreak response, where mass gatherings (including religious gatherings), burial practices, and insensitivity of aid workers to local culture can inhibit local cooperation, spark skepticism in response, and ultimately lead to increased incidence and underreporting of cases.²⁹

Beyond human behavior and sociodemographic factors, infectious disease outbreaks are naturally influenced by the nature of the disease itself—namely, pathology, routes of transmission, and etiological factors, all contributing to how the disease proliferates and moves through an affected population. The EVD outbreak under study in the North Kivu and Ituri provinces of the DRC specifically involves the Zaire Ebolavirus strain, 1 of 5 Ebola virus species that have been identified, and 1 of 4 known to cause severe hemorrhagic fever in humans.³⁰ Excluding index cases, where animal-to-human transmission usually occurs through handling of wild mammals or bush meat, the virus is transmitted person-to-person via direct contact with infected blood, urine, vomit, fecal matter, bodily secretions, or semen, making it highly communicable once infected individuals begin shedding the virus. Burial practices that involve direct contact with the body of the deceased have also been known to contribute to disease spread, reinforcing the idea that local culture and practices contribute to disease transmission.^23,24

Aside from an incubation period of approximately 2 to 21 days, EVD outbreaks may be particularly susceptible to complex security environments for a variety of reasons.²⁴ First, because of concentrated physical contact during violent events, as well as multiple routes of exposure and transmission, the frequency and duration of violence can increase the probability of disease spread.²⁸ That said, regional and isolated violence and instability can also indirectly influence disease spread, as in situations where medical supply lines are disrupted, when medical facilities are attacked resulting in an impact on treatment operations, and even in healthcare workers are exposed in improvised treatment facilities with substandard infection control measures due to lacking advanced medical infrastructure. Additionally, quarantine (or lack thereof) is another major issue contributing to disease incidence in security complex environments—specifically, identifying exposed individuals for quarantine presents challenges, combined with hesitancy in quarantine compliance among already resistant populations.^28,29

Results from this study suggest a plausibly causal relationship between violent events targeted against aid workers and increased EVD incidence over the course of the ongoing DRC EVD outbreak. Lags of 8 to 14 days and 22 to 29 days were found to be significant between violent events targeted against aid workers and rises in EVD cases. These lags align with expected etiologic and epidemiologic characteristics of EVD. Specifically, assuming the average incubation period for EVD of 4 to 10 days, and adding 3 to 5 days for symptom manifestation before the individual seeks medical attention, the early lags of 8 to 14 days correspond closely to the 7 to 15 days it would likely take for a person who contracted EVD to present for medical care. Further, the later lags between 22 and 29 days could correspond to a longer incubation period (21 days), with an additional 3 to 5 days for symptom manifestation before an individual seeks medical treatment, placing the expected time-to-presentation, 24 to 26 days, within the window of plausibility. Alternatively, the later lag period could also be explained through secondary transmission. Assuming a first wave of patients who present between 8 and 14 days postevent/exposure, approximately doubling that time to correspond with a second wave of infections (16 to 28 days) would also fall within the 22- to 29-day significant lag window.

Another possible explanation for the significant lag ranges between violent events and EVD incidence is that there is an impact on operations because of the violent events. Depending on the severity and nature of a given violent incident, the impact on operational functioning during and after the incident is likely to be highly variable. Repetitious targeted violent events that damage facilities or attack supply lines (such as those observed on December 12 and 27, 2018, as well as February 24 and 27-28, March 9 and 14, and April 18, 2019) could potentially cause significant delays in treatment of patients and recording of cases, explaining the later significant lag-range.^5,31 Likewise, attacks against personnel, including harassment, abduction, and murder, as observed on April 19 when a WHO epidemiologist was killed in an armed attack on Butembo University Hospital, may also manifest as delays in operations related to staffing, as well as heightened security for personnel and facilities, all potentially resulting in delays in treatment.^9,31

In practice, this study reinforces academic literature as well as government and media reporting, suggesting that infectious disease outbreaks in developing regions and complex security environments are susceptible to regional violence that affects humanitarian response. As demonstrated, results from this study can also be used to quantitatively assess the magnitude of relationships among violent events, violence-related fatalities, and EVD incidence, characterizing not only the existence, but also the extent of association over time. Future research efforts involving a framework similar to this study for assessing violence in relation to infectious disease spread could extrapolate on the VAR, Granger-cause, and impulse response measures used in this study by incorporating forecasting into the modeling process to predict how changes in violence patterns will influence case incidence.

Limitations

Given the difficulty in obtaining data from outbreaks in developing regions, this study has several limitations regarding data accuracy or comprehensiveness, as well as the statistical methods used to inform conclusions. First, given the level of violence and overall lack of medical infrastructure in the DRC region at this time, it is likely that case counts are underreported. Additionally, in identifying violent events associated with humanitarian and medical aid operations in the region, there were limitations in both the overall ACLED dataset as well as the process for subsetting the dataset. The ACLED database is primarily derived from news reporting and is known to be incomplete in reporting of events in developing regions. Further, given that this study used a keyword search process for selecting events specifically involving aid workers or aid organizations operating in the region, it is possible that some relevant events were overlooked.

To mitigate the latter concern, following the selection process, records were cross-checked for validity with secondary sourcing as well as for consistency in event frequency per day with the GDELT database, another open-source dataset with aggregated regional violent event information. Additionally, all records were further reviewed to confirm contextual relevance based on the identified keywords, adding to the confidence in the subsetting methodology.³²

Finally, given that vector autoregression has primarily been used in economic forecasting and econometrics, the employment of VAR modeling in epidemiologic modeling is arguably nontraditional. Although univariate relationships are common in VAR modeling, there are likely many confounding factors influencing the relationships under study that limit our ability to establish epidemiologic cause, but rather Granger causality, purely based on quantitative association.

Conclusion

Beginning only weeks after the first cases of EVD were reported, international organizations began to recognize regional violence and instability as barriers to responding to the rapidly developing 2019 EVD outbreak in the North Kivu and Ituri provinces of the Democratic Republic of the Congo. While authorities observed increasing case counts amid attacks against aid workers, destruction of treatment facilities, and theft of supplies, it was unclear whether regional violence was contributing directly to disease incidence, the impact on aid operations, or some combination of these. This study used a time series framework to model the interaction between violent events against aid workers, fatalities associated with those events, and case incidence, using lagged variables to examine the relationships both instantaneously and temporally. Results found plausible Granger causality relationships between violent events and case incidence in the study region in both early and later lags, reinforcing the conclusion that regional violence, and specifically violence against aid workers and operations, can significantly influence case incidence both etiologically, as well as logistically in terms of impact on operations, in both early and later lags. While the concept of violence influencing disease in complex security environments has been well studied, the novelty of this research study stems from the quantitative framework used to establish plausible Granger causality. This study contributes significantly to the mounting evidence implicating violence and regional instability in case incidence associated with infectious disease outbreaks and lays a framework for future researchers to combine research methodologies for examining sociopolitical influences on infectious disease outbreaks from both qualitative and quantitative perspectives.

References

World Health Organization. Cluster of presumptive Ebola cases in North Kivu in the Democratic Republic of the Congo [news relaease]. World Health Organization August 1, 2018. https://www.who.int/news-room/detail/01-08-2018-cluster-of-presumptive-ebola-cases-in-north-kivu-in-the-democratic-republic-of-the-congo. Accessed November 4, 2019.

World Health Organization. Ebola virus disease – Democratic Republic of the Congo (7 September 2018, 4 October 2018, 11 October 2018, 18 October 2018). https://www.who.int/csr/don/archive/year/2018/en/. Accessed November 4, 2019.

United Nations. DR Congo violence puts Ebola humanitarian response at risk, UN food relief agency warns. UN News November 20, 2018. https://news.un.org/en/story/2018/11/1026131. Accessed November 4, 2019.

United Nations. UN will do ‘utmost to prevent and mitigate any risk of violence’ in DR Congo, pledges Mission chief. UN News March 18, 2019. https://news.un.org/en/story/2019/03/1034911. Accessed November 4, 2019.

World Health Organization. WHO expresses concern over damage to Ebola treatment facilities in the Democratic Republic of the Congo. February 28, 2019. https://www.afro.who.int/media-centre/statements-commentaries/who-expresses-concern-over-damage-ebola-treatment-facilities. Accessed November 4, 2019.

World Health Organization. Ebola response in Democratic Republic of the Congo risks slowdown [news release]. February 26, 2019. https://www.who.int/news-room/detail/26-02-2019-ebola-response-in-democratic-republic-of-the-congo-risks-slowdown. Accessed November 4, 2019.

Vinck

, Pham

, Bindu

, Bedford

, Nilles

. Institutional trust and misinformation in the response to the 2018-19 Ebola outbreak in North Kivu, DR Congo: a population-based survey. Lancet Infect Dis, 2019:19(5):529-536.

World Health Organization. Ebola situation worsening while support lags. April 30, 2019. https://www.who.int/news-room/detail/30-04-2019-statement-from-who-high-level-mission-to-butembo-drc-ebola-situation-worsening-while-support-lags. Accessed November 4, 2019.

World Health Organization. WHO Ebola responder killed in attack on the Butembo hospital.April 19, 2019. https://www.who.int/news-room/detail/19-04-2019-who-ebola-responder-killed-in-attack-on-the-butembo-hospital. Accessed November 4, 2019.

10.

World Health Organization. WHO reaffirms commitment to Democratic Republic of the Congo as Ebola outbreak nears 1000 cases amid increased violence. March 23, 2019. https://www.who.int/news-room/detail/23-03-2019-who-reaffirms-commitment-to-the-democratic-republic-of-the-congo-as-ebola-outbreak-nears-1-000-cases-amid-increased-violence. Accessed November 4, 2019.

11.

World Health Organization. Doubling down on Ebola. March 19, 2019. https://www.who.int/news-room/feature-stories/detail/doubling-down-on-ebola. Accessed November 4, 2019.

12.

World Health Organization. WHO Nord Kivu DRC EVD Outbreak Situation Reports. 2019. https://www.who.int/emergencies/diseases/ebola/drc-2019/situation-reports.

13.

RDC MdlS. Ministry of Health Nord Kivu EVD Outbreak Situation Reports. 2018.

14.

Dhillon

, Srikrishna

, Kelly

. Deploying RDTs in the DRC Ebola outbreak. Lancet, 2018; 391(10139):2499-2500.

15.

Armed Conflict Location & Event Data Project (ACLED) Codebook. 2015. https://www.acleddata.com/wp-content/uploads/dlm_uploads/2017/10/ACLED_Codebook_2019FINAL_pbl.pdf. Accessed November 4, 2019.

16.

Sims

CA.

Macroeconomics and reality. Econometrica: journal of the Econometric Society. 1980:1-48.

17.

Pfaff

. VAR, SVAR and SVEC models: implementation within R package vars. J Stat Softw, 2008; 27(4):1-32.

18.

Lütkepohl

. New Introduction to Multiple Time Series Analysis. Berlin, Heideelberg: Springer-Verlag; 2005.

19.

Granger

CWJ

. Investigating causal relations by econometric models and cross-spectral methods. Econometrica, 1969; 37(3):424-438.

20.

Zeileis

, Grothendieck

. zoo: S3 infrastructure for regular and irregular time series. arXiv preprint math/0505527. 2005.

21.

Trapletti

, Hornik

, LeBaron

. Tseries: time series analysis and computational finance. R package version 010-11. 2007.

22.

Hyndman

, Athanasopoulos

, Bergmeir

, et al. Package ‘forecast.’ 2019. https://cran r-project org/web/packages/forecast/forecast pdf. Accessed November 4, 2019.

23.

Kadanali

, Karagoz

. An overview of Ebola virus disease. North Clin Istanb, 2015; 2(1):81-86.

24.

World Health Organization. Ebola virus disease. May 30, 2019. https://www.who.int/news-room/fact-sheets/detail/ebola-virus-disease. Accessed November 4, 2019.

25.

Koop

, Pesaran

, Potter

. Impulse response analysis in nonlinear multivariate models. J Econom, 1996; 74(1):119-147.

26.

Benjamini

, Hochberg

. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol, 1995; 57(1):289-300.

27.

Reiner

, Yekutieli

, Benjamini

. Identifying differentially expressed genes using false discovery rate controlling procedures. Bioinformatics, 2003; 19(3):368-375.

28.

Gayer

, Legros

, Formenty

, Connolly

. Conflict and emerging infectious diseases. Emerg Infect Dis, 2007; 13(11):1625-1631.

29.

Foghammar

, Jang

, Kyzy

, et al. Challenges in researching violence affecting health service delivery in complex security environments. Soc Sci Med, 2016; 162:219-226.

30.

Heymann

, ed. Control of Communicable Diseases Manual. 20th ed. Washington, DC: American Public Health Association; 2016. https://ccdm.aphapublications.org/. Accessed November 4, 2019.

31.

Raleigh

, Dowd

. ACLED dataset: all events within the Democratic Republic of the

Congo between 08-03-2018 and 04-30-2019

. 2019. https://www.acleddata.com/data/.

32.

GDELT. The Global Data on Events, Language, and Tone (GDELT) Project. 2019. https://www.gdeltproject.org/. Accessed November 4, 2019.