Abstract
In this reply we offer a response to the Mercer and Reed (2015) replication of Toya and Skidmore (2007). The replication study extends the original work in several important dimensions and is a very useful contribution. Nevertheless, we offer additional evaluation and discussion of the robustness portion of their replication. In particular, we consider the fixed effects and interval regression estimates.
In this replication study, Mercer and Reed (2015) reexamine the relationship between measures of economic development and natural disaster impacts. The authors fully replicate the original study by Toya and Skidmore (2007) and then conduct a rigorous robustness evaluation of the original work. The robustness analysis (1) uses updated data, (2) addresses potential sample selection and overdispersion using the interval regression technique, and (3) adds a time indicator variables and country-fixed effects. The replication study is thorough and the analysis is carefully done. The researchers conclude that once these robustness issues are addressed, the only measure of economic development that is robustly related to disaster-induced fatalities is income. While the replication study extends the original work in several important dimensions and is a very useful contribution, we offer our response to the robustness evaluation portion of the replication.
In the robustness section of the article, the authors use interval regression techniques because they are concerned about sample selection issues (disaster observations may be systematically omitted from the database due to the criteria used for inclusion) as well as overdispersion. The authors treat 17 percent of the observations at the low end of the distribution (five or fewer deaths) as censored. About 2.5 percent on the upper end of the distribution (1,000 or more deaths) are also treated as censored because these observations are considered to be outliers. With regard to censoring on the low end, Mercer and Reed (2015) state, “If economic development variables are successful in mitigating loss of life from natural disasters, then these events may be omitted from the disaster sample in higher income countries. This will cause the impact of economic development variables to be underestimated.” However, Kahn (2005) conducts a careful examination of this issue, demonstrating that the probability of a disaster event occurring and being recorded in the Office of Foreign Disaster Assistance (OFDA)/Centre for Research on the Epidemiology of Disasters (CRED) database is generally independent of the level of development. With the exception of floods, high- and low-income countries are equally likely to experience a naturally occurring disaster event and be included in the OFDA/CRED database. Kahn’s result suggests that sample selection may not be a critical issue.
In addition, with the interval regression technique, the authors acknowledge that the left- and right-side cutoffs for the censoring are at the discretion of the researcher and are “ad hoc.” 1 We therefore want to determine whether the lack of robustness in the relationship between the measures of economic development and disaster-induced deaths is sensitive to alternative left- and right-censoring points. To explore sensitivity of the coefficient estimates to this issue, we tested different censoring thresholds on the left and right sides in the interval regression. In table 1, we present a series of interval regressions in which we change the censoring point from five deaths on the left side to four, three, two, and one deaths, and change the censoring point from 1,000 deaths on the right side to 5,000, 10,000, 20,000, and 40,000 deaths.
Checking for Robustness (Fatalities)—Interval Regressions.
Note: GDP = gross domestic product. The dependent variable is Ln(Deaths). The first column reproduces the ordinary least squares results from table 4a, column (3) above. The second column estimates the same specification using interval regression, where observations are categorized as censored whenever the number of disaster-related fatalities was 5 or fewer (617 observations), or 1,000 or more (93 observations), which repeats the result in table 5a. All four estimating equations include Ln(Population), Ln(Area), and a series of dummy variables to indicate disaster type. Numbers in parentheses are t-values based on cluster-robust standard errors (robust to country-specific serial correlation and heteroscedasticity).
In column (1) of table 1, we present the original estimates from table 4a of Mercer and Reed (2015), which uses the same approach as Toya and Skidmore (2007) but with updated data. We were able to exactly replicate this regression. In column (2), we present the interval regression estimates from table 5a of Mercer and Reed (2015). From our reading, Mercer and Reed indicate that this regression does not include time indicator variables. However, we were only able to obtain exact coefficient estimates by including time effects in the regression. In column (3), we present the same interval regression as in column (2) except that we exclude the time indicator variables. In this regression, the coefficients on schooling years, openness, and money supply are all statistically significant. In columns (4–6), we incrementally reduce the number of left- and right-censored observations. These regressions show that, with the exception of income, the statistical significance of the variables increases as fewer observations are treated as censored.
From our evaluation, it appears that the inclusion of time indicator variables is the primary reason for the reduced significance of the coefficient estimates in the interval regressions, not the censoring. In the context of this type of disaster impact evaluation, there are trade-offs associated with including time- and country-fixed effects. Time indicator variables and fixed country effects mop up general trends and cross-country variation, and so it is perhaps not overly surprising that adding them to the regressions reduces the precision of the coefficient estimates. Perhaps more importantly, there is a significant time lag between improved economic circumstances and improved disaster safety. Kahn (2005) chooses not to include fixed effects in his analysis of disaster impacts because “economic adjustment…is unlikely to quickly take place…Though rising national income can be measured, there is likely to be a long latency between economic development and improved average quality of infrastructure as new homes and new infrastructure are built of higher quality than the existing capital stock.” In the context of identifying specific policy impacts, the use of panel data and controlling for fixed jurisdiction and time effects is very important, particularly when policies are expected to generate immediate effects. However, in the context of natural disasters, the translation of economic development to improved disaster safety is a slow process that is unlikely to be captured with year over year within country changes in economic conditions.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The original research was partially funded by a grant from the Japanese Government Ministry of Education, Science, Sports, and Culture, Grant-in-Aid for Scientific Research (C), 16530182, 2004–2005. The reply to the replication was not funded.