Acinetobacter baumannii 2002–2008: Increase of Carbapenem-Associated Multiclass Resistance in the United States

Abstract

The study consisted of data for 55,330 U.S. Acinetobacter baumannii isolates from The Surveillance Network^® database for the period 2002–2008. Risk factors were time, age, sex, census region, location (Ward or ICU), and isolate source. Antimicrobial susceptibility data were available for carbapenems, cephalosporins, aminoglycosides, fluoroquinolones, and β-lactam/β-lactamase inhibitor combinations. Multiclass resistance was defined as nonsusceptibility to carbapenems and two or more additional classes. Odds of resistance were obtained using a logistic regression model with cubic splines. Carbapenem-associated multiclass resistance has had a 3.7-fold (95% confidence interval [CI] 3.4–4.3) increase from 20.6% in 2002 to 49.2% in 2008. Among blood isolates the increase was by 2.2 times (95% CI 1.7–2.9). Subjects <18 years old had significantly (p < 0.001) lower rates in 2002 (6.9%) than those 65 years or older (21.5%), but by 2008 this difference diminished as rates increased to 44.2% and 54.2%, respectively. A similar divergence was also observed between ICU and Ward, with no differences in 2002, whereas in 2008 ICU isolates had significantly higher rates (55.2%, 95% CI 53.6%–56.9%) than Ward isolates (45.6%, 95% CI 44.2%–47.0%). Over half of all A. baumannii–resistant isolates were carbapenem and multiclass resistant in 2008. Rates among subjects <18 years old have increased faster than those of the elderly, and in the ICU as compared to Ward.

Introduction

A cinetobacter baumannii is currently the most medically relevant representative of the genus Acinetobacter, as it has emerged as a problematic human pathogen for healthcare institutions globally.²¹ Initially, the problem was primarily confined to the ICU setting. However, during the last few years, an increased number of patients have been reported with acquired nosocomial infections outside the ICU.²⁸ Although most of the patients had respiratory tract infections, a minority had bacteremia with an unknown primary site.⁸

The reported increased incidence of A. baumannii infections¹² has been attributed to the advances in support of critically ill and frail patients.¹⁴ This rise in the incidence of A. baumannii infections was noted first in the ICU setting,⁶ although other areas of the hospital seem to be reporting an increased number of infections with this pathogen.¹¹

To add to the concern over the rise in A. baumannii infections, Acinetobacter species have the remarkable ability to upregulate or acquire resistant determinants to many classes of antibiotics.¹⁸ Due to their better activity, carbapenems have been preferred for use in serious Acinetobacter infections. However, the prevalence of carbapenem-resistant strains has recently been increasing in hospital settings,⁹ especially among critically ill patients.⁴

There is an additional threat of carbapenem-resistant strains becoming multidrug resistant (MDR) or pandrug resistant, which could affect the clinical outcome of serious Acinetobacter infections. In fact, MDR Acinetobacter infection has been found to significantly prolong the duration of ICU stay by 6 days and the median duration of hospitalization by 18 days.²⁶

The objective of this study was to use a large surveillance database of hospitalized patients in the United States to characterize the changes in A. baumannii carbapenem–associated multidrug resistance (Carb XR).

Methods

The Surveillance Network^® (TSN) [Eurofins Medinet] was the data source used for this investigation and analysis. TSN is an electronic database of strain-specific, qualitative, and quantitative antimicrobial susceptibility test data reported by clinical laboratories in the United States that has been used extensively²⁵ to evaluate various trends regarding antimicrobial susceptibility. Methodological details regarding collection and identification have been published elsewhere.²³ In addition to antimicrobial susceptibility profiles, other query parameters that were used individually or in any combination for analysis of antimicrobial susceptibility data included organism identification, the nine regions of the U.S. Census Bureau, age, gender, location (ICU or Ward), and specimen source. Duplicates from the same subject and source were removed from the analysis.

This study focused specifically on A. baumannii and used TSN data collected at the hospital level from 2002 to 2008. The definition proposed by Peleg et al.²¹ of multidrug resistance was applied, and Carb XR was further defined with the benefit of the prior characterization as follows: nonsusceptibility to carbapenems (imipenem or meropenem) and two or more of the following four drug classes: antipseudomonal cephalosporins (ceftazidime or cefepime), β-lactam and β-lactamase inhibitor combinations (piperacillin-tazobactam or ticarcillin-clavulanate), fluoroquinolones (ciprofloxacin or levofloxacin), and aminoglycosides (gentamicin, tobramycin, or amikacin).

Only strains tested simultaneously against at least one antimicrobial agent of the different classes were included in the analysis of prevalence and distribution of multidrug resistance. The overall prevalence of A. baumannii isolated from Ward and ICU specimens was calculated by using all organisms isolated from each patient group as the denominator. The same CLSI⁵ breakpoints were used for the categorization of resistance for all years.

Age was categorized into three groups: <18 years old, 18–64 years old, and 65 or older. Four types of clinical specimen source were analyzed: blood, sputum, bronchial (bronchoalveolar lavage and transtracheal aspirate), and other.

The levels and trends in antimicrobial resistance are presented as yearly prevalence ± a 95% confidence interval (CI). To evaluate these trends, cubic splines were used to model the changes over time. Each antimicrobial has its own intercept and curve shape, given by a three knot cubic spline that hypothesizes an initial rise and subsequent acceleration in the rates of resistance. The cubic spline is used to get the best shape for the curves and interactions between the risk factors and time.

The cubic splines were included in a logistic regression model to examine the association between time and various risk factors such as age group, source of the isolate, location (inpatient or outpatient), and census region. p-Values are not mentioned whenever 95% CIs are present since any overlap in the CI when comparing two proportions is considered not significant. Any measure of association such as the odds ratio (OR) whose CI includes 1 is also considered not significant. STATA 11^® was used to estimate adjusted resistance rates and 95% CIs.

Results

The study consisted of 55,330 U.S. A. baumannii isolates from the years 2002 to 2008. Complete information for multiple resistance including carbapenems was available for 47,342 isolates. Range per year varied from 5,413 (2008) to 8,052 (2004) isolates, and averaged 6,763 isolates per year.

The components of Peleg's multiclass resistance definition can be seen in Fig. 1. There was a steep increase in carbapenem nonsusceptibility over the period of observation, and shallower increases of antipseudomonal cephalosporins, β-lactams, and fluoroquinolones. Resistance to aminoglycosides has stayed relatively stable in the period 2002–2008.

FIG. 1.

Multiclass resistance components.

Carb XR more than doubled in prevalence in 6 years (2002–2008), OR 3.7, 95% CI 3.4–4.3. Moreover, 91.2% of all carbapenem-resistant isolates were also resistant to two or more additional antimicrobial classes with a bias toward pan resistance: 64.9% to four, 23.1% to three, and 3.2% to two antimicrobial classes (all groups include the carbapenem class).

Unadjusted Carb XR increased from 20.6% in 2002 to 49.2% in 2008. A statistical model that takes into account age, gender, source of the isolate, whether the isolate came from a patient in an ICU or in a Ward, and region of the country fits the data very well (Fig. 2). The Carb XR nonsusceptibility rates adjusted for any changes in risk factors over time can also be seen in Fig. 2 and will be used to describe changes over time.

FIG. 2.

Carbapenem-associated multiclass resistance (Carb XR) by year, and unadjusted and adjusted rates.

The multivariate model with cubic splines accurately depicted the acceleration of the resistance increase over time.

In the multivariate model changes over time were significantly different by age group as well as source of the isolate, location (ICU/Ward), and State and Census Region in the United States. ORs and 95% CIs for the risk factors can be seen in Table 1. There were no significant changes by gender, and therefore it is removed from further models.

Table 1.

Carbapenem-Associated Multidrug Resistance and Risk Factors

Factor	Odds ratio	Standard error	95% Confidence interval	p-Value^a
Linear time	1.25	0.020	1.21–1.29	0.0000
Nonlinear time	1.06	0.020	1.02–1.10	0.0030
Age
<18 years	Referent	—	—	—
18–65	2.05	0.092	1.88–2.23	0.0000
65+	2.30	0.107	2.10–2.52	0.0000
Gender	0.99	0.022	0.95–1.03	0.6560
ICU	1.15	0.027	1.10–1.20	0.0000
Source
Blood/CSF	Referent	—	—	—
Bronchial	1.23	0.055	1.12–1.34	0.0000
Other	1.25	0.049	1.16–1.35	0.0000
Sputum	1.56	0.063	1.44–1.69	0.0000
Region
East/West South Central	Referent	—	—	—
New England–Mid Atlantic	3.71	0.154	3.42–4.03	0.0000
Mountain-Pacific	1.80	0.076	1.66–1.96	0.0000
South Atlantic	2.09	0.086	1.92–2.26	0.0000
East/West North Central	2.29	0.097	2.11–2.49	0.0000

p-Values are small due to large sample sizes.

CSF, cerebrospinal fluid.

Cubic splines showed time changes using two coefficients, one for linear time (year to year changes) and one for the acceleration in the prevalence of Carb XR over time.

Subjects <18 years old had significantly (p < 0.001) lower adjusted rates in 2002 (6.9%) than adults (19.1%) and the elderly (65 years +, 21.6%). These large differences between age groups continued until 2006 when rates among those <18 years old started to climb faster (19.9%, 95% CI 18.5%–21.3%) than those of adults (34.8%, 95% CI 34.0%–35.5%) and the elderly (39.2%, 95% CI 38.2–40.2). By 2008 the difference between age groups had diminished even more as rates for all three groups converged. Note the overlap in the 95% CIs in 2008 and the slight divergence of adults and the elderly (Fig. 3).

FIG. 3.

Carb XR rates and age groups.

In 2002, there were small nonsignificant differences by source of the isolate, with prevalences of Carb XR resistance between 19% and 22%. Differences between sources started to diverge in 2005, and by 2008 sputum isolates had a significantly higher rate of Carb XR resistance (57.3%, 95% CI 55.5%–59.8%) compared with 2002 (19.2%, 95% CI 17.9%–20.1%).

Among blood isolates the increase during the period of observation was by 2.2 times (95% CI 1.7–2.8), reaching levels of 36.6% by 2008 (95% CI 33.6%–39.5%). Bronchial isolates increased similarly by 2.4 times (95% CI 1.9–2.9) during the period of observation (Fig. 4).

FIG. 4.

Carb XR rates and source of the isolate.

The changes in Carb XR differ in time when age groups and source were considered together. Age and source were significant effect modifiers over the changes in time. A statistical model that considers these changes finds that among subjects <18 years of age, bronchial and blood isolates increased significantly during the observation period, but sputum isolates grew much faster to levels of 47.8%, (95% CI 38.3%–57.2%). Adults and the elderly had much larger starting values for blood and bronchial isolates and also greater rates of change (Fig. 5).

FIG. 5.

Carb XR rates by age group and source interaction.

Among the elderly, the 2008 Carb XR resistance level of blood isolates increased 1.9 times between 2002 and 2008 (95% CI 1.2–3.1), while in bronchial isolates the OR was 3.2 (95% CI 2.3–4.6). For adults the ORs for the change during the period of observation were 2.2, 95% CI 1.6–2.9 for blood and 2.1, 95% CI 1.6–2.8 for bronchial.

There was a divergence in the rate of change of resistance observed among ICU isolates, with no difference between ICU and Ward in 2002, whereas in 2008 ICU isolates had significantly higher rates (55.2%, 95% CI 53.6%–56.9%) than Ward isolates (45.6%, 95% CI 44.2%–47.0%).

A multivariate model that allows for each census region in the country to have its own curve shape over time indicates that the highest resistance for 2008 is found in the Midwest (55.7%, 95% CI 53.4%–57.9%) and the South Atlantic (52.6%, 95% CI 50.5%–55%) regions of the United States. The Gulf states except Florida started with low resistance rates in 2002 (5.9%, 95% CI 5.23%–6.62%) but increased rapidly after 2005 to reach levels of 42.9%, (95% CI 39.8%–46.2%) in 2008. The Mountain-Pacific started not as low in 2002 as the Gulf states (12.2%, 95% CI 11.3%–13.1%) but experience an immediate and sustained increase to reach levels of 51.5% (95% CI 49.1%–53.8%) in 2008. The New England–Mid Atlantic regions had a very high resistance in 2002 (32.5%, 95% CI 31.1%–33.9%) but increased only slightly in 6 years (39.4%, 95% CI 36.9%–41.8%). Comparisons between regions can be seen in Table 1.

Discussion

This study has provided evidence that the number of carbapenem-resistant isolates in a U.S. surveillance study has increased from one out of five to about half of all A. baumannii isolates in the span of 6 years. These findings are in line with prior observations from a case–control study,¹⁵ where imipenem-resistant isolates were significantly more often resistant to eight antimicrobial agents than the imipenem-susceptible group.

An earlier set of isolates from the same database was previously published,⁹ but only imipenem resistance was discussed. This analysis has the advantage of more years of observation, the consideration of multiple resistance, a proper multivariate model, and more advanced statistical treatment of the changes over time.

The rise of antibiotic resistance in clinical isolates has been documented on a global scale as far back as 1999.¹⁰ Zarrilli et al.²⁹ have reviewed surveillance studies worldwide and found a gradual increase in carpbapenem-resistant A. baumannii over the past 10 years in Europe, North America, and Latin America. In the United States, the rise in carbapenem resistance had been recognized by Gaynes¹² and later by ongoing surveillance studies from hospitals in New York City¹⁶ and two medical centers in Pennsylvania.¹⁷ Of note is the tendency of A. baumannii infections happening during the warmer summer months.²⁰

This concomitant resistance against multiple antimicrobial agents has become a great concern for the treatment of serious Acinetobacter infections. A recent study¹⁹ has shown that patients with MDR Acinetobacter bacteremia had a higher mortality rate, a longer hospital stay, and greater medical costs compared with non-MDR Acinetobacter bacteremia.

We have found an acceleration in the rate of acquisition of Carb XR among ICU isolates when compared to general Ward, which may be due to the ability of A. baumannii to rapidly respond to antimicrobial pressure in an environment of high antimicrobial use, such as the spread of several bla_OXA genes.²² Moreover, ICU-related risk factors such as the severity of the underlying disorder, pneumonia as the source of bacteraemia, septic shock, disseminated intravascular coagulation, acute respiratory distress syndrome, mechanical ventilation, and immunosuppressive status have been found to be independently associated with an unfavorable outcome in Acinetobacter bacteremia.^1,3,24,27

A systematic review⁷ of several studies found that the attributable in-hospital mortality due to A. baumannii infection was 7.8%–23% and that the ICU mortality was 10%–43%. These data suggest that infections by A. baumannii may be associated with considerable attributable mortality and increased length of ICU stay. Other studies^13,26 also suggest an increased risk of mortality² for patients with MDR A. baumannii colonization or infection. This finding may indicate that the acquisition and expression of multiple antibiotic-resistance factors does not compromise the pathogenicity of this organism.

The multivariate analysis allowing for curves of different shapes from this study has demonstrated that although the elderly have the highest rates, Carb XR has been catching up among the young (<18 years old). There was a threefold difference favoring those older than 65 in 2002, but by 2008 that difference has disappeared, with rates equally high for all age groups.

Faster resistance growth has also been demonstrated for sputum isolates, which may signal faster increases among colonizing A. baumannii than bacteremias (blood) or pneumonias (sterile lower respiratory sources). The examination of the interaction between age and source provides a sharper but more complicated picture, with sputum isolates among young patients growing very fast, whereas blood and bronchial isolates lagged, while nonetheless attaining levels similar to other categories in 2008. The fastest growing resistance seems to be among sputum and bronchial isolates of subjects older than 65 years.

There was roughly a 15% difference in resistance rates for 2008 when comparing different areas of the country between the highest (Midwest) and the lowest (New England). Areas that started with a particularly low prevalence of resistance such as the Mountain-Pacific region have experienced notable growth, with no region in the country having <39% resistance prevalence in 2008. These differences may be driven by clones among certain hospitals.

Putting all those elements together, the epidemic of Carb XR isolates has been centered around Midwest sputum and bronchial isolates (presumably pneumonias) in the elderly at the ICU, spreading fast to other age groups. That the dramatic increase in Carb XR is spread across ages, sources, locations, and regions adds evidence that A. baumannii has a remarkable capacity to acquire and disseminate resistance to many classes of antibacterials and that it does not appear to have tremendous fitness cost associated with this resistance.

Limitations of this study include the fact that isolates come from passive surveillance that is not population based. The isolates from different census regions may not be completely representative as they are subject to selection bias. Most isolates represent subjects with a variety of infections.

This large surveillance study has indicated a remarkable increase in the prevalence of Carb XR in the period 2002–2008. Rates among subjects <18 years old have increased faster than those of the elderly, and the same effect has been observed among sputum isolates when compared to blood isolates, and ICU isolates as compared to Ward isolates.

Footnotes

Acknowledgment

This work was supported by GlaxoSmithKline.

Disclosure Statement

R.M. Mera, H.A. Madsen, and L.A. Miller are GlaxoSmithKline employees. D.F. Sahm is an employee of Eurofins Medinet. No competing financial interests exist.

References

Chen

H.P.

, Chen

T.L.

, Lai

C.H.

, Fung

C.P.

, Wong

W.W.

, Yu

K.W.

, Liu

C.Y.

2005. Predictors of mortality in Acinetobacter baumannii bacteremia. J. Microbiol. Immunol. Infect., 38:127–136.

Choi

J.Y.

, Park

Y.S.

, Kim

C.O.

, Yoon

H.J.

, Shin

S.Y.

, Kim

Y.A.

, Song

Y.G.

, Yong

, Lee

, Kim

J.M.

2005. Mortality risk factors of Acinetobacter baumannii bacteraemia. Intern. Med. J., 35:599–603.

Cisneros

J.M.

, Reyes

M.J.

, Pachón

, Becerril

, Caballero

F.J.

, García-Garmendía

J.L.

, Ortiz

, Cobacho

A.R.

1996. Bacteremia due to Acinetobacter baumannii: epidemiology, clinical findings, and prognostic features. Clin. Infect. Dis., 22:1026–1032.

Cisneros

J.M.

, Rodriguez-Baño

2002. Nosocomial bacteremia due to Acinetobacter baumannii: epidemiology, clinical features and treatment. Clin. Microbiol. Infect., 8:687–93.

[CLSI] Clinical, Laboratory Standards Institute. 2008. Performance standards for antimicrobial susceptibility testing; eighteenth informational supplement. CLSI document M100-S18. Wayne, PA.

Crowe

, Ispahani

, Humphreys

, Kelley

, Winter

1998. Bacteraemia in the adult intensive care unit of a teaching hospital in Nottingham, UK, 1985–96. Eur. J. Clin. Microbiol. Infect. Dis., 17:377–384.

Falagas

M.E.

, Kopterides

, Siempos

I.I.

2006. Attributable mortality of Acinetobacter baumannii infection among critically ill patients. Clin. Infect. Dis., 43:389.

Ferrara

A.M.

2006. Potentially multidrug-resistant non-fermentative gram-negative pathogens causing nosocomial pneumonia. Int. J. Antimicrob. Agents., 27:183–195.

Hoffman

M.S.

, Eber

M.R.

, Laxminarayan

2010. Increasing resistance of acinetobacter species to imipenem in United States Hospitals, 1999–2006. Infect Control Hosp. Epidemiol., 31:196–197.

10.

Gales

A.C.

, Jones

R.N.

, Forward

K.R.

, Liñares

, Sader

H.S.

, Verhoef

2001. Emerging importance of multidrug-resistant Acinetobacter species and Stenotrophomonas maltophilia as pathogens in seriously ill patients: geographic patterns, epidemiological features, and trends in the SENTRY Antimicrobial Surveillance Program (1997–1999) Clin. Infect. Dis., 32:S104–S113.

11.

Garcia-Garmendia

J.L.

, Ortiz-Leyba

, Garnacho-Montero

, Jiménez-Jiménez

F.J.

, Pérez-Paredes

, Barrero-Almodóvar

A.E.

, Gili-Miner

2001. Risk factors for Acinetobacter baumannii nosocomial bacteraemia in critically ill patients: a cohort study. Clin. Infect. Dis., 33:939–946.

12.

Gaynes

, Edwards

J.R.

the National Nosocomial Infections Surveillance System. 2005. Overview of nosocomial infections caused by gram-negative bacilli. Clin. Infect. Dis., 41:848–854.

13.

Gkrania-Klotsas

, Hershow

R.C.

2006. Colonization or infection with multidrug resistant Acinetobacter baumannii may be an independent risk factor for increased mortality. Clin. Infect. Dis., 43:1224–1225.

14.

Joly-Guillou

M.L.

Clinical impact and pathogenicity of Acinetobacter. 2005. Clin. Microbiol. Infect., 11:868–873.

15.

Kwon

K.T.

, Oh

W.S.

, Song

J.H.

, Chang

H.H.

, Jung

S.I.

, Kim

S.W.

, Ryu

S.Y.

, Heo

S.T.

, Jung

D.S.

, Rhee

J.Y.

, Shin

S.Y.

, Ko

K.S.

, Peck

K.R.

, Lee

N.Y.

2007. Impact of imipenem resistance on mortality in patients with Acinetobacter bacteraemia. J. Antimicrob. Chemother., 59:525–530.

16.

Landman

, Bratu

, Kochar

, Panwar

, Trehan

, Doymaz

, Quale

2007. Evolution of antimicrobial resistance among Pseudomonas aeruginosa, Acinetobacter baumannii and Klebsiella penumoniae in Brooklyn, NY. J. Antimicrob. Chemother., 60:78–82.

17.

Lautenbach

, Synnestvedt

, Weiner

M.G.

, Bilker

W.B.

, Vo

, Schein

, Kim

Epidemiology, impact of imipenem resistance in Acinetobacter baumannii. 2009. Infect. Control Hosp. Epidemiol., 30:1186–1192.

18.

Lee

C.M.

, Lim

H.K.

, Liu

C.P.

, Tseng

H.K.

2005. Treatment of pan-drug resistant Acinetobacter baumannii. Scand. J. Infect. Dis., 37:195–199.

19.

Lee

, Lee

, Ko

2005. Impact of multidrug antimicrobial resistance on the health outcomes and economic burden in cases with Acinetobacter baumannii bacteremia. Abstracts of the 45th Interscience Conference on Antimicrobial Agents and Chemotherapy, Washington, DC, 2005; 336. American Society for Microbiology: Washington, DCAbstract K1303.

20.

McDonald

L.C.

, Banerjee

S.N.

, Jarvis

W.R.

1999. Seasonal variation of Acinetobacter infections: 1987–1996. Clin. Infect. Dis., 29:1133–1137.

21.

Peleg

A.Y.

, Seifert

, Paterson

D.L.

2008. Acinetobacter baumannii: emergence of a successful pathogen. Clin. Microbiol. Rev., 21:538–582.

22.

Poirel

, Naas

, Nordmann

2010. Diversity, epidemiology, and genetics of class D β-lactamases. Antimicrob. Agents Chemother., 54:24–38.

23.

Sahm

D.F.

, Marsilio

M.K.

, Piazza

1999. Antimicrobial resistance in key bloodstream bacterial isolates: electronic surveillance with The Surveillance Network Database-USA. Clin. Infect. Dis., 29:259–263.

24.

Seifert

, Strate

, Pulverer

1995. Nosocomial bacteremia due to Acinetobacter baumannii. Clinical features, epidemiology, and predictors of mortality. Medicine, 74:340–349.

25.

Styers

, Sheehan

D.J.

, Hogan

, Sahm

D.F.

2006. Laboratory-based surveillance of current antimicrobial resistance patterns and trends among Staphylococcus aureus: 2005 status in the United States. Ann. Clin. Microbiol. Antimicrob., 5:2.

26.

Sunenshine

R.H.

, Wright

M.O.

, Maragakis

L.L.

, Harris

A.D.

, Song

, Hebden

, Cosgrove

S.E.

, Anderson

, Carnell

, Jernigan

D.B.

, Kleinbaum

D.G.

, Perl

T.M.

, Standiford

H.C.

, Srinivasan

2007. Multidrug-resistant Acinetobacter infection mortality rate and length of hospitalization. Emerg. Infect. Dis., 13:97–103.

27.

Tilley

P.A.

, Roberts

F.J.

1994. Bacteremia with Acinetobacter species: risk factors and prognosis in different clinical settings. Clin. Infect. Dis., 18:896–900.

28.

Wisplinghoff

, Bischoff

, Tallent

S.M.

, Seifert

, Wenzel

R.P.

, Edmond

M.B.

2004. Nosocomial bloodstream infections in US hospitals: analysis of 24 179 cases from a prospective nationwide surveillance study. Clin. Infect. Dis., 39:309–317.

29.

Zarrilli

, Giannouli

, Tomasone

, Triassi

, Tsakris

2009. Carbapenem resistance in Acinetobacter baumannii: the molecular epidemic features of an emerging problem in health care facilities. J. Infect. Dev. Ctries., 3:335–341.