Abstract
We report on Phase 1 efforts of the Hierarchical Taxonomy of Psychopathology (HiTOP) measurement subgroup tasked with developing provisional scales for the somatoform spectrum and eating disorders. In Study 1, items were written to assess five somatoform spectrum constructs (bodily distress symptoms, conversion symptoms, health anxiety, disease conviction, and somatic preoccupation). Scale development analyses were conducted on 550 university students. The conversion symptom items were too infrequently endorsed and were set aside for Phase 2. Analyses of the other items yielded four scales corresponding closely to their hypothesized structure. In Study 2, we delineated 15 specific feeding and eating disorder constructs. A sample of 400 university students were administered candidate items and several eating disorder questionnaires for criterion validity. Analyses yielded six scales capturing previously described constructs, tapping content related to body image and weight concerns, restricting and purging, cognitive restraint, binging, excessive exercise, and muscle building. Two scales representing additional constructs deemed to be of high clinical import—negative attitude towards obesity and avoidant/restrictive food intake disorder—were retained for Phase 2, for a total of eight scales. Overall, we concluded that Phase 1 had been successful at generating a comprehensive set of provisional scales for inclusion in Phase 2.
The Hierarchical Taxonomy of Psychopathology (HiTOP) is an international consortium that is using data from various factor-analytic studies to develop a quantitative system of nosology (Kotov et al., 2017). Assessment is an important part of this effort and a new comprehensive self-report inventory is the first step in this process. Although self-report scales have been developed to measure almost all forms of mental disorder, and these measures could theoretically be combined to create a comprehensive assessment battery, they use a diverse range of instructions, time frames, and response formats, which would complicate both structural analyses and clinical interpretation. Furthermore, in the case of somatic symptom disorders and eating disorders (EDs), one of two foci of the current article, the HiTOP instrument would also constitute the most comprehensive and yet efficient method to assess the full range of theoretical constructs. There are no current measures available that would cover all the constructs developed here in a comprehensive and systematic manner. For EDs, the second focus of this article, we considered using the Eating Pathology Symptom Inventory (EPSI; Forbush et al., 2013) given that it was originally designed to comprehensively test a full range of ED symptoms in a variety of populations. However, since the publication of the Diagnostic and Statistical Manual of Mental Disorders–Fifth edition (DSM-5), feeding disorders were introduced and integrated with the EDs (e.g., avoidant/restrictive food intake disorder [ARFID]) and certain clinical content of importance was insufficiently captured by the EPSI (e.g., weight phobia, body checking, and orthorexia).
As articulated by Simms et al. (this issue), in light of the magnitude and complexity of creating a comprehensive assessment of psychopathology, the HiTOP Measurement Workgroup decided to construct this measure in multiple phases. In Phase 1, five different subgroups were formed to develop preliminary sets of scales falling within the major spectra identified in structural research (Kotov et al., 2017); specifically, these five subgroups were charged with the assessment of (a) internalizing, (b) disinhibited and antagonistic externalizing, (c) thought disorder, (d) detachment, and (e) somatoform and eating-related psychopathology. Each group was given substantial autonomy in creating preliminary Phase 1 scales, although efforts were made to minimize the overlap between them. In Phase 2 of the project, the preliminary scales from these five groups will be subjected to joint analyses to begin the process of creating a comprehensive measure.
This article details the efforts of the HiTOP Somatoform Spectrum and Eating Disorders measurement subgroup. This subgroup was formed because HiTOP considers somatoform psychopathology as separate from other forms of psychopathology (Kotov et al., 2017), including internalizing (cf. Krueger et al., 2003). Although initial evidence suggested that somatoform disorders were part of the broader internalizing spectrum (Krueger et al., 2003; Simms et al., 2012), subsequent evidence has emerged using different measurement modalities indicating that when a sufficient number of indicators are present, internalizing and somatoform represent separate spectra (e.g., Forbes et al., 2017; Kotov et al., 2011; Marek et al., 2020; Sellbom, 2017; Watson et al., in press). However, these sets of findings are not contradictory. Indeed, Forbes et al. (2017) showed convincingly that internalizing and somatoform form the same spectrum at very broad levels of abstraction in the psychopathology hierarchy, but at more specific (lower) levels of the hierarchy (viz., moving from three to four spectra), somatoform separates from internalizing. As such, the HiTOP consortium’s measurement workgroup formed a separate committee (subgroup) for the assessment of somatoform psychopathology.
Some readers might wonder why EDs were grouped with somatoform disorders, particularly when evidence has indicated that (1) EDs load with other internalizing disorders (Forbush et al., 2010; Forbush & Watson, 2013; Kotov et al., 2017; Mitchell et al., 2014; Watson et al., in press) and (2) EDs have similar lifetime prevalence rates in community-based studies as other common internalizing disorders, such as depression (Allen et al., 2013; Stice et al., 2013). For purely pragmatic reasons, however, eating pathology was assigned to the Somatoform Group due to its members having greater expertise in this area. EDs are also associated with an assessment literature that is distinct from internalizing psychopathology in general, and therefore, the allocation of construct development and measurement to our subgroup was straightforward. In Phase 2 of the HiTOP assessment project, the somatoform and eating pathology items from our group will be combined with those written by the other groups (including internalizing), which will allow the consortium to determine their proper placement within the hierarchical structure.
The main objective of this project was to report on the development of preliminary scales for the somatoform spectrum and for EDs, which resulted from the HiTOP measurement workgroup’s Phase 1 (see Simms et al., this issue). Because somatoform and ED scale development occurred separately, we report two separate studies. In each study, we describe how we arrived at our measurement model of target constructs, as well as report the main findings from preliminary scale development.
Study 1
Up to two thirds of somatic symptoms presented in primary medical care defy a clear biomedical explanation (e.g., Steinbrecher et al., 2011) and prevalence rates of medically unexplained symptoms appear equally high in more specialized secondary medical settings (e.g., neurology and gastroenterology; Nimnuan et al., 2001). For chronic and disabling conditions marked by “medially unexplained somatic symptoms,” DSM-III first coined the descriptive term “somatoform disorders” (i.e., “indicating a somatic condition, albeit biomedical disorders do not explain the symptoms,” Rief & Martin, 2014, p. 341). With estimated prevalence rates between 10% and 20% in primary care patients in the United States and European countries (e.g., Kroenke et al., 1997; Steinbrecher et al., 2011), somatoform disorders (according to DSM-IV) are among the most frequent and most expensive mental health conditions. Despite its epidemiological frequency and clinical relevance, the term “somatoform” represents one of the most controversial diagnostic labels in the realm of medicine and psychology alike (Witthöft et al., 2018). Largely due to the associated stigma and the problematic Cartesian mind–body dualism innate in its conceptualization, somatoform disorders were fundamentally revised and relabeled to “somatic symptom and related disorders” in the DSM-5.
Historically, the concept “somatoform” traces back to the concept of “hysteria” (a concept rooted in ancient Egypt and early Greek medical tradition) with the first systematic scientific attempts to describe and explain this phenomenon taking place in 19th century neurology and psychiatry (Kapfhammer, 2001). Originating in neurology and particularly in the work of Charcot, Janet, Breuer, and Freud, the two concepts of dissociation (i.e., the disintegration of certain parts of sensory and motor information processing due to disturbing traumatic events) and conversion (i.e., a defense mechanisms marked by the transformation/suppression of psychological conflicts into somatic symptoms) represented the dominant explanations for “somatoform” symptoms (Brown, 2004) and can be considered as precursors of the later concept of “somatization” by Lipowski (1968; i.e., “the tendency to experience or express psychological distress as the symptoms of physical illness,” Brown, 2004, p. 798).
Although these explanatory accounts remain prominent in the psychodynamic medical and psychotherapeutic tradition, cognitive, and cognitive–behavioral models of somatoform disorders rooted in learning theory, the cognitive revolution in psychology as well as the biopsychosocial model in (behavioral) medicine were developed in the late 20th and early 21st centuries (Brown, 2004; Kirmayer & Taillefer, 1997; Witthöft & Hiller, 2010). According to these accounts, medically unexplained (i.e., “somatoform”) symptoms are the results of cognitive (e.g., attention allocation, interpretation, and attribution) as well as dysfunctional behavioral processes (e.g., withdrawing from physical activity or excessively searching for reassurance). The predictive processing model (Van den Bergh et al., 2017) represents the most recent theoretical advancement in the area of somatoform disorders and conceptualizes the experience of chronic somatic symptoms as an active constructive process based on prior expectations and current (afferent) sensory input.
The somatoform spectrum thus considers the psychological preoccupation, anxiety, and/or experience of chronic somatic symptoms of unknown etiology that causes significant distress. This literature on somatoform (now, somatic symptom) disorders was extensively reviewed with the aim of identifying underlying construct dimensions. Our review was meant to be overinclusive rather than underinclusive to ensure that we sampled the constructs of interest as comprehensively as possible. We reviewed both the conceptual literature, including the current primarily categorical nosologies of the DSM-5 (American Psychiatric Association, 2013) and the International Statistical Classification of Diseases and Related Health Problems (ICD-10; World Health Organization, 2016), as well as the structural/psychometric literature (e.g., Longley et al., 2005; Witthöft et al., 2018), that has identified various dimensions related to somatoform symptomatology. Traditionally, somatization disorder, pain disorder, hypochondriasis, and conversion disorder are the primary diagnostic categories within the very heterogeneous (and arbitrarily defined) area of somatoform disorders (Witthöft & Hiller, 2010). Although DSM-5 introduced different terms and broader diagnostic categories, the central constructs of chronic somatic symptom distress (as the hallmark of the former concept of somatization disorder), health/illness anxiety (as the core feature of hypochondriasis), and functional neurological/pseudoneurological symptoms (as the defining feature of conversion disorder) were largely maintained in DSM-5 (Rief & Martin, 2014). Research has indeed identified support for a “bodily distress syndrome” that incorporates all functional somatic symptoms separate from “health/illness anxiety” (e.g., Budtz-Lilly et al., 2015; Fink et al., 2004; Fink & Schröder, 2010; Petersen et al., 2020; see also Fink, 2017, for a review). However, in light of historical and contemporary theoretical considerations, as well as the current DSM-5 bifurcation, we separated functional neurological symptoms (i.e., conversion) from bodily distress in the Phase 1 construct development.
Additionally, psychometric studies provide evidence for distinct subfactors within hypochondriasis, such that multidimensional structures comprising different subfactors (depending on the respective scale) have been proposed. Accordingly, disease phobia/illness worries (i.e., health anxiety) and disease conviction have been proposed as two distinguishable constructs (Fergus & Valentiner, 2010; Longley et al., 2005). Furthermore, a behavioral (e.g., reassurance seeking) as well as a perceptual component (focus on bodily sensations) of hypochondriasis have been identified (Longley et al., 2005). The perceptual component particularly is closely related to the concept of somatosensory amplification (i.e., “the tendency to experience bodily sensations as intense, noxious, and disturbing”; Barsky, 1992, p. 28) as an explanatory construct for distress and symptom chronicity in a wide range of mental and physical conditions (Barsky, 1992).
As a result of these categorical nosological considerations as well as the existing psychometric structural evidence, the following five constructs were identified by the workgroup as the core components of the somatoform spectrum: (1) bodily distress symptoms (i.e., the experience of a range of persistent symptoms such as headache, fatigue, etc.); (2) conversion symptoms (i.e., symptoms of sensory, perceptual, or motor abnormalities without a clear medical explanation such as loss of vision, hearing, paralysis, etc.); (3) health anxiety (i.e., anxiety about one’s health status or a potential fatal illness); (4) disease conviction (i.e., the persistent belief that one has a serious illness despite contrary medical information); and (5) somatic preoccupation (i.e., a strong focus on bodily sensations and physical changes). 1 In a second step, international experts (psychiatrists and clinical psychologists) in the field of somatoform disorders were consulted and invited to comment on the identified set of constructs and to refine the respective definitions (see online supplemental materials for the final detailed definitions of the five constructs). Following the typical practice in somatic symptom assessment (e.g., Longley et al., 2005), these constructs were conceptualized as unipolar in nature, such that all items were written in the keyed direction.
The current project aimed to develop preliminary scales to assess various aspects of the somatoform spectrum by writing items to assess each of these five constructs. We used the model set forth by the HiTOP measurement workgroup and, therefore, we did not have any a priori expectations regarding the ultimate structure of the preliminary somatoform spectrum scales that would emerge from our item pool.
Method
Participants were 550 university students enrolled in first- or second-year psychology courses at a large public university in New Zealand. They completed the study measures (including the 70 somatoform candidate items) in groups of up to 15, monitored by a trained research assistant, and received course credit for their participation. The sample consisted of 115 men (20.9%), 433 (78.7%) women, 1 person who identified as transgender, and 1 person who did not report gender. They ranged in age from 17 to 51 years (M = 19.76; SD = 2.75). The majority of participants reported their ethnicity as being New Zealand European (73%), with 17% being Other European (including Australian), 10% New Zealand Māori, 7% Chinese, 3% Pacific Islander, and 3% Indian, and 13% selecting “other.” These values add up to greater than 100% because some selected more than one option.
In addition to the 70 Phase 1 candidate items (see next), these individuals also completed the Minnesota Multiphasic Personality Inventory–2–Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2008/2011). The MMPI-2-RF is a 338-item self-report inventory that includes scales that measure the somatoform spectrum (see Sellbom, 2019). More specifically, we included Restructured Clinical Scale 1 (Somatic Complaints), which measures general somatoform pathology, and the Specific Problems scales Malaise (MLS), Gastrointestinal Complaints (GIC), Head Pain Complaints (HPC), Neurological Complaints (NUC), and Cognitive Complaints (COG), as external criteria for validation purposes. To evaluate discriminant validity, we also included three scales that measure other psychopathology spectra, specifically, internalizing (Emotional/Internalizing Dysfunction [EID]), thought disorder (Thought Dysfunction [THD]), and externalizing (Behavioral/Externalizing Dysfunction [BXD]).
Scale Development and Results
Candidate Item Generation
Using the five construct descriptions as a guide, most authors (M.S., K.F., K.M., D.W., and M.W.) were asked to independently contribute at least five candidate items for each construct. Each author relied on the construct definition and content coverage in generating these items. The first author then culled any items that were identical or near-identical in wording. This produced an initial list of 37 Bodily Distress Symptoms items, 39 Conversion Symptoms items, 31 Health Anxiety items, 41 Disease Conviction items, and 24 Somatic Preoccupation items. Next, each author independently rated the items with respect to the quality of the item (1-5 scale; 5 being excellent) for content and style and was also asked to select their top 15 items. Items that received at least a 3.6 average rating and were selected by multiple raters as their top 15 were included as candidate items, with a maximum of 15 items per construct. This process resulted in 15 items for three constructs (Bodily Distress Symptoms, Health Anxiety, and Disease Conviction), 14 for Conversion Symptoms, and 11 for Somatic Preoccupation. The final set of candidate items are included in the online supplemental materials.
These 70 items were administered to the university sample. We used the standard instructions that were developed for the overall HiTOP measure (see Simms et al., this issue). Specifically, participants were asked whether there had been “significant times during the last 12 months during which the following statements applied to you”; they responded using a 4-point scale (not at all, a little, moderately, and a lot). The Phase 1 “flexible” data analysis protocol was executed, as articulated for the overall measurement development process (see Simms et al., this issue, for greater detail).
Initial Item Screening
Prior to the main analyses, we first examined item response frequencies and inter-item correlations for each individual construct. Items were removed if response frequencies revealed little or no variability (i.e., extreme range restriction). Items were considered for removal if correlations were greater than .70 and were too similar in item content with other items. For each such item pair, the item deemed to have most redundancy with other items was tentatively removed unless there was a conceptual reason otherwise.
In terms of item frequency, it was determined that all of the Conversion Symptoms items were too restricted in range. All but two items had >92% of participants responding “not at all,” with 84% and 88% for the other two items. We deemed that there was insufficient variability to examine this scale with much confidence in this sample; all conversion symptom items were, therefore, set aside and retained for Phase 2 of the HiTOP measurement project. For the other four constructs, two Health Anxiety items were removed (98% and 99% responded “not at all”). The Disease Conviction items had low item response frequency (for most items, >80% selected “not at all”), but this problem was not deemed as severe as for the Conversion Symptoms; these items were therefore retained. Three items (one each from Health Anxiety, Disease Conviction, and Somatic Preoccupation) were removed due to excessive item correlations with other items.
Phase 1 Analyses
Simms et al. (this issue) provides an overview of the overall analytical framework, which we followed. First, we scored the four remaining constructs by averaging the retained items to determine whether a higher order structure would emerge. Unsurprisingly, as there were only four indicators, an exploratory factor analysis with robust maximum likelihood estimation in Mplus 8 yielded a one-factor solution, with acceptable model fit (comparative fit index [CFI] = .98, Tucker–Lewis index[TLI] = .94, root mean square error of approximation [RMSEA] = .07, standardized root mean square residual [SRMR] = .02). Loadings ranged from .57 (Somatic Preoccupation) to .74 (Health Anxiety).
Next, we executed an item-level factor analysis using all items from the four constructs, as there was only one higher order factor. Because the item responses constituted ordered categorical data, we used a diagonally weighted least squares estimator (WLSMV in Mplus 8). A parallel analysis based on a polychoric correlation matrix using Revelle’s psych package in R (fa.parallel) yielded a four-factor solution, which was also associated with good model fit (CFI = .96, TLI = .95, RMSEA = .04, SRMR = .05). Following instructions from the overall analytic plan (Simms et al., this issue), we retained only items with loadings > |.40| and at least |.20| difference from the largest cross-loading. The four-factor structure clearly and almost perfectly represented the four original constructs, with acceptable factor loadings: Bodily Distress Symptoms (loadings: .57-.80), Health Anxiety (loadings: .47-.93), Disease Conviction (loadings: .39-.96), and Somatic Preoccupation (loadings: .35-.92). One Disease Conviction item was removed as its loading was higher on the Health Anxiety factor, albeit not sufficiently distinctive to be considered for that factor. Two Somatic Preoccupation items were removed because loadings were < |.40|. Supplementary Table S1 (available online) provides the complete set of loadings across the four factors.
Next, we assessed individual item-based factors. This step involved CFAs of the items of each individual factor to determine items that potentially detracted from unidimensionality (estimated via McDonald’s ω coefficient). Then we estimated Samejima’s graded response IRT (item response theory) models in STATA 15 to reduce each item set into 8 to 10 item scales. Per instructions from the overall analytic plan (Simms et al., this issue), we (a) consulted McDonald’s ω and IRT-based item information curves to identify items to remove, (b) iteratively dropped items and recalculated omegas and item information curves, and (c) stopped iterating when either omega fell below .85 or 10 items were identified, whichever came first.
Bodily distress symptoms
The 15 item one-factor model was associated with adequate model fit (CFI = .92, TLI = .91, RMSEA = .10) and McDonald’s ω was .92. Item loadings ranged from .48 to .79. We iteratively removed five items due to lowest discrimination parameter and/or the item clearly did not contribute incremental information at any level of latent construct per the item information functions. Item content was also considered to ensure that construct breadth was not sacrificed. Test information functions (TIFs) for the 15-item and 10-item versions of the scale are available in the online supplement (see Figures S1.1 and S1.2). The final 10-item scale, which is included in Supplementary Table S2, had final factor loadings that ranged from .62 to .80, with a final McDonald’s ω of .91.
Health anxiety
The 12-item one-factor model was associated with good model fit (CFI= .98, TLI = .97, RMSEA = .07) and ω = .95. Factor loadings ranged from .45 to .94. Two items were removed per the aforementioned procedures, which resulted in a final 10-item scale. TIFs are available in the online supplement (Figures S2.1 and S2.2) and the final 10-item scale, which is included in Supplementary Table S3 (available online), had factor loadings that ranged from .63 to .88, with a McDonald’s ω of .95.
Disease conviction
The 14-item one-factor model was associated with good model fit (CFI = .98, TLI = .97, RMSEA = .07) and ω = .97. Factor loadings ranged from .67 to .94. Four items were removed per the aforementioned procedures, which resulted in a final 10-item scale. TIFs are available in the online supplement (Figures S3.1 and S3.2) and the final 10-item scale, which is included in Supplementary Table S4 (available online), had factor loadings that ranged from .79 to .95, with a McDonald’s ω of .97.
Somatic preoccupation
Because this scale only had eight remaining items, the model was associated with good relative fit (CFI = .97, TLI = .96), albeit poor absolute fit (RMSEA = .15), good IRT parameters, and the McDonald’s ω was .93, we did not make any further modifications. The final eight-item scale is included in Supplementary Table S5 (available online).
The intercorrelations among the final four scales (see Table 1) were weak to moderate in magnitude. None of these correlations reached a large magnitude (i.e., r ≥ .50), indicating that they measured relatively distinct constructs.
Correlations Between HiTOP Somatoform Scales and External MMPI-2-RF Criteria (n=550).
Note. r = |.14| is statistically significant (p < .001). Correlations appearing in bold typeface are those conceptually most relevant to each HiTOP construct. HiTOP = Hierarchical Taxonomy of Psychopathology; MMPI-2-RF = Minnesota Multiphasic Personality Inventory–2–Restructured Form; BDS = Bodily Distress Symptoms; HA = Health Anxiety; DC = Disease Conviction; SP = Somatic Preoccupation; CS = Conversion Symptoms (all candidate items).
Criterion Validity
Table 1 shows the correlations between the provisional Phase 1 somatoform scales and the MMPI-2-RF somatic/cognitive scales. For this analysis, we also scored an aggregate of the 14 Conversion Symptoms items to evaluate criterion validity (while recognizing the likely impact of severe range restriction). We expected that all scales would correlate meaningfully (set at a medium effect size estimate [r ≥ .30] to account for shared method variance) with RC1 as an index of broader somatization; indeed, all but Somatic Preoccupation reached a medium effect size, with Bodily Distress Symptoms having a large correlation with this MMPI-2-RF scale. Moreover, GIC, HPC, and COG are all indicators of Bodily Distress Symptoms conceptually and, as expected, the Bodily Distress Symptoms scale correlated most strongly with these MMPI-2-RF scales. Interestingly, and despite its restricted range, the Conversion Symptoms aggregate evidenced the largest correlation with NUC, which is a measure of functional neurological symptoms. It is noteworthy that Somatic Preoccupation did not correlate meaningfully with any of the MMPI-2-RF scales, which is evidence for discriminant validity as none of the latter scales specifically measure somatic preoccupation as defined in the HiTOP measurement framework. Finally, associations with MMPI-2-RF scales that measure other psychopathology spectra (i.e., EID, THD, and BXD) were generally below our threshold for meaningfulness in nearly every instance. The main exception was Health Anxiety, which correlated with EID to a similar degree as RC1.
Brief Discussion
Although the conversion symptom items were too restricted in range to be factor analyzed, the remaining four constructs emerged as cohesive. Our findings generally support dimensional somatoform constructs that are rooted in the proposed bodily distress syndrome (e.g., Fink, 2017; Petersen et al., 2020), health anxiety, and two separate aspects of the old hypochondriasis literature, disease conviction and somatic preoccupation. Our preliminary criterion and discriminant validity results were also quite consistent with theoretical expectations, though it is noteworthy that the Somatic Preoccupation scale did not correlate meaningfully with a broad somatoform index and was the most weakly associated with the other HiTOP somatoform scales. Future analysis in Phase 2 of this project will need to determine if the conversion symptoms fall under the broader bodily distress syndrome (e.g., Petersen et al., 2020) or represent a separate construct. It will also be important to determine if health anxiety remains under the somatoform spectrum or whether it will be better situated under the internalizing spectrum (as indicated from its correlation with a scale [EID] measuring this spectrum; see Table 1), or perhaps is interstitial.
Study 2
The second study examined the development of the preliminary ED scales. Despite the fact that EDs include several categorical diagnoses, there is substantial heterogeneity within ED diagnoses (Forbush et al., 2018) and high rates of “migration” among purportedly different EDs over time, which indicates that the current categorical ED constructs have poor temporal stability (Forbush et al., 2018; Schaumberg et al., 2019; Tozzi et al., 2005). In addition, feeding disorders, traditionally thought to be limited to childhood and, therefore, distinct from EDs, are now understood to exist across the lifespan, particularly ARFID. Currently, ARFID and other feeding disorders are grouped with EDs in the DSM-5, although few studies have tested whether this conceptually based grouping is empirically supported.
We began by identifying specific ED constructs as described in the DSM-5 Eating and Feeding Disorders. Next, we extensively reviewed the literature on eating and feeding disorders and consulted expert clinicians, given the sparsity of validated assessment tools for feeding disorder symptoms that were available at the time we undertook this project. Expert clinicians were ED treatment providers at a Midwestern academic center. Experts included psychologists, social workers, dietitians, and medical providers. Creation of items for the ED scales began by reviewing the dimensions contained within the EPSI (Forbush et al., 2013), given that the EPSI was created in a similar fashion as the overall HiTOP assessment project, and demonstrated evidence for strong psychometric properties (Forbush et al., 2013; Forbush et al., 2014). For example, the original item pool for the EPSI included over 20 homogeneous item composites (potential scales) that were developed to comprehensively assess the full content domain of EDs based on careful reviews of the theoretical and empirical literature. To expand the construct domains beyond what is included in the EPSI, we first had several team meetings to identify potential narrow-band constructs based on the DSM-5 and relevant theory. Next, a group of expert clinicians (see above) was asked to review our provisional scales and suggest items they thought needed to be added. A clinician credentialed as a Certified Eating Disorder Specialist by the International Association for Eating Disorders Professionals (S.G.) was involved in all phases of item writing and development. This process yielded a total of 15 possible constructs, which are defined in the online supplemental materials. Again, following the typical practice in the assessment of eating pathology (e.g., Forbush et al., 2013), these constructs were conceptualized as unipolar in nature.
The goal of this study, similar to Study 1, was to oversample potential content and theoretical areas and to be mindful of developmental considerations when assessing eating and feeding disorder symptoms. Care was taken to include items representing all symptoms of each eating and feeding disorder and to incorporate information from recent research reflecting subgroups within the heterogeneous diagnostic category of ARFID (e.g., disinterest in food, sensory selectivity, and special diets; Bryson et al., 2017; Sharp & Stubbs, 2019).
Method
Participants and Procedures
Participants were 400 university students enrolled in first- or second-year psychology courses at a large public university in New Zealand. They completed study measures (including the 170 ED candidate items) in groups up to 12, monitored by a trained research assistant, and received course credit for their participation. The sample consisted of 84 men (21.0%), 315 (78.8%) women, and 1 person who identified as intersex. Participants ranged in age from 17 to 51 years (M = 19.75; SD = 3.16). The majority of participants reported their ethnicity as being New Zealand European (75%), with 16% being Other European (including Australian), 12% New Zealand Māori, 8% Chinese, 4% Indian, 2% Pacific Islander, and 11% selecting “other.” These values add up to greater than 100% because some selected more than one option. Because we had access to an established diagnostic self-report measure for ED (Eating Disorder Diagnostic Scale for DSM-5 [EDDS-DSM-5]; Stice et al., 2000), we could, based on clinical cut-offs, estimate that 19.7% met probable diagnostic criteria for any ED. More specifically, 10.8% met threshold for Bulimia Nervosa, 6.1% for Other Specified Feeding or Eating Disorder, 1.8% for Anorexia Nervosa, and 1.3% for Binge Eating Disorder.
The participants were also administered psychometrically supported ED and body image questionnaires that were used for criterion-related validity. These included the Eating Disorder Examination–Questionnaire (Fairburn & Beglin, 1994), EDDS-DSM-5 (Stice et al., 2000), EPSI (Forbush et al., 2013), Binge Eating Scale (Gormally et al., 1982), and Body Image Acceptance and Action Questionnaire (Sandoz et al., 2013).
Scale Development and Results
The same approach as for Study 1 was implemented for Study 2 unless otherwise stated. Therefore, in the interest of brevity, we will not re-explain every step for the scale development process and results.
Candidate Item Generation
Each author independently provided a list of at least three items for each of the 15 constructs. After the items has been screened for redundancy, each author independently reviewed each list and selected the items they thought had the highest quality, as well as items they thought were poor and should not be considered further. This process resulted in 10 to 15 candidate items (M = 11.3) per construct, with a total of 170 candidate items being subject to data collection. The final sets of candidate items for each construct are included in the online supplemental materials.
Initial Item Screening
We used the same screening criteria as for Study 1. All items that pertained to Pica (>97% responded “not at all” to all items) and Rumination Disorder (>92% responded “not at all” to all items) were extremely rare (range restricted) and, therefore, were dropped from the item pool. One item concerning steroid use from Muscle Building was also removed (>99% responded “not at all”). In total, 23 items were also removed because of excessive statistical redundancy with other items; more specifically, three from Body Dissatisfaction, two from Binge Eating, three from Cognitive Restraint, three from Restricting, one from Excessive Exercise, one from Purging, one from Muscle Building, one from Negative Attitudes Towards Obesity, two from Body Checking, two from Prepubertal Body Dissatisfaction, and four from Weight Phobia. One item from Prepubertal Body Dissatisfaction was removed because it did not correlate meaningfully with any other item.
Phase 1 Analyses
Our first step was to score all constructs and conduct an exploratory factor analysis. We scored 13 of 15 constructs given the removal of pica and rumination disorder from the analyses. We used robust maximum likelihood estimation and parallel analysis (1,000 replications) to extract factors. Parallel analysis revealed a two-factor solution. Model fit was borderline (CFI = .91; TLI = .86; RMSEA = .09; SRMR = .04), and although a three-factor solution was associated with improved model fit, it was inconsistent with the parallel analysis and actually covered fewer constructs as the data analysis plan had set an a priori loading ≥ |.50| as meaningful. For the two-factor solution, Body Dissatisfaction (.88), Binge Eating (.63), Cognitive Restraint (.79), Restricting (.72), Purging (.64), Body Checking (.83), Prepubescent Body Dissatisfaction (.72), and Weight Phobia (.83) loaded above the |.50| threshold on the first factor. For the second factor, Excessive Exercise (.68) and Muscle Building (.53) loaded meaningfully. Negative Attitudes Toward Obesity (.25, .06), AFRID (.39, .03), and Special Diets/Orthorexia (.36, .25) failed to load above threshold on either factor and were set aside for subsequent analyses. The full factor solution is included in the online supplement (see Table S6).
Next, we conducted item-level EFAs of the item pool within each of the two construct-based factors. We used WLSMV estimation in Mplus. For the first item pool, a parallel analysis of the polychoric correlation matrix revealed up to six factors; however, an inspection of a six- and even five-factor solution indicated nonsensical factor structures with bloated specifics (repeated coverage of items with the same basic content). Consequently, a four-factor structure was deemed to make most conceptual sense, and was associated with acceptable model fit (CFI = .95, TLI = .94, RMSEA = .05, SRMR = .06). The full factor solution is presented in the online supplement (Table S7). Following instructions from the overall analytic plan (Simms et al., this issue), we retained only items with loadings > |.40| and at least a difference of |.20| from the largest cross-loading. The first factor included items representing Body Dissatisfaction, Body Checking, Prepubescent Body Dissatisfaction, and Weight Phobia (λs = .55-.95); the second factor had only Binge Eating items (λs = .66-.81); the third factor had Restricting and Purging Items (λs = .53-.91); and the fourth factor had five Cognitive Restraint items (λs = .47-.85).
The item pool associated with the second construct-based factor yielded a clear two-factor solution, with acceptable model fit (CFI=.96, TLI = .95, RMSEA = .07; SRMR = .06). The full rotated solution appears in Table S8 (available online). All 10 remaining Excessive Exercise items loaded on the first factor (λs = .70-.87) and 10 of the 11 remaining Muscle Building items on a second factor (λs = .62-.93). One Muscle Building item was discarded because it loaded equally well on both factors. This process yielded six potential scales in total that were further refined in the next step of the data analysis protocol.
The next step involved culling items from each of the six scales to bring them down to 8 to 10 items with acceptable omega values (i.e., >.85) and acceptable coverage of individual item information curves derived from graded response IRT models. We used the same process as for Study 1.
Body image and weight concerns
This scale started with 23 items, which were systematically culled down to 11 items (we stopped at 11 items to preserve content breadth). We consulted item discrimination parameters (i.e., which items yielded the most information), item information curves (which items provided complementary coverage across the full range of the construct), and item content to preserve content breadth (particularly, body dissatisfaction, body checking, weight phobia, overvaluation of weight and shape). TIFs for the 23-item and 11-item versions are available in the online supplement (Figures S4.1 and S4.2). The final scale, which is included in Table S9 (see online supplement), had an ω = .95.
Binge eating
This scale only had eight items and the McDonald’s ω was .95; thus, we did not make any further modifications. On final content review, we decided to add a previously removed item (due to statistical redundancy) because the item included critical content related to the DSM-5 definition of binge eating (i.e., loss of control over eating), which did not affect other item parameters. The final nine-item scale is available in Table S10 (see online supplement).
Restricting and purging
This scale started with 18 items, which were systematically culled down to 11 items using the same procedure (we again stopped at 11 items to preserve content breadth). TIFs for the 18-item and 11-item versions are available in the online supplement (Figures S5.1 and S5.2). The final scale, which is included in the online Supplementary Table S11, had an ω = .94.
Cognitive restraint
This scale started with five items, which was below our goal of 8 to 10. We, therefore, revisited previously excluded Cognitive Restraint items, and added three items from the first item-based factor analysis that had loaded meaningfully on the cognitive restraint factor (i.e., ≥.40), but within the .20 separation from another factor. This eight-item version, which is included in the online supplementary materials (Table S12), had good IRT parameters, and an ω = .93.
Excessive exercise
Because this scale had 10 items and the McDonald’s ω was .95, we did not make any further modifications. The final scale is available in Table S13 (see online supplement).
Muscle building
Because this scale had 10 items and the McDonald’s ω was .94, we did not make any further modifications. The final scale is available in Table S14 (see online supplement).
Additional Provisional Constructs
Five of the original 15 ED constructs did not contribute to any of the aforementioned six scales. Two (pica and rumination disorder) were removed from further consideration because of extreme range restriction and their likely nonapplicability to the sample under study. These items might require further testing in specific populations with intellectual or other developmental disabilities. Among the other three, we opined that two constructs should be provisionally retained for Phase 2 given their clinical applicability to individuals with feeding and eating pathology: Negative Attitudes Toward Obesity and ARFID.
Negative Attitudes Toward Obesity
This scale had 10 remaining items after the initial item review, and the McDonald’s ω was .95, so we did not make any further modifications. The final scale is available in Table S15 (see online supplement).
Avoidant/Restrictive Food Intake Disorder
This scale started with 15 items; these were systematically culled down to 10 items using the same procedure as the other scales. TIFs for the 15-item and 10-item versions are available in the online supplement (Figures S6.1 and S6.2). The final scale, which is included in Table S16 (see online supplement), had an ω = .95.
The intercorrelations between the final six scales and these two provisional scales are shown in Table 2. Four core scales—Body Image and Weight Concerns, Binge Eating, Restricting and Purging, and Cognitive Restraint—were moderately to strongly related to one another. Correlations among these four scales ranged from .41 to .69, with an average value (after r-to-z transformation) of .57. The four remaining scales tended to be more modestly correlated with the other scales, with coefficients ranging from −.01 to .45 (mean r = .22).
Intercorrelations Among Eight Provisional Eating Disorder Scales (n = 400).
Note. r = |.17| is statistically significant (p < .001). BIWC = Body Image and Weight Concerns; BE = Binge Eating; REP = Restrictive Eating and Purging; CR = Cognitive Restraint; EE = Excessive Exercise; MB = Muscle Building; NATO = Negative Attitudes Toward Obesity; ARFID = Avoidant/Restrictive Food Intake Disorder.
Criterion Validity
Table 3 includes criterion-related validity against established measures of eating pathology and body-image concerns. As evident from this table, the eight provisional scales were highly correlated with external criteria in a manner that is consistent with conceptual expectations (see bold typeface in Table 3). Some specific findings are noteworthy. Muscle Building, which tends to be deemphasized in most ED measures, was only correlated with its EPSI counterpart (r = .66) and EPSI Excessive Exercise (r = .43). Negative Attitudes Toward Obesity had a very large correlation with its EPSI counterpart (r = .82), but failed to achieve a moderate correlation with other scales. Although the EPSI content related to restricting and purging loaded on separate factors in previous research (Forbush et al., 2013; Forbush et al., 2014), these constructs loaded together on a common factor in the current study. Future research is needed to determine whether the inclusion of restricting and purging behaviors within the same scale lowers criterion-related validity for distinguishing among traditional anorexia nervosa and bulimia nervosa patients. Finally, ARFID did not evidence a large correlation with any of the measures, but was most strongly associated with EPSI Restricting and EDDS Restricting/Fasting.
Correlations Between HiTOP Eating Pathology Scales and External Criteria (n = 400).
Note. Correlations appearing in bold typeface are those conceptually most relevant to each HiTOP construct. r = |.17| is statistically significant (p < .001). HiTOP = Hierarchical Taxonomy of Psychopathology; EPSI = Eating Pathology Symptom Inventory; EDE-Q = Eating Disorder Examination–Questionnaire; EDDS-DSM-5 = Eating Disorder Diagnostic Scale for DSM-5; BIAAQ = Body Image Acceptance and Action Questionnaire; BIWC = Body Image and Weight Concerns; BE = Binge Eating; REP = Restricting and Purging; CR = Cognitive Restraint; EE = Excessive Exercise; MB = Muscle Building; NATO = Negative Attitudes Towards Obesity; ARFID = Avoidant/Restrictive Food Intake Disorder.
Composite of compensatory behavior items on EDE-Q. bEDDS Binge Eating is the sum of responses to DSM-5 Criterion B symptoms of binge eating disorder. We also calculated correlations among HiTOP constructs and EDDS objective binge eating items, which yielded nearly identical results (data available on request from the corresponding author).
Brief Discussion
The Phase 1 process yielded six provisional scales for Phase 2 along with the addition of two scales representing constructs deemed clinically meaningful and requiring further evaluation before deciding if it is appropriate to remove them from the overall HiTOP inventory. The six constructs we derived clearly represent broad transdiagnostic themes that underlie traditional feeding and eating disorders in the DSM-5 and are consistent with previous empirically identified ED constructs (Forbush et al., 2013). The absence of large correlations between the ARFID scale and existing measures of ED symptoms is consistent with expectations, given that ARFID is currently characterized as a feeding, not eating, disorder in DSM-5. Thus, the lack of substantial correlations between the ARFID scale and existing ED symptoms demonstrates discriminant validity, which may aid clinicians in differential diagnosis between ARFID and nonweight phobic anorexia nervosa.
These new feeding and ED scales, pending Phase 2 of the HiTOP measurement project, have the potential to aid clinical practice, given that research already demonstrates that clinicians tailor treatment to specific symptoms, rather than to a categorical diagnosis (e.g., Waszczuk et al., 2017), and view symptom dimensions as more informative than DSM-5 diagnoses for treatment selection (e.g., Hopwood et al., 2020).
General Discussion
Across the two studies, we developed 10 Phase 1 scales: four assessing somatoform and six measuring eating-related pathology. The scales correspond to previously established constructs in the literature. The four somatoform scales correspond closely to their a priori hypothesized structure, comprising content related to Bodily Distress Symptoms, Health Anxiety, Disease Conviction, and Somatic Preoccupation. The eating-related scales similarly capture previously described constructs in the literature, tapping content related to Body Image and Weight Concerns, Restricting and Purging, Cognitive Restraint, Binge Eating, Excessive Exercise, and Muscle Building. The 10 scales evinced sound psychometric properties in the samples examined, including promising criterion validity, providing an empirical foundation for further investigation in Phase 2.
In addition, three sets of items—the conversion items, and two eating-related groups—were retained for further study in Phase 2. Consistent with prior literature (Benbadis & Hauser, 2000; Grant & Krasnik, 2015), the conversion items had very low endorsement rates, precluding modeling them in Phase 1. Relatedly, some eating-related item sets were not strongly associated with ED-related factors in structural analyses. Two of them were retained for further study in Phase 2, however, due to their importance in the domain: Negative Attitudes Toward Obesity, and items related to ARFID.
These initial results help characterize the basic psychometric contours of the HiTOP somatoform and ED scales. However, these scales are preliminary and should not be considered final. Phase 2 will help better characterize the measures by replicating the current results, determining measurement redundancy in light of other HiTOP scales developed for other spectra, and by extending information on their properties to both general population samples (with greater demographic diversity than those used in the current project) and higher risk samples where they presumably will have more symptom variation. Crucially, superordinate structural analyses will also help determine how the somatoform and eating-related scales relate to other forms of psychopathology, especially internalizing psychopathology. Based on previous results, we tentatively expect that the majority of eating-related scales will fit within the internalizing dimension (Forbush et al., 2010; Forbush et al., 2018; Forbush & Watson, 2013; Watson et al., in press); the somatoform scales should form a separate domain at the spectrum level (e.g., Kotov et al., 2011; Marek et al., 2020), but then combine with internalizing to form a superordinate emotional dysfunction dimension at the higher-order level, for example, the big three (emotional dysfunction, externalizing, and thought disorder; Forbes et al., 2017; Kotov et al., 2020; Watson et al., in press).
Our methods were associated with some limitations. First, only university students, with a high proportion of females, from New Zealand were sampled, which restricts generalizability to other community and patient populations and, perhaps, even other countries. Indeed, some of the range restriction observed for certain items, especially in Study 1, is likely owing to sampling. However, the results of Study 2 should be less affected in that regard given the high incidence of eating pathology among university students (e.g., American College Health Association, 2008; Eisenberg et al., 2011), including those in New Zealand. Second, we only had a limited range of criterion measures available in Study 1 to examine validity, and like those available in Study 2, they were exclusively other self-report questionnaires. Shared method variance might have artificially inflated correlations between these new HiTOP Phase 1 scales and criterion measures to an unknown degree. It will therefore be important to include other types of participants (psychiatric and general medical patients) and to take a multimethod approach to validation as this work continues through Phase 2 and beyond.
Supplemental Material
sj-pdf-1-asm-10.1177_10731911211020825 – Supplemental material for HiTOP Assessment of the Somatoform Spectrum and Eating Disorders
Supplemental material, sj-pdf-1-asm-10.1177_10731911211020825 for HiTOP Assessment of the Somatoform Spectrum and Eating Disorders by Martin Sellbom, Kelsie T. Forbush, Sara R. Gould, Kristian E. Markon, David Watson and Michael Witthöft in Assessment
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: A grant from the University of Minnesota Press funded the data collections.
Supplemental material
Supplemental material for this article is available online.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
