Low Overlap and High Heterogeneity Across Common Measures of Eating Disorder Pathology: A Content Analysis

Abstract

This study evaluated symptoms assessed in common measures of eating disorder pathology and tested overlap to evaluate the extent to which measures may be interchangeable. Six measures were included: Bulimia Test-Revised, Eating Attitudes Test-26, Eating Disorder Diagnostic Scale, Eating Disorder Examination Questionnaire, Eating Pathology Symptoms Inventory, and Questionnaire for Eating Disorder Diagnoses. Content overlap was quantitatively estimated using the Jaccard Index. Mean overlap was low (.195), likely due to the wide range of symptoms (87) assessed. The mean overlap of each measure with all others was .117 − .267, and the overlap among individual measures was .083 − .382. Implications of low overlap among measures include variable characterization of eating disorder phenotypes and the risk for lower generalizability of findings due to measurement variability.

Keywords

eating disorders content analysis assessment

Get full access to this article

View all access options for this article.

References

American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders (3rd ed., rev.).

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.).

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing.

Backhaus

Leijten

Meinck

Gardner

(2022). Different instruments, same content? A systematic comparison of child maltreatment and harsh parenting instruments. Trauma, Violence and Abuse, 24(5), 3546–3563. https://ora.ox.ac.uk/objects/uuid:9ff2366e-cd8f-4052-bb62-01e4df4a15d6

Berg

K. C.

Peterson

C. B.

Frazier

Crow

S. J.

(2015). Psychometric evaluation of the Eating Disorder Examination and Eating Disorder Examination-Questionnaire: A systematic review of the literature. International Journal of Eating Disorders, 45(3), 428–438. https://doi.org/10.1002/eat.20931

Burton

A. L.

Abbott

M. J.

Modini

Touyz

(2016). Psychometric evaluation of self-report measures of binge-eating symptoms and related psychopathology: A systematic review of the literature. International Journal of Eating Disorders, 49(2), 123–140. https://doi.org/10.1002/eat.22453

Chrobak

A. A.

Krupa

Dudek

Siwek

(2021). How soft are neurological soft signs? Content overlap analysis of 71 symptoms among seven most commonly used neurological soft signs scales. Journal of Psychiatric Research, 138, 404–412. https://doi.org/10.1016/j.jpsychires.2021.04.020

Chrobak

A. A.

Siwek

Dudek

Rybakowski

J. K.

(2018). Content overlap analysis of 64 (hypo)mania symptoms among seven common rating scales. International Journal of Methods in Psychiatric Research, 27(3), e1737. https://doi.org/10.1002/mpr.1737

Crombez

Paepe

A. L. D.

Veirman

Eccleston

Verleysen

Ryckeghem

D. M. L. V

. (2020). Let’s talk about pain catastrophizing measures: An item content analysis. PeerJ, 8, e8643. https://doi.org/10.7717/peerj.8643

10.

Fairburn

C. G.

(2008). Eating Disorder Examination Questionnaire (EDE-Q 6.0). In Fairburn

C. G.

(Ed.), Cognitive behavior therapy and eating disorders (pp. 309–313). The Guilford Press.

11.

Fairburn

C. G.

Beglin

(1994). Assessment of eating disorders: Interview or self-report questionnaire? International Journal of Eating Disorders, 16(4), 363–370. https://doi.org/10.1002/1098-108X(199412)16:4<363::AID-EAT2260160405>3.0.CO;2-%23

12.

Forbush

K. T.

Gould

S. R.

Chapa

D. A. N.

Bohrer

B. K.

Hagan

K. E.

Clark

K. E.

Sorokina

D. A.

Perko

V. L.

(2017). New horizons in measurement: A review of novel and innovative approaches to eating-disorder assessment. Current Psychiatry Reports, 19(10), 76. https://doi.org/10.1007/s11920-017-0826-2

13.

Forbush

K. T.

Hagan

K. E.

Kite

B. A.

Chapa

D. A.

Bohrer

B. K.

Gould

S. R.

(2017). Understanding eating disorders within internalizing psychopathology: A novel transdiagnostic, hierarchical-dimensional model. Comprehensive Psychiatry, 79, 40–52.

14.

Forbush

K. T.

Richson

B. N.

Swanson

T. J.

Thomeczek

M. L.

Negi

Johnson

S. N.

Chapa

D. A.

Morgan

R. W.

O’Brien

C. J.

Gould

S. R.

(2022). Screening for eating disorders across genders in college students: Initial validation of the brief assessment of stress and eating. International Journal of Eating Disorders, 55(11), 1553–1564.

15.

Forbush

K. T.

Siew

C. S. Q.

Vitevitch

M. S.

(2016). Application of network analysis to identify interactive systems of eating disorder psychopathology. Psychological Medicine, 46(12), 2667–2677.

16.

Forbush

K. T.

Wildes

J. E.

Hunt

T. K.

(2014). Gender norms, psychometric properties, and validity for the Eating Pathology Symptoms Inventory. International Journal of Eating Disorders, 47(1), 85–91.

17.

Forbush

K. T.

Wildes

J. E.

Pollack

L. O.

Dunbar

Luo

Patterson

Petruzzi

Pollpeter

Miller

Stone

Bright

Watson

(2013). Development and validation of the Eating Pathology Symptoms Inventory (EPSI). Psychological Assessment, 25(3), 859–878. https://doi.org/10.1037/a0032639

18.

Fried

E. I.

(2017). The 52 symptoms of major depression: Lack of content overlap among seven common depression scales. Journal of Affective Disorders, 208, 191–197. https://doi.org/10.1016/j.jad.2016.10.019

19.

Fried

E. I.

(2020). Corrigendum to “The 52 symptoms of major depression: Lack of content overlap among seven common depression scales,” [Journal of Affective Disorders, 208, 191-197]. Journal of Affective Disorders, 260, 744. https://doi.org/10.1016/j.jad.2019.05.029

20.

Garner

D. M.

Olmstead

M. P.

Polivy

(1983). Development and validation of a multidimensional eating disorder inventory for anorexia nervosa and bulimia. International Journal of Eating Disorders, 2(2), 15–34. https://doi.org/10.1002/1098-108X(198321)2:2<15::AID-EAT2260020203>3.0.CO;2-6

21.

Garner

D. M.

Olmsted

M. P.

Bohr

Garfinkel

P. E.

(1982). The Eating Attitudes Test: Psychometric features and clinical correlates. Psychological Medicine, 12(4), 871–878. https://doi.org/10.1017/S0033291700049163

22.

Goel

N. J.

Burnette

C. B.

Weinstock

Mazzeo

S. E.

(2022). Eating Disorder Examination-Questionnaire: Evaluating factor structures and establishing measurement invariance with Asian/Hawaiian/Pacific Islander, Black, and White American college men. International Journal of Eating Disorders, 55(4), 481–493.

23.

Habashy

Benning

S. D.

Renn

B. N.

Borgogna

N. C.

Lawrence

E. M.

Kraus

S. W.

(2023). Psychometric properties of the Eating Disorder Examination Questionnaire: Factor analysis and measurement invariance by race/ethnicity and gender. Eating Behaviors, 48, 101696. https://doi.org/10.1016/j.eatbeh.2022.101696

24.

Jaccard

(1908). Nouvelles recherches sur la distribution florale [New research on floral distribution]. Bulletin de la Société vaudoise des Sciences Naturelles, 44, 223–270.

25.

Jenkins

P. E.

Rienecke

R. D.

(2022). Structural validity of the Eating Disorder Examination-Questionnaire: A systematic review. International Journal of Eating Disorders, 55(8), 1012–1030.

26.

Karhulahti

V.-M.

Martončik

Adamkovič

(2021). Measuring internet gaming disorder and gaming disorder: A qualitative content validity analysis of validated scales. Assessment, 30(2), 402–413. https://doi.org/10.1177/10731911211055435

27.

Karstoft

K.-I.

Armour

(2022). What we talk about when we talk about trauma: Content overlap and heterogeneity in the assessment of trauma exposure. Journal of Traumatic Stress, 36(1), 71–82. https://doi.org/10.1002/jts.22880

28.

Kotov

Krueger

R. F.

Watson

Achenbach

T. M.

Althoff

R. R.

Bagby

R. M.

Brown

T. A.

Carpenter

W. T.

Caspi

Clark

L. A.

Eaton

N. R.

Forbes

M. K.

Forbush

K. T.

Goldberg

Hasin

Hyman

S. E.

Ivanova

M. Y.

Lynam

D. R.

Markon

. . .Zimmerman

(2017). The Hierarchical Taxonomy of Psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126(4), 454–477. https://doi.org/10.1037/abn0000258

29.

Krabbenborg

M. A. M.

Danner

U. N.

Larsen

J. K.

Veer

van der Elburg

A. A.

van Ridder

D. T. D.

de Evers

Stice

Engels

R. C. M. E

. (2012). The Eating Disorder Diagnostic Scale: Psychometric features within a clinical population and a cut-off point to differentiate clinical patients from healthy controls. European Eating Disorders Review, 20(4), 315–320. https://doi.org/10.1002/erv.1144

30.

Lavender

J. M.

Brown

T. A.

Murray

S. B.

(2017). Men, muscles, and eating disorders: An overview of traditional and muscularity-oriented disordered eating. Current Psychiatry Reports, 19(6), 1–7.

31.

Levinson

C. A.

Williams

B. M.

(2020). Eating disorder fear networks: Identification of central eating disorder fears. International Journal of Eating Disorders, 53(12), 1960–1973. https://doi.org/10.1002/eat.23382

32.

Levinson

C. A.

Williams

B. M.

Christian

Hunt

R. A.

Keshishian

A. C.

Brosof

L. C.

Vanzhula

I. A.

Davis

G. G.

Brown

M. L.

Bridges-Curry

Sandoval-Araujo

L. E.

Ralph-Nearman

(2023). Personalizing eating disorder treatment using idiographic models: An open series trial. Journal of Consulting and Clinical Psychology, 91(1), 14–28. https://doi.org/10.1037/ccp0000785

33.

Meule

(2021). Reconsidering the use of cut-off scores for the Eating Disorder Examination-Questionnaire. Eating Disorders, 29(5), 480–484.

34.

Mintz

L. B.

O’Halloran

M. S.

(2000). The Eating Attitudes Test: Validation with DSM-IV eating disorder criteria. Journal of Personality Assessment, 74(3), 489–503. https://doi.org/10.1207/S15327752JPA7403_11

35.

Mintz

L. B.

O’Halloran

M. S.

Mulholland

A. M.

Schneider

P. A.

(1997). Questionnaire for Eating Disorder Diagnoses: Reliability and validity of operationalizing DSM-IV criteria into a self-report format. Journal of Counseling Psychology, 44, 63–79. https://doi.org/10.1037/0022-0167.44.1.63

36.

Monteleone

A. M.

Cascino

(2021). A systematic review of network analysis studies in eating disorders: Is time to broaden the core psychopathology to non specific symptoms. European Eating Disorders Review, 29(4), 531–547. https://doi.org/10.1002/erv.2834

37.

Murray

S. B.

Nagata

J. M.

Griffiths

Calzo

J. P.

Brown

T. A.

Mitchison

Blashill

A. J.

Mond

J. M.

(2017). The enigma of male eating disorders: A critical review and synthesis. Clinical Psychology Review, 57, 1–11.

38.

Purgato

Singh

Acarturk

Cuijpers

(2021). Moving beyond a “one-size-fits-all” rationale in global mental health: Prospects of a precision psychology paradigm. Epidemiology and Psychiatric Sciences, 30, e63. https://doi.org/10.1017/S2045796021000500

39.

R Core Team. (2022). R: A language and environment for statistical computing (4.2.2) [Computer software]. https://www.R-project.org/

40.

Richson

B. N.

Forbush

K. T.

Chapa

D. A. N.

Gould

S. R.

Perko

V. L.

Johnson

S. N.

Christensen

K. A.

Swanson

T. J.

Tregarthen

(2021). Measurement invariance of the Eating Pathology Symptoms Inventory (EPSI) in adolescents and adults. Eating Behaviors, 42, 101538. https://doi.org/10.1016/j.eatbeh.2021.101538

41.

Richson

B. N.

Forbush

K. T.

Schaumberg

Crosby

R. D.

Peterson

C. B.

Crow

S. J.

Mitchell

J. E.

(2020). Are the Criterion B binge-eating symptoms interchangeable in understanding binge-eating severity? An item response theory analysis. International Journal of Eating Disorders, 53(12), 1983–1992. https://doi.org/10.1002/eat.23383

42.

Scharmer

Donahue

J. M.

Heiss

Anderson

D. A.

(2020). Factor structure of the Eating Disorder Examination—Questionnaire among heterosexual and sexual minority males. Eating Behaviors, 38, 101403. https://doi.org/10.1016/j.eatbeh.2020.101403

43.

Schaumberg

Anderson

D. A.

Anderson

L. M.

Reilly

E. E.

Gorrell

(2016). Dietary restraint: What’s the harm? A review of the relationship between dietary restraint, weight trajectory and the development of eating pathology. Clinical Obesity, 6(2), 89–100. https://doi.org/10.1111/cob.12134

44.

Skogestad

I. J.

Kirkevold

Indredavik

Gay

C. L.

Lerdal

(2019). Lack of content overlap and essential dimensions—A review of measures used for post-stroke fatigue. Journal of Psychosomatic Research, 124, 109759. https://doi.org/10.1016/j.jpsychores.2019.109759

45.

Smith

A. R.

Forrest

L. N.

Duffy

M. E.

Jones

P. J.

Joiner

T. E.

Pisetsky

E. M.

(2020). Identifying bridge pathways between eating disorder symptoms and suicidal ideation across three samples. Journal of Abnormal Psychology, 129(7), 724–736. https://doi.org/10.1037/abn0000553

46.

Stice

Fisher

Martinez

(2004). Eating disorder diagnostic scale: Additional evidence of reliability and validity. Psychological Assessment, 16(1), 60–71.

47.

Stice

Shaw

H. E.

(2002). Role of body dissatisfaction in the onset and maintenance of eating pathology: A synthesis of research findings. Journal of Psychosomatic Research, 53(5), 985–993. https://doi.org/10.1016/S0022-3999(02)00488-9

48.

Stice

Telch

C. F.

Rizvi

S. L.

(2000). Development and validation of the Eating Disorder Diagnostic Scale: A brief self-report measure of anorexia, bulimia, and binge-eating disorder. Psychological Assessment, 12(2), 123–131. https://doi.org/10.1037//1040-3590.12.2.123

49.

Thelen

M. H.

Farmer

Wonderlich

S. A.

Smith

(1991). A revision of the Bulimia Test: The BULIT-R. Psychological Assessment, 3(1), 119–124. https://doi.org/10.1037/1040-3590.3.1.119

50.

Thelen

M. H.

Mintz

L. B.

Vander Wal

J. S.

(1996). The Bulimia Test-Revised: Validation with DSM-IV criteria for bulimia nervosa. Psychological Assessment, 8, 219–221. https://doi.org/10.1037/1040-3590.8.2.219

51.

Tiego

Martin

E. A.

DeYoung

C. G.

Hagan

K. E.

Cooper

S. E.

Pasion

Satchell

Shackman

A. J.

Bellgrove

Fornito

(2022). Precision behavioral phenotyping as a strategy for uncovering the biological correlates of psychopathology. OSF Preprints. https://doi.org/10.31219/osf.io/geh6q

52.

Túry

Güleç

Kohls

(2010). Assessment methods for eating disorders and body image disorders. Journal of Psychosomatic Research, 69(6), 601–611. https://doi.org/10.1016/j.jpsychores.2009.05.012

53.

Vartanian

L. R.

Porter

A. M.

(2016). Weight stigma and eating behavior: A review of the literature. Appetite, 102, 3–14. https://doi.org/10.1016/j.appet.2016.01.034

54.

Visontay

Sunderland

Grisham

Slade

(2019). Content overlap between youth OCD scales: Heterogeneity among symptoms probed and implications. Journal of Obsessive-Compulsive and Related Disorders, 21, 6–12. https://doi.org/10.1016/j.jocrd.2018.10.005

55.

Wall

A. D.

Lee

E. B.

(2022). What do anxiety scales really measure? An item content analysis of self-report measures of anxiety. Journal of Psychopathology and Behavioral Assessment, 44(4), 1148–1157. https://doi.org/10.1007/s10862-022-09973-9

56.

Wang

S. B.

Jones

P. J.

Dreier

Elliott

Grilo

C. M.

(2019). Core psychopathology of treatment-seeking patients with binge-eating disorder: A network analysis investigation. Psychological Medicine, 49(11), 1923–1928. https://doi.org/10.1017/S0033291718002702