A Review of Missing Data Reporting Practices in the Field of Internationalization of Higher Education

Abstract

Missing data are inevitable in quantitative research, yet the issue of missing data does not always receive adequate attention. Unless researchers report missing data, research consumers are left to assume default software protocols handled missing data—protocols that are frequently inappropriate, risk compromising a study's conclusions, and needlessly discard data shared by participants. This study reviews missing data reporting practices within the internationalization of higher education literature to determine how often researchers address missing data due to survey item nonresponse and which methods they used. Among the 35 sampled studies from the literature, only 37% explicitly addressed missing data and only 3% used a modern method endorsed by missing data researchers, signally the need to improve missing data reporting practices. To that end, this study concludes with a discussion on guidelines for reporting missing data, how missing data can be used to researchers’ advantage, and areas for future research.

Keywords

Missing data reporting practices internationalization multiple imputation maximum likelihood estimation

Get full access to this article

View all access options for this article.

References

Abayomi

Gelman

Levy

(2008). Diagnostics for multivariate imputations. Journal of the Royal Statistical Society: Series C (Applied Statistics), 57(3), 273–291. https://doi.org/10.1111/j.1467-9876.2007.00613.x

Allison

P. D.

(2002). Missing data. Sage.

American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed). American Psychological Association.

Baraldi

A. N.

Enders

C. K.

(2010). An introduction to modern missing data analyses. Journal of School Psychology, 48(1), 5–37. https://doi.org/10.1016/j.jsp.2009.10.001

Bennett

D. A.

(2001). How can I deal with missing data in my study? Australian and New Zealand Journal of Public Health, 25(5), 464–469. https://doi.org/10.1111/j.1467-842X.2001.tb00294.x

Brick

J. M.

Kalton

(1996). Handling missing data in survey research. Statistical Methods in Medical Research, 5(3), 215–238. https://doi.org/10.1177/096228029600500302

Burton

Altman

D. G.

(2004). Missing covariate data within cancer prognostic studies: A review of current reporting and proposed guidelines. British Journal of Cancer, 91(1), 4–8. https://doi.org/10.1038/sj.bjc.6601907

Cheema

J. R.

(2014). A review of missing data handling methods in education research. Review of Educational Research, 84(4), 487–508. https://doi.org/10.3102/0034654314532697

Chwialkowska

(2020). Maximizing cross-cultural learning from exchange study abroad programs: Transformative learning theory. Journal of Studies in International Education, 24(5), 535–554. https://doi.org/10.1177/1028315320906163

10.

Collins

L. M.

Schafer

J. L.

Kam

C.-M.

(2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6(4), 330–351. https://doi.org/10.1037/1082-989X.6.4.330

11.

Couper

M. P.

(2017). New developments in survey data collection. Annual Review of Sociology, 43, 121–145. https://doi.org/10.1146/annurev-soc-060116-053613

12.

de Leeuw

E. D.

Hox

Huisman

(2003). Prevention and treatment of item nonresponse. Journal of Official Statistics, 19(2), 153–176.

13.

de Wit

Hunter

(2015). The future of internationalization of higher education in Europe. International Higher Education, 83, 2–3. https://doi.org/10.6017/ihe.2015.83.9073

14.

Downe-Wamboldt

(1992). Content analysis: Method, applications, and issues. Health Care for Women International, 13(3), 313–321. https://doi.org/10.1080/07399339209516006

15.

Enders

C. K.

(2001). A primer on maximum likelihood algorithms available for use with missing data. Structural Equation Modeling, 8(1), 128–141. https://doi.org/10.1207/S15328007SEM0801_7

16.

Enders

C. K.

(2010). Applied missing data analysis. The Guilford Press.

17.

Enders

C. K.

(2025). Missing data: An update on the state of the art. Psychological Methods, 30(2), 322–339. https://doi.org/10.1037/met0000563

18.

Enders

C. K.

Gottschall

A. C.

(2011). The impact of missing data on the ethical quality of a research study. In Panter

A. T.

Sterba

S. K.

(Eds.), Handbook of ethics in quantitative methodology (pp. 357–381). Routledge.

19.

Fowler

F. J.

Jr. (2013). Survey research methods (5th ed.). Sage.

20.

Graham

J. W.

(2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549–576. https://doi.org/10.1146/annurev.psych.58.110405.085530

21.

Graham

J. W.

Taylor

B. J.

Olchowski

A. E.

Cumsille

P. E.

(2006). Planned missing data designs in psychological research. Psychological Methods, 11(4), 323–343. https://doi.org/10.1037/1082-989X.11.4.323

22.

Hardt

Herke

Leonhart

(2012). Auxiliary variables in multiple imputation in regression with missing X: A warning against including too many in small sample research. BMC Medical Research Methodology, 12(184), 1–13. https://doi.org/10.1186/1471-2288-12-184

23.

Haupt

Ogden

A. C.

Rubin

(2018). Toward a common research model: Leveraging education abroad participation to enhance college graduation rates. Journal of Studies in International Education, 22(2), 91–107. https://doi.org/10.1177/1028315318762519

24.

Hayward

Hoelscher

(2011). The use of large-scale administrative data sets to monitor progression from vocational education and training into higher education in the UK: Possibilities and methodological challenges. Research in Comparative and International Education, 6(3), 316–329. https://doi.org/10.2304/rcie.2011.6.3.316

25.

Headley

M. G.

Swoboda

C. M.

Foote

(2016). What’s missing in longitudinal studies conducted in the U.S. With implications for mathematics education? International Journal of Research & Method in Education, 39(4), 383–400. https://doi.org/10.1080/1743727X.2016.1184245

26.

Heyl

J. D.

Hunter

F. J. H.

(2019). The senior international officer as change agent (2nd ed.). Stylus Publishing.

27.

Horton

N. J.

Kleinman

K. P.

(2007). Much ado about nothing: A comparison of missing data methods and software to fit incomplete data regression models. The American Statistician, 61(1), 79–90. https://doi.org/10.1198/000313007X172556

28.

Hsieh

H.-F.

Shannon

S. E.

(2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687

29.

Hunter

Rumbley

L. E.

(2016). Exploring a possible future for the scholar-practitioner. In Streitwieser

Ogden

A. C.

(Eds.), International higher education’ s scholar-practitioners: Bridging research and practice (pp. 297–308). Symposium. https://doi.org/10.15730/books.96

30.

Jeličić

Phelps

Lerner

R. M.

(2009). Use of missing data methods in longitudinal studies: The persistence of bad practices in developmental psychology. Developmental Psychology, 45(4), 1195–1199. https://doi.org/10.1037/a0015665

31.

Kehm

B. M.

Teichler

(2007). Research on internationalization in higher education. Journal of Studies in International Education, 11(3-4), 260–273. https://doi.org/10.1177/1028315307303534

32.

Krosnick

J. A.

(1999). Survey research. Annual Review of Psychology, 50, 537–567. https://doi.org/10.1146/annurev.psych.50.1.537

33.

Lee

J. J.

(2010). International students’ experiences and attitudes at a US host institution: Self-reports and future recommendations. Journal of Research in International Education, 9(1), 66–84. https://doi.org/10.1177/1475240909356382

34.

Little

R. J. A.

(1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198–1202. https://doi.org/10.2307/2290157

35.

Little

R. J.

(2021). Missing data assumptions. Annual Review of Statistics and Its Applications, 8, 89–107. https://doi.org/10.1146/annurev-statistics-040720-031104

36.

Little

T. D.

Jorgensen

T. D.

Lang

K. M.

Moore

E. W. G.

(2014). On the joys of missing data. Journal of Pediatric Psychology, 39(2), 151–162. https://doi.org/10.1093/jpepsy/jst048

37.

Little

R. J. A.

Rubin

D. B.

(2020). Statistical analysis with missing data (3rd ed.). John Wiley & Sons.

38.

Mackinnon

(2010). The use and reporting of multiple imputation in medical research—A review. Journal of Internal Medicine, 268(6), 586–593. https://doi.org/10.1111/j.1365-2796.2010.02274.x

39.

Manly

C. A.

Wells

R. S.

(2015). Reporting the use of multiple imputation for missing data in higher education research. Research in Higher Education, 56, 397–409. https://doi.org/10.1007/s11162-014-9344-9

40.

McKinley

K. M.

(2019). Assessing mixed methods research quality in the European student mobility literature. Research in Comparative and International Education, 14(4), 433–449. https://doi.org/10.1177/1745499919889216

41.

McKinley

K. M.

(2024). Increasing survey response rates: Framing international students as potentially vulnerable and hard-to-reach. Research in Comparative and International Education, 19(2), 239–257. https://doi.org/10.1177/17454999241253059

42.

Meitinger

K. M.

Johnson

T. P.

(2020). Power, culture and item nonresponse in social surveys. In Brenner

P. S.

(Ed.), Understanding survey methodology: Sociological theory and applications (pp. 169–191). Springer. https://doi.org/10.1007/978-3-030-47256-6_8

43.

Mustillo

(2012). The effects of auxiliary variables on coefficient bias and efficiency in multiple imputation. Sociological Methods & Research, 41(2), 335–361. https://doi.org/10.1177/0049124112452392

44.

Ogden

A. C.

Brewer

(2019). U.S. education abroad: Historical perspectives, emerging trends, and changing narratives. In Brewer

Ogden

A. C.

(Eds.), Education abroad and the undergraduate experience: Critical perspectives and approaches to integration with student learning and development (pp. 15–40). Stylus Publishing.

45.

Peng

C.-Y. J.

Harwell

Liou

S.-M.

Ehman

L. H.

(2006). Advances in missing data methods and implications for educational research. In Sawilowsky

S. S.

(Ed.), Real data analysis (pp. 31–78). Information Age Publishing.

46.

Peters

C. L. O.

Enders

(2002). A primer for the estimation of structural equation models in the presence of missing data. Journal of Targeting, Measurement and Analysis for Marketing, 11(1), 81–95. https://doi.org/10.1057/palgrave.jt.5740069

47.

Peugh

J. L.

Enders

C. K.

(2004). Missing data in educational research: A review of reporting practices and suggestions for improvement. Review of Educational Research, 74(4), 525–556. https://doi.org/10.3102/00346543074004525

48.

Randall

Stern

(2021). Five ethical risks to consider before filling missing race and ethnicity data. Urban Institute. https://www.urban.org/research/publication/five-ethical-risks-consider-filling-missing-race-and-ethnicity-data

49.

R Core Team. (2024). R: A language and environment for statistical computing [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org/

50.

Rhemtulla

Hancock

G. R.

(2016). Planned missing data designs in educational psychology research. Educational Psychologist, 51(3-4), 305–316. https://doi.org/10.1080/00461520.2016.1208094

51.

Rubin

D. B.

(1976). Inference and missing data. Biometrika, 63(3), 581–592. https://doi.org/10.1093/biomet/63.3.581

52.

Rubin

D. B.

(1987). Multiple imputation for survey nonresponse. John Wiley & Sons.

53.

Schafer

J. L.

Graham

J. W.

(2002). Multiple imputation: Our view of the state of the art. Psychological Methods, 7(2), 147–177. https://doi.org/10.1037//1082-989X.7.2.147

54.

Schafer

J. L.

Olsen

M. K.

(1998). Multiple imputation for multivariate missing-data problems: A data analyst's perspective. Multivariate Behavioral Research, 33(4), 545–571. https://doi.org/10.1207/s15327906mbr3304_5

55.

Sinharay

Stern

H. S.

Russell

(2001). The use of multiple imputation for the analysis of missing data. Psychological Methods, 6(4), 317–329. https://doi.org/10.1037/1082-989X.6.4.317

56.

Souto-Otero

Huisman

Beerkens

Wit

H. d.

Vujić

(2013). Barriers to international student mobility: Evidence from the Erasmus program. Educational Researcher, 42(2), 70–77. https://doi.org/10.3102/0013189X12466696

57.

Steinmetz

Thompson

Marshall

(2020). Surveying international university students: The case of the 5% response rate. Issues in Educational Research, 30(3), 1105–1125. http://www.iier.org.au/iier30/steinmetz.pdf

58.

Sterne

J. A. C.

White

I. R.

Carlin

J. B.

Spratt

Royston

Kenward

M. G.

Wood

A. M.

Carpenter

J. R.

(2009). Multiple imputation for missing data in epidemiological and clinical research: Potential and pitfalls. BMJ, 338, b2393. https://doi.org/10.1136/bmj.b2393

59.

Streitwieser

B. T.

Rust

(2012). Research on study abroad, mobility, and student exchange in comparative education scholarship. Research in Comparative and International Education, 7(1), 5–19. https://doi.org/10.2304/rcie.2012.7.1.5

60.

Teddlie

Tashakkori

(2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Sage.

61.

van Buuren

(2007). Multiple imputation of discrete and continuous data by fully conditional specification. Statistical Methods in Medical Research, 16(3), 219–242. https://doi.org/10.1177/0962280206074463

62.

van Buuren

(2018). Flexible imputation of missing data (2nd ed.). Chapman and Hall/CRC Press.

63.

van Buuren

Groothuis-Oudshoorn

(2011). Mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1–67. https://doi.org/10.18637/jss.v045.i03

64.

Van Mol

(2017). Improving web survey efficiency: The impact of an extra reminder and reminder content on web survey response. International Journal of Social Research Methodology, 20(4), 317–327. https://doi.org/10.1080/13645579.2016.1185255

65.

Wang

Lin

(2019). Traditional Chinese views on education as perceived by international students in China: International student attitudes and understandings. Journal of Studies in International Education, 23(2), 195–216. https://doi.org/10.1177/1028315318797356

66.

White

I. R.

Royston

Wood

A. M.

(2011). Multiple imputation using chained equations: Issues and guidance for practice. Statistics in Medicine, 30(4), 377–399. https://doi.org/10.1002/sim.4067

67.

Wilkinson

& American Psychological Association Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54(8), 594–604. https://doi.org/10.1037/0003-066X.54.8.594

68.

Woods

A. D.

Gerasimova

Van Dusen

Nissen

Bainter

Uzdavines

Davis-Kean

P. E.

Halvorson

King

K. M.

Logan

J. A. R.

Vasilev

M. R.

Clay

J. M.

Moreau

Joyal-Desmarais

Cruz

R. A.

Brown

D. M. Y.

Schmidt

Elsherif

M. M.

(2024). Best practices for addressing missing data through multiple imputation. Infant and Child Development, 33(1), 1–37. https://doi.org/10.1002/icd.2407

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.91 MB