Redefining Teacher Measurement Literacy in a Transforming Assessment Landscape: A Validated Three-Dimensional Framework

Abstract

While assessment literacy has long guided teacher’ assessment practice, the rise of digital assessment presents new challenges in educational measurement, necessitating a renewed focus on Teacher Measurement Literacy (TML). This study reconceptualizes TML and develops a validated framework to support teachers in this evolving context. Using the Delphi method, a three-dimensional, 14-element TML framework was refined through two rounds of expert consultation, demonstrating increased consensus. A corresponding instrument was then administered to 306 pre-service teachers. Confirmatory factor analysis supported the proposed three-dimensional structure. Furthermore, likelihood ratio tests comparing nested Rasch models indicated that a three-dimensional model provided a significantly better fit to the data than a unidimensional alternative. Cross-validation with a sample of 297 in-service teachers further supported the robustness of the multidimensional framework. These findings validate the proposed three-dimensional conception of TML and offer empirical grounding to strengthen teachers’ capacity in the context of evolving measurement practices.

Keywords

teacher measurement literacy conceptual framework Delphi method empirical validation

Get full access to this article

View all access options for this article.

References

Aburizaizah

S. J.

(2021). Data-informed educational decision making to improve teaching and learning outcomes of EFL. Journal of Education and Learning, 10(5), 17–29. https://doi.org/10.1111/bjet.13407

Ackerman

T. A.

Bandalos

D. L.

Briggs

D. C.

Everson

H. T.

A. D.

Lottridge

S. M.

Wind

S. A.

Sinharay

Rodriguez

M. C.

Russell

von Davier

A. A.

(2024). Foundational competencies in educational measurement. Educational Measurement: Issues and Practice, 43(3), 7–17. https://doi.org/10.1111/emip.12581

Adams

R. J.

Wilson

Wang

(1997). The multidimentional random coefficients multinominal logit model. Applied Psychological Measurement, 21(1), 1–23. https://doi.org/10.1177/0146621697211001

Alkharusi

Kazem

A. M.

Al-Musawai

(2011). Knowledge, skills, and attitudes of preservice and inservice teachers in educational measurement. Asia-Pacific Journal of Teacher Education, 39(2), 113–123. https://doi.org/10.1080/1359866X.2011.560649

Binkley

Erstad

Herman

Raizen

Ripley

Miller-Ricci

Rumble

(2012). Defining twenty-first-century skills. In: Assessment and Teaching of 21st Century Skills, (pp. 17–66). Springer Science & Business Media. https://doi.org/10.1007/978-94-007-2324-5_2

Bond

T. G.

Fox

C. M.

(2013). Applying the rasch model: Fundamental measurement in the human sciences. Psychology Press. https://doi.org/10.4324/9781410614575

Brookhart

S. M.

(2011). Educational assessment knowledge and skills for teachers. Educational Measurement: Issues and Practice, 30(1), 3–12. https://doi.org/10.1111/j.1745-3992.2010.00195.x

Brookhart

S. M.

(2024). Educational assessment knowledge and skills for teachers revisited. Education Sciences, 14(7), 751. https://doi.org/10.3390/educsci14070751

Brousselle

Buregeya

J. M.

(2018). Theory-based evaluations: Framing the existence of a new theory in evaluation and the rise of the 5th generation. Evaluation, 24(2), 153–168. https://doi.org/10.1177/1356389018765487

10.

Bulut

Beiting-Parrish

Casabianca

J. M.

Slater

S. C.

Jiao

SongMorilova

D. P.

(2024). The rise of artificial intelligence in educational measurement: Opportunities and ethical challenges. arXiv Preprint arXiv:2406.18900. https://doi.org/10.59863/MIQL7785

11.

Butler-Henderson

Crawford

(2020). A systematic review of online examinations: A pedagogical innovation for scalable authentication and integrity. Computers & Education, 159(5), 104024. https://doi.org/10.1016/i.compedu.2020.104024

12.

Celik

Dindar

Muukkonen

Järvelä

(2022). The promises and challenges of artificial intelligence for teachers: A systematic review of research. TechTrends, 66(4), 616–630. https://doi.org/10.1007/s11528-022-00715-y

13.

Chan

K. W.

Ali

Park

Sham

K. S. B.

Tan

E. Y. T.

ChongSze

F. W. C. G. K.

(2025). Automatic item generation in various STEM subjects using large language model prompting. Computers and Education: Artificial Intelligence, 8(6), 100344. https://doi.org/10.1016/j.caeai.2024.100344

14.

Clough

Shehabi

Morgan

(2016). Medical risk assessment in dentistry: Use of the American society of anesthesiologists physical status classification. British Dental Journal, 220(3), 103–108. https://doi.org/10.1038/sj.bdj.2016.87

15.

Comrey

A. L.

Lee

H. B.

(1992). A first course in factor analysis (2nd ed.). Psychology Press. https://doi.org/10.4324/9781315827506

16.

Coombe

Davidson

(2022). Language assessment literacy. In Mohebbi

Coombe

(Eds.), Research questions in language education and applied linguistics: A reference guide (pp. 343–347). Springer. https://doi.org/10.1007/978-3-030-79143-8

17.

Daniel

L. G.

King

D. A.

(1998). Knowledge and use of testing and measurement literacy of elementary and secondary teachers. The Journal of Educational Research, 91(6), 331–344. https://doi.org/10.1080/00220679809597563

18.

Delgado

H. O. K.

de Azevedo Fay

Sebastiany

M. J.

Silva

A. D. C.

(2020). Artificial intelligence adaptive learning tools: The teaching of English in focus. BELT-Brazilian English Language Teaching Journal, 11(2), e38749. https://doi.org/10.15448/2178-3640.2020.2.38749

19.

DeLuca

Chavez

Cao

(2013). Establishing a foundation for valid teacher judgement on student learning: The role of pre-service assessment education. Assessment in Education: Principles, Policy & Practice, 20(1), 107–126. https://doi.org/10.1080/0969594X.2012.668870

20.

DeLuca

Klinger

Searle

Shulha

(2010). Developing a curriculum for assessment education. Assessment Matters, 2, 133–155. https://doi.org/10.3316/INFORMIT.330301976917998

21.

DeLuca

LaPointe-McEwan

Luhanga

(2016). Teacher assessment literacy: A review of international standards and measures. Educational Assessment, Evaluation and Accountability, 28(3), 251–272. https://doi.org/10.1007/s11092-015-9233-6

22.

Duckworth

A. L.

Yeager

D. S.

(2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44(4), 237–251. https://doi.org/10.3102/0013189X15584327

23.

Ebel

R. L.

(1961). Improving the competence of teachers in educational measurement. The clearing house. A Journal of Educational Strategies, Issues and Ideas, 36(2), 67–71. https://doi.org/10.1080/00098655.1961.11475810

24.

Eisinga

Grotenhuis

M. T.

Pelzer

(2013). The reliability of a two-item scale: Pearson, cronbach, or spearman-brown? International Journal of Public Health, 58(4), 637–642. https://doi.org/10.1007/s00038-012-0416-3

25.

Estaji

Banitalebi

Brown

G. T.

(2024). The key competencies and components of teacher assessment literacy in digital environments: A scoping review. Teaching and Teacher Education, 141, 104497. https://doi.org/10.1016/j.tate.2024.104497

26.

Foster

Piacentini

(Eds.), (2023). Innovating assessments to measure and support complex skills. OECD Publishing. https://doi.org/10.1787/e5f3e341-en

27.

Gotch

C. M.

(2012). An investigation of teacher educational measurement literacy. Washington State University. https://www-proquest-com-s.web.bisu.edu.cn/dissertations-theses/investigation-teacher-educational-measurement/docview/1112071363/se-2

28.

Griffin

Care

Harding

S. M.

(2014). Task characteristics and calibration. In Assessment and teaching of 21st century skills: Methods and approach (pp. 133–178). Springer Netherlands. https://doi.org/10.1007/978-94-017-9395-7_7

29.

Gummer

E. S.

Mandinach

E. B.

(2015). Building a conceptual framework for data literacy. Teachers College Record: The Voice of Scholarship in Education, 117(4), 1–22. https://doi.org/10.1177/016146811511700401

30.

Ibna Seraj

P. M.

Chakraborty

Mehdi

Roshid

M. M.

(2022). A systematic review on pedagogical trends and assessment practices during the COVID‐19 pandemic: Teachers’ and Students’ perspectives. Education Research International, 2022(1), 1534018. https://doi.org/10.1155/2022/1534018

31.

Inman

T. F.

Roberts

J. L.

(2021). Authentic, formative, and informative: Assessment of advanced learning. In Modern curriculum for gifted and advanced academic students (pp. 205–236). Routledge.

32.

Jarr

K. A.

(2012). Education practitioners' interpretation and use of assessment results, Doctoral dissertation. University of Iowa. https://doi.org/10.17077/etd.35vh2oc1

33.

Kershaw IV

(1993). Ohio vocational education teachers' perceived use of student assessment information in educational decision-making. The Ohio State University. https://www-proquest-com-s.web.bisu.edu.cn/dissertations-theses/ohio-vocational-education-teachers-perceived-use/docview/304060852/se-2

34.

Kubiszyn

Borich

G. D.

(2024). Educational testing and measurement (pp. 1–23). John Wiley & Sons.

35.

Lambert

N. M.

(1991). The crisis in measurement literacy in psychology and education. Educational Psychologist, 26(1), 23–35. https://doi.org/10.1207/s15326985ep2601_2

36.

Looney

Cumming

van Der Kleij

Harris

(2018). Reconceptualising the role of teachers as assessors: Teacher assessment identity. Assessment in Education: Principles, Policy & Practice, 25(5), 442–467. https://doi.org/10.1080/0969594X.2016.1268090

37.

Looney

J. W.

(2009). Assessment and Innovation in Education. OECD Education Working Papers, No. 24. OECD Publishing (NJ1). https://doi.org/10.1787/222814543073

38.

Maity

Deroy

(2024). The future of learning in the age of generative ai: Automated question generation and assessment with large language models. arXiv Preprint. https://doi.org/10.48550/arXiv.2410.09576

39.

Marion

S. F.

Pellegrino

J. W.

Berman

A. I.

(2024). Reimagining balanced assessment systems. National Academy of Education.

40.

Mertler

C. A.

(2003). Preservice versus inservice teachers' assessment literacy: Does classroom experience make a difference? https://policycommons.net/artifacts/14881227/preservice-versus-inservice-teachers-assessment-literacy/15779931/

41.

Mertler

C. A.

(2004). Secondary teachers' assessment literacy: Does classroom experience make a difference? American Secondary Education, pp. 49–64. https://www-jstor-org-443.web.bisu.edu.cn/stable/41064623

42.

Newton

P. E.

Shaw

S. D.

(2014). Validity in educational and psychological assessment. Sage.

43.

Pastore

Andrade

H. L.

(2019). Teacher assessment literacy: A three-dimensional model. Teaching and Teacher Education, 84, 128–138. https://doi.org/10.1016/j.tate.2019.05.003

44.

Plake

B. S.

Impara

J. C.

Fager

J. J.

(1993). Assessment competencies of teachers: A national survey. Educational Measurement: Issues and Practice, 12(4), 10–12. https://doi.org/10.1111/j.1745-3992.1993.tb00548.x

45.

Popham

W. J.

(2011). Assessment literacy overlooked: A teacher educator's confession. The Teacher Educator, 46(4), 265–273. https://doi.org/10.1080/08878730.2011.605048

46.

Popham

W. J.

(2018). Assessment literacy for educators in a hurry. ASCD.

47.

Randall

Rios

J. A.

Jung

H. J.

(2021). Graduate training in educational measurement and psychometrics: A curriculum review of graduate programs in the US. Practical Assessment, Research & Evaluation, 26, 2. https://doi.org/10.7275/y1v0-wm37

48.

Rudner

L. M.

Schafer

W. D.

(2002). What teachers need to know about assessment. Washington, DC: National Education Association.

49.

Russell

Ludlow

O'Dwyer

(2019). Preparing the next generation of educational measurement specialists: A call for programs with an integrated scope and sequence. Educational Measurement: Issues and Practice, 38(4), 78–86. https://doi.org/10.1111/emip.12285

50.

Schafer

W. D.

(1991). Essential assessment skills in the professional education of teachers. Educational Measurement: Issues and Practice, 10(1), 3–6. https://doi.org/10.1111/j.1745-3992.1991.tb00170.x

51.

Stiggins

(1991). Assessment literacy. The Phi Delta Kappan, 72(7), 534–539. https://www.elibrary.ru/item.asp?id=1583759.

52.

Sudakova

N. E.

Savina

T. N.

Masalimova

A. R.

Mikhaylovsky

M. N.

Karandeeva

L. G.

Zhdanov

S. P.

(2022). Online formative assessment in higher education: Bibliometric analysis. Education Sciences, 12(3), 209. https://doi.org/10.3390/educsci12030209

53.

Taylor

(2009). Developing assessment literacy. Annual Review of Applied Linguistics, 29, 21–36. https://doi.org/10.1017/S0267190509090035

54.

Timmis

Broadfoot

Sutherland

Oldfield

(2016). Rethinking assessment in a digital age: Opportunities, challenges and risks. British Educational Research Journal, 42(3), 454–476. https://doi.org/10.1002/berj.3215

55.

Villegas

A. M.

Irvine

J. J.

(2010). Diversifying the teaching force: An examination of major arguments. The Urban Review, 42(3), 175–192. https://doi.org/10.1007/s11256-010-0150-1

56.

von Davier

A. A.

Mislevy

R. J.

Hao

(Eds.), (2022). Computational psychometrics: New methodologies for a new generation of digital learning and assessment. Springer Nature.

57.

Willis

Adie

Klenowski

(2013). Conceptualising teachers' assessment literacies in an era of curriculum and assessment reform. The Australian Educational Researcher, 40(2), 241–256. https://doi.org/10.1007/s13384-013-0089-9

58.

Woolfolk

(2016). Educational psychology. Pearson. https://thuvienso.hoasen.edu.vn/handle/12345678919007

59.

Wylie

E. C.

Heritage

(2024). Assessment literacy and professional learning. In Marion

Pellegrino

Berman

(Eds.), Reimagining balanced assessment systems. National academy of education. National Academy of Education. https://doi.org/10.31094/2024/1/5

60.

Brown

G. T.

(2016). Teacher assessment literacy in practice: A reconceptualization. Teaching and Teacher Education, 58, 149–162. https://doi.org/10.1016/j.tate.2016.05.010

61.

(2022). Urban public-school teacher assessment literacy in China Zhengzhou city. Education, 51(1), 3–13. https://doi.org/10.1080/03004279.2021.2025128