The acquisition of teacher digital competence is a key aspect in the initial training of teachers. However, most existing evaluation instruments do not provide sufficient evidence of this teaching competence. In this study, we describe the design and development process of a three-dimensional (3D) virtual environment for evaluating the teacher digital competence of future teachers, through a performance-based, collaborative and contextual evaluation. This environment, named ETeach3D, has been constructed using the educational design research approach. It is based on successive iterative cycles and is in accordance with the criteria of usefulness, validity, and effectiveness. In addition to the research team responsible for the project, participating in this study were 187 Spanish undergraduate students of Education and 22 experts in the field of educational technology. Results show that these environments, in addition to other characteristics, should (a) function smoothly and have simple interfaces, realistic scenes, and interactive activities and (b) follow a systematic evaluation procedure that integrates several strategies and levels of complexity. This research helps to improve the initial training of preservice teachers and contributes to the growing number of educational design research studies that focus in the field of evaluation of the curriculum domain.
AlmåsA. G.KrumsvikR. (2007) Digitally literate teachers in leading edge schools in Norway. Journal of In-Service Education33(4): 479–497. doi:10.1080/13674580701687864.
2.
AndrewsD. H.WulfeckW. H. (2014) Performance assessment: Something old, something new. In: SpectorJ. M.MerrillM. D.ElenJ.BishopM. J. (eds) Handbook of research on educational communications and technology, New York, NY: Springer, pp. 303–310.
3.
AtkinsC. (2009) Virtual experience: Observations on second life. In: PurvisM.SavarimuthuB. (eds) Lecture notes in computer science: Computer-mediated social networking, Berlin, Germany: Springer, pp. 7–17. doi:10.1007/978-3-642-02276-0_2.
4.
BanisterS.ReinhartR. (2012) Assessing NETS-T performance in teacher candidates: Exploring the Wayfind teacher assessment. Journal of Digital Learning in Teacher Education29(2): 59–65.
5.
Cela-RanillaJ. M.Esteve-GonzalezV.Esteve-MonF.Gisbert-CerveraM. (2014) 3D simulation as a learning environment for acquiring the skill of self-management: An experience involving Spanish university students of education. Journal of Educational Computing Research51(3): 295–309. doi:10.2190/ec.51.3.b.
6.
ChauM.WongA.WangM.LaiS.ChanK. W.LiT. M.SungW.-K. (2013) Using 3D virtual environments to facilitate students in constructivist learning. Decision Support Systems56: 115–121. doi:10.1016/j.dss.2013.05.009.
7.
ChristensenR.KnezekG.Tyler-WoodT.GubsonD. (2011) Simschool: An online dynamic simulator for enhancing teacher preparation. International Journal of Learning Technology6(2): 201–220.
8.
Clarke, J., Code, J., Zap, N., & Dede, C. (2011). Exploring the utility of a virtual performance assessment. Paper presented at SREE Fall 2011, Advancing Education Research, Washington, DC.
9.
ClarkeJ.DedeC. (2010) Assessment, technology, and change. Journal of Research on Technology in Education42(3): 309–328.
10.
CodeJ.Clarke-MiduraJ.ZapN.DedeC. (2011) Virtual performance assessment in immersive virtual environments. In: WangH. (ed.) Interactivity in e-learning: Case studies and frameworks, Hershey, PA: IGI Global, pp. 230–252. doi:10.4018/978-1-61350-441-3.ch011.
11.
CodeJ.Clarke-MiduraJ.ZapN.DedeC. (2013) The utility of using immersive virtual environments for the assessment of science inquiry learning. Journal of Interactive Learning Research24(4): 371–396.
12.
CollinsA.JosephD.BielaczycK. (2004) Design research: Theoretical and methodological issues. Journal of the Learning Sciences13(1): 15–42. doi:10.1207/s15327809jls1301_2.
13.
CovelloS. (2010) A review of digital literacy assessment instruments, New York, NY: Syracuse University, School of Education.
14.
DalgarnoB.LeeM. J. W. (2010) What are the learning affordances of 3-D virtual environments?British Journal of Educational Technology41(1): 10–32.
15.
DedeC.NelsonB.KetelhutD. J.ClarkeJ.BowmanC. (2004) Design-based research strategies for studying situated learning in a multi-user virtual environment. In: KafaiY.SandovalW.EnyedyN. (eds) Proceedings of the 6th international conference on learning sciences, Santa Monica, CA: International Society of the Learning Sciences, pp. 158–165.
16.
De FreitasS. (2008) Serious virtual worlds. A scoping study, Bristol, England: Joint Information Systems Committee.
17.
DowseC.HowieS. (2013) Promoting academic research. Writing with South African master's student in the field of education. In: PlompT.NieveenN. (eds) Educational design research. Part B: Illustrative cases, Enschede, the Netherlands: Netherlands Institute for Curriculum Development (SLO), pp. 951–879.
18.
Enlaces (2011) Competencias y estándares TIC para la profesión docente [ICT competences and standards for the teaching profession]. Centro de Educación y Tecnología (Enlaces), Santiago, Chile: Ministerio de Educación, Gobierno de Chile.
19.
EseryelD.GuoY.LawV. (2012) Interactivity design and assessment framework for educational games to promote motivation and complex problem-solving skills. In: IfenthalerD.EseryelD.GeX. (eds) Assessment in game-based learning, New York, NY: Springer, pp. 257–285. doi:10.1007/978-1-4614-3546-4_14.
20.
Esteve-Mon, F. M. (2015). La competencia digital docente: Análisis de la autopercepción y evaluación del desempeño de los estudiantes universitarios de educación por medio de un entorno 3D [Teachers’ digital competence: Self-perception and performance assessment of pre-service teachers’ digital competence through a 3D environment] (Doctoral dissertation). Rovira i Virgili University, Tarragona.
21.
Esteve-MonF. M.Adell-SeguraJ.Gisbert-CerveraM. (2014) Diseño de un entorno 3D para el desarrollo de la competencia digital docente en estudiantes universitarios: Usabilidad, adecuación y percepción de utilidad [Design of 3D environment to develop pre-service teachers’ digital competence: Usability, adequacy and perceived usefulness]. RELATEC, Revista Latinoamericana de Tecnología Educativa13(2): 35–47.
22.
Esteve-MonF. M.Gisbert-CerveraM. (2013) La competencia digital en la educación superior: Instrumentos de evaluación y nuevos entornos [Digital competence in higher education: Assessment tools and new environments]. Enl@ce: Revista Venezolana De Información, Tecnología y Conocimiento10(3): 29–43.
23.
European Commission (2013) Analysis and mapping of innovative teaching and learning for all through new Technologies and Open Educational Resources in Europe. SWD(2013) 341 final, Brussels, Belgium: Author.
24.
FerrariA. (2012) Digital competence in practice: An analysis of frameworks, Sevilla, Spain: Joint Research Centre (JRC), European Commission.
25.
GilsterP. (1997) Digital literacy, New York, NY: Wiley Computer.
26.
GregoryS.DalgarnoB.CrispG.ReinersT.MastersY.DreherH.KnoxV. (2013) VirtualPREX: Innovative assessment using a 3D virtual world with pre-service teachers, Sydney, Australia: Office for Learning and Teaching, Australian Government.
27.
GutiérrezA.PalaciosA.TorregoL. (2010) La formación de los futuros maestros y la integración de las TIC en la educación: Anatomía de un desencuentro [School Teacher training and ICT integration in education: Anatomy of a mismatch]. Revista de Educación352: 267–293.
28.
Hall, R., Atkins, L., & Fraser, J. (2014). Defining a self-evaluation digital literacy framework for secondary educators: The DigiLit Leicester project. Research in Learning Technology, 22. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.3402/rlt.v22.21440.
29.
ISTE (2008) National educational technology standards for teachers, Washington, DC: International Society for Technology in Education.
30.
KabakçiI. (2009) A proposal of framework for professional development of Turkish teachers with respect to information and communication technologies. Turkish Online Journal of Distance Education10(3): 204–216.
31.
KetelhutD. J. (2007) The impact of student self-efficacy on scientific inquiry skills: An exploratory investigation in river city, a multi-user virtual environment. Journal of Science Education and Technology16(1): 99–111. doi:10.1007/s10956-006-9038-y.
32.
KrumsvikR. (2008) Situated learning and teachers’ digital competence. Education and Information Technologies13(13): 279–290. doi:10.1007/s10639-008-9069-5.
33.
KrumsvikR. (2009) Situated learning in the network society and the digitised school. European Journal of Teacher Education32(2): 167–185. doi:10.1080/02619760802457224.
34.
KuoC.-Y.WuH.-K. (2013) Toward an integrated model for designing assessment systems: An analysis of the current status of computer-based assessments in science. Computers & Education68: 388–403. doi:10.1016/j.compedu.2013.06.002.
35.
LankshearC.KnobelM. (2008) Digital literacies: Concepts, policies and practices, New York, NY: Peter Lang.
36.
McKenney, S. (2001). Computer-based support for science education materials developers in Africa: Exploring potentials (Doctoral dissertation). University of Twente, Enschede.
37.
McKenneyS.NieveenN.van den AkkerJ. (2006) Design research from a curriculum perspective. In: van den AkkerJ.GravemeijerK.McKenneyS.NieveenN. (eds) Educational design research, London, England: Routledge, pp. 110–143.
38.
MislevyR. J. (2011) Evidence-centered design for simulation-based assessment, Los Ángeles, CA: The National Center for Research on Evaluation, Standards, and Student Testing (CRESST), University of California.
39.
MorphewV. N. (2012) A constructivist approach to the national educational technology standards for teachers, Arlington, VA: International Society for Technology in Education (ISTE).
40.
NelsonB.KetelhutD. J.ClarkeJ.BowmanC.DedeC. (2005) Design-based research strategies for developing a scientific inquiry curriculum in a multi-user virtual environment. Educational Technology45(1): 21–27.
41.
NelsonB. C.ErlandsonB.DenhamA. (2011) Global channels of evidence for learning and assessment in complex game environments. British Journal of Educational Technology42(1): 88–100. doi:10.1111/j.1467-8535.2009.01016.x.
42.
NokelainenP. (2006) An empirical assessment of pedagogical usability criteria for digital learning material with elementary school students. Educational Technology & Society9(2): 178–197.
43.
OhE.FrenchR. (2004) Pre-service teachers perceptions of an introductory instructional technology course. Electronic Journal for the Integration of Technology in Education3(1): 37–48.
44.
OlasojiR.Henderson-BeggS. (2010) Summative assessment in second life: A case study. Journal of Virtual Worlds Research3(3): 3–13.
45.
PlompT.NieveenN. (2009) An introduction to educational design research, Enschede, the Netherlands: Netherlands Institute for curriculum development (SLO).
46.
QuellmalzE. S.SilberglittM. D.TimmsM. J. (2011) How can simulations be components of balanced state science assessment systems (Policy Brief), San Francisco, CA: WestEd.
47.
QuellmalzE. S.TimmsM. J.SilberglittM. D.BuckleyB. C. (2012) Science assessments for all: Integrating science simulations into balanced state science assessment systems. Journal of Research in Science Teaching49(3): 363–393. doi:10.1002/tea.21005.
48.
RedeckerC. (2013) The use of ICT for the assessment of key competences, Sevilla, Spain: Joint Research Centre, Institute for Prospective Technological Studies, European Commission.
49.
RedeckerC.Ala-MutkaK.BacigalupoM.FerrariA.PunieY. (2009) Learning 2.0: The impact of web 2.0 innovations on education and training in Europe (JCR Scientific and Technical Report), Seville, Spain: Institute for Prospective and Technological Studies, Joint Research Centre.
50.
RicheyR. C.KleinJ. D. (2005) Developmental research methods: Creating knowledge from instructional design and development practice. Journal of Computing in Higher Education16(2): 23–38.
51.
Rodríguez EspinarS.PradesA. (2009) Guía para la evaluación de competencias en el área de ciencias sociales [Guide for skills assessment in the Social Sciences area], Barcelona, Spain: AQU Catalunya.
52.
RoelofsE.SandersP. (2007) Towards a framework for assessing teacher competence. European Journal of Vocational Training40: 123–139.
53.
ShuteV. J.MasdukiI.DonmezO. (2010) Conceptual framework for modeling, assessing, and supporting competencies within game environments. Technology, Instruction, Cognition, and Learning8(2): 137–161.
54.
SilvaJ. (2012) Estándares tic para la formación inicial docente: Una política pública en el contexto chileno [ICT standards for initial teacher training: A public policy in the Chilean context]. Archivos Analíticos de Políticas Educativas20(7): 1–36.
55.
SparrowJ. L.BlevinsS. J.BrennerA. M. (2011) Faculty development for and in virtual worlds. In: HinrichsR.WankelC. (eds) Transforming virtual world learning, Bingley, England: Emerald, pp. 47–65. doi:10.1108/S2044-9968(2011)0000004007.
56.
StufflebeamD. (2002) Foundational model for 21st century program evaluation. In: StufflebeamD.MadausG.KellaghanT. (eds) Evaluation models viewpoints on educational and human services evaluation, New York, NY: Kluwer Academic Publishers, pp. 33–84.
57.
TessmerM. (1993) Planning and conducting formative evaluations: Improving the quality of education and training, London, England: Kogan Page.
58.
UNESCO (2008) Estándares de competencia en TIC para docents [ICT Competency standards for teachers], Londres, England: Author.
59.
van den AkkerJ. (2002) The potential of development research for improving the relation between curriculum research and curriculum development. In: RosenmundM.FriesA. V.HellerW. (eds) Comparing curriculum-making process, Berlin, Germany: Peter Lang, pp. 1–12.
60.
van den AkkerJ.GravemeijerK.McKenneyS.NieveenN. (2006) Educational design research, London, England: Routledge.
61.
WilsonK. A.BedwellW. L.LazzaraE. H.SalasE.BurkeC. S.EstockJ. L.ConkeyC. (2009) Relationships between game attributes and learning outcomes: Review and research proposals. Simulation Gaming40(2): 217–266.
62.
WoollardJ.WankelC. (2011) Initial teacher training in a virtual world. In: HinrichsR. (ed.) Transforming virtual world learning, Bingley, England: Emerald, pp. 29–46. doi:10.1108/S2044-9968(2011)0000004006.
63.
Zulkardi. (2002). Developing a learning environment on realistic mathematics education for Indonesian student teachers (Doctoral dissertation). University of Twente, Enschede.
