Major educational initiatives in the world place great emphasis on fostering rich computer-based environments of assessment that make student thinking and reasoning visible. Using thinking tools engages students in a variety of critical and complex thinking, such as evaluating, analyzing, and decision making. The aim of this study was to explore patterns in student critical thinking performance and motivation in Evidence-Centered Concept Map (ECCM) mode, compared to basic notepad mode. One hundred ninety 14-year-old students from the United States, United Kingdom, Singapore, and South Africa participated in the study. Students in both modes were able to analyze a multifaceted dilemma by using similar information resources. In the ECCM mode, students used ECCM to organize their thinking; in another mode, students were provided with a basic online notepad to make records as needed. Overall, the findings showed that students assessed in ECCM mode outperformed their peers in notepad mode in critical thinking skills. Students who worked with ECCM provided more informed recommendations by using supporting evidence from the available resources and discussing alternative points of view on the topic. The findings showed a significantly positive relationship between student recommendation score and the ability to develop ECCM. In addition, the results demonstrated that it did not matter for students' motivation whether they analyzed the dilemma with or without ECCM. Directions for future research are discussed in terms of their implications for large-scale assessment programs, teaching, and learning.
Get full access to this article
View all access options for this article.
References
1.
AllA. C.HuyckeL. I.FisherM. J. (2003). Instructional tools for nursing education: Concept maps. Nursing Education Perspectives, 24(6), 311–317.
2.
BellerM. (2013). Technologies in large-scale assessments: New directions, challenges, and opportunities. In von DavierM.GonzalezE.KirschI.YamamotoK. (Eds.), The role of international large-scale assessments: Perspectives from technology, economy, and educational research (pp. 25–46). New York, NY: Springer.
3.
BennettR. E. (1999). Using new technology to improve assessment. Educational Measurement: Issues and Practice, 18, 5–12.
4.
BennettR. E.PerskyH.WeissA.JenkinsF. (2007). Problem solving in technology rich environments: A report from the NAEP Technology-based Assessment Project, Research and Development Series (NCES 2007-466). Washington, DC: U.S. Department of Education, National Center for Education Statistics.
5.
BinkleyM.ErstadO.HermanJ.RaizenS.RipleyM.Miller-RicciM. (2012). Defining twenty-first century skills. In GriffinP.McGawB.CareE. (Eds.), Assessment and teaching of 21st century skills (pp. 17–66). Dordrecht, Netherlands: Springer.
6.
BonkC. J.SmithG. S. (1998). Alternative instructional strategies for creative and critical thinking in the accounting curriculum. Journal of Accounting Education, 16(2), 261–293.
7.
CubanL. (2003). Why is it so hard to get good schools?New York, NY: Teachers College.
8.
CubanL. (2006). 1:1 laptops transforming classrooms: Yeah, sure. New York, NY: Teachers College Record.
9.
DedeC. (2009). Immersive interfaces for engagement and learning. Science, 323, 66–69.
10.
EklöfH. (2006). Development and validation of scores from an instrument measuring student test-taking motivation. Educational and Psychological Measurement, 66(4), 643–656.
11.
EnnisR. H. (1985). A logical basis for measuring critical thinking skills. Educational Leadership, 43(2), 44–48.
12.
EnnisR. H. (1993). Critical thinking assessment. Theory into Practice, 32(3), 179–186.
13.
FischerS. C.SpikerV. A.RiedelS. L. (2009). Critical thinking training for army officers, volume 2: A model of critical thinking. (Technical Report). Arlington, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.
14.
GerstnerS.BognerF. (2009). Concept map structure, gender and teaching methods: An investigation of students' science learning. Educational Research, 51(4), 425–438.
15.
HalpernD. F. (1998). Teaching critical thinking for transfer across domains: Disposition, skills, structure training, and metacognitive monitoring. American Psychologist, 53, 449–455.
16.
HoeftR.JentschF.HarperM.EvansA.BowersC.SalasE. (2003). TPL-KATS—Concept map: A computerized knowledge assessment tool. Computers in Human Behavior, 19(6), 653–657.
17.
HsuL. L. (2004). Developing concept maps from problem-based learning scenario discussions. Journal of Advanced Nursing, 48(5), 510–518.
18.
JonassenD. H. (1996). Computers in the classroom: Mindtools for critical thinking. Englewood Cliffs, NJ: Prentice-Hall, Inc.
19.
JonassenD. H. (2006). Modeling with technology: Mindtools for conceptual change. Columbus, OH: Merrill/Prentice-Hall.
20.
JonassenD. (2008). Meaningful learning with technology. Upper Saddle River, NJ: Pearson Merrill Prentice Hall.
21.
JonassenD.ReevesT. (1996). Learning with technology: Using computers as cognitive tools. In JonassenD. (Ed.), Handbook of research for educational communications and technology (pp. 694–719). New York, NY: Macmillan.
22.
KahnemanD. (2011). Thinking fast and slow. New York, NY: Farrar, Strauss and Giroux.
23.
KinchinI. M. (2000). Concept mapping in biology. Journal of Biological Education, 34(2), 61–68.
24.
KuK. Y. (2009). Assessing students' critical thinking performance: Urging for measurements using multi-response format. Thinking Skills and Creativity, 4(1), 70–76.
25.
KuncelN. R.HezlettS. A. (2010). Fact and fiction in cognitive ability testing for admissions and hiring decisions. Current Directions in Psychological Science, 19(6), 339–345.
26.
LaiE. R.VieringM. (2012, April). Assessing 21st century skills: Integrating research findings. Paper presented at the annual meeting of the National Council on Measurement in Education, Vancouver, Canada.
27.
LajoieS. P. (2000). Computers as cognitive tools: No more walls (Vol. 2). Mahwah, NJ: Lawrence Erlbaum Associates.
28.
LangerE. J. (1989). Mindfulness. Reading, MA: Addison-Wesley.
29.
LangerE. J. (1997). The art of mindful learning. Reading, MA: Addison-Wesley.
30.
MayerR. E. (2001). Multimedia learning. New York, NY: Cambridge University Press.
31.
MayerR. E.WittrockM. C. (2006). Problem solving. In AlexanderP. A.WinneP. H. (Eds.), Handbook of educational psychology (pp. 287–303). Mahwah, NJ: Lawrence Erlbaum Associates.
32.
McClureJ.SonakB.SuenH. (1999). Concept map assessment of classroom learning: Reliability, validity, and logistical practicality. Journal of Research in Science Teaching, 36, 475–492.
33.
MossP. A.KoziolS. M. (1991). Investigating the validity of a locally developed critical thinking test. Educational Measurement: Issues and Practice, 10(3), 17–22.
34.
National Research Council. (2011). Assessing 21st century skills. Washington, DC: National Academies Press.
35.
NorrisS. P. (1989). Can we test validly for critical thinking?Educational Researcher, 18(9), 21–26.
36.
NovakJ. D. (1998). Learning, creating, and using knowledge: Concept maps as facilitative tools in schools and corporations. Mahwah, NJ: Lawrence Erlbaum Associates.
PellegrinoJ. W.ChudowskyN.GlaserR. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.
41.
PerkinsD. (1993). Person plus: A distributed view of thinking and learning. In SalomonG. (Ed.), Distributed cognitions (pp. 88–110). New York, NY: Cambridge University Press.
42.
RosenY. (2009). Effects of animation learning environment on knowledge transfer and learning motivation. Journal of Educational Computing Research, 40(4), 439–455.
43.
RosenY. (2011, August). Effects of constructivist teacher-led digital platform on learning achievement. Paper presented at the biennial meeting of the European Association of Researchers on Learning and Instruction, Exeter, UK.
44.
RosenY.Beck-HillD. (2012). Intertwining digital content and one-to-one laptop learning environment. Journal of Research on Technology in Education, 44(3), 223–239.
45.
RosenY.SalomonG. (2007). The differential learning achievements of constructivist technology-intensive learning environments as compared with traditional ones: A meta-analysis. Journal of Educational Computing Research, 36(1), 1–14.
46.
Ruiz-PrimoM. A. (2004). Examining concept maps as an assessment tool. Concept maps: Theory, methodology, technology. Proceeding of the First International Conference on Concept Mapping.
47.
SalomonG.PerkinsD. N. (2005). Do technologies make us smarter? Intellectual amplification with, of, and through technology. In PreissD. D.SternbergR. (Eds.), Intelligence and technology (pp. 71–86). Mahwah, NJ: LEA.
48.
SmithG. F. (2002). Thinking skills: The question of generality. Journal of Curriculum Studies, 34(6), 659–678.
49.
TuckerB. (2009). Beyond the bubble: Technology and the future of educational assessment. Washington, DC: Education Sector.
50.
U.S. Department of Education. (2010). Transforming American education—Learning powered by technology: National Education Technology Plan 2010. Washington, DC: Office of Educational Technology, U.S. Department of Education. Retrieved from http://www.ed.gov/sites/default/files/netp2010.pdf
51.
WestonM.BainA. (2010). The end of techno-critique: The naked truth about 1:1 laptop initiatives and educational change. Journal of Technology, Learning, and Assessment, 9(6), 5–25.
52.
WiseS. L.DeMarsC. E. (2005). Low examinee effort in low-stakes assessment: Problems and potential solutions. Educational Assessment, 10(1), 1–17.
