In this article, we aim to describe the practical aspects of research actions that should be taken into account when two methods—the think-aloud technique for data collection and protocol analysis as an analysis technique—are combined with research of decision making in acute clinical settings. These methods are rarely used together or written about in health care research. In addition, careful consideration of the pros and cons of study sampling, data collection, data management, and analysis techniques or approaches is still lacking when these methods are used in health care research. We discuss the background of the think-aloud technique and protocol analysis and review previous studies that have used these methods. The examples in the article are based on our study, in which we investigated the decision making of critical care experts. We find that the combination of the think-aloud technique and protocol analysis is applicable when investigating complex and overlapping decision-making processes and rapid, ad hoc decisions made by critical care experts.
Aitken, L.M. ( 2003). Critical care nurses’ use of decision-making strategies . Journal of Clinical Nursing, 12, 476-483.
2.
Aitken, L.M., & Mardegan, K.J. (2000). "Thinking aloud": Data collection in the natural setting. Western Journal of Nursing Research, 22, 841-853.
3.
Boren, M.T., & Ramey, J. ( 2000). Thinking aloud: Reconciling theory and practice. IEEE Transactions on Professional Communication, 43(3), 261-278.
4.
Branch, J.L. ( 2000). The trouble with think alouds: Generating data using verbal protocols. In A. Kublik (Ed.), Proceedings of the 28th Annual Conference of the Canadian Association for Information Science. Dimensions of a global information science , Canada. Retrieved July 20, 2009, from http://www.cais-acsi.ca/search.asp?year=2000
5.
Buckingham, C., & Adams, A. ( 2000). Classifying clinical decision making: A unifying approach . Journal of Advanced Nursing, 32, 981-989.
6.
Burla, L., Knierim, B., Barth, J., Liewald, K., Duetz, M., & Abel, T. ( 2008). From text to codings. Nursing Research , 57(2), 113-117.
7.
Carter, S.M., & Little, M. ( 2007). Justifying knowledge, justifying method, taking action: Epistemologies, methodologies, and methods in qualitative research. Qualitative Health Research, 17, 1316-1328.
8.
Caulton, D.A. ( 2001). Relaxing the homogeneity assumption in usability testing . Behaviour & Information Technology, 20, 1-7. Cioffi, J., & Markham, R. ( 1997). Clinical decision-making by midwives: Managing case complexity . Journal of Advanced Nursing, 25, 265-272.
9.
Corbin, J., & Strauss, A.C. ( 2007). Basics of qualitative research: Techniques and procedures for developing grounded theory (3rd ed.). Thousand Oaks, CA: Sage.
10.
Curran, M., Campbell, J., & Rugg, G. ( 2006). An investigation into the clinical reasoning of both expert and novice podiatrists. Foot, 16(1), 28-32.
11.
Daly, W.M. ( 2001). The development of an alternative method in the assessment of critical thinking as an outcome of nursing education. Journal of Advanced Nursing, 36, 120-130.
12.
Defeng, L. ( 2004). Trustworthiness of think-aloud protocols in the study of translation processes. International Journal of Applied Linguistics, 14(3), 301-313.
13.
Duncker, K. ( 1926). A qualitative (experimental and theoretical) study of productive thinking (solving of comprehensible problems). Pedagogical Seminar, 33, 642-708.
14.
Ericsson, K.A., & Simon, H.A. ( 1993). Protocol analysis. Verbal reports as data (1st rev. ed.) . Cambridge, MA: MIT Press.
15.
Fisher, A., & Fonteyn, M.E. ( 1995). An exploration of an innovative methodological approach for examining nurses’ heuristic use in clinical practice. Scholarly Inquiry for Nursing Practice, 9(3), 263-276.
16.
Fonda, S.J., Paulsen, C.A., Perkins, J., Kedziora, R.J., Rodbard, D., & Bursell, S.-E. (2008). Usability test of Internet-based informatics tool for diabetes care providers: The comprehensive diabetes management program. Diabetes Technology & Therapeutics, 10(1), 16-24.
17.
Fonteyn, M.E., & Grobe, S.J. ( 1994). Expert system development in nursing: Implications for critical care nursing practice. Heart & Lung: The Journal of Critical Care, 23(1), 80-87.
18.
Fonteyn, M.E., Kuipers, B., & Grobe, S.J. ( 1993). A description of think aloud method and protocol analysis . Qualitative Health Research, 3, 430-441.
19.
Fowler, L.P. ( 1997). Clinical reasoning strategies used during care planning . Clinical Nursing Research, 6(4), 349-361.
Funkesson, K.H., Anbäcken, E.-M., & Ek, A.-C. ( 2007). Nurses’ reasoning process during care planning taking pressure ulcer prevention as an example. A think-aloud study. International Journal of Nursing Studies, 44, 1109-1119.
22.
Garro, L.C. ( 1998). On the rationality of decision making studies: Part 1: Decision models of treatment choice. Medical Anthropology Quarterly, 12(3), 319-340.
23.
Göransson, K., Ehrenberg, A., Ehnfors, M., & Fonteyn, M. ( 2006). The use of qualitative data analysis software (QDAS) to manage and support the analysis of think aloud (TA) data. Studies in Health Technology and Informatics, 122, 143-146.
24.
Greenwood, J., & King, M. ( 1995). Some surprising similarities in the clinical reasoning of "expert" and "novice" orthopaedic nurses: Report of a study using verbal protocols and protocol analyses. Journal of Advanced Nursing , 22, 907-913.
25.
Greenwood, J., Sullivan, J., Spence, K., & McDonald, M. ( 2000). Nursing scripts and the organizational influences on critical thinking: Report of a study of neonatal nurses’ clinical reasoning . Journal of Advanced Nursing, 31, 1106-1114.
26.
Grobe, S.J., Drew, J.A., & Fonteyn, M.E. ( 1991). A descriptive analysis of experienced nurses’ clinical reasoning during a planning task. Research in Nursing & Health, 14(4), 305-314.
27.
Hall, M., De Jong, M., & Steehouder, M. (2004). Cultural differences and usability evaluation: Individualistic and collectivistic participants compared . Technical Communication, 51(4), 489-503.
28.
Hammond, K.R. ( 1996). Human judgement and social policy. Irreducible uncertainty, inevitable error, unavoidable injustice. New York: Oxford University Press.
29.
Hoppmann, T.K. (in press). Examining the "point of frustration": The think-aloud method applied to online search tasks. Quality and Quantity.
30.
Hsieh, H.-F., & Shannon, S.E. ( 2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15, 1277-1288.
31.
Jones, J.A. ( 1989). The verbal protocol: A research technique for nursing . Journal of Advanced Nursing, 14, 1062-1070.
32.
Kuiper, R.A. ( 2004). Nursing reflections from journaling during a perioperative internship. AORN Journal, 79(1), 195-218.
33.
Kuipers, B., & Kassirer, J.P. (1984). Causal reasoning in medicine: Analysis of a protocol. Cognitive Science, 8, 363-385.
34.
Lauri, S., & Salanterä, S. (2002). Developing an instrument to measure and describe clinical decision making in different nursing fields . Journal of Professional Nursing, 18(2), 93-100.
35.
Lewis, J.R. ( 1994). Sample size for usability studies: Additional considerations . Human Factors, 36, 369-378.
36.
Linden, D.E.J. ( 2007). The working memory networks of the human brain. Neuroscientist, 13(3), 257-267.
37.
Malterud, K. ( 2001). Qualitative research: Standards, challenges, and guidelines . Lancet, 358, 483-488.
38.
Mercado-Martinez, F.J., Tejada-Tayabas, L.M., & Springett, J. ( 2008). Methodological issues in emergent evaluations of health programs: Lessons from Iberoamerica. Qualitative Health Research , 18, 1277-1288.
39.
Miles, M.B., & Huberman, A.M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage.
40.
Newell, A., & Simon, H.A. ( 1972). Human problem solving. Englewood Cliffs, NJ: Prentice Hall.
41.
Nielsen, J. ( 1994). Estimating the number of subjects needed for a thinking aloud test. International Journal of Human-Computer Studies , 41, 385-397.
42.
Nielsen, J., Clemmensen, T., & Yssing, C. ( 2002). Getting access to what goes on in people’s heads? Reflections on the think-aloud technique. Proceedings of the Nordic Conference on Human-Computer Interaction, Denmark, 31, 101-110. Retrieved July 20, 2009, from http://portal.acm.org/citation.cfm?id=572033
43.
Offredy, M. ( 2002). Decision-making in primary care: Outcomes from a study using patient scenarios. Journal of Advanced Nursing, 40, 532-541.
44.
Parahoo, K. ( 2006). Nursing research. Principles, process and issues (2nd ed.). New York: Palgrave Macmillan.
45.
Phansalkar, S., Patel, V.L., Hoffman, J.M., Pharm, D., & Hurdle, J.F. ( 2006). Use of verbal protocol analysis for identification of ADE signals [Abstract]. Proceedings of the American Medical Informatics Association, USA, 2006, 1063.
46.
Qiu, Y., & Yu, P. ( 2007). Nursing information systems- Applying usability testing to assess the training needs for nursing students. Methods of Information in Medicine, 46(4), 416-419.
47.
QSR International. (2006). NVivo7: Qualitative research computer software and manual. Retrieved July 20, 2009, from http://www.qsrinternational.com/products_previous-products_nvivo7.aspx
Simmons, B., Lanuza, D., Fonteyn, M., Hicks, F., & Holm, K. ( 2003). Clinical reasoning in experienced nurses. Western Journal of Nursing Research, 25, 720-724.
50.
Smagorinsky, P. ( 1989). The reliability and validity of protocol analysis. Written Communication, 6(4), 463-479.
51.
Stratman, J.F., & Hamp-Lyons, L. (1994). Reactivity in concurrent think-aloud protocols: Issues for research. In P. Smagorinsky (Ed.), Speaking about writing: Reflections on research methodology (pp. 89-111). Thousand Oaks, CA: Sage.
52.
Taylor, C. ( 2000). Clinical problem-solving in nursing: Insights from the literature. Journal of Advanced Nursing, 31, 842-849.
53.
van den Haak, M.J., De Jong, M.D.T., & Schellens, P.J. (2003). Retrospective vs. concurrent think-aloud protocols: Testing the usability of an online library catalogue . Behaviour & Information Technology, 22, 339-351.
54.
van Someren, M.W., Barnard, Y.F., & Sandberg, J.A.C. (1994). The think aloud method. A practical guide to modeling cognitive processes. London : Academic Press.
55.
Yang, S.C. ( 2003). Reconceptualizing think-aloud methodology: Refining the encoding and categorizing techniques via contextualized perspectives. Computers in Human Behavior, 19, 95-115.
