Verbal protocols have been used for many years in different research contexts, but there still is no clear consensus about the validity of the technique and methods for maximizing validity in an applied setting; how to standardize the collection and analysis of protocols; and last but certainly not least, whether the resulting data is worth the effort.
This panel discussion is a companion to a symposium at this conference which presents empirical studies and human factors applications of verbal protocol techniques. The panel will focus in more depth on issues raised in that earlier session, with the goal of providing guidance for practical applications of the technique.
Get full access to this article
View all access options for this article.
References
1.
BhaskarR.SimonH.A. (1977). Problem solving in semantically rich domains: An example from engineering thermodynamics. Cognitive Science, 1, 193–215.
2.
EricssonK.A.SimonH.A. (1980). Verbal reports as data. Psychological Review, 87, 215–251.
3.
EricssonK.A.SimonH.A. (1984). Protocol analysis.Cambridge: MIT Press.
4.
FisherC. (1987). Advancing the study of programming with computer-aided protocol analysis. In OlsonG.MSheppardS.SolowayE. (Eds.), Empirical studies of programmers: Second workshop (198–216). Norwood, N.J.: Ablex.
5.
GugertyL.OlsonG.M. (1986). Comprehension differences in debugging by skilled and novice programmers. In SolowayE.IyengarS. (Eds.), Empirical studies of programmers (13–27). Norwood, N.J.: Ablex.
6.
JeffriesR. (1982, March) A comparison of the debugging behavior of expert and novice programmers. Paper presented at the Conference of the American Educational Research Association.
7.
KatzI.R. (1988). Transfer of knowledge in programming. Unpublished doctoral dissertation, Carnegie-Mellon University, Pittsburgh, PA.
8.
KesslerC.M.AndersonJ.R. (1986). A model of novice debugging in LISP. In SolowayE.IyengarS. (Eds.), Empirical studies of programmers (198–212). Norwood, N.J.: Ablex.
9.
KlahrD.CarverS.M. (1988). Cognitive objectives in a LOGO debugging curriculum: Instruction, learning, and transfer. Cognitive Psychology, 20, 362–404.
10.
LuekeE.PagereyP.D.BrownC.R. (1987). User requirements gathering through verbal protocol analysis. In SalvendyG. (Ed.), Cognitive engineering in the design of human-computer interaction and expert Systems (523–528). Amsterdam: Elsevier.
11.
MacKayW.E. (1989). EVA: An experimental video annotator for symbolic analysis of video data. SIGCHI Bulletin, 21, 68–71.
12.
PitmanK.M. (1985). CREF: An editing facility for managing structured text (A.I. Memo No. 829). Massachusetts Institute of Technology, Artificial Intelligence Laboratory, Cambridge, MA.
13.
PoltrockS.E.NasrM.G. (1989). Protocol analysis: A tool for analyzing human-computer interactions. (Technical Report Number ACT-HI-186–89) Microelectronics and Computer Technology Corporation, Austin, TX.
14.
SandersonP.M.JamesJ.M. (1990). MacSHAPA: A tool for exploring and analyzing complex sequential data. (Technical Report in preparation) Engineering Psychology Research Laboratory, Department of Mechanical and Industrial Engineering, University of Illinois at Urbana-Champaign.
15.
SandersonP.M.JamesJ.M.SeidlerK.S. (1989). SHAPA: An interactive software environment for protocol analysis. Ergonomics, 32, 1271–1302.
16.
WatermanD.A.NewellA. (1971). Protocol analysis as a task for artificial intelligence. Artificial Intelligence, 2, 285–318.
17.
WatermanD.A.NewellA. (1972). Preliminary results with a system for automatic protocol analysis. (C.I.P. Working Paper No. 211). Pittsburgh, PA: Carnegie-Mellon University.
18.
WatermanD.A.NewellA. (1973). PAS-II: An interactive task-free version of an automatic protocol analysis system. In Proceedings of the Third ITCAI (pp. 431–445). Menlo Park, CA: Stanford Research Institute.
