Enhancing the Performance of Interface Evaluators Using Non-Empirical Usability Methods

Heuristic Evaluation has been shown to be a quick cost-effective methodology that can lead to early identification of many of the same user interface errors as laboratory usability studies. In this paper, we describe a method designed to enhance the performance of expert, system developer, and non-expert evaluators. The evaluators most proficient at Heuristic Evaluation are Human-Factors Experts (Desurvire, Lawrence and Atwood, 1991; Desurvire, Kondziela and Atwood, 1992; Jeffries, Miller, Wharton and Uyeda, 1991) and most notably, “double experts” (Nielsen, 1992). Similar results were obtained for the Cognitive Walkthrough developed by Lewis, Polson, Wharton and Rieman, 1990 (Desurvire, et al., 1992; Jeffries, et al., 1991). We were interested in whether a non-empirical method could be developed in which evaluators other than Human Factors Experts can perform nearly as well as Experts.

Desurvire, et al. (1992) found that Heuristic Evaluation and Cognitive Walkthroughs not only predicted problems observed in laboratory studies but also encouraged evaluators to suggest improvements. In addition, non-empirical methods stimulated evaluators to point out problems that would be likely to occur in actual use, but would not be observed in laboratory studies. We were interested in expanding this finding by developing a method that encouraged a broader scope of thinking, and thus a broader evaluation. In this paper, we describe the method Programmed Amplification of Valuable Experts (PAVE) and how it enhanced the performance of System Developers and Non-Human-Factors-Expert evaluators. Future work is discussed in which real users in the field will be compared to these results.