This paper presents a user-centred design and evaluation methodology for ensuring the usability of IR interfaces. The methodology is based on sequentially performing: a competitive analysis, user task analysis, heuristic evaluation, formative evaluation and a summative comparative evaluation. These techniques are described, and their application to iteratively design a prototype IR interface, which was then evaluated, is described. After each round of testing, the prototype was modified as needed. The user-centred methodology had a major impact in improving the interface. Results from the summative comparative evaluation suggest that users’ performance improved significantly in our prototype interface compared with a similar competitive system. They were also more satisfied with the prototype design. This methodology provides a starting point for techniques that let IR researchers and practitioners design better IR interfaces that are both easy to learn to use and remember. The paper concludes with some principles of interface design for IR systems.
Ahmed, S. M. Z. , McKnight, C. and Oppenheim, C. (2004) ‘A Study of Users’ Performance and Satisfaction with the Web of Science IR Interface’ , Journal of Information Science30 (5): 459-468 .
2.
Ahmed, S. M. Z. , McKnight, C. and Oppenheim, C. (2005) ‘A Study of Learning and Retention with a Web-based IR Interface’ , Journal of Librarianship and Information Science37 (1): 7-16 .
3.
Bailey, G. (1993) ‘Iterative Methodology and Designer Training in Human-computer Interface Design’ , in INTERCHI ’93: Conference Proceedings in Human Factors in Computing Systems, 24-29 April, Amsterdam, The Netherlands, pp. 198-204 . New York: ACM.
4.
Baldonado M. Q. W. (2000) ‘A User-centered Interface for Information Exploration in a Heterogeneous Digital Library’ , Journal of the American Society for Information Science51 (3): 297-310 .
5.
Barker, A. L. (1997) ‘DataStar Web: Living up to the Hype? An Evaluation of the Interface and Search System’ , in Online Information 97: Proceedings of the 21st International Online Information Meeting, 9-11 December, London, pp. 213-222 . Oxford: Learned Information.
6.
Chin, J. P. , Diehl, V. A. and Norman, K. L. (1988) ‘Development of an Instrument Measuring User Satisfaction of the Human-computer Interface’ , in CHI ’88: Proceedings of the Conference on Human Factors in Computing Systems, 15-19 May, Washington, DC, pp. 213-218 . New York: ACM.
7.
Cogdill, K. (1999) MEDLINEplus Interface Evaluation: Final Report. Unpublished Report, Human-Computer Interaction Lab (HCIL), University of Maryland , URL (consulted January 2006): http://www.wam.umd.edu/%7Ekcogdill/medlineplus.htm
8.
Cuomo, D. L. and Bowen, C. D. (1994) ‘Understanding Usability Issues Addressed by Three User-system Interface Evaluation Techniques’ , Interacting with Computers6 (1): 86-108 .
9.
Desurvire, H. , Kondziela, J. and Atwood, M. (1992) ‘What is Gained and Lost when Using Evaluation Methods other than Empirical Testing’ , in A. Monk , D. Diaper and M. D. Harrison (eds) Proceeding of the HCI ’92: People and Computers VII, 15-18 September, York, UK, pp. 89-102 . Cambridge: Cambridge University Press.
10.
Doubleday, A. , Ryan, M. , Springett, M. and Sutcliffe, A. (1997) ‘A Comparison of Usability Techniques for Evaluating Design’ , in DIS ’97: Proceedings of the Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, 18-20 August, Amsterdam, The Netherlands, pp. 101-110 . New York: ACM.
11.
Fenichel, C. H. (1981) ‘Online Searching Measures that Discriminate among Users with Different Types of Experience’ , Journal of the American Society for Information Science32 (1): 23-32 .
12.
Frants, V. I. , Shapiro, J. , Taksa, K. and Voiskunskii, V. G. (1999) ‘Boolean Search: Current State and Perspectives’ , Journal of the American Society for Information Science50 (1): 86-95 .
13.
Hsieh-Yee, I. (1993) ‘Effects on Search Experience and Subject Knowledge on the Search Tactics of Novice and Experienced Searchers’ , Journal of the American Society for Information Science44 (3): 161-174 .
14.
Jeffries, R. , Miller, J. R. , Wharton, C. and Uyeda, K. M. (1991) ‘User Interface Evaluation in the Real World: A Comparison of Four Techniques’ , in CHI ’91: Proceedings of the Conference on Human Factors in Computing Systems - Reaching through Technology, 27 April-2 May, New Orleans, LA, pp. 119-124 . New York: ACM.
15.
Marchionini, G. and Komlodi, A. (1998) ‘Design of Interfaces for Information Seeking’ , Annual Review of Information Science and Technology38, pp. 89-130 , Medford, NJ: Information Today .
16.
Meyyappan, N. , Chowdhury, G. G. and Foo, S. (2001) ‘Use of a Digital Work Environment Prototype to Create a User-centred University Digital Library’ , Journal of Information Science27 (4): 249-264 .
17.
Meyyappan, N. , Foo, S. and Chowdhury, G. G. (2004) ‘Design and Evaluation of a Task-based Digital Library for the Academic Community’ , Journal of Documentation60 (4): 449-475 .
18.
Morgan, E. L. (1999) ‘MyLibrary@NCState: The Implementation of a User-centered, Customizable Interface to a Library’s Collection of Information Resources’ , in SIGIR’99: 22nd International Conference on Research and Development in Information Retrieval, 15-19 August, University of California, Berkeley, URL (consulted January 2006): http://dewey.library.nd.edu/mylibrary/sigir-99/
19.
Mulhem, P. and Nigay, L. (1996) ‘Interactive Information Retrieval Systems: From User Centered Interface Design to Software Design’ , in Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 18-22 August, Zurich, Switzerland, pp. 326-334 . New York: ACM.
20.
Nielsen, J. (1990) ‘Paper Versus Computer Implementations as Mockup Scenarios for Heuristic Evaluation’ , in IFIP INTERACT ’90: Proceedings of the 3rd International Conference on Human-Computer Interaction, 27-31 August, Cambridge, UK, pp. 315-320 . Amsterdam: North Holland.
21.
Nielsen, J. (1992) ‘Finding Usability Problems through Heuristic Evaluation’ , in CHI ’92: Proceedings of the Conference on Human Factors in Computing Systems, 3-7 May, Monterey, CA, pp. 373-380 . New York: ACM.
22.
Nielsen, J. (1993) ‘Iterative User Interface Design’ , IEEE Computer26 (11): 32-41 .
23.
Nielsen, J. (1994) ‘Heuristic Evaluation’, in J. Nielsen and R. L. Mack (eds) Usability Inspection Methods, pp. 25-62. New York: John Wiley .
24.
Peng, L. K. , Ramaiah, C. K. and Foo, S. (2004) ‘Heuristic-based User Interface Evaluation at Nanyang Technological University in Singapore’ , Program: Electronic Library and Information Systems38 (1): 42-59 .
25.
Penhale, S. J. and Taylor, N. (1986) ‘Integrating End-user Searching into a Bibliographic Instruction Program’ , RQ27 (2): 212-220 .
26.
Petrelli, D. , Hansen, P. , Beaulieu, M. , Sanderson, M. , Demetriou, G. and Herring, P. (2004) ‘Observing Users - Designing Clarity: A Case Study on the User-centred Design of a Cross-language Retrieval System’ , Journal of the American Society for Information Science and Technology55 (10): 923-934 .
27.
Rudd, J. , Stern, K. and Isensee, S. (1996) ‘Low vs. High-fidelity Prototyping Debate’ , Interactions3 (6): 76-85 .
28.
Shneiderman, B. (1998) Designing User Interface: Strategies for Effective Human-Computer Interaction. Reading, MA: Addison-Wesley.
29.
Spyridakis, J. H. (1992) ‘Conducting Research in Technical Communication: The Application of True Experimental Design’ , Technical Communication39 (4): 607-624 .
30.
Sullivan, M. V. , Borgman, C. L. and Wippern, D. (1990) ‘End-user, Mediated Searches, and Front-end Assistance Programs on Dialog: A Comparison of Learning, Performance and Satisfaction’ , Journal of the American Society for Information Science41 (1): 27-42 .
31.
Sutcliffe, A. , Ennis, M. and Watkinson, S. J. (2000) ‘Empirical Studies of End-user Information Searching’ , Journal of the American Society for Information Science51 (13): 1211-1231 .
32.
Van House, N. A. , Butler, M. H. , Ogle, V. and Schiff, L. (1996) ‘A User-centered Iterative Design for Digital Libraries: The Cypress Experience’ , D-Lib Magazine, URL (consulted January 2006): http://www.dlib.org/dlib/february96/02vanhouse.html
33.
Virzi, R. A. (1992) ‘Refining the Test Phase of Usability Evaluation: How many Subjects is Enough?’ , Human Factors34 (4): 457-468 .
34.
Virzi, R. A. , Sokolov, J. L. and Karis, D. (1996) ‘Usability Problems Identification Using both Low- and High- fidelity Prototypes’ , in CHI ’96: Proceedings of the conference on Human Factors in Computing Systems, 13-18 April, Vancouver, Canada, pp. 236-243 . New York: ACM.
35.
Zhang, X. and Chignell, M. (2001) ‘Assessment of the Effects of User Characteristics on Mental Models of Information Retrieval Systems’ , Journal of the American Society for Information Science52 (5): 445-459 .
