Using Eye Tracking to Evaluate Alternative Search Results Interfaces

Abstract

Surveys have shown that 75% of users get frustrated with search engines and only 21% find relevant information. Inability to find relevant results can be partially attributed to cluttered results pages and failure in constructing Boolean queries. This research used sixteen subjects to evaluate two types of search results interfaces using four tasks while measuring performance and studying their ocular behavior using a Tobii 1750 eye-tracker. The two interfaces used were list interface, commonly seen on many search engines and a tabular interface presenting information in discrete categories or elements of the result's abstract. Quantitative comparisons of two interfaces are made on performance metrics such as time and errors, process metrics such as fixation durations, number of fixations, and eye movement transitions from one element or category of the abstract. Subjective data was collected through post-task and post-test questionnaires. The results did not show any significant difference in performance between the two interfaces, however, eye movements analysis provide some insights into importance of search result's abstract elements such as title, summary, and URL of the interface while searching.

Get full access to this article

View all access options for this article.

References

AllTheWeb. (2000). Retrieved on 09/26/2004 from http://www.alltheweb.com .

Bandos

Resnick

M. L

. (2004). Improving user search with embedded Boolean search hints. Proceedings of Human Factors Ergonomics Society 48th Annual Meeting, New Orleans, LA, USA, 1523–1527.

Broder

(2002). A taxonomy of web search. ACM SIGIR Forum, 36(2): 3–10.

Drori

(2003). How to display search results in digital libraries-user study. Proceedings of New Developments in Digital Libraries, 13–28.

Duchowski

A. T.

(2002). A Breadth-First Survey of Eye Tracking Applications. Behavior Research Methods. Instruments. & Computers (BRMIC). 34(4): 55–470.

Dumais

Cutrell

Chen

(2001). Optimizing Search by Showing Results in Context. Proceedings of ACM CHI Conference, 3(1): 277–284.

Fallow

(2005). Internet searchers are confident, satisfied and trusting - but they are also unaware and naive. Pew / Internet and American Life Project. Retrieved February 2005, from http://www.pewinternet.org/pdfs/PrPSearchengineusers.pdf Forrester report. (October 2000). Retrieved on 09/26/ 2004 from http://www.forrester.com .

Goldberg

J. H.

Kotval

X. P.

(1999). Computer interface evaluation using eye movements: Methods and constructs. International Journal of Industrial Ergonomics, 24, 631–645.

Granka

Hembrooke

Gay

Feunser

(2004). Eye-Tracking Anaysis of User Behavior in WWW Search. Proceedings of the 27th annual international conferenceon Research and development in information retrieval, 478–479.

10.

Klockner

Wirschum

Jameson

(2004). Depth and Breadth-First Processing of Search Results Lists. Proceedings of ACM CHI Late breaking Results Poster, 1539.

11.

Kaki

(2005). Findex: Search Result Categories Help Users When Document Ranking Fails. Proceedings of ACM CHI Conference. 131–140.

12.

RealNames. (2000). Retrieved on 09/26/2004 from http://www.realnames.com .

13.

Resnick

M. L.

Maldonado

C. A.

Santos

J. M.

Lergier

(2001). Modeling On-line Search Behavior Using Alternative Output Structures. Proceedings of the Human Factors and Ergonomics Society 45th Annual Conference, Minneapolis, MN, USA, 1166–1171.

14.

Salogarvi

Kojo

Jaana

Kaski

(2003). Can relevance be inferred from eye movements in information retrieval? Proceedings of the Workshop on Self-organizing Maps, Hibikino, Kitakyushu, Japan, 261–266.

15.

WebTop. (2000). Retrived on 09/26/2005 from http://searchenginewatch.com/sereport/article.php/2163451 .