Searching the web is a common activity of web users. English and non-English speakers utilize international or local search engines so as to satisfy their information needs. Most of the attempts at evaluation of search engines focus on English queries and on English document collections. In this paper an evaluation methodology is presented and the capabilities of international and local web retrieval systems using Greek queries are evaluated based on this method. We aim at identifying difficulties and knowledge requirements when using a Greek supporting search engine. The importance of interface localization and the effects of standard information retrieval techniques such as case insensitivity, stopword removal and simple stemming are studied in international and local search engines. The evaluation methodology is applicable to other non-English natural languages as well.
Global Reach, Global Internet Statistics (by Language) (2004). Available at: www.global-reach.biz/globstats (accessed 31 July 2006).
2.
R. Baeza-Yates and B. Ribeiro-Neto (eds), Modern Information Retrieval (Addison Wesley/ACM Press, New York , 1999).
3.
C.W. Cleverdon , J. Mills and E.M. Keen, An Inquiry in Testing of Information Retrieval Systems (Aslib Cranfield Research Project, College of Aeronautics , Cranfield, 1966).
4.
S.E. Robertson , The parameter description of retrieval systems: overall measures, Journal of Documentation25 (1969) 93—107.
5.
J. Gwizdka and M. Chignell, Towards Information Retrieval Measures for Evaluation of Web Search Engines (1999). Available at: www.imedia.mie.utoronto.ca/~jacekg/pubs.html (accessed 31 July 2006). [Unpublished manuscript, 1999]
6.
H. Chu and M. Rosenthal, Search engines for the World Wide Web: a comparative study and evaluation methodology. In: S. Hardin (ed.), Proceedings of the Annual Conference for the American Society for Information Science, Baltimore, MD, 1996 (Information Today, Medford, NJ, 1996) 127—35.
7.
M. Gordon and P. Pathak, Finding information on the World Wide Web: the retrieval effectiveness of search engines, Information Processing and Management35(2) (1999) 141—80.
8.
M.D. Dunlop, Time, relevance and interaction modelling for information retrieval. In: N.J. Belkin et al. (eds), Proceedings of ACM/SIGIR, Philadelphia, PA, 1997, (ACM Press, 1997) 206—13.
9.
L. Su, H. Chen and X. Dong, Evaluation of Web-based search engines from an end-user's perspective: a pilot study. In: Proceedings of the 61st Annual Meeting of the American Society for Information Science, Pittsburgh, PA., 1998, 348—61.
10.
I. Hsieh-Yee, The retrieval power of selected search engines: how well do they address general reference questions and subject questions?The Reference Librarian28/60 ( 1998) 27—47.
11.
C. Oppenheim , A. Morris and C. McKnight, The evaluation of WWW search engines , Journal of Documentation56(1) (2000) 71—90.
12.
M. Courtois, W. Baer and M. Stark, Cool tools for searching the Web: a performance evaluation, Online19(6) (1995) 14—32.
13.
W. Chung, Y. Zhang, Z. Huang, G. Wang, T. Ong and H. Chen, Internet searching and browsing in a multilingual world: an experiment on the Chinese Business Intelligence portal (CBizPort), Journal of the American Society for Information Science and Technology55(9) (2004) 818—31.
14.
M. Sroka , Web search engines for Polish information retrieval: questions of search capabilities and retrieval performance, International Information & Library Research32(2) (2000) 87—98.
15.
H. Moukdad, Lost in cyberspace: how do search engines handle Arabic queries? In: Proceedings of the 32nd Annual Conference of the Canadian Association for Information Science, Winnipeg, 2004 . Available at: www.cais-acsi.ca/proceedings/2004/moukdad_2004.pdf (accessed 31 July 2006).
16.
J. Bar-Ilan and T. Gutman, How do search engines respond to some non-English queries?Journal of Information Science31(1) (2005) 13—28.
17.
H. Moukdad and H. Cui, How do search engines handle Chinese queries?Webology2(3) (2005) Article 17. Available at: www.Webology.ir/2005/v2n3/a17.html (accessed 31 July 2006).
18.
F. Lazarinis , Evaluating the searching capabilities of Greek e-commerce Web sites, Online Information Review Journal (forthcoming).
19.
F. Lazarinis , Do search engines understand Greek or do user requests “sound Greek” to them? In: M. Beigbeder and W.G. Yee (eds), Open Source Web Information Retrieval Workshop in conjunction with IEEE/WIC/ACM International Conference on Web Intelligence & Intelligent Agent Technology, Compiegne, France, 19 September 2005 (IEEE, 2005) 43—6.
20.
B. Jansen and A. Spink, An analysis of Web searching by European AlltheWeb.com users, Information Processing and Management41(2) ( 2005) 361—81.
21.
Cross-Language Evaluation Forum (CLEF). Available at: www.clef-campaign.org/ (accessed 20 February 2007).
22.
C. Silverstein , M. Henzinger, J. Marais and M. Moricz, Analysis of a Very Large Alta Vista Query Log, Technical Report 1998—014 (COMPAQ Systems Research Center, Palo Alto, CA, 1998).
23.
D. Sullivan, Nielsen NetRatings: Search Engine Ratings (2006). Available at: http://searchenginewatch.com/reports/article.php/2156451 (accessed 31 July 2006).
24.
F. Lazarinis , Searching the Greek Web space: an instructional approach for effective utilization of search engines, Journal of Science Education7 (2006) 28—30. [Special Issue II]
25.
T.Z. Kalamboukis , Suffix stripping with modern Greek, Program29(3) (1995) 313—21.
26.
G. Ntais, Development of a stemmer for the Greek language (MSc Thesis, Stockholm University, 2006). Available at: www.dsv.su.se/~hercules/papers/Ntais_greek_stemmer_thesis_final.pdf (accessed 31 July 2006).
27.
J. Savoy , A stemming procedure and stopword list for general French corpora, Journal of the American Society for Information Science50(10) (1999) 944—52.
28.
S. Tomlinson , Finnish, Portuguese and Russian retrieval with Hummingbird SearchServerTM, CLEF 2004, 221—32. Available at: http://clef.iei.pi.cnr.it/2004/working_notes/WorkingNotes2004/21.pdf (accessed 20 January 2007).
29.
F. Zou, F.L. Wang, X. Deng and S. Han, Automatic identification of Chinese stop words, Research on ComputingScience18 (2006) 151—62. [Special issue on Advances in Natural Language Processing]
