Supporting information use and task accomplishment: What system features do users like and expect?

Abstract

Information systems have been improving in helping users find information. However, they have been less attended to regarding helping searchers in using located information. This research attempts to address the issue of information use by investigating what information systems and features searchers think are helpful in using located information to accomplish information tasks. In all, 32 college students were invited to an information interaction lab, first being interviewed on a recently completed task and then working on a to-be-finished task, both being their real-life tasks of their own choices. Through questionnaires, the study discovered the most favoured existing and expected features helpful for users’ task completion. Users expected convenient citations, note taking in search result pages and being kept on task. Findings in this study have implications on designing search systems that can better support task accomplishment, in addition to returning search results.

Keywords

Information use task accomplishment user expectation

1. Introduction

Information search is more and more indispensable in modern society. Meanwhile, information search is closely connected with other information activities. Take an example of a college student searching in library databases. Conducting the search is typically driven by the goal of finding sources to write a course article. During the searching, the student may save useful articles and take notes. After the search, the student uses the found information to write the article. It can be clearly seen that searching is not an isolated activity but is surrounded by others.

That information search is not a solitary task has been well illustrated in the literature throughout the years. Kuhlthau [1] included in her Information Seeking Process model a ‘presentation’ phase, when search is to complete and the problem is to be solved. Wilson’s [2] model of human information behaviour noted that information processing and use is another important component in the information behaviour cycle. Vakkari [3] addressed the use of sources for task outcome as part of activities in working on complex tasks. ALA’s [4] standards for information literacy clearly include effectively using information to accomplish a specific purpose, besides accessing and evaluating information and so on. In the Interactive Information Retrieval (IIR) research community, tasks involving information use are called ‘work tasks’, in comparison with ‘search tasks’ that deal merely with searching [5,6]. The concept of work task implies that besides information search, information use and work task accomplishment are also what systems can support.

Following this way of thinking, search systems are restrained when they target only at returning results to search queries. Being a ubiquitous, primary and essential tool for handling information tasks, the search system would be more robust if capable of assisting people with their work tasks beyond their search tasks. Järvelin et al. [6] have proposed that the ultimate goal of information interaction tools is to support the performance of work tasks. Particularly, information use (the dotted area in Figure 1) could also be supported by search systems, in addition to information search and information saving that have already been supported. A system proactively helping people use the found information is likely to save user effort and increase user satisfaction.

Figure 1.

The process of information behaviours in finishing a work task.

The use of information can be mental (e.g. learning, generate thoughts) or physical (e.g. generate concrete products such as essays, reports). Järvelin et al. [6] pointed out that it is not enough in a typical complex task scenario to just learn about a topic, but it is important to document the outcome of cognitive processing of information. In the current article, information use is defined as making use of the found and/or saved information to generate a concrete task product, including to document the outcome of cognitive process of information such as making notes. We also define the other two types of information behaviours in Figure 1: information searching is defined as searching and/or finding information; information saving is defined as saving the whole information object (article or webpage). Information use is a process that follows information searching and/or information saving.

Work tasks have been extensively employed in IIR user experiments for studying information search behaviours. However, the majority of previous research has been limited to using work tasks only to provide the context of information search tasks/questions. Consequently, the focus has been mainly on information search. The issue of information use, as well as the relationship between search task and work task, has not been sufficiently addressed. In recent years, a few studies have examined particularly on work task outcome/product as well as the relationship between search tasks and work tasks, which reflect the relationship between information search and information use [7,8]. That being said, research effort has rarely been spent exploring system features that support information use and work task accomplishment.

In the industry, search engines have attempted to provide more functions to help with user tasks than simply returning search results. Google Notebook [9], Yahoo! Search Pad [10] and Microsoft Thumbtack [11] are examples among the approaches to enabling the system’s automatic or the user’s manual saving of search results or notes. Unfortunately, these functions have either been shut down or not attracted much use. Regardless, these functions were primarily information saving related instead of information use techniques. It remains an open area of great potential for search systems to be equipped with features that support information use.

The above illustrated gaps call for research effort. The current study aimed at eliciting what features the users favour and what features they expect the systems to have in order to help them accomplish their tasks. The study comprises two parts: one interviewed users about a recently finished real-life task and the other asked users to work on a to-be-finished real-life task on site. Both parts were guided by questionnaires. Taking the two parts together, the study addresses the following research questions:

RQ1. What existing system features do users think helpful for accomplishing their work tasks?

RQ2. What system features do users expect for helping them accomplish their work tasks?

2. Related literature

Traditionally, search systems’ goal has been to provide relevant search results to user queries, that is, to support information search. However, as introduced in Section 1, search systems’ ultimate goal has been proposed to support work tasks and information use [6]. This target switch in search systems’ goal from information search to information use warrants some additional background and discussion about the relationships between information search and information use, which is presented in the first subsection. Following that, the second subsection reviewing search engines attempts to support tasks through searching. While it is not exactly what the current study is about, that is, supporting information use, this line of efforts is related in that the aim was to support tasks. Similarly, some systems have been going beyond supporting search and attempted to support information saving, which is addressed in the third subsection.

2.1. The relationship between information search and information use

In recent years, some researchers [7,8,12 –14] have attempted to examine user information interaction from a more holistic perspective than focusing on information search only – they examined the relationship between searching and use or searching and the product of work tasks. Findings of these studies help improve the understanding about how search systems can better support information task accomplishment.

An early piece in this line of studies is Huuskonen and Vakkari [15], which examined how a client information system supports social workers’ tasks in child protection. Although the system is not an information search system that the current article focuses on (e.g. search engines or library databases), the study’s focus on information use made it distinct from those consider only information search. The study made recommendations on how the system can more sufficiently support social workers’ tasks, for example, making the system to be simpler and easy to use.

The work by Kumpulainen and Järvelin [13] was among the pioneers that explore approaches to supporting information use in the work task accomplishment context. The researchers found that in information seeking, people used multiple sources and channels, including web search engines, databases and websites. They suggested that information integration is needed for system development. While integrating information with multiple sources and channels is the ultimate goal, one interim and/or parallel strategy could be to focus on single information systems and explore the means by which the systems can assist information use.

Some researchers investigated search behaviours in completing the task products. For example, Hagen et al. [16] examined writers’ searching behaviours while they wrote non-fictional texts. Users were found to often submit recurring anchor queries to avoid losing the main themes or to explore new directions. Fact-checking queries were found to often conclude a writing task, but the number of submitted queries was not a good indicator for task completion. Findings of these types are unique as compared with findings on search behaviours without working on task products and can be indicative of the work task process/stages.

Some researchers investigated the relationship between search performance and work task performance. Vakkari and Huuskonen [8] found that the effort in the search process degraded search precision but improved the essay writing task outcome. The authors suggested that in addition to the precision of search results, the evaluation of search systems should also consider measures for assessing search process and task outcome. Liu and Belkin [7] found that some measures typically used in evaluating search performance, such as the number of viewed useful webpages, the number of issued queries and task completion time, were not correlated with the performance of report writing. Instead, the performance of report writing was found to be affected by users’ knowledge of task topics and their previous experience on the task types. In addition, the report writing performance was positively correlated with users’ effectiveness of finding useful webpages and their time allocated to writing. Liu and Belkin [14] further found that users with different levels of task topic knowledge performed differently in different tasks. Particularly, users with higher level knowledge tended to perform better in the parallel-structured task (subtasks being in parallel with each other) than in the dependent-structured task (some subtasks being dependent upon the completion of others); in comparison, users with lower level knowledge tended to perform better in the dependent-structured task than in the parallel-structured task.

More recently, Vakkari et al. [17] attempted to predict the success of information retrieval from information use behaviours for writing tasks, together with search behaviours such as querying and clicking. Two main indicators for the usefulness of search results were the number of words reused from the clicked search results and the number of pastes. Increased search result usefulness was also found to associate with decreasing effort to edit the pastes for the essay, which was consistent with findings in Vakkari and Huuskonen [8].

A common finding in the above studies is that information search and information use are two different activities whose performances are not even correlated. This strongly endorses the importance of researching and designing systems that support information use, in order for the systems to better help people accomplish work tasks.

2.2. Search systems as tools to support tasks through searching

Although not being exactly what the current research addresses, search engines have actually been used to support work tasks through searching. This subsection reviews related literature along this line.

Brandt et al. [18,19] investigated the roles of online sources in support of programming. Through a controlled lab study and a larger scale web log analysis, Brandt et al. [18] found that programmers used web search and web sources for a number of purposes: just-in-time learning and gaining high-level conceptual knowledge through web tutorials, as a ‘translator’ for exact terminology or syntax through web search and as external memory for complicated syntax that can be accessed as needed. Brandt et al. [19] designed an interface called Blueprint which embedded a task-specific search engine in the code development environment. Evaluation from both a comparative laboratory study and a 3-month usage log analysis found that Blueprint enabled programmers to write better code, find example code faster than with a standard Web browser and changed how and when programmers search the Web.

Fourney [20] pointed out that when relying on web resources to support their work, users interact with systems/objects in three environments: (1) the search engine, (2) retrieved documents and (3) the application’s user interface. Brandt et al.’s [19] approach to supporting programming with search engines touched on the interface aspect.

Some approaches have attempted to assist users’ work tasks by better providing search results or documents. Budzik and Hammond [21] designed a system named Watson that could suggest documents as a searcher writes a article. Golovchinsky et al. [22] used readers’ annotations on documents as queries, and the experiment results received better performance than using queries derived from users’ judgement of document relevance.

Norman [23] noted that search engines have evolved into answer engines that provide answers to queries such as ‘define: embodiment’ and ‘time in Nagoya’ (p. 44). One specific way that search engines are used is to support low-level, language-related tasks, such as spell checking, grammar checking, disambiguation between homonyms, definitions and pronunciations [24]. Through two surveys, one distributed by emails asking respondents to reflect on a recently finished task and one embedded directly in search result pages (SERPs), Fourney et al. [24] collected data with a group of highly educated people who reported having good fluency in both spoken and written English. Their study found that language-related searches were indeed common, accounting for at least 2.7% of queries in their data set. They also found that the convenience of search engines, the richness of the information search engines can provide and people’s familiarity and confidence with search engines contributed to the common use of search engines as a language tool.

The support that search systems provide as above described is either low level, factual or simple tasks for which search engines are generally good at providing answers or purpose-/activity-specific tasks for which a search engine was built in the activity (e.g. programming) interface. It remains unknown how other groups of users than those having been examined (e.g. programmers), and how people in general, want search systems to help with their tasks.

There has been some work aiming at helping users take notes. Ahn et al. [25] designed an interface called TaskSieve that supports task accomplishment. Specifically, it provides a ‘Task Model Notes’ panel on the right side of the search interface, on which users could paste and print notes. Similarly, He et al. [26] designed an interface that provides a shoebox that users can assemble text fragments. These approaches proposed one decade ago were well targeted to help with information use and were found to improve task performance in their evaluation experiments; however, up to date, approaches like these have not been prevailing in search systems in general. Furthermore, it would be helpful to explore what other features would be expected by users for information use and task accomplishment.

2.3. Search engine approaches to saving and collecting information

Leading search engines have developed services that support information activities beyond information search. In 2006, Google released an application called Google Notebook [9], which allows users to save and organise clips of information while conducting research online. It was shut down in 2012, and all user-created Notebooks contents were exported to Google Docs [9]. In 2008, Microsoft launched its online research collection tool, Thumbtack, which was intended as a repository for all the information one collects when surfing the web [11]. In 2009, Yahoo! released a similar function called Search Pad [10,27]. The difference between this and other similar services was that this was designed to automatically save search results: it observes the searchers’ activities, recognises research when they click on various links related to specific topics and starts to automatically save the search results in an online dossier. Despite of all the good intensions, these services have not been used widely.

Regardless of the current status of these services, they do not have the same focus as the current research. Except for the deceased Google Notebook, which allowed users to take their own notes, all other services are basically about information saving and collecting rather than information use. To the best of our knowledge, current commercial search systems do not generally and extensively show features that help searchers use information to create task outcome and accomplish their tasks. One reasonable explanation is that system features supporting task accomplishment may depend, to a large degree, on the specific tasks with specific task requirements. For example, writing an essay is different from finding a restaurant or creating a vacation plan in many aspects, such as the type of information sources, the format of the final products and the writing/creating process. The reliance on specific tasks poses a challenge to search engines that are generic in nature. This also formulates a rationale for the current research to choose a target user group and focus on their main tasks and task types (see also Section 3).

3. Method

To better support the exploratory nature of the research and better engage the participants, the study used real-life tasks instead of experimenter-designed tasks. To keep the tasks relatively manageable from the innumerable real-life task domains and types, the study focused on one user group. College students were selected based on several considerations: (1) they use not only a good amount but also a good variety of online systems; (2) their information-related tasks have highly stereotyped categories connected to the main themes in their everyday life: school work and non-school work, which presumably require different types of search systems; (3) their information behaviours have been well studied, which provides a rich background for possible connections between the current study and the literature and (4) recruitment was convenient with the researchers’ academic affiliation.

3.1. Participants

Thirty-two students (20 females, 12 males) from the first author’s university participated in this study. They were recruited by student email listserv from randomly selected academic programmes, including accounting, biomedical engineering, chemistry, English, journalism and social work. There were 23 undergraduates and 9 graduate students. Their average age was 23 years. In general, they were experienced searchers, with an average of 10.5 years of online searching experience and 2.24 h per day on online searching. Each participant received US$15 upon completing the experiment session.

3.2. Tasks

The study had two parts, each having one task. The first was a recall of a recently finished task. The second was a to-be-finished task that the participant worked on in the study. Each participant was asked to think of one real-life task in the beginning of each part. Both tasks were required to involve information search and importantly have concrete task product/outcome (e.g. a blog, a research article). The tasks could be school, work or life related.

Having these two parts of tasks in the study were based on several considerations. First, the task numbers doubled the approach of having only one task for each participant, considering that it would have been harder to recruit a sample pool of the doubled size. Second, this is more effective and efficient: having only Task 1 type would have lost the actual experience that participants went through in Task 2 (like most of the survey/questionnaire studies); having only Task 2 type would have greatly extended the experiment duration. There could possibly be differences between the results of the two task parts; however, the main focus of the current article is to explore the features that users like and expect, without considering the differences in the two tasks.

Since all tasks were participants’ own tasks, the 64 (32 × 2) tasks in the study were all different from each other. A more detailed description of the tasks and their types is available in Section 4.1.

3.3. Procedure and data collection

Participants were invited individually to an on-campus information interaction lab for the experiment. The lab had a Windows desktop computer, equipped with high-speed cable Internet connection. The computer had two monitors: a main monitor for the participants to search and work on their tasks and a side monitor for experimental questionnaires.

Upon arrival, each participant was first given a article version consent form to sign. Then they completed a background questionnaire on the side monitor. Next, participants were shown the interview questionnaire for Task 1. They were asked to recall a real-life task that they had recently finished and to describe how they finished it. The questionnaire asked open-ended questions about system features that helped them to accomplish the task. They also answered what features they had expected the systems to have. Task 1 took about 20–30 min.

Participants then worked on Task 2. A pre-task questionnaire asked them to find a task that they needed to complete and describe the task. They were then given up to 30 min to work on the task. They were encouraged to work on it as they would do in their life should they not come to the experiment. They were allowed to freely use any sources and tools/software that were accessible and available on the computer. Their interaction with the computer in the task process was recorded by logging software Morae (https://www.techsmith.com/morae.html). After the task, a post-task questionnaire asked them about system features that were helpful completing the task and what features they had expected the systems to have. The whole experiment took about 1 h.

3.4. Data analysis

Data analysis was conducted by coding the participants’ responses to the questions in both tasks’ questionnaires regarding what system features were helpful and what features were expected to finish the tasks. Helpful and expected system features were extracted from all the responses through multiple rounds of coding. The two authors first coded all participants’ responses individually. Reviews and discussions were then conducted until an agreement was reached between the two coders for every response.

4. Results

4.1. An overview of tasks and systems used

As mentioned earlier, in total, there were 64 different tasks (32 participants, 2 tasks each). The tasks fell in two broad categories: (1) scholarly (SC) work and (2) non-scholarly (NS) work.

In general, SC work usually involves serious study or research, and the outcome examples were course articles, essays and so on. For SC work, students most probably use library sources, for example, databases, although they could also use the Internet sources such as Google, Google Books, Google Scholar or Wikipedia, to name a few. NS work is most often non-academic related, such as writing a blog for a personal interest; but it could be academic-related work too, such as editing a photo for a media class or conducting a fundraising project for an internship purpose. For NS work, students do not generally use library sources but they rely heavily on the Internet.

Since participants were college students, it is reasonable to see that about two-thirds of the tasks in the study (68.8%; 44 out of 64) were SC tasks. Table 1 presents the frequencies and examples of the task types, as well as the main systems/sources used.

Table 1.

Tasks: types, frequencies, examples and systems used.

Task type	Freq.	Example	Main systems/sources used
SC	44	Write a 10-page article on solar energy materials with references as one of the course requirements	Library sources: The University library catalogue, the University library databases (e.g. PubMed, JSTOR, EBSCOhost, Social Work Abstract), NIH library site, course site (blackboard.com), class PPT slides The general Internet: Google, Google Books, Google Scholar, Google Maps, Google Reads, YouTube, Wikipedia, Amazon, Pinterest, Pokemon.com, Sas.com, various SAS blogs, Owl Purdue website
NS	20	Write a blog about vegan make-up companies	Google, Google Scholar, Google Books, YouTube, Foodnet.com, Reddit.com, Craigslist, Bing, Pinterest, Wordpress.com, Karmakidsyoga.com, Childlightyoga.com, sources professor suggested, Photoshop, personal blogs
Total	64	–	–

SC: scholarly; NS: non-scholarly.

4.2. Helpful search system features

Table 2 shows search system features and their frequencies identified by the participants as helpful for accomplishing their work tasks. The reported features were about multiple activities: finding, saving (saving the original complete sources such as PDFs, webpages) and using the information (e.g. copy-paste passages, taking notes) to generate task outcomes. Although only a couple of activities were about information use, they were rarely examined in previous studies. It can also be seen that the reported features spread across various aspects: search general feature, search tips, querying, SERP features, information source features and some features in specific systems (e.g. some databases and YouTube).

Table 2.

Helpful search systems and their features.

Activity helpful for	Aspect	Helpful feature	Frequency
Finding	General search feature	(Google) simplicity of search	1
		(Google) speed of search	1
	Search tips	Allowing quotation mark to search exact word(s)	1
	Querying	Query auto-completion	1
	SERP (mainly search engines)	SERP highlighting keywords in the snippets	2
		Option to filter (by subject, resource type, etc.)	2
		SERP distinguish result type (forums) to help filter out unhelpful results	1
		Providing snippets	1
		Returning a large amount of results	1
		More reliable sources being at the top	1
		(Google scholar) identifying journal articles	1
		Access to specific types of references (non-profit)	1
		Having results from .org/.gov which are more credible	1
		The ability to search for abstracts and reviews as well as the original material itself	1
	Information sources	Information source full text available	2
		Peer-reviewed sources available	2
		(Database) articles being more credible	1
Saving	Database	Allowing to save article links in the system	1
		(EBSCOhost) option to email articles	1
	YouTube	(YouTube) ‘share’ function that provides the URL	2
Use	SERP	SERP providing URL for copy/paste	2
	Database	(Side bar) pre-prepared citations	8

SERP: search result page.

As can be seen, while most of the favoured feature had a frequency of 1 or 2, the feature ‘pre-prepared citations’ appeared eight times. This was perhaps because of the great number of SC tasks which typically require references.

4.3. Expected system features

Participants’ expected features mainly fell in three categories: (1) general or unspecified search systems, (2) commercial search engines and (3) library catalogue and databases. These are reported in the following.

4.3.1. Expected features for general or unspecified search systems

As Table 3 shows, except for two features being reported in two tasks, all other expected features were reported in only one task. The low frequency and high variance may indicate that participants brainstormed with their own thoughts for what they expected.

Table 3.

Search systems in general or unspecified: expected features, frequency and current usage status.

Activity helpful for	Aspect	Features	Frequency	Current usage status
Finding	Querying	Spell checking	2	M
		Query recommendation	1	M
	SERP content	Links to alternate but similar information	2	M
		A link to the original source from a review piece	1	R
	SERP presentation	Sorting results by some attributes (e.g. date)	1	M
		Having notification about duplicate articles	1	R
	SERP function	Providing social recommendation (e.g. people who looked at this item/article also looked at these)	1	M
		Narrowing down the results*	1	M
		Tips for how to narrow down results*	1	M
		Bringing back more relevant results*	1	W
		Prioritising search results according to relevance*	1	W
Saving	SERP function	Providing recently visited pages on the side (would not have to bookmark or use the back arrow)	1	R
		Being able to star websites on Google and articles on EBSCOhost to review later without having to save them as a bookmark	1	R
		Saving searches*	1	W
Use	SERP function	Citations being more universally available in systems	1	M
	Others	A way for the system to keep the user on task	1	R
		Enabling users to highlight passages	1	R
		Providing a means to type information or notes	1	R

SERP: search result page.

R: rarely used; M: moderately used; W: widely used.

Features in italic for those too general.

Among the features reported, four of them were about information use, three were about information saving, and all others were about information finding. Of special interest are those features about information use. Although each having a frequency of one, they provided ways for improving the systems to assist information use in the aspects of citations being more universally available, keeping users on task, enabling highlighting passages and providing a way to take notes.

Expected features addressed multiple aspects including querying (spell checking and query recommendation), SERP content (e.g. links to alternate but similar information), SERP presentation (e.g. results being sortable) and SERP functions (e.g. providing social recommendation). Some features about SERP functions (e.g. saving searches) appeared to be too general, which are presented in italic font in Table 4. In addition, three features did not belong to either querying or SERP related: a way for the system to keep the user on task, enabling users to highlight passages and providing a means to type information or notes. These three features were all about information use.

Table 4.

Commercial search engines: expected other features, frequency and current usage status

Activity helpful for	Aspect	Features	Frequency	Current usage status
Finding	Querying	Placing query suggestions on the top of the page	1	R
		Allowing users to search by fields (e.g. title)	1	M
		(Google scholar) having advanced search	1	W
		Providing guide for search tips like how to use * and ‘’	1	M
	SERP content	Eliminating certain source types (e.g. Wikipedia, blogs, Twitter/Facebook account, results behind pay walls, advertisement)	5	R
		Providing accessibility to professional journals, books, full-text articles (Google Scholar), more full pages (GoogleBooks)	4	R
		Results having more scholarly articles and PDFs	1	R
		Returning specific items (e.g. webpages and videos) and information (e.g. time and ease to prepare a food)	3	M
		Having ways of narrowing down search results, especially for credibility check or narrowing by price range	2	M
	SERP presentation	(Google) putting results with more relevant keywords in the snippets higher on the list	1	R
		(Google) ranking forums by dates	1	R
	SERP function	(Google and Bing) having social likeness for search results	1	R
		(Google) having a better way to distinguish between subjects with similar names but different topics (author note: like authority control)	1	R
	Others	A combined search engine that can browse different types of sources in the same tab, but still separate them within the page	1	R
		Highlighting keywords in the passage (in addition to the SERPs) to help locate the relevant passages	1	R
		(Google) advertising their applications and shortcuts (e.g. Google scholar)	1	R
Saving	Others	Bookmarks acting more like Pinterest for browsing through	1	R
		Saving information gathered from various systems into one database to refer to	1	R
Use	Others	Having a way of taking notes on the side of pages (an add on opening a notepad on the side of the page and saving notes in a folder on the top bar of the search engine)	1	R
		A side bar for taking notes	1	R
		Placing a word document into the webpage in the unused space on a webpage	1	R
		Being able to take notes directly from Google and then comprise them into a document	1	R
		Highlighting passages online and having them dropped into a Google doc instantly that you can save to your desktop	1	R

SERP: search result page.

R: rarely used; M: moderately used; W: widely used.

Due to the limitation of personal knowledge, some features reported by the participants as expected features might in fact be available in some systems. Therefore, the features were labelled according to their current status (to the authors’ best knowledge) into several categories: (1) rarely used (R) for those being used by a small number of systems at the most (including features not being used), (2) moderately used (M) for those being used by some but not the majority of systems and (3) widely used (W) for those being used by a great number of systems (including universally used features). Figure 2 illustrates the features for information finding, saving and use, respectively, with different background for different usage status: dark grey for widely used (W), light grey for moderately used (M) and white for rarely used (R).

Figure 2.

Search systems in general or unspecified: expected features for information finding, saving and use (dark grey for widely used, light grey for moderately used and white for rarely used).

4.3.2. Expected features for search engines

Besides general search systems, some features were reported for search engines only (Table 4). There were a total of five features about information use, most of which were about assisting users to take notes.

For querying, users wanted the advanced search option (again, some systems have this feature but not all users know this). One other interesting expected feature was suggested queries being placed on the top of the SERP – for now, leading search engines place them down below the search results.

For SERP-related expectations, participants mentioned a total of five times about eliminating certain source types, including Wikipedia, blogs, Twitter/Facebook account, results behind pay walls and advertisement; four times about providing accessibility to professional journals, books, full-text articles and more full pages in Google Books; three times about returning specific items and information and two times about narrowing search results. These correspond to the limitations in commercial search engines.

Expected features also included the ranking of search results, putting results with more relevant keywords in the snippets higher on the list, and ranking forums by dates. Providing social likeness for the results, as well as differentiating search results with similar names, was also expected.

In addition, participants mentioned a number of expected features beyond querying and SERP. Five of them were about note taking: on the side of the page, having a side bar, placing a Word document in the blank space of SERP, taking notes on Google and highlighting passages and having the system drop them into Google docs and save to desktop. One participant mentioned saving information gathered from various systems into one database to refer to.

Figure 3 illustrates the expected features for information finding, saving and use in commercial systems.

Figure 3.

Commercial search engines: expected features for information finding, saving and use (dark grey for widely used, light grey for moderately used and white for rarely used).

4.3.3. Expected features for library systems/databases

For library databases, there was only one expected feature reported for information use (Table 5), no features were reported for information saving and all other features (13 in total) were about information finding. This could be due partially to the fact that library systems have already had a way to provide citations (the most frequently mentioned features).

Table 5.

Library systems/databases: expected features, frequency and current usage status.

Activity helpful for	Aspect	Features	Frequency	Current usage status
Finding	Querying	(Databases) query auto-completion	1	R
		Guiding users build advanced search syntax on the advanced search page or available as a mouse-over popup	1	R
	SERP content	Having less access restrictions on articles	2	R
		Easier to tell accurately if a library has a full text of the articles	1	M
		Students not taking classes could download e-books from the library	1	M
		(Library) offering access to more resources so that no interlibrary loan is needed	1	R
		Related citations of the topic	1	M
	SERP presentation	(Library site) result list providing more information about the source (e.g. types of articles, publications)	1	M
		Identifying articles having been opened to avoid wasting time looking at the same article	1	R
		E-journal having a link for full access to all volumes	1	M
	Others	Not having to walk through an elaborate process of logging into various sites over and over if I leave them sit for too long while I switch to another tab	1	R
		Pinpointing certain information in the article that was relevant to the task	1	R
		Library databases integrated together instead of being separated	1	M
Use	Others	Providing a citation button allowing to copy and paste the citation	1	M

SERP: search result page.

R: rarely used; M: moderately used; W: widely used.

Expected features again covered multiple aspects, many of which are already in use in commercial search systems such as Google, for example, query auto-completion. It was also the case that some items in library systems were not accessible, hence some expected features were mentioned about having the full-text access to items, for example, less access restrictions on articles. Another expectation about SERP content was to provide related citations of the topics. About SERP presentation, participants expected features such as providing more information about the source (e.g. types of articles, publications), identifying articles having been opened to avoid wasting time looking at the same article and providing a link for full access to all volumes of e-journals.

Beyond querying and SERPs, users expected other functions including library databases being integrated together instead of being separated, providing a button allowing to copy and paste citations more easily, pinpointing certain information in the article that was relevant to the task and keeping users logged on in case of extended period of page inactivity.

Figure 4 illustrates the expected features for information finding and use (no features reported for information saving) with library systems.

Figure 4.

Library systems: expected features for information finding and use (grey for moderately used and white for rarely used; no features widely used).

5. Discussion

Although the study targeted at one user group with a limited number of participants, the findings are fruitful and inspiring and have implications on designing systems that can better support information use and task accomplishment. Several points of interest are discussed in the following.

5.1. Existing and expected system features supporting information use

The majority of helpful existing search system features were about information finding, and only a few were about information use. This indicates that search systems have a long way to go in order to better support information use. Existing system features that support information use were mainly reported for databases. The feature of having pre-prepared citations (in side bar) was identified as a helpful feature for eight times, showing its popularity among users.

Regarding expected search system features, pre-prepared citations were again expected to be available in the systems. Furthermore, it would be even more convenient if the citations could be copy-pasted by clicking a button, as one participant wished. In fact, citations typically follow certain rules, and systems can automatically build citation formats by following the rules.

One other feature being mentioned multiple times was systems enabling users to take notes on the side of the SERPs. Another feature is users being able to highlight passages. These two are alike in the way that users determine the passages are useful after reading, evaluating and thinking. Note taking on SERPs or highlighting passages saves users effort and time from opening another application (e.g. MS Word). It should be noted that note taking of online sources is different from simply saving (e.g. Search Pad, Thumbtack) or copy-pasting the whole resources for saving purposes, and it is also different from note taking on article for recording purposes – note taking of online sources often involves more critical thinking. Not only could people learn in the note taking process, but they also make decisions on what parts are needed and helpful for the task outcome, which fosters information use and task accomplishment.

Another feature was a way for the systems to keep the users on task. One participant (p02) commented that she

started out looking for the PDF of the book and got caught up in looking at research with professors … I am glad that it lead me to do something else productive, but at the same time, I lost sight of the task I had initially set forward to do.

This could be viewed as echoing one finding by Hagen et al. [16], that users submitted recurring queries not to lose the main theme of the task. It is quite obvious that this requires users’ additional cognitive load and effort. Given Liu and Belkin’s [7] findings that task performance was positively correlated with time spent on writing, keeping users on task has the potential to improve users’ task performance.

In summary, system features supporting information use are mainly in three categories: convenient citations, note taking embedded in SERP and keeping users on task. Future system design could consider implementing these features. In addition, as a matter of fact, during the revision of this article, commercial systems like Microsoft Research have introduced better system management systems to better keep users on task [28].

5.2. Innovative system features supporting task accomplishment

Participants identified a number of features that are rarely or even not being used in current search systems, which are quite innovative. These features spread across multiple aspects including querying, SERP features and others and were about all activities of information finding, saving and use.

5.2.1. Innovative features about querying

For querying, one expected feature was placing suggested queries towards the top on SERPs. There may be good rationales for current search engines to place the suggested queries on the bottom part of SERPs, nevertheless, some users liked to have them on the top. No matter in what area(s) of the SERP users prefer to see suggested queries, systems aimed at providing personalised search to various types of users might want to consider putting query suggestions on the top section for those who like them to be there.

Participants also wanted databases to provide query auto-completion and offer help with building advanced search syntax. These expectations indicated that querying in databases are not as easy as that in search engines, which corresponds to the common findings in the literature that search engines are simple and easy to use [29 –31]. Technology wise, library databases want to make search easier for users.

5.2.2. Innovative features about SERPs

For SERPs, the findings showing users’ expectations and desires imply a number of points, as listed and discussed in the following. Some of them are basic rules in interface design [32], and others are also thought provoking. Having both the theoretical and the practical significance, these points are beneficial to the design of search systems that can better support task accomplishment.

Users want access. If they find relevant sources, they want the full text to be available. This is the case for both search engines and library systems. Although providing full-text contents may sometimes involve more issues than interface design (e.g. copyright), access to the full text of located resources is what users anticipate.

Users want the related sources to be together, for example, a link to the original source from a review piece. This echoes the semantic web concept [33], which emphasises the connections between related objects.

Following the above point, however, users want similar items to be differentiated from each other. They want search engines to have a better way distinguishing between subjects with similar names but different topics. This is much alike the concept of authority control [34] in library information organisation.

Users want to save effort. They do not want to open and view duplicate articles/pages or reopen previously or recently visited items.

Users want to have only the item types that they need. If the system returns more types than needed, users want to be able to filter those unwanted out. For example, for SC tasks, users want SC articles and PDFs, and they want to filter out sources from Wikipedia, personal blogs and Twitter/Facebook account.

A related feature to the different types of results is having a combined search engine that can browse different types of sources in the same tab, but still separate them within the page. This reflects that users want things to be organised and easily browsable.

A similar feature to the filtering option was systems ranking time-sensitive sources by dates, for example, for threads in discussion forums, the most recent could be more useful than those dated ones. All these desires for filtering could indicate that users want some sort of control on the SERPs.

5.2.3. Other innovative features

Expected features were reported in other aspects than querying and SERPs. One example is bookmarks acting more like Pinterest.¹ Some expected features are not easy or even feasible to do, but they reflect users’ expectation of convenience. Some examples are as follows: saving information gathered from various systems into one database to refer to, which requires cooperation between systems, and highlighting keywords in the passage to help locate relevant passages, which in fact is not controlled by search systems but by the information resources themselves.

5.3. Existing system features: those unfamiliar to users

As the results show, not all expected features are innovative – some are in fact being used in search systems moderately or even widely. These features were still reported by participants as expected features, which could be due to two reasons, either because the features are used in some but not all systems or because the users lack knowledge of the existing features.

One example demonstrating the first reason is library databases being integrated together instead of being separated. So far, this has been done in some but not all library systems. Those libraries not offering this feature might want to consider implementing it, and those already offering this feature might want to consider better advertising it.

Also, social network elements appeared to be expected features. One is social recommendation, for example, ‘people who looked at this article also looked at these articles’. Another is having social likeness for search results. These have been much used in some domains such as online shopping, but not much in the article finding domain. These features could be considered for personalization purposes.

Some existing features were reported as expected due to the user’s lack of knowledge. For example, Google and Google Scholar were expected by some participants to provide the advanced search option. As a matter of fact, these systems have the advanced search option available. Meanwhile, it is also the fact that Google’s advanced search option is not on its search homepage, and it is not easy to be found. It would be good to make it better known or easily accessible by some users.

5.4. Limitations and future directions

This study was able to explore only one group of users for the systems that they used to accomplish their tasks. This was mainly attributed to the nature of domain and task type dependency of this research. It can be seen from the results that some system features helpful for information use are quite generic and are relatively independent of task types or subject domains, for example, note-taking. However, some features are specific and are more subject domain and task-type-dependent, for example, systems providing citations for SC articles. Future studies may consider other group of users, who also typically have salient task products/outcomes by using found information, such as teachers (course preparation), journalists (news writing), etc.

The current study explored helpful system features reported by the participants in the questionnaires. Future research will examine the logged user–computer interaction data for Task 2 for what features were actually used and compare with the self-reported useful features. Future research will also compare the features reported in the recently completed task (Task 1) and those reported/used in the on-site task (Task 2) and examine the similarities and differences, if any, and if so, why.

Based on the findings and their implications on system design, future research will implement the expected and innovative features into a search system prototype. The prototype will be evaluated regarding its overall usefulness as well as the usefulness of the new features.

6. Conclusion

Through a laboratory user study, this research elicited a variety of existing and expected systems and features that college students thought are helpful for using located information to accomplish their real-life tasks. The most favoured existing search system feature was pre-prepared citations, which was reasonable given that the majority of participants’ tasks were scholarly. Expected features helpful for information use fell in three categories: convenient citations, note taking embedded in SERPs and keeping users on task. Some expected features were in fact being currently available in some systems, indicating a gap between existing system features and users’ awareness. A number of innovative expected features were identified about querying, SERPs and others. Findings in this research have two-fold contributions: not only do they advance our understanding about users’ desires but they also have implications on designing and improving search systems that better support information use and task accomplishment, beyond providing relevant information.

Footnotes

Acknowledgements

We thank Dr. Samantha K. Hastings for her support on the research and Hansan Zamir for his help with data collection.

Author’s Note

The author is now affiliated with University of Texas M.D. Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX 77054, USA.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article was partially sponsored by an ASPIR-I grant from the Office of the Vice President for Research at the University of South Carolina.

ORCID iD

Jingjing Liu

Notes

References

Kuhlthau

. Inside the search process: information seeking from the user’s perspective. J Am Soc Inf Sci Technol 1991; 42: 361–371.

Wilson

. Human information behavior. Inform Sci 2000; 3(2): 49–55.

Vakkari

. Information search processes in complex tasks. In: Proceedings of the 2018 conference on human information interaction and retrieval, New Brunswick, NJ, 11–15 March 2018, p.1. New York: ACM.

ALA. Information literacy competency standards for higher education, http://www.ala.org/Template.cfm?Section=Home&template=/ContentManagement/ContentDisplay.cfm&ContentID=33553#stan (accessed 26 April 2019).

Ingwersen

Järvelin

. The turn: integration of information seeking and retrieval in context. Berlin: Springer, 2005.

Järvelin

Vakkari

Arvola

et al. Task-based information interaction evaluation: the viewpoint of program theory. ACM Trans Inform Syst 2015; 33(1): 1–30.

Liu

Belkin

. Searching vs. writing: factors affecting information use task performance. In: Proceedings of the annual conference of the American society for information science & technology (paper 127), Baltimore, MD, 28–31 October 2012, pp.1–10, https://asistdl.onlinelibrary.wiley.com/doi/pdf/10.1002/meet.14504901127

Vakkari

Huuskonen

. Search effort degrades search output but improves task outcome. J Am Soc Inf Sci Technol 2012; 63(4): 657–670.

Google Notebook, https://en.wikipedia.org/wiki/Google_Notebook (accessed 20 March 2018).

10.

Needleman

. Yahoo Search Pad: an online notebook that watches you, https://www.cnet.com/news/yahoo-search-pad-an-online-notebook-that-watches-you/ (2009, accessed 20 March 2018).

11.

Brown

. Thumbtack: a bookmarking service from Microsoft, https://www.cmswire.com/cms/social-media/thumbtack-a-bookmarking-service-from-microsoft-003665.php (2008, accessed 20 March 2018).

12.

Kumpulainen

Järvelin

. Information interaction in molecular medicine: integrated use of multiple channels. In: Proceedings of the third symposium on information interaction in context, New Brunswick, NJ, 18–21 August 2010, pp.95–104. New York: ACM.

13.

Kumpulainen

Järvelin

. Barriers to task-based information access in molecular medicine. J Am Soc Inf Sci Technol 2012; 63: 86–97.

14.

Liu

Belkin

. Multi-aspect information use task performance: the roles of topic knowledge, task structure, and task stage. In: Proceedings of the annual conference of the American society for information science & technology, Seattle, WA, 31 October–5 November 2014, pp.1–10. New York: ACM.

15.

Huuskonen

Vakkari

. Client information system as an everyday information tool in child protection work. In: Proceedings of the third symposium on information interaction in context, New Brunswick, NJ, 18–21 August 2010, pp.3–12. New York: ACM.

16.

Hagen

Potthast

Völske

et al. How writers search: analyzing the search and writing logs of non-fictional essays. In: Proceedings of the 2016 conference on human information interaction and retrieval, Carrboro, NC, 13–17 March 2016, pp.193–202. New York: ACM.

17.

Vakkari

Völske

Potthast

et al. Predicting retrieval success based on information use for writing tasks. In: Proceedings of the international conference on theory and practice of digital libraries, Porto, 10–13 September 2018, pp.161–173. Berlin: Springer.

18.

Brandt

Guo

Lewenstein

et al. Two studies of opportunistic programming: interleaving web foraging, learning, and writing code. In: Proceedings of the SIGCHI conference on human factors in computing systems, Boston, MA, 4–9 April 2009, pp.1589–1598. New York: ACM.

19.

Brandt

Dontcheva

Weskamp

et al. Example-centric programming: integrating web search into the development environment. In: Proceedings of the SIGCHI conference on human factors in computing systems, Atlanta, GA, 10–15 April 2010, pp.513–522. New York: ACM.

20.

Fourney

. Web search, web tutorials & software applications: characterizing and supporting the coordinated use of online resources for performing work in feature-rich software. PhD Dissertation, University of Waterloo, ON, Canada, 2015.

21.

Budzik

Hammond

. User interactions with everyday applications as context for just-in-time information access. In: Proceedings of the 5th international conference on intelligent user interfaces, New Orleans, LA, 9–12 January 2000, pp.44–51. New York: ACM.

22.

Golovchinsky

Price

Schilit

. From reading to retrieval: freeform ink annotations as queries. In: Proceedings of the 22nd annual international ACM SIGIR conference on research and development in information retrieval, Berkeley, CA, 15–19 August 1999, pp.19–25. New York: ACM.

23.

Norman

. The next UI breakthrough: command lines. Interactions 2007; 14(3): 44–45.

24.

Fourney

Morris

White

. Web search as a linguistic tool. In: Proceedings of the 26th international conference on world wide web, Perth, WA, Australia, 3–7 April 2017, pp.549–557. New York: ACM.

25.

Ahn

Brusilovsky

et al. Personalized web exploration with task models. In: Proceedings of the 17th international conference on world wide web, Beijing, China, 21–25 April 2008, pp.1–10. New York: ACM.

26.

Brusilovsky

Ahn

et al. An evaluation of adaptive filtering in the context of realistic task-based information exploration. Inform Process Manage 2008; 44: 511–533.

27.

Donato

Bonchi

Chi

et al. Do you want to take notes? In: Proceedings of the 19th international conference on world wide web, Raleigh, NC, 26–30 April 2010, pp.321–330. New York: ACM.

28.

Microsoft 365 Team. Keep on tasks with easy-to-use task management software, 2019, https://www.microsoft.com/en-us/microsoft-365/growth-center/resources/keep-on-task-with-easy-to-use-task-management-software (accessed 20 January 2020).

29.

De Rosa

Cantrell

Cellentani

et al. Perceptions of libraries and information resources: a report to the OCLC membership. Dublin, OH: Online Computer Library Center (OCLC), 2005.

30.

Dervin

Reinhard

Adamson

et al. Sense-making the information confluence: the whys and hows of college and university user satisficing of information needs. Phase I: project overview, the Three-Field Dialogue project, and state-of-the-art reviews. Report on National Leadership Grant LG-02-03-0062-03, to Institute of Museum and Library Services, 2006. Washington, DC: School of Communication; Columbus, OH: The Ohio State University.

31.

Georgas

Google vs. the library: student preferences and perceptions when doing research using Google and a federated search tool. Libr Acad 2013; 13(2): 165–185.

32.

Nielsen

. Usability 101: introduction to usability. Nielsen Norman Group, https://www.nngroup.com/articles/usability-101-introduction-to-usability/ (2016, accessed 20 March 2018).

33.

Berners-Lee

Hendler

Lassila

. The semantic web. Sci Am 2001; 284(5): 29–37.

34.

Taylor

. Research and theoretical considerations in authority control. Catalog Classif Quart 1989; 9(3): 29–56.