Abstract
Online retailers are providing a large amount of products over the Internet for potential customers. Given the opportunity of accessing vast amounts of products online, customers usually encounter difficulties to choose the right product or service for themselves. Obtaining advice from the Internet is both time consuming and most of the time unreliable. Therefore, some kind of intelligent software is needed to act on behalf of customers in such situations. Recommender agents are intelligent software providing easily accessible, high-quality recommendations for online consumers. They either track online customer behaviour implicitly or obtain information from the customer explicitly and provide the products or services in which the customer might be interested. By utilizing such systems, online retailers not only increase their sales but also assist their customers in finding the products or services they seek. This study assessed the influence of knowledge-based recommender agents on the online-consumer decision-making process. Shopping duration, purchase of desired item, effort spent in searching for the desired product and the decision quality of online consumers were assessed by exposing the participants to a knowledge-based recommender system which has been integrated into one of the online shopping systems developed in the scope of this study. Only objective measures have been utilized in this research; that is, shopping system log data has been used to measure the influence of recommender agents on the consumer decision-making process. Study findings have shown that knowledge-based recommender agents improve the consumer decision-making process by reducing the shopping duration and effort spent in searching for suitable products. Also, it was found that decision quality and the number of consumers who purchase the desired item increase with their use of such systems. The results of this study provide additional proof of the potential benefits of integrating such systems into online web stores.
Keywords
Knowledge-based recommender agents help to improve the online consumer decision-making process
Introduction
The number of Internet users worldwide has been increasing every year. Merchants, being aware of potential customers over the Internet, have started to provide more and more goods over the Internet. When customers are given the opportunity of accessing vast amounts of products over the Internet, they usually encounter difficulties in choosing the right product or service among so many different options. Furthermore, since they are shopping online, they have no chance to ask for advice from a sales representative about the products and services. Without any kind of professional advice, online customers usually encounter difficulties in selecting products that fulfill their needs. Customers who try to obtain advice from the Internet have realized that obtaining information from the web is both time-consuming and unreliable most of the time. Therefore, some kind of intelligent software is needed to act on behalf of the customer in such situations. There exist recommender agents which exactly fulfill this need on online trading. Recommender agents (RAs) are intelligent software which provide easily accessible, high-quality recommendations for the online consumer. Recommender agents either track online customer behavior implicitly or obtain information from the customer explicitly and provide the products or services in which the customer might be interested (Jannach et al., 2011). By utilizing such systems, online retailers increase their sales and assist their customers to find the products that match their criteria. These intelligent agents not only improve the customers’ decision-making process by the reducing amount of information burden and complexity in searching, but also increase the consumers’ decision quality by suggesting products and services in which the customer might be interested (Chiasson et al., 2002). Consumers’ decision effort in online shopping context is usually measured by the time spent for decision-making and the extent of the product search (Xiao and Benbasat, 2007). Previous studies showed that recommender agents reduce the time required for customers to find suitable products and make a purchase decision (Pedersen, 2000; Hostler et al., 2005). In addition, such intelligent systems narrow the limits of product search by decreasing the total number of products that customers will analyze (Dellaert and Haubl, 2005). By integrating such intelligent software into their online stores, merchants shift the tedious job of screening, filtering and sorting large amounts of items from the user to the recommender agent, and customers use the time saved to make quality decisions.
One of the most common types of recommender agents is a knowledge-based one. The objective of this study is to assess the impact of knowledge-based recommender agents on the online consumer decision-making process. In this study, only objective measures have been utilized in order to find answers to research questions. That is, developed shopping systems tracked and logged participants’ behavior during a simulated shopping session and the log data was then used to measure the influence of recommender agents on the consumer decision-making process.
The next section presents the literature review on recommender agents. Then the theoretical framework and research methodology are given. Statistical results and discussions are given in the final sections of this paper.
Literature review on recommender agents
Information overload is a common issue in the modern information society; therefore some kind of intelligent software is required to provide the most relevant data according to online users’ needs. Recommender agents are intelligent software which collects information from users either directly or indirectly and recommends items based on customers’ usage patterns, choices, priorities and needs. Recommender agents aim to support and guide customers during online decision-making processes by providing easily accessible, high quality recommendations. There are different types of recommender agents; collaborative, content-based, knowledge-based and hybrid agents are those most commonly utilized by online retailers.
In collaborative systems, items are suggested to customers based on the ratings given by other users with similar tastes (Hostler et al., 2012). The idea of collaborative filtering is actually to automate word-of-mouth recommendations (Shardan and Maes, 1995). Such systems use statistical formulae to locate customers with similar tastes. For the system to be effective, customers need to rate a few items they have had experience with or for which they have a purchasing history. By using these ratings or purchasing histories, collaborative filtering finds a reference customer and recommends items based on the reference customer’s rating scores. If there are few ratings per item or if there are few ratings per user then the system cannot provide useful recommendations (Schafer et al., 2007). In collaborative filtering more and more user ratings are required as an input in order to receive useful recommendations. In this system, recommended items are based on the users’ evaluation of them and those evaluations show the quality of the products, so quality can be considered in such systems (Funakoshi and Ohguru, 2000).
Content-based systems consider product features and customer profiles while suggesting products to the customers. Attributes and specifications of the items rated by the customers are used to build individual interest profiles and these profiles are utilized in generating product recommendations for the customers (Mladenic, 1999). In other words, during the recommendation process, attributes of the user profile are matched against the content of the item. Content-based recommenders come with several shortcomings. Firstly, some products have attributes such as quality and taste that cannot be easily identified with current technology to be matched with a user profile in order to generate recommendations. Secondly, suggested products tend to be similar to previously rated products because of the systems’ tendency to recommend items scoring highly against the users’ profile. Thirdly, in general, recommender systems have a mechanism that requires users to rate items in order to receive relevant recommendations and some users may not be willing to give feedback related to the item they had used (Balabanovic and Shoham, 1997).
Collaborative-filtering systems suggest products according to the assigned ratings while content-based RAs suggest items based on user profile and content of the item. In certain situations, these two approaches give undesirable results. For example, certain items like electronic devices might be outdated; that is, technology develops so fast that ratings for certain products might not be valid after some time (Jannach et al., 2011). In addition, a certain number of ratings are required from a particular user in order for the system to understand the pattern in that user’s ratings and generate recommendations (Burke, 2000). The problems mentioned above do not exist in a knowledge-based system since it takes into consideration neither the rating of the items nor the characteristics of the particular user. In other words, it does not need any pre-established database of user preferences and item ratings. In such systems, users specify their needs and the system searches the database and shows the most suitable items to the user. For instance, if customers want to purchase a new car, they input their preferences into the system such as price, fuel efficiency, environmentally friendly, etc. By considering user-entered specifications, the most suitable items are presented to the customer by the system. Customers have an ability to change the feature of the desired car in order to receive different alternatives. Strong interaction between the customer and the system is needed in such systems; in other words, customers are considered an integral part of knowledge-based systems (Burke, 2000). Knowledge-based systems require very good product domain knowledge and this knowledge needs to be stored, organized and engineered in such a way that it can be easily retrieved (Chun and Hong, 2001).
Finally, the most common type of recommender agent is the hybrid. When analyzed individually, each recommender technique has its own limitations. Restricted content and feature analysis, new product problems, and new customer problems are just a few of them. The solution to the problems mentioned above is to combine different recommender techniques in order to generate more precise recommendations by avoiding the drawbacks of the individual techniques. For example, if knowledge about the behavior, tastes, etc. of a large community of other users is known and if there is detailed information about the items, the recommendation process can be enhanced by combining the collaborative filtering and content-based techniques (Jannach et al., 2011). In another approach, collaborative, content-based and demographic techniques are merged to overcome the ‘cold start’ problem in which there exists no data for the given customer. The demographic recommender technique categorizes customers on the basis of personal attributes and generates suggestions accordingly. By using the demographic characteristics, new users are categorized into clusters and items are recommended based on the cluster that a particular user belongs to (Chikhaoui et al., 2011). There are different ways of combining collaborative and content-based recommender techniques. They can be implemented separately and their results can be combined; characteristics of one technique can be incorporated into another one, or a unifying model can be developed which incorporates characteristics of both techniques (Puntheeranurak and Tsuji, 2007).
Theoretical framework
The proposed model of the study shown in Figure 1 was developed by considering previous models and studies on the issue. There exist four constructs in the conceptual model: shopping duration, search effort, decision quality and purchase of desired item. Use of a recommender agent is an independent variable.
Proposed study model’.
Time spent during shopping and effort spent in searching and analyzing products point to the amount of total effort spent by the customer. Two factors used to measure this total effort are shopping time and extent or depth of product search (Xiao and Benbasat, 2007). Studies have showed that users who are assisted by recommender systems have spent considerably less time in selecting products for purchase than users who are not assisted by such systems (Pedersen, 2000; Hostler et al., 2005). The extent of product search as a measure of effort has been analyzed in another study and the results showed that users assisted by recommender agents analyzed substantially fewer product details in simulated online store environments than users who had not been assisted by such systems (Haubl and Trifts, 2000).
Therefore, the following hypotheses are put forward:
Decision quality can be a subjective or objective quality of a consumer’s purchase decision (Xiao and Benbasat, 2007). Researchers (Haubl and Trifts, 2000; Vijayasarathy and Jones, 2001; Olson and Widing, 2002; Swaminathan, 2003; Haubl and Murray, 2006) analyzed whether or not there exists any correlation between RA use and the decision quality of an online consumer by conducting both objective and subjective studies. In the literature, the subjective approach to measure decision quality was measured by considering the user’s confidence level in the purchasing decision. Some authors have shown that users assisted by recommender agents were more confident in their purchasing decisions than those who were not so assisted (Haubl and Trifts, 2000; Olson and Widing, 2002). However, another study found contrary results (Vijayasarathy and Jones, 2001). Another study showed that such intelligent systems increase online consumers’ decision quality when the perceived risk associated with the product is greater and when consumers have in-depth knowledge about the product category they are about to purchase (Swaminathan, 2003).
In the literature (Haubl and Trifts, 2000; Olson and Widing, 2002; Haubl and Murray, 2006), several objective approaches were utilized to measure decision quality of an online consumer. Haubl and Trifts (2000) showed that recommender systems increase the quality of consumers’ decisions by raising the total number of non-dominated products in the alternative sets which the customers may seriously consider for purchase. Non-dominated products are the products which are not inferior to any alternative in a particular product set. Other studies showed that the number of users who changed their mind and purchased another product when given a chance was smaller when an RA system was used than when it was not (Haubl and Trifts, 2000; Olson and Widing, 2002). Another objective approach research showed that recommender agent use increases decision quality of the user in terms of the attractiveness of the chosen product to that user (Haubl and Murray, 2006).
Based on the discussion above it is proposed that:
Information overload causes consumers to mistakenly purchase items that do not match their preferences or that they have never intended to buy. In that respect, recommender agents improve the customer’s decision-making process by reducing information load and search complexity. Recommender agents increase consumer decision quality by recommending products and services in which a customer was interested (Hanani et al., 2001; Chiasson et al., 2002). In this study, purchase of the desired item by participants refers to the case that whether a final product which participants purchase matches their desired camera category which they have stated before starting the simulated shopping task. Purchase of a desired item by a consumer can be considered as another measure of overall decision quality of an online consumer.
Therefore, it is proposed that:
Research methodology
Experimental design
In order to test the developed conceptual model, data was collected through a simulated online shopping experiment. Two online shopping systems were developed by using the ASP.NET framework, which is a web application framework developed, maintained and marketed by Microsoft Corporation. One of the developed systems was integrated with a knowledge-based RA and the other utilized a basic product filtering system.
When compared with other recommender systems, knowledge-based systems are highly interactive. Such systems require strong interaction between the user and the system. They are also different from the simple filtering systems which we see in most online shopping stores. Simple filtering systems just focus on item specifications and they do not consider whether users have any knowledge or experience with the product domain. The knowledge-based system collects user requirements by asking easily comprehensible questions, rather than asking participants which technical details they want the product to possess. Based on the answers given to the questions the system generates recommendations with specifications that meet user requirements.
Interface and functionality of a non-RA integrated shopping system is similar to most shopping systems which we encounter on the web. In this system the user simply selects a product by using a product filtering functionality to input technical details which they want the product to possess. The shopping system searches across the product database and retrieves products based on the user’s inputs. Later, users can sort the retrieved results with a sorting functionality.
Students of Middle East Technical University (METU) were invited via email to participate in the survey. Those who accepted the invitation to participate were randomly sent the Uniform Resource Locator (URL) of one of the shopping systems. The treatment group was sent the URL of a RA integrated shopping system, while the control group was sent the URL of a simple product filtering integrated shopping system. In the simulated shopping, task subjects were required to purchase a digital camera. Before starting the experiment, the purpose of the study and instructions on how to use the simulated stores were explained to the subjects. In addition, participants were told to consider the purchase as if they had average income to purchase only one digital camera.
Two different data sets were collected during the survey. The first was shopping system log data, which was collected and saved to the database by the shopping system. This data set consisted of shopping duration, search effort (number of pages viewed), decision quality and purchase of the desired item by the participant. The second data set includes demographic and technological background details of study participants. Before beginning the survey, the subjects were instructed to input a nickname to the system and to specify the same nickname in the questionnaire so that the two different data sets could be linked.
Experimental shopping agent
In this study, the RA and non-RA integrated shopping systems had similar welcome screens in which subjects could become familiar with different types of camera. In the next step, subjects were instructed to enter their nickname and the type of digital camera they desire to purchase.
The recommender agent integrated system asked the participants seven questions on price, ease of use, purpose of use, usage environment (indoor, outdoor), printing choices, memory of camera, and battery type. Based on the answers, the system searched across 649 digital cameras stored in the product database and displayed the most suitable ones. (The same camera database was used for both shopping systems). For example, in the knowledge-based system users only input that they need a camera for large printing purpose and the system searches the database for cameras meeting the specifications for large printing. However, in the non-RA system users need to know the product specifications for large printing such as pixel and lens, and they need to select the right specifications themselves. Such shopping systems assume that users have product domain knowledge.
Sampling
Non-probability convenience sampling technique was used to select respondents of the study. Respondents were chosen based on their ease of access. All of them were the students of Middle East Technical University (METU), located in the capital of Turkey, Ankara. By sending e-mail invitations, students were invited to voluntarily participate in the survey. Necessary permissions were taken from METU Ethics Centre before sending the invitations and online surveys to the potential respondents. During one month, 1100 invitations were sent and 223 usable and complete responses were obtained, giving a response rate 20.27 percent. Demographic characteristics of the respondents, including gender, age, education and faculties are listed in Table 1. Gender distribution shows that 57 percent of the respondents were male. Most of the respondents (47 percent) were between age 23 and 25. As for the level of education, the majority of the respondents were bachelor students (66 percent), followed by master’s students (29 percent) and PhD students (5 percent). The respondents were mainly from economics and administrative science faculties (43 percent), followed by education faculty (26 percent).
Demographic characteristics of respondents.
Measurement of variables
Four constructs have been utilized in order to assess the influence of knowledge-based RA on the online consumer decision-making process. The first construct is shopping duration of participants. Both the shopping systems tracked how many seconds participants spent in finalizing the shopping task and saved the calculated duration to the database.
The second construct is the participant’s searching effort. This construct is measured by the number of pages viewed by the participant during the shopping session. Both of the shopping systems tracked and saved the total number of pages viewed by the participant.
The third construct is the quality of the participant’s purchasing decision. There are two approaches to assess the decision quality of participants in the relevant literature – subjective and objective. In this study one of the objective approaches was utilized. Decision quality was assessed by giving participants an opportunity to change their final selection and switch to a product which belonged to a completely different category at the end of the shopping task. If they changed their selection this was saved to the database as “Yes” and if decision was not changed it was saved as “No” to the database. “Yes” shows poor initial decision quality while “No” refers to the opposite (Haubl and Trifts, 2000).
The final construct is the purchase of an intended item by the participant. Before starting the simulated shopping session the participant was required to choose the type of digital camera they desired to purchase. After finalizing the shopping, the purchased item was compared with the type of camera that the participant had originally specified. If these two camera types matched it was saved to the database as “Yes”, otherwise it is saved as “No”.
Data analysis
In this study, parametric and non-parametric statistical tools are utilized to test the study hypotheses. As the measurement scale of two dependent variables follows a continuous data pattern, a parametric independent-samples t-test is employed. The chi-square test is used for the other dependent variables since they have a nominal measurement scale. A non-parametric Mann-Whitney U Test is the third statistical tool used in this study. This test is generally used to determine mean differences on an ordinal dependent variable between two groups of an independent variable.
Results
The study sample was composed of 223 undergraduate and graduate students, of whom 126 were male and 97 were female. There were 115 students in the treatment group (recommender agent users) and 108 students in the control group (non-recommender agent users). The average age of participants was 22.83 years.
Homogeneity test between groups
In order to see differences between the control and treatment groups in various areas, participants were required to complete 8 pretest items before starting the simulated shopping session. These were: participants’ computer experience, frequency of computer usage, frequency of Internet usage, frequency of visiting shopping websites, frequency of purchasing products online, knowledge level of camera technology, camera usage experience, and frequency of using a camera. A non-parametric Mann-Whitney U was used to test the possible differences that might exist between groups. The results of the test are given in Table 2. Since the pretest items’ p values (Asymptotic Sig.) derived from the Mann-Whitney U test are all greater than 0.05, it can be said that there was no statistically significant difference between the scores of the two groups for the 8 pretest items. The reason for conducting the homogeneity test was to ensure that there was no difference between the control and treatment groups due to the individual participants’ varying levels of experience related to computers, the Internet, online shopping and digital cameras. Homogeneity between groups in between-subjects experimental design is very important for producing reliable results and conclusions from the subsequent statistical tests.
Result of homogeneity test between groups.
Hypotheses testing
Since the measurement scale of shopping time and search effort followed a continuous data pattern, a parametric independent-samples t-test was used to determine whether there is a statistically significant difference between the control and treatment groups. Summary of the independent sample t-test results for shopping time and search effort (number of page-view) is given in Table 3.
Independent samples T-Test (Shopping Time and Search Effort).
The mean shopping time of the treatment group was 189.64 seconds, as compared with 278.75 seconds for the control group. In order to determine whether this mean difference was statistically significant or not, an independent-samples t-test analysis was run. The results indicated a significant difference between the control and treatment groups in shopping duration with a significance level of p < 0.001. This statistical result supports Hypothesis 1, that there is a negative relationship between the use of a recommender agent and shopping duration of the user. The reason why non-RA users spent more time completing the shopping task is mainly because they either tried to examine available items one by one, or they tried to retrieve items based on a product filtering function which presented lots of options to them. On the other hand, rather than examining available products one by one, or with the assistance of a filtering function, RA assisted users quickly retrieved the list of available items based on their preferences. Later, they saved time by focusing on the small set of available products which they might seriously consider for purchasing.
The mean page-view of the treatment group is 16.10 while this figure is 24.89 for the control group. That is, participants in the treatment group viewed fewer pages in the simulated shopping task than participants in the control group. An independent-samples t-test was used to see whether this difference is significant. The results indicated a significant difference between the control and treatment groups in the number of pages viewed, with a significance level of p < 0.001. This supports Hypothesis 2, that there is a negative relationship between the use of recommender agents and the search effort of the user. The main reason behind the significant difference in the number of page viewed by the non-RA and RA-assisted groups is that participants in the former group had to examine more cameras in order to find one that met their requirements. However, RA assisted participants viewed fewer pages mainly because the intelligent shopping system initially obtained participants’ preferences and later displayed products in which they might be interested. That is, the recommender agent shifted the tedious job of searching, filtering and extracting suitable products from the participant to the system itself.
Since measures of the decision quality and the purchase of a desired item by users are categorical (nominal) data, a chi square (χ2) test was used to determine whether there is a statistically significant difference between the responses obtained from the control and treatment groups. Summary of the chi-square test results for decision quality and purchase of desired item is given in Table 4.
Chi-Square Test (Decision Quality and Purchase of Desired Product).
In the treatment group 71 (61.74 percent) of 115 participants did not switch to another product at the final stage of the shopping task when given an opportunity. However, in the control group this figure was 48 (44.44 percent) out of 108 participants. These results imply that recommender agent users are less likely to change the item they have selected with the assistance of the system for a randomly offered item. The results of the chi-square test indicated a significant statistical difference in decision quality between the control and treatment groups, with a significance level of p = 0.01. This supports Hypothesis 3, that use of a recommender agent is positively related to the decision quality of the user. During the shopping simulations, participants examined different kinds of cameras by spending a certain amount of time and effort. After examining several alternatives and deciding on the final product, participants’ willingness to switch a completely different and randomly offered camera type is an indication of their poor decision quality. That is, all the effort and time spent by the participants was wasted when they switched to a different product without considering its suitability for their needs. The non-RA assisted users’ tendency to change their final selection was an indication of poor decisions made by them during the shopping simulation. Most of them changed their final selection because they were not sure that the product they selected actually met their needs.
RA users were less likely to change their final selection mainly because they were confident that randomly offered products could not be superior to those they had carefully examined and selected with the assistance of the RA.
‘Purchase of Desired Item’ by participants is another dependent variable that was assessed by a chi-square test to see whether there is a statistically significant difference between the two groups. In the treatment group, 67 (58.26 percent) of 115 participants purchased the item that they had indicated before starting the shopping task. In the control group, this figure is 44 (40.74 percent) out of 108 participants. The results of the chi square test indicated a significant difference in the purchase of the intended item between the two groups with a significance level of p = 0.009. This supports Hypothesis 4, that use of a recommender agent is positively related to the purchase of the intended item by the user.
Discussion and conclusion
Unlike other relevant studies in the literature, this study assessed the influence of knowledge-based e-commerce product recommender systems on the online-consumer decision-making process by solely utilizing objective measures. Most of the studies in the relevant literature utilize subjective approaches by relying on participants’ self-reported data in order to measure their behavior. However, this study tried to predict consumer behavior by taking into account only actual behavior data extracted from simulated shopping systems’ log records. The reason for using objective measures in this study is because participants’ self-reported data sometimes does not reflect their actual behavior.
Several previous studies have shown that intelligent RA systems improve consumer decision quality and reduce overall effort; however, other studies have found opposite results. Therefore, this study tried to contribute to the relevant literature by providing additional findings related to the potential benefits of RAs.
In this study, consumers’ shopping duration, effort spent in searching for products, decision quality and purchase of the desired item by participants have been assessed. The results of statistical tests showed that there is a negative relationship between using RA and shopping duration of participants. That is, in simulated shopping tasks, recommender-assisted participants spent statistically significantly less time than did non-recommender-assisted participants. This is the same result as the findings of Hostler et al. (2005) and Pedersen (2000). On the other hand, these results contradict with study findings of Olson and Widing (2002), which showed that RA-assisted participants had longer actual and perceived decision times.
Statistical tests also showed that use of a recommender agent negatively and significantly influenced the search effort of participants in a simulated shopping session. Recommender agent users viewed and analyzed statistically significantly fewer pages than non-recommender users. This statistical result is similar to the study findings of Haubl and Trifts (2000), which showed that participants assisted by such systems analyzed substantially fewer product details than those who had not been assisted by such systems.
Statistical tests have indicated that use of recommender agents positively and significantly influenced the decision quality of participants in a simulated shopping session. The number of participants who changed their mind and purchased another random product when given a chance was statistically significantly less in the presence of recommender agents than in the absence of such intelligent systems. This is the same result as the findings of Haubl and Trifts (2000) and Olson and Widing (2002).
Statistical tests also showed that recommender agents positively and significantly influenced the purchase of the desired item by participants. The number of participants who purchased the camera that matched their initially desired item was significantly more in the presence of recommender agents than in the absence of such intelligent systems.
Several limitations exist in this study. Firstly, this study was carried out by using only knowledge-based recommender agent. Therefore, readers should be careful in generalizing the results of this study to other types of recommender agents. Secondly, participants of this study were limited to university students. Thirdly, this study utilized a simulated shopping environment; that is, participants pretended they were really purchasing the product from an online store. Replicating this study in real life situations might bring out interesting results. Finally, this study has not considered the possible effects of moderating factors on the study results. Possible moderating factors could be participants’ product expertise, product type, product complexity, risks involved in purchasing a given product, participants’ familiarity with recommender agents, etc. It is recommended that future studies analyze the impact of recommender agents on the consumer decision-making process by considering the moderating factors mentioned above.