Propensity to trust and the influence of source and medium cues in credibility evaluation

1.1. Trust in online environments

A common definition of trust is ‘the willingness of a party to be vulnerable to the actions of another party based on the expectation that the other will perform a particular action important to the trustor, irrespective of the ability to monitor or control that other party’ [9]. This definition implies that a certain risk is taken when someone trusts someone else [10]; this trust may prove to be unjustified. In the online domain, trust is an especially relevant concept, as Internet users often interact with parties they do not have prior experiences with. Consider, for instance, online financial transactions (e.g. buying a product through a web shop): the consumer is at risk of losing their money when the vendor fails to meet their expectations.

Four levels have been proposed at which trust may be studied [10], namely individual (as a personality trait), interpersonal (one actor trusting another), relational (mutual trust) and societal (trust in a community). When considering trust in information, the appropriate level is interpersonal trust, as the reader puts their trust in the author of the information.

In order to reduce the risk associated with trusting someone or something, a credibility evaluation may be performed. In such an evaluation, the ‘trustor’ searches for cues on the credibility of the ‘trustee’. Such evaluations are largely heuristic processes, as the user often lacks the motivation and/or ability for a systematic (thorough) evaluation [11]. According to the MAIN model [8], today’s information technology has resulted in numerous affordances in which credibility cues can be found. Such cues may trigger cognitive heuristics; simple judgment rules to estimate the various dimensions of the quality of information. These dimensions also play an important role in the judgment of credibility.

1.2. Trust in information

Another model that clarifies the use of various cues of credibility is the 3S-model of information trust [6]. This model asserts that the most direct strategy for evaluating credibility is to search for semantic cues in the information. By doing so, Internet users try to answer the question: ‘Is this information correct?’ Cues such as factual accuracy, neutrality or completeness of the information are considered by users who follow this strategy. This implies that some domain knowledge of the topic at hand is required. However, a typical information search concerns information that is new to the user, as it normally does not make any sense to search for information one already has. This means that users may often lack the required domain expertise to evaluate the semantics of the information, which makes it impossible to apply this strategy.

To work around this deficit, users may also consider surface cues of the information to evaluate credibility. This strategy concerns the manner of presentation of the information. Examples of cues evaluated when following this strategy are the writing style, text length or number of references in the information. While it is a less direct way to evaluate credibility, no domain knowledge is needed. Instead, by considering surface cues, users bring to bear their information skills. Such skills involve knowledge of how certain cues (e.g. a lengthy text, numerous images) relate to the concept of credibility.

Following dual-processing theory [12], it is tempting to see the strategy of evaluating semantic cues as a systematic evaluation and the strategy of evaluating surface cues as heuristic. However, both strategies can be performed at various levels of processing. For instance, recognizing something one already knows (semantic) is considered largely heuristic behaviour [13]. On the other hand, checking the validity of each of the references of an article on Wikipedia (surface) can be seen as largely systematic. The choice between systematic or heuristic processing in credibility evaluation primarily depends on the motivation and ability of the user [11].

Thus, the cues used in credibility evaluation depend heavily on user characteristics. This has also been proposed in the unifying framework of credibility assessment [14], which suggests that there are three levels of credibility evaluation between an information seeker and an information object (namely the construct, heuristics and interaction layers). The first layer is the construct layer, in which it is posited that each user has their own definition of credibility, which means that different elements of the information object are salient to different users when evaluating credibility (see also Fogg [15]).

1.3. Trust in the source

A third, more passive way of evaluating credibility is also posited in the 3S-model [6], namely the strategy of considering the source of information. Following this strategy, earlier interactions with a particular source may serve as a cue for the credibility of the current information. For instance, if someone has numerous positive experiences with information from a particular website, this user may choose to trust new information from that source without actively evaluating its credibility. The opposite is also possible: when one has negative experiences with a source, one may choose to avoid new information from this source without even looking at it (at the semantic or surface level).

The approach of transferring the credibility of the source of information to the credibility of the information itself only works well when the credibility of information from a source is stable over time. However, in online environments, information from one source may vary greatly in credibility. Consider again Wikipedia: information quality is generally very high [16], but numerous examples of incorrect information from Wikipedia are readily available [17–19]. This means that trusting this source involves taking the risk of encountering false information. On the other hand, distrusting this source means that the user may miss out on much high-quality, valuable information. Nevertheless, it has been shown that, also in the case of Wikipedia, the source-strategy is applied very often. It was found that around 25% of experts on the topic and 33% of novices trusted or distrusted information solely because it came from Wikipedia [6]. This is an indication that users weighed the benefits of Wikipedia (much information) against its risks (poor information). For some users, the benefits clearly outweighed the risks. For others, they did not.

A second drawback of considering the source of information is that nowadays it is often difficult to determine a single author who is responsible for the information. Online news, for instance, is often carried through multiple sources (e.g. a blogger writing a piece on something she read on Facebook, which was a reaction on an article on the CNN news page). This concept is known as ‘source layering’ [4], and makes it increasingly difficult to determine which source is responsible for the credibility of the information. However, a recent study on this phenomenon [5] has shown that only highly involved users considered more distal sources when evaluating credibility. Users with low involvement were only influenced by the credibility of the most proximate source (i.e. the website on which they read the news). For this reason, we consider the most proximate source (i.e. Wikipedia) as ‘the source’ of information in this study.

Two key factors for the credibility of a source have been identified [20]. First, sources should have the appropriate knowledge (expertise) to provide correct information. For instance, a doctor is able provide credible health information, whereas a patient may not be. Second, sources should be trustworthy, that is, have the intention to supply correct information. To clarify this concept, consider the difference between a manufacturer of a product and an independent party testing this product. Both may provide similar information about the product, but have very different intentions. The manufacturer wants to sell the product, whereas the tester wants to provide consumer advice. This may have large consequences for the credibility of the information supplied.

1.4. Trust in the medium

Traditional linear communication models generally encompass a source (sender) of information, who transmits a message through a medium to a receiver. However, it has been shown that a medium may also be treated as a more general type of source by information seekers [4]. People tend to say that they got information ‘off the Internet’, or even ‘off the computer’ rather than naming one specific website [8]. As such, credibility may also be attributed to a medium rather than a single source.

Examples of different media channels are the Internet (or a subset, such as Internet vendors [21]), television, newspapers or school books. It has been shown that trust in the Internet is primarily influenced by experience [7]. It is hardly possible to assign a value to the credibility of online information without having used the Web. Such experience with the Web means that users have interacted with various online sources (websites). The experiences in these interactions are accumulated into trust in the Internet as a whole.

Trust in a medium can be brought to bear when encountering a new source on this medium (i.e. an unfamiliar website). Users may evaluate the credibility of this website, as well as the information on it, but trust in the Internet in general may serve as a baseline.

In the context of research on the Internet, this medium is often compared with traditional sources such as books or newspapers [22]. Differences are found at various levels, such as organization, usability, presentation and vividness. In various instances, the Internet has been shown to be more credible (e.g. political information [23]) and less credible (e.g. health information [24]) than traditional media.

1.5. Propensity to trust

As stated earlier, when studying trust in information, the appropriate level is interpersonal trust [10]. However, this does not mean that trust on the other levels has no influence. Consider, for instance, trust at the individual level, or ‘propensity to trust’. Propensity to trust is a personality trait, a stable factor within a person, that affects someone’s likelihood to trust [6].

One’s propensity to trust, or dispositional trust, serves as a starting point, upon which more case-specific trust builds [25]. In an experiment with an X-ray screening task with automation available, trust in the automation moved from dispositional to history-based [25]. In other words, trust became more calibrated to the automation. Owing to the heuristic character of online credibility evaluation, we expect that the propensity to trust will also be visible in more case-specific trust, such as trust in the medium, source or actual information.

The relationship between propensity to trust and trust in online information has been studied before. Propensity to trust has been shown to be among the most influential factors predicting consumers’ trust in Internet shopping [21, 26]. In these studies, propensity to trust is seen as a mediating factor between trustworthiness of the vendors and the external environment on the one hand, and trust on the other. Some researchers distinguish a propensity to trust from a propensity to distrust [27]. Again, in the context of e-commerce, it was shown that the former has an influence on trust in low-risk situations, whereas the latter influences trust in high-risk situations.

It is not surprising that much research on trust in online environments has focused on e-commerce. In this domain, users take a direct, measurable risk (of losing money), which makes trust a very important construct. This risk may be less salient (or at least measurable) in other domains, such as online information search, as it heavily depends on the purpose of the information. However, wrong decisions may be taken based on this information, which makes online information search an important area of study.

1.6. Proposed research model

In the literature discussed here, the concepts of trust in information, sources and media, and a general propensity to trust are mostly studied in isolation from each other. Some exceptions can be found (e.g. [6, 28]), but an integrated approach featuring all these concepts in one study is yet to be seen. In this study, we present a novel model of trust in information, explaining how these concepts are related to each other.

As presented in Figure 1, we hypothesize that not all types of trust discussed here have a direct influence on trust in each particular piece of information. Instead, we suggest a layer model, in which trust is built from a general propensity to trust to case-specific trust in a particular piece of information. In this model, we consider general propensity to trust as the general baseline of trust of a person [25] in all situations, hence not only for trust in (online) information, but also trust in, for example, others, society or technology. With each layer, trust becomes more specific for a single situation (i.e. evaluating the credibility of a single piece of information).

The second layer is labelled ‘trust in the medium’ and concerns trust of a user in a particular type of medium (e.g. newspapers, the Internet). While this is clearly a more case-specific form of trust than a general propensity (at least one feature of the information at hand is considered), it is still a generalization of trust in the source of the information [4]. Trust in a medium can also be seen as trust in a collection of sources.

Trust in the medium is followed by the layer ‘trust in the source’. Again, trust is further specified, as the specific source of the information is considered rather than the medium through which the information is communicated. Considering the source of information is perhaps the most traditional form of credibility evaluation [6].

The most specific form of trust is trust in the information itself. Especially when the credibility of the source is doubtful, users may search for cues in the information itself to estimate credibility [28]. In sources where the credibility varies between different pieces of information (e.g. Wikipedia), trust is best calibrated with the actual information quality when cues from the information itself are considered, rather than cues from the source or medium [6].

In this study, we seek initial validation for this model of trust, by evaluating the influence of each of these layers on trust in Wikipedia articles. Thereby, we assume that all participants actively evaluate the credibility of the information to a certain extent (as this is the task imposed on them). However, the layer at which credibility is evaluated in practice may largely vary between users and contexts. Motivation and ability to evaluate have been identified as important factors for the extent to which credibility is evaluated [11]. Only relying on one’s propensity to trust does not require any effort when encountering a piece of information. Each next layer requires more effort from the user to evaluate credibility. Hence, in situations with a low risk of poor information, or with users with a low motivation or ability, the outer layers may have a larger influence on trust in the information than when the risk, motivation or ability are higher.

We thus hypothesize a direct influence of each layer on the next. Moreover, we expect that the influence of each layer can also be observed in more distant layers (e.g. the influence of trust in a medium on trust in information). However, we hypothesize that this influence is mediated by the intermediate layer, which can better explain how the two non-adjacent layers are related (e.g. trust in a source explains the relationship between trust in a medium and trust in information). Hence, the following hypotheses are tested through mediation analysis in order to examine the validity of the proposed research model. The following three hypotheses can be derived from the hypothesized model:

2.1. Participants

Invitations for participation in an online experiment were posted on several online forums and social media, and via direct email contact. This resulted in a total of 152 participants who completed the whole experiment. Three participants were excluded for bogus participation; they gave the same answer to every question. Of the remaining 149 participants, the majority (81.9%, n = 122) was male. The average age was 25.7 years (standard deviation, SD = 10.1). The participants came from Europe (67.8%), North America (25.5%), Australia (2.7%), South America (2.0%) and Asia (2.0%). It was ensured that each participant could only partake once by registering their IP-addresses and placing a cookie on their computer.

2.2. Task and procedure

The experiment was conducted using an online questionnaire. When following the link to the questionnaire in the invitations, an explanation of the task was presented first. Participants were informed that they had to evaluate the credibility of two Wikipedia articles, without specifying how to perform this task (Wikipedia Screening Task, [29]). The explanation also stated that, after the evaluation, a few questions about trust and Internet use would be asked. Moreover, the participants were warned that they could not use the back and forward buttons of their browser.

After reading the instructions, the participants could decide to participate by clicking on the ‘next’ button. After doing this, they were asked to enter their gender, age, and nationality on the subsequent page.

The actual experiment started after the participants clicked on the ‘next’ button again. A full-page screenshot of a Wikipedia article was shown in the questionnaire. Underneath the article, the participants had to answer three questions. First, they had to rate how much trust they had in the article on a seven-point Likert scale. Second, they could provide an explanation for their answer through an open-ended question. This explanation was mainly used as an indicator for bogus participation, but the explanations were subsequently also categorized according to the 3S-model. Third, the participants had to rate how much they already knew about the topic at hand, again on a seven-point Likert scale. After answering these questions and clicking ‘next’, the procedure was repeated for the second article.

After evaluating both articles, three separate webpages asked for (1) propensity to trust in general, (2) trust in Wikipedia and the Internet, and (3) general remarks on the experiment.

2.3. Stimuli

Each participant viewed one article of high quality and one article of low quality. The ratings of the Wikipedia Editorial Team [30] were used to make this distinction. For the high-quality articles, the highest quality class (Featured Articles) was used whereas for the low-quality articles, the second lowest quality class (Start-class Articles) was used. The lowest quality class (Stub articles) was deliberately avoided, as these are often single-sentence articles.

Next to quality, length of the articles was also taken into account. Some featured articles tend to be extremely lengthy. This could cause problems in the experiment, as it could take too much time for the participants to evaluate such articles. Therefore, only featured articles with fewer than 2000 words were selected. Moreover, we enssured that the articles in the poor-quality condition were sufficiently long to perform a meaningful credibility evaluation (i.e. they contained enough cues to evaluate). Therefore, only start-class articles with more than 300 words were selected.

Following these considerations, three topics with a typically encyclopedic character were used, namely:

The first of each pair served as a low-quality article, and the latter as a high-quality article. Each participant was randomly assigned to one of the topics and evaluated two articles on this topic. The order of articles was counterbalanced between subjects.

2.4. Measures

2.5. Data analyses

For each of the constructs measured through questionnaires, its reliability was calculated using Cronbach’s α. We took an α of at least 0.70 as an acceptable value for all three constructs (propensity to trust, trust in the Internet and trust in Wikipedia).

In order to find validation for our research model, bootstrapping mediation analysis was performed [33, 34] to estimate direct and indirect effects with multiple mediators using the PROCESS toolkit for SPSS [35]. The advantages of this technique are that all mediators can be tested simultaneously, normal distribution does not need to be assumed, and the number of inferential tests is minimized (reducing the risk of a type 1 error). Following the proposed research model (see Figure 1), a model with trust in the medium and source as sequential mediators was tested.

The motivations for trust in the articles that could be provided by the participants were classified in accordance with the 3S-model [6]. This means that each comment was categorized as referring to a semantic, surface or source feature. Comments that could not be categorized as referring to any of these features were classified as ‘other’. Half of the comments were categorized by two raters. Based on this overlap, Cohen’s k was calculated to ensure inter-rater reliability. A k of 0.91 indicated a near-perfect agreement.

3.1. Validity of the questionnaires

Cronbach’s α for the participants’ propensity to trust derived from the NEO-PI-R questionnaire [31] was 0.82, indicating good reliability. For the trust in the Internet scale, Cronbach’s α was 0.70, indicating acceptable reliability, and for the trust in Wikipedia scale, Cronbach’s α was 0.88, again indicating good reliability.

3.2. Trust

Propensity to trust, as measured through the eight questions on this construct on the NEO-PI-R [31], is divided into five categories, displayed in Table 1.

OPEN IN VIEWER

Table 1.

Participants in each of the five categories of the NEO-PI-R trust scale

Category	Percentage
Very high trust	10.7% (n = 16)
High trust	17.4% (n = 26)
Average trust	45.6% (n = 68)
Low trust	16.8% (n = 25)
Very low trust	9.4% (n = 14)

Trust in the Internet ranged from 1.17 to 5.33 (on a Likert-scale from 0 to 6), with an average of 3.63 (SD = 0.79). Trust in Wikipedia ranged from 0.00 to 5.83 (on a Likert-scale from 0 to 6), with an average of 3.51 (SD = 1.11).

Trust in high-quality information (mean, M = 4.91, SD = 1.64) was higher than trust in low-quality information (M = 4.42, SD = 1.63), t(148) = 2.82, p ≤ 0.01. Average trust in the information was 4.67 (SD = 1.24).

3.3. Validity of the research model

Figure 2 shows a cross section of the layer model presented in Figure 1, with unstandardized regression coefficients between all constructs. Trust in the information was entered as the dependent variable, propensity to trust as the predictor variable and trust in the medium and trust in the source as (sequential) mediators.

OPEN IN VIEWER

Figure 2.

Cross section of the proposed layer model, showing unstandardized regression coefficients between all proposed constructs. Coefficients marked with three asterisks are significant at the 0.001 level; other coefficients were not significant.

Basic regression analysis showed that the effect of propensity to trust on trust in the information was 0.20 (p ≤ 0.05). However, when (either or both of) the two mediating variables were entered into the model, this direct effect became insignificant (0.05, p = 0.56).

The total indirect effect was estimated at 0.08, with a 95% bias-corrected bootstrap (1000 samples) confidence interval of 0.03–0.16. Hence, trust in the medium and trust in the source mediated the effect of propensity to trust on trust in the information. Moreover, a model with only trust in the medium or trust in the source as mediating variable proved to be less valid, with a total indirect effect of respectively 0.05 (95% CI −0.02–0.16) and 0.01 (95% CI −0.05–0.08).

More light can be shed on the relationship between trust in Wikipedia and trust in information when considering the difference between high-quality and low-quality information. A median split on trust in Wikipedia was performed to distinguish participants with high and low trust in this source. Based on this split, we performed a repeated-measures ANOVA with article quality as a within-subject variable and trust in Wikipedia as a between-subject variable. A main effect of article quality and trust in Wikipedia on trust in the information was found, respectively F(1, 147) = 7.45, p ≤ 0.01 and F(1, 147) = 31.71, p ≤ 0.001. An interaction effect between article quality and trust in Wikipedia on trust in the information was only significant at the 10% significance level, F(1, 147) = 3.02, p = 0.08. Visual inspection of the data suggested that users with low trust in Wikipedia were less influenced by article quality than users with high trust in Wikipedia. As expected, the same analysis applying a median split on propensity to trust and trust in the medium did not yield a significant interaction effect, F(1, 147) = 0.37, p = 0.55.

3.4. Motivations for trust

A total of 131 of the 149 participants entered a motivation for their trust in the article on at least one occasion. This resulted in a total of 252 comments, which were categorized in accordance with the 3S-model [6]. Table 2 gives an overview of these comments.

OPEN IN VIEWER

Table 2.

Motivations given for trust in the articles, coded in accordance with the 3S-model [6]

Motivation	Percentage
Semantic features	11.5% (n = 29)
Surface features	59.9% (n = 151)
Source features	13.9% (n = 35)
Other	14.7% (n = 37)

As can be expected from a user group with limited domain knowledge on the topic at hand, most comments could be classified as referring to surface features (e.g. ‘This article is well-cited’). However, semantic features (e.g. ‘It appears to be historically correct, as far as my knowledge of the subject goes’) and source features (e.g. ‘Wikipedia has yet to fail me’) were also mentioned as a motivation to trust the article. A remainder of 14.7% of the comments could not be classified in the 3S-model (e.g. ‘I have no reason not to trust this particular article’).

4.1. Limitations

Only one medium (the Internet) and one source (Wikipedia) were taken as a case study to demonstrate the validity of the research model in this experiment. Future research in this direction could utilize the same model, but with different media and/or sources to verify its validity in other contexts.

The interaction effect found between trust in the source and quality of the information on trust in the information was not significant at the customary 5% level. However, the trend found in this experiment suggests a larger risk of undertrust in Wikipedia than overtrust. More research on the effect of the source of information on trust should confirm whether this is actually the case.

In this experiment, trust on various levels was measured through (partially validated) questionnaires. In future research, attempts should be made to manipulate trust more systematically, in order to rule out the effect of potentially confounding variables (e.g. age, Internet experience).

We cannot rule out the possibility that the order of the experiment (specifically the general questions on trust after the administration of the stimuli) had an influence on the answering of the questions. However, we are convinced that a reverse order in which the general questions would have been presented before the stimuli would have influenced the answering of the questions regarding the stimuli to an even larger extent, as this would have primed the participants on the issues of source, medium and propensity to trust. Future research could completely preclude this potential issue by counter-balancing the order of questions.

Finally, the results found here may not generalize to the entire population of Wikipedia (or Internet) users, as the sample is demographically biased towards European and North-American males. Gender differences in trust in Wikipedia [38] and in general [39] have been shown before. Moreover, six different articles were used as stimuli in this experiment. Although we attempted to rule out specific effects of this selection (e.g. very short or long articles), we cannot be sure that exactly the same effects are found when other articles are selected.