Modelling trust formation in health information contexts

Abstract

This study explores trust formation in the context of health information. Trust as an interpersonal notion, when formed in a vulnerable state, is a response or belief about how the trusted will behave towards the trustor. This study focuses on the process of assessing the trustworthiness of information, in a dependency state of information need, through the identification of the many factors influencing this assessment. A set of propositions are developed to suggest the criteria by which trustworthiness is assessed as well as the factors that influence these judgements. The proposed model is tested in a large-scale survey using a trust inventory with factor analysis to explore the constructs of trust formation. Structural equation modelling is used to explore the relationship among the identified criteria and their influencing factors. The resulting framework contributes to the understanding of trust formation in digital information contexts on the criteria of usefulness and credibility and further research into the influencing factors is recommended.

Keywords

Health information seeking information evaluation information-seeking behaviour relevance judgements trust formation

1. Introduction

On what basis do we trust information found in health information-seeking contexts? Kohn states that ‘[m]aking sense of trust requires deciding what trust is’ but that there exists a community of meanings labelled as ‘trust’ [1]. Trust is ‘complex and multidimensional’ [2], can ‘have various meanings’ and is ‘viewed differently’ in different situations and contexts [3]. When trust is formed in a dependency state, for example to resolve a query on a health issue, as a judgement it is likely to be affected by many different attributes of the information and its delivery. The judgement may also be moderated by the context in which the information is sought. In this study, it is proposed that the key to studying trust is to identify the criteria on which trust judgements are formed and identify, in turn, the factors that influence these criteria. That is, according to Yi et al. [4], to disentangle the criteria used to evaluate the information in forming trust from the factors that influence them. Previous studies of trust have identified a comprehensive list of the many factors that might influence trust formation. Disentangling and thus identifying the criteria and modelling the relationship of these and the influencing factors aims to develop a measurement scale for the study of trust in a variety of digital information contexts.

This study aims to contribute to understanding ‘trust’ as a central concept in the assessment of health information encountered in digital information contexts. The approach taken departs from previous research which, in general, adopts exploratory methods to identify the core factors as having major influence on formation of trust, and takes a positivist perspective to identify the relative importance and the relationships between the factors that are identified in previous research. Informed by a set of propositions, a psychometric questionnaire is used to identify the evaluative judgements that young people (students) employ in considering the trustworthiness of health information found on the web. More specifically, this study adopts a critical incident approach to identify the core criteria and factors when asked to recall which were most important in assessing the trustworthiness of the information. The critical incident approach has been used extensively in information-seeking research as it has the advantage that the incident is relevant to the searcher, and is therefore closer to ‘real life’ than an experimental setting, and it also facilitates the collection of a larger dataset, thereby permitting more rigorous quantitative analysis. In this study the responses to the questionnaire regarding the assessment of information were analysed to identify the core criteria on which trust is formed and the relationship among the influencing factors relating to the document attributes and user perceptions of the information. The resulting framework contributes to the understanding of trust formation in digital information contexts on the criteria of usefulness and credibility and further research into the influencing factors is recommended. In the following sections the literature is reviewed to define trust and to identify the core propositions of trust. Identifying the core factors leads to the research model and the development of the measurement scale. The empirical study to test the scale and the proposed two-factor model on the criteria of usefulness and credibility is presented from the data analysis. The limitations of the study are discussed and the further development and use of the model is identified for the study of the formation of trust in digital information contexts.

2. Review of related work

2.1. Trust as an attitude

Trust in electronic environments has been variously studied [2, 3, 5 –7] and as the Internet becomes more pervasive in everyday life, the issue of information found online and how much to trust it becomes an increasing concern [6]. Chopra and Wallace [2] comment that trust in information is a ‘social attitude’ that tends to relate to a particular ‘technological artefact’, for instance, information found on the Internet. As an attitude, trust may be understood to be a belief that what others are saying or doing is correct, a ‘general expectancy held by an individual that the word, promise oral or written statement of another individual or group can be relied upon’ [8]. As a cognitive issue, trust appears as a reliance on some sort of ‘mutual interest’ or a ‘want’ to maintain the relationship [9]. In the context of online information, the user (the ‘trustor’) looking for or wanting information has a particular need to fulfil, and the information (in which the user places trust) possesses the potential to satisfy this need. This state of dependence between the ‘trustor’ and the ‘trustee’ satisfies the precondition that Chopra and Wallace [2] propose is required for trust. They further outline this precondition for trust in stating that acting on this dependence must entail a risk with potential uncertainty about the outcome and a vulnerability to the potential loss from an undesirable outcome. This is to say that, given these preconditions, we can assume that trust is likely to be formed in the context of looking for and finding information on the web in response to some health issue, concern or need. Furthermore, from an analysis of commonalities in a set of definitions of trust, Rowley and Johnson [10] have suggested that the positive belief formed must be verifiable and will lead to a state of ‘intention to use’. Thus the users’ judgement of the trustworthiness of the information, in a state of dependency and intention to use, is central to understanding the effective provision and use of digital information. Not surprisingly then many studies have used ‘trust’ as a measure or metric to increase understanding of information use and behaviour in various contexts.

This basis for this study identifies trust to be a multidimensional concept that is formed and is dependent on the trustor’s particular situation or information need. In the consideration of trust as multidimensional, cognitive and seemingly dynamic concept we pose the question – what are the constructs of this judgement and what are the factors that influence or affect such a judgement? Thus the aim of this study is not to measure trust, but to identify its constructs to propose a model of its formation for use in further research.

2.2. Factors influencing trust

Extant research has, in general, adopted exploratory methods to identify the many core factors as having major influence on formation of trust. Gouge and Gilson [11] review the qualitative research that explores and identifies the attributes that promote perceptions of trustworthiness, in interpersonal trust and in the health sector, such as a person’s perceived competency or honesty. In digital information trust contexts, various characteristics and attributes of the information have been identified as promoting trust formation. Corritore et al. [12] have shown that trust in health information websites was significantly explained by users’ perceptions of website credibility, ease of use and risk. Ye [13] explored the extent of correlation of consumer trust in online health information with income, education, health status, engagement in social network sites, ease of location of information, understandability and trust in health information through other media (doctors, family and friends, print media). Sillence et al. [14] found that, in the context of health websites, the top five trust markers were: the site was easy to use; the advice came from a knowledgeable source; the advice appeared to be prepared by an expert; the advice appeared to be impartial and independent; and the reasoning behind the advice was explained. They also recognized that the situation was dynamic, and affected by the source available and user experience. In another diary-based study, Sillence et al. [15] found that the factors contributing to the selection and trust of web sites can be divided into design factors (clear layout, good navigation aids, interactive features) and content factors (e.g. informative content, unbiased information, clear, simple language). In their guidance on how to investigate (specifically interpersonal) trust, Gouge and Gilson [11] drew attention to the value of quantitative inquiry to test the identified factors influencing trust and to understand trust building over time and its dependence on context, particular (health) issues or individuals involved [16]. However, despite the acknowledged importance of trust in online information environments, researchers have wrestled with the development of adequate measurement scales. Wang and Emurian [17] point out considerable variability in the way in which researchers operationalize the concept and the way in which they explore the relationship between trust and its antecedents and consequents. They go on to suggest that trust is:

often conceptualised by researchers according to the features of the particular context [and that] it is often used interchangeably with related concepts such as credibility, reliability, or confidence. (p. 108)

With the exception of the respective18- and 24-item scales used by Briggs et al. [18] and Sillence et al. [15] to measure trust in the context of health information, measurement scales of trust in online information are generally relatively simple and often focus on specific cues of health website trustworthiness, rather than taking a broader conceptual approach to trust [12]. As a result, previous research appears to suggest that the cues or the characteristics of information (or a source) invoke some placement of trust, but fail to develop insight into the process of trust formation. As O’Neill [19] explains, trust is formed in the process of assessing trustworthiness, suggesting that it is the judgements made on some criteria (for example, on the information attribute of credibility) that promote the formation of trust. A few studies have sought to resolve this by proposing theoretical frameworks [6, 17, 20 –22]. Whilst these attempt to build a model of the relationships between trust and its preceding judgements in various contexts, there is considerable diversity in the relationships proposed and tested. For example, Cugelman et al. [23] examined the dimensions of website credibility (identified as expertise, visual appeal, and trustworthiness), with active trust. Shen et al. [24] studied trust in the context of student use of Wikipedia, and built a model with sources credibility and information quality (including completeness and format) as antecedents to information usefulness, which in turn was an antecedent to trust in Wikipedia and information adoption. Harris et al. [25] proposed and tested a predictive model of trust in internet-based health information and advice, with information quality, personalization, impartiality and credible design as antecedents to trust, and corroboration and threat as mediating variable. Wathen and Burkell [22] differentiate between evaluation of surface characteristics, which includes appearance/presentation, usability/interface design and organization of information, and evaluation of message credibility, which includes source and message components. Lucassen and Schraagen [20] propose and test the 3-S model of information trust, which suggests that trust formation is influenced by information characteristics, and user characteristics (domain expertise, information skills and source experience). Their study suggests that these user characteristics lead to different features of the information being used in trust judgements in different contexts.

The lack of consensus on how trust is conceptualized presents a potential for confusion in any attempt to identify the attributes of the information used in assessing its trustworthiness. This may arise when, for example, credibility and information quality are both identified as factors that promote the formation of trust. This may be the case but the model fails to identify any relationship between the judgement formed on the information credibility, as an antecedent to trust, and the assessment of the information quality. That is to say, information quality as an attribute would not simply invoke trust, but may however influence the judgement of trustworthiness. Put simply, assessing trustworthiness requires some ‘action’ or judgement from the trustor, and to understand the trust formation in digital information, and its dependency on contexts, it is necessary to identify the core criteria as the antecedents of trust and to distinguish the attributes or inherent properties of the information that may influence the judgement. This identifies a possible shortcoming in these models that may appear to lack a consensus on how trust is conceptualized, but also it may be suggested that they fail to reflect how users might think about trust, which according to Fergie et al. [26] and Sillence et al. [27] often involves rapid and heuristic based judgements.

3. Research model

3.1. Introduction

Key questions remain in the endeavour to understand trust in specific contexts. In particular, what are the core constructs of the trust judgement when formed in the context of digital information? In order to make sense of trust in digital information, Section 3.2 presents a set of propositions deduced from previous research on the judgements made in information-seeking contexts. Focusing on the process of trust formation and on the assessment of information trustworthiness, the aim is to identify the factors influencing trust, and crucially, what are the core criteria on which trust is formed to distinguish these judgements from their influencing factors or determinants. This conceptualizes trust formation in digital information as a process, involving the user in making judgements and dependent on the context of the users’ information need. This is based upon, and is further explained by, drawing a comparison with long-standing research into the concept of relevance as a dynamic judgement in information science [28, 29]. Relevance is conceptualized as a subjective judgement of the information’s ‘topicality’ as perceived by the user in relation to an information need, and its ‘usefulness’ with regards to its actual use in directly resolving the given problem. The judgement as a continuum can assume topicality, an attribute of the document, as a basic requirement of a users’ judgement of a document as relevant. Further along in a continuum of relevance, the user may be expected to make a judgement of the document’s usefulness in relation to the problem in hand. Where this distinction is helpful is in its identification of the core criteria for relevance judgements, such as usefulness, utility and pertinence as well as the identification of the independent variables such as aspects of the document, for example its topicality, style and novelty in providing new information to the user. Thus the perceived attributes of the document (or its information) contribute to the user’s overall relevance evaluation and this judgement is formed on the criteria of usefulness (and its possible surrogates such as pertinence) as the dependent variable. As a framework for the study of trust, the information characteristics and the criteria for the formation of trust are distinguished and treated at different levels. This framework can then be further developed by distinguishing the types of influences and modelling their relations to the criteria used in trust formation. Drawing on the Elaboration Likelihood Model (ELM), which has been used to explain individuals’ changes to the credibility assessment as they encounter, specifically, persuasive communication [30, 31], an important distinction is made between content and otherwise peripheral cues [32]. The content of the argument quality may be one way to influence an individual’s attitude to a persuasive message. ELM states that, equally so, the peripheral clues, such as source reputation, may also be used to influence attitude. Both Cacioppo and Petty [32] and Sundar [31] have suggested that individuals may base their judgement of a message not on its content but on peripheral cues when the motivation to engage in the mental effort in analysing a message based on content is missing or lacking. As the factors that have been previously identified as influencing or even determining trust formation range from content to others that may be regarded as ‘peripheral’, such as ease of use, it is important that any model of trust formation allows for this distinction to be made across the influencing factors. In particular, as trust is identified as a dynamic judgement made in the context of an information need, the ELM implies that this context might determine the type of factors used in trust.

To propose the model of trust, the following propositions are presented for the development of a multi-item scale to identify the criteria and influencing factors (both content and peripheral) in trust formation in information ‘use’ contexts.

3.2. Key propositions

P1 − Assessing information credibility is an antecedent to trust.

One of the fundamental conceptual debates in relation to trust in digital information is on the relationship between trust and credibility. The assertion regarding the interchangeable use of trust and credibility as if they are the same thing was made by Wang and Emurian [17] and Tseng and Fogg [33], and there is plenty of evidence of this in key empirical studies in the field; for example, Rieh and Hilligoss [30] and Hargittai et al. [34] use the terms interchangeably, whilst Sillence et al. [14] do not define either trust or credibility. Other researchers show trust or trustworthiness as an element of credibility [22, 23], whereas others view online trust as a complex construct that includes credibility. Shankar et al. [35] and Tseng and Fogg [33] recognized this dilemma, and proposed the following distinction:

Trust indicates a positive belief about the perceived reliability of, dependability of, and confidence in a person, object or process. Credibility is synonymous with believability. (p. 41)

However, Kelton et al. [6] dispute Tseng and Fogg’s definitions of trust and credibility, and argue that their definition of trust is too narrow − they claim that the relationship between trust and credibility is an open area for further research, and generally propose that there is need for more in-depth exploration of the components of the trust concept. Here we argue first and foremost that it is important to take a position on the relationship between trust and credibility, and that whilst, on occasions, it may be appropriate to use them interchangeably, this use should be informed and deliberate. In proposing the framework for the study of trust we take the position that trust is the super-ordinate concept and that it subsumes credibility.

P2 − Assessing information usefulness is an antecedent to trust.

In a similar vein we propose that trust subsumes usefulness as a core criterion used in the assessment of the information sought in some dependency state. The concept ‘relevance’ from traditional information retrieval theory is generally defined as the extent to which a document or piece of information solves a problem or satisfies an information need. In systems terms, relevance could be viewed as the match between a set of documents and the user’s query. However, other types of relevance are more subjective and are more likely to be important to and employed by users. These include relevance to a subject (topical relevance), relevance to the user’s existing state of knowledge (cognitive relevance) and relevance to a task or problem (situational relevance). A distinction is usually made between subject and situation relevance − specifically that the ‘aboutness’ of a document in relation to some information need is subject relevance and the usefulness or utility of the information is with regards to the information need or task in hand [29]. Simply put, the document may be topically relevant, but irrelevant with regards to situation relevance if it fails to contribute to the problem or task in hand. The centrality of the judgement of ‘usefulness’ is confirmed for example in Lim’s [36] exploration of responses to a range of anticipated outcomes (expectations) as factors related to information use. A number of factors including past experience, disposition to information, information utility and verbal persuasion were tested on a series of statements such as ‘I will find useful information’ [36: 2192]. The findings indicated that information utility will be a core factor related to the outcome expectations and use of the information. However, Hertzum et al. [37] suggest that relevance, even taking into account different types of relevance, is not the only factor that users use when assessing documents. She proposes that two other key factors are trust and the ‘least-effort principle’. We propose that trust subsumes usefulness along with credibility judgements in the sense that these are preconditions made in the process of trust formation and with respect to the information need context. The premise that trust formation in digital information is based on the judgements of the credibility and usefulness of the information allows for the possibility that a variety of factors will influence these judgements, the importance of which is likely to be determined by the context. These further factors that may relate to the core criteria are identified in propositions P3 and P4.

P3 −Perceiving information quality is pivotal to trust formation.

Chopra and Wallace [2] define users’ trust in information as a social attitude towards an item where there is an expectation that the information will be reliable and valid and will be assessed on a rational basis using quality indicators such as accuracy, coverage, timeliness and depth. They suggest that trust in digital information can develop based on such indicators. Further studies of trust formation suggests that authorial authority is pivotal to trust formation in digital information with user interest in the authority associated with authorship, and indeed author names may well be recognized, or have affiliations that lend further confidence in the reliability of the digital information. The other factor, which emerges repeatedly from the empirical evidence on trust formation in digital environments and related studies, is information presentation. This is represented in slightly different ways in different studies, and, information presentation factors are more obviously evident in studies on website credibility, such as that conducted by Tseng and Fogg [33]. This study investigated how users evaluate the credibility of web sites in 10 categories, including e-commerce, finance, search and health; top of their list of the themes that occurred most frequently in connection with credibility judgements were design look, information design/structure and information focus. Similarly, in Sillence et al.’s [15] diary study of how patients evaluate and make use of online health information, design factors, such as clear layout, good navigation aids and interactive features, were found to contribute to the selection and trust of web sites. Rowley and Johnson [10], found that students expressed considerable interest in the structure, style and quality of writing of Wikipedia articles, and grammatical or other editorial or proofreading oversights, and these impacted on their trust judgements.

P4 − Peripheral cues influence trust formation.

Previous research into trust formation identifies a set of possible peripheral or contextual ‘information’-related cues that may influence trust. We tend to trust big brands and this extends to digital information contexts with respect to the information providers. Pan et al. [38] suggested that people were more likely to trust the results from Google and furthermore claimed these results to be superior. Bailey et al. [39] similarly found that when branding was removed users were unable to tell the difference in quality between the search results from two popular search engines. The influence of recommendation is also of interest as a factor employed by users to potentially reduce the complexity in decision-making in uncertain situations (whether to trust information) and so used as a mechanism as such. Further influencing factors may be identified from the empirical evidence from both the trust and credibility literatures suggesting that, in digital environments, verification or checking the information content against other sources is at the core of trust formation. In a large study of first-year undergraduate students, amongst the factors that Hargittai et al. [34] identified as most important in credibility assessment were sources for validation, authorship and linking sites. Students in Iding et al. [40] study sought corroboration with other web sites. Rieh and Hilligoss [30] examined college students’ credibility judgement in the information-seeking process using an interview-based study and confirmed that students certainly undertook evaluation of information, based on their current knowledge on the topic, quality control mechanism (refereeing, editing) and verification (through using multiple sources and co-referencing).

P5 − Trust formation can be modelled by its core criteria and determining factors.

Our research model focuses on the process of trust formation. Previous research identifies many different factors that may influence trust. The propositions P1–P4 relate the ongoing process of trust formation as that of assessing the trustworthiness of the information and this provides a framework to identify the core criteria of the usefulness and the credibility of the information. Information quality, variously expressed as reliability, accuracy and authorial authority, is a pivotal factor in assessing trustworthiness, as are brand, recommendation, verification and ease of access. The framework crucially distinguishes the core criteria by which information is assessed and their influences, both from the information content and peripheral cues, and which may be variously applied according to different contexts and situations. In this way, trust is conceptualized as formed as a process, a series of judgements, as events involving the user and modelled on the core criteria that may invoke trust which, in turn, are determined by the various factors.

4. Methodology

4.1. Research design

The empirical study adopted a quantitative, survey-based research design, in order to gather sufficient data to be able to develop measurement items and explore relationships between variables in the process of trust formation. Questionnaires were chosen to collect data as they were deemed suitable for gathering large amounts of data and collecting accurate information.

A four-sided, paper-based questionnaire was developed. The core of this questionnaire was a bank of Likert-style statements, designed to investigate respondents’ perceptions of the relative importance of various aspects of web health information in their evaluation of its trustworthiness. Items (questions) to reflect the constructs for credibility, usefulness, content, authority, style, verification, brand, ease of use and recommendation were designed with a five-point scale. Whilst these constructs may not be observed directly, they may manifest through responses to questions regarding the construct. For example, to measure the impact of ‘style’ we ask survey participants about the importance of the way that the information is structured. The inclusion of specific factors and the questions (or items) on which they are based were largely drawn from previous research, adapted from Sillence et al. [15], Metzger [21], Walraven et al. [41], Hargittai et al. [34] and Wang and Emurian [17], as shown in Table 2. This, to some extent, ensures content validity in that the items used in the survey provide representative coverage of the possible questions and reflect the possible aspects of the construct. However, since few researchers have proposed multi-item scales of trust in digital information and its associated factors, in many instances it was necessary to generate new items in pursuit of proposing and validating a new scale. For face validity the researchers discussed the phrasing of the questions and ran a pilot that led to some minor changes. Thus the aim was to include a set of items reflecting different aspects of the construct such that the responses to the items could be taken as a manifestation of the construct.

Questionnaires were distributed to students in class settings. After a brief introduction, students were invited to complete the questionnaire. Prior to responding to the Likert-style statements, respondents were asked to think about a specific instance when they had looked for health or medical information on the internet. This domain was chosen to provide a type of critical incident as it was assumed that all, at some point, would have looked up information relating to a health issue and, aided by the trust inventory, would respond to the items reflecting their responses to the underlying constructs in assessing trustworthiness. They were then invited to indicate whether their search was triggered by general interest, or because they or a member of their family had a specific complaint.

At the end of the questionnaire, respondents were asked about their disposition to trust, and their health status, before being asked to provide basic demographic data such as gender, age, course level, course subject and student status (UK or International). Completed questionnaires were collected immediately by the researchers.

4.2. Participants

Participants were third-year undergraduate students on courses in Business and Sport Science at a large metropolitan university in the UK. Consistent with previous research studies on student and young people’s health information-seeking behaviour and trust judgements in digital environment [34, 41 –44], convenience sampling was employed to maximize response rate. Most respondents were aged between 18 and 21; there was a relatively even distribution on gender, and between the main subject categories. Most participants had either a level of involvement that fell into the general interest category (41%) or were answering in respect of their information searching with regard to a recent personal health issue with 43% very serious and 15% serious. On health status, 73.7% either agreed or strongly agreed that they were generally healthy, whilst 21.9% either agreed or strongly agreed that they had recently experienced a major health issue (Table 1).

Table 1.

Profile of participants.

		No.	%
Gender	Male	165	56.5
Gender	Female	127	43.5
Subject	Business	136	46.6
Subject	Sport	156	53.4
Level of involvement	General interest	121	41.4
	Not serious	126	43.2
	Serious	45	15.4
Health status	Generally healthy	215	73.7
	Recent major issues	64	21.9
	Not responded	13	4.4

4.3. Data analysis

Table 2 presents the descriptive statistics for all of the items in the measurement scale. Cronbach’s alpha coefficients for all intended constructs were higher than the minimum cut-off of 0.70 [45] and with the deletion of five items the coefficients ranged from 0.705 to 0.845, indicating internal consistency. The descriptive statistics show that most of the items have a mean between 3 and 4, which would indicate an upward bias in the sample. As we asked participants to respond to the questions based on information they had found in relation to some health information need we might assume a minimal rating on each of these constructs (i.e. that the information was considered to be useful and credible). Also we observe that the overall means for all of the constructs are similar, suggesting that all make a similar level of contribution to trust judgements and that there appears to be a range of factors influencing these judgements, confirming assertions that these judgements are complex. Specifically, on the basis of the means, the intended constructs credibility (4.27), ease of use (4.24) and content (4.03) could be considered to be most important in influencing trust judgement. Recommendation (3.93) and authority (3.90) were also high and brand (3.59) was the least important. Items with the highest rating within credibility, ease of use and content were: Whether I feel I can believe the information (4.42); The quality of the information (4.40); That the information was free (4.36); How easy it was to access the information (4.28); The comprehensiveness of the information (4.42); and The accuracy of the information (4.32).

Table 2.

Descriptive statistics of all means and constructs.

Variable (CA value)	Item	Code	Variable mean	Mean	SD
Authority (0.708) The standing of the author or organization responsible for providing the information	That the author/organization is known to me	AU1	3.82	4.14	1.01
	That the author appears to be knowledgeable	AU2		3.54	1.20
	That the author/org responsible for the information can be easily identified	AU3		3.74	1.00
	That the information appears to be objective	AU4		3.99	0.99
	That the author’s qualifications and/or expertise are indicated	AU5		3.71	1.16
Construct adapted from: Fogg et al. [46]; Sillence et al. [14]; Metzger [21]; Walraven et al. [41]; Hargittai et al. [34].
Style (0.756) The way in which the information is presented and written	The ease with which I can understand the information	ST1	3.87	4.21	0.90
	The clarity of the structure of the information	ST2		4.24	0.85
	The ease with which I can read the information	ST3		4.02	0.89
	The quality of the presentation of the information	ST4		3.79	0.99
	The presence of a table of contents	ST6		3.09	1.18
Construct adapted from Fogg et al. [46]; Wang and Emurian [17]; Metzger [21]; Sillence et al. [15]; Walraven et al. [41].
Content (0.778) Core characteristics of the information, such as reliability and accuracy	The currency of the information	CO1	4.03	3.70	0.97
	The reliability of the information	CO2		3.90	0.83
	The comprehensiveness of the information	CO3		4.42	0.85
	The accuracy of the information	CO4		4.32	0.94
	Mention of recent developments in treatment	CO6		3.80	0.93
Construct adapted from Fogg et al. [46]; Metzger [21]; Walraven et al. [41]; Hargittai et al. [34].
Usefulness (0.782) The extent to which the user is informed by and can make use of the information	That the information tells me most of what I need to know	UF1	3.80	4.49	0.68
	That the information helps me to understand the issue better	UF2		4.42	0.72
	That I have not found similar information before	UF3		3.22	1.02
	That I find the information interesting	UF4		3.43	1.11
	Whether the article gives me information that I can use	UF5		3.70	0.92
	Whether the article adds to my previous knowledge	UF6		4.09	0.83
	Whether the information seems to be tailored to me personally	UF7		3.08	1.08
	The advice seemed to be offered in my best interest	UF8		3.75	0.92
	The extent to which I felt that the information helped me	UF9		4.05	0.84
Construct adapted from Fogg et al. [46]
Brand (0.797)	The information source (e.g. website) carries advertising	BR1	3.59	3.54	1.16
	The information source carries the logo of a well-known brand	BR2		3.46	1.14
	The information source brand has a good reputation	BR4		3.90	1.01
	The information is on the website of a specialist health charity	BR5		3.45	1.10
Construct adapted from Fogg et al. [46] ; Sillence et al. [15]; Walraven et al. [41]; Hargittai et al. [34].
Ease of Use (0.821) Ease in access and use of the information	How easy it was to access the information	EU1	4.24	4.28	0.81
	How easy it was to find the information	EU2		4.23	0.85
	That the information was free	EU3		4.36	1.00
	The speed with which I found the information	EU4		4.10	0.94
Construct adapted from Metzger [21].
Recommendation (0.838) Recommendations regarding the information from known person	Family and friends have recommended the source to me	RE1	3.93	3.24	1.22
	A health professional has recommended the source to me	RE2		3.93	1.14
	I have seen online recommendations from users of the source	RE3		3.33	1.06
	I have seen recommendations from a social network community	RE4		2.96	1.09
	I have been advised against using a certain source	RE5		3.50	1.12
	My friends and family use the source	RE6		3.40	1.13
Construct adapted from Kelton et al. [6]; Rieh and Hillgoss [30];Hargittai et al. [34]; Lim [36].
Credibility (0.845)	Whether I feel I can believe the information	CR1	4.27	4.42	0.75
	The objectivity of the information	CR2		4.16	0.81
	The impartiality of the information	CR3		4.01	0.88
	The quality of the information	CR4		4.40	0.76
	The extent to which contains facts rather than opinions	CR5		4.34	0.86
Construct adapted from Sillence et al. [15]; Menchen-Trevino and Hargittai [44]; Lim [36].
Triangulation (0.705)	The inclusion of references to related sources	TR1	3.77	3.82	1.04
	Hyperlinks through to other web pages and documents	TR2		3.44	1.00
	Consistency with information that I have found elsewhere	TR3		4.02	0.91
	Consistency with my prior knowledge	TR4		3.79	0.94
Construct adapted from Wang and Emurian [17]; Metzger [21]; Rieh and Hillgoss [30]; Hargittai et al. [34]; Menchen-Trevino and Hargittai [44]; Lim [36].
Familiarity (0.744)	My familiarity with the topic	FA1	3.64	3.58	0.99
	Previous positive experience with info. from the same source	FA2		3.76	0.94
	Whether the information is on my favourite health web-site	FA3		3.32	1.16
	My confidence that I will understand information that I find	FA4		3.89	0.96
Construct adapted from Lucassen and Schraagen [20].

In order to verify the trust inventory in the questionnaire, factor analysis with principal component analysis (PCA) was used to analyse the relationships among the items and where these converge, to identify and extract the underlying factors in the participants’ responses. Based on items used in previous research, the trust scale to identify the likely factors influencing trust is shown in Table 2. The extracted factors from PCA are re-labelled according to the items that have converged in the analysis, as effectively measuring a common underlying or latent construct. These derived factors are presented in Table 4 labelled to reflect the propositions that these are characteristics and attributes of the information used in the process of assessing trustworthiness. Prior to conducting PCA, the suitability of the data for this test was established. The KMO value was 0.874, which is greater than the recommended value of 0.6. Bartlett’s test was test is statistically significant with a value of 0.000. Furthermore, if the questions (items) for a construct are well designed they should converge and form a major factor [46]. The major principal components (with eigenvalue >1) were extracted as constructs and to satisfy convergent validity, that all items intended to measure a construct do reflect that construct, the factor loadings were >0.5. Items with low loading or cross loading were removed as follows: CO1, CO6, AU1, AU2, ST6, TR4, UF5 and UF9. The principal components extracted suggest a good fit of 35 items to seven latent constructs accounting for 53.6% of total variance. Table 3 lists the eigenvalues associated with these seven factors, and the variance in the evaluation process associated with trust judgements explained by each of the factors. Specifically, factor 1 explains 25.0% of the total variance; factor 2, 7.7%; factor 3, 6.1%; factor 4, 4.4%; factor 5, 3.7%; factor 6, 3.5%; and factor 7, 3.1%. These were labelled follows, factors 1, Reliable Content; 2, Assessing Credibility; 3, Personal Recommendation; 4, Ease of Use − Access; 5, Assessing Usefulness; 6, Style − Readable; and 7, Branded − Logo. Thus ‘Reliable Content’ with items relating to the authorial authority and to the comprehensiveness and accuracy of the information accounted for the greatest total variance. Two factors were not formed on the survey items and these were intended to measure triangulation/verification and familiarity. As this is explained by the method adopted for data collection, it is discussed in concluding section of the paper.

Table 3.

Variance explained by each of the factors for trust.

	Initial eigenvalues			Extraction sums of squared loadings			Rotation sums of squared loadings
Component	Total	Percentage variance	Cumulative percentage	Total	Percentage variance	Cumulative percentage	Total	Percentage variance	Cumulative percentage
1	12.51	25.02	25.02	12.51	25.02	25.02	4.64	9.27	9.27
2	3.87	7.74	32.76	3.87	7.74	32.76	3.60	7.20	16.47
3	3.05	6.11	38.87	3.05	6.19	38.87	3.50	7.00	23.47
4	2.22	4.45	43.31	2.22	4.45	43.31	3.25	6.51	29.97
5	1.87	3.74	47.06	1.87	3.74	47.06	2.85	5.71	35.68
6	1.74	3.48	50.54	1.74	3.48	50.54	2.74	5.47	41.15
7	1.54	3.09	53.62	1.54	3.09	53.62	2.69	5.37	46.52

The results from the PCA suggest a good fit of items to latent constructs To further determine the items as attributable to a factor, Confirmatory Factor Analysis was used to test the factor structure extracted and ensure instrument quality, prior to its use in structural equation modelling. According to Segars and Grover [48], the measurement model should be evaluated first and then re-specified as necessary to generate the ‘best-fit’ model. Therefore the pre-specified construct–item correspondences must be significant with composite factor reliability (CFR) >0.7 and Cronbach’s alpha >0.5 [47] and with the construct explaining >50% of the total variance (with the average variance extracted for an item >0.5). This process led to a refined measurement model with seven factors and 21 items (Table 4). Item reliability ranged from 0.65 to 0.97, thus exceeding the acceptable value of 0.5. Composite reliability for these seven factors ranged from 0.761 to 0.926, which is >0.7. Finally, the average variance extracted ranged from 0.516 to 0.805, and for all items exceeded the threshold value of 0.5 recommended by Fornell and Larcker [49]. Together these indices show that the model has an appropriate level of reliability, convergent validity and determinant validity.

Table 4.

Convergent validity of the measurement model.

		CFR	Cronbach’s alpha
Factor 1 Reliable Content	AU4 − That the information appears to be objective (i.e. no hidden agendas)	0.65	0.826
	CO3 − The reliability of the information	0.73
	CO2 − The comprehensiveness of the information	0.74
	CO4 − The accuracy of the information (such as the absence of errors)	0.77
Factor 2 Assessing Credibility	CR5 − The extent to which the source contains facts rather than opinions	0.66	0.844
	CR3 − The impartiality of the information	0.69
	CR1 − Whether I feel I can believe the information	0.70
	CR4 − The quality of the information	0.75
	CR2 − The objectivity of the information	0.81
Factor 3 Personal Recommendation	RE4 − I have seen recommendations from members of a social network	0.71	0.787
	RE1 − Family and friends have recommended the source to me	0.73
	RE6 − My friends and family use the source	0.79
Factor 4 Ease of Use − Access	EU1 − How easy it was to access the information	0.89	0.923
Factor 4 Ease of Use − Access	EU2 − How easy it was to find the information	0.97	0.923
Factor 5 Assessing Usefulness	UF1 − That the information tells me most of what I need to know	0.78	0.815
Factor 5 Assessing Usefulness	UF2 − That the information helps me to understand the issue better	0.88	0.815
Factor 6 Style − Readable	ST3 − The clarity of the structure of the information	0.67	0.848
	ST1 − The ease with which I can understand the information	0.85
	ST2 − The ease with which I can read the information	0.94
Factor 7 Branded − Logo	BR1 − The information source features the logo of a respected brand	0.90	0.891
Factor 7 Branded − Logo	BR2 − The information source carries the logo of a well-known brand	0.90	0.891

4.4. Structural relationships among the factors

When all constructs are measured with multiple questions in the survey, and the ‘latent meaning’ extracted, the proposed relationships between the independent (e.g., Style − Readable and Personal Recommendation) and the dependent (i.e. Assessing Credibility and Assessing Usefulness) can be tested. Structural equation modelling in AMOS specifies both item–construct correspondence and construct–construct causal relationships. If the hypotheses are supported, we expect significant regression coefficients when they are solved with maximum likelihood estimation. This procedure of structural equation modelling was followed for the relationship between Assessing Usefulness and Credibility and the other variables.

The fit of the data model can be shown with the root mean square error of approximation (RMSEA) <0.10 at 0.059 and 0.052 and the goodness of fit index >0.9 at 0.961 and 0.978. The model (Figure 1) would seem to support the propositions that Assessing Usefulness and Assessing Credibility are criteria in (the formation of) trust judgement and are determined by factors relating to document properties or contextual influences. The factor Assessing Usefulness held the strongest associations with Reliable Content (0.74), Style − Readable (0.87) and Ease of Use − Access (0.66), whereas the factor Assessing Credibility held an association with Reliable Content (0.76), Style − Readable (0.61) and to lesser extent Ease of Use − Access (0.46). Branded − Logo held a weak association with both Assessing Credibility and Usefulness. The regression for the factor labelled Personal Recommendation was not significant nor obtained on either of the dependent factors of Assessing Credibility or Usefulness.

Figure 1.

Trust formation by core criteria (Assessing Credibility and Usefulness) determined by Information Quality and Peripheral factors.

4.5. Relationships among the factors

The conceptualization of trust formation presented in this study stems, in the main, from the recognition that trust in digital information is formed in a dependency state and, as such, the judgements made on criteria such as usefulness or credibility may depend on the context, particularly user or task characteristics. To illustrate the potential use of the model of trust formation, as a framework for such further investigation, a regression analysis was run on each of the factors that influence Assessing Usefulness and Credibility. With the intention to gain insight into the dependency of the judgements, this analysis was carried out on data from participants who reported a high involvement and from those who reported a low involvement in health information. In these two contexts it may be assumed that the different levels of involvement, or interest invested in the information, will have some impact on the assessment of trustworthiness. Significant regression coefficients were obtained on Assessing Usefulness and Assessing Credibility with the factors Reliable Content, Style- Readable, Ease of Use − Access and Branded − Logo when run across participant data with high level of involvement and participants with low level of involvement (with high level of involvement the factors accounted for 50% of variance in Usefulness (p < 0.001 d.f. = 42, anova = 12.821, F = 9.612) and 37.6% in Credibility (p < 0.005, d.f. = 42, anova = 18.663, F = 5.736); with a low level involvement the predictors accounted for 43% of variance in Usefulness (p < 0.001, d.f. = 234, anova = 71.033, F = 43.33) and 34.5% in Credibility (p < 0.001, d.f. = 237, anova = 94.784, F = 30.742)). Of particular interest, however, is the contribution of each predictor variable to the criteria (Table 5). With participants with a high level of involvement only Style − Readable held a significant coefficient in association with Assessing Usefulness (β = 0.483, p < 0.005) and only Reliable Content with Credibility (β = 0.548, p < 0.005). With a low level of involvement the regression model for Assessing Usefulness shows each of the four factors contributing in a similar way to the coefficient associations of Reliable Content (β = 0.238, p < 0.001), Style − Readable (β = 0.231, p < 0.001), Branded − Logo (β = 0.213, p < 0.001) and Ease of Use − Access (β = 0.201, p < 0.005). On Credibility an association was held on all factors except Style − Readable with the coefficient associations of Reliable Content (β = 0.398, p < 0.001), Branded − Logo (β = 0.127, p < 0.05) and Ease of Use − Access (β = 0.243, p < 0.001). It appears that with a high involvement, trust may be formed on Assessing Usefulness and Credibility and the factors that might influence these judgements relate to the Information Quality, and in this dataset these were Style − Readable with Assessing Usefulness, and Reliable Content with Assessing Credibility. With a low involvement, trust is also formed on Assessing Usefulness and Credibility and both the Information Quality and peripheral factors might influence these judgements, as in this dataset: Reliable Content, Style − Readable, Branded − Logo and Ease of Use − Access related to Assessing Usefulness, and Reliable Content, Ease of Use − Access and Branded Logo related to Assessing Credibility. It is tempting to conclude that a low involvement leads to the use of many and various factors in assessing information usefulness and credibility, whereas with a high involvement specific use is made of only information quality, for example Style − Readable. The analysis carried out here is for illustrative purposes to suggest that modelling of the factors, as determinants of the core criteria in assessing trustworthiness, may be used to explore such variations in the process invoked by different task contexts.

Table 5.

Coefficients for each factor to usefulness and credibility across user level of involvement/interest.

	Reliable Content	Style − Readable	Ease of Use − Access	Branded − Logo
Assessing Usefulness
Low involvement	β = 0.238, p < 0.001	β = 0.231, p < 0.001	β = 0.201, p < 0.005	β = 0.213, p < 0.001
High involvement	β = 0.215, p = 0.147	β = 0.483, p < 0.005	β = 0.129 p = 0.371	β = 0.003, p = 0.981
Assessing Credibility
Low involvement	β = 0.398, p < 0.001	β = 0.016, p = 0.814	β = 0.243, p < 0.001	β = 0.127, p < 0.05
High involvement	β = 0.548, p < 0.005	β = 0.007, p = 0.966	β = 0.088, p = 0.587	β = 0.053, p = 0.703

5. Discussion and conclusions

The results of the exploratory and confirmatory factor analysis present the seven constructs involved in the formation of trustworthiness judgement of information (in health domains). These explain 53.6% of the variance in the data on the trust scales, suggesting that these factors are comprehensive in explaining trust judgements. The premise was that the relationship held between these constructs of trust would be such that the criteria as antecedents to trust formation are determined by the independent variables of the document properties (or perceptions of) and/or by other peripheral factors. The results show that the constructs of content, style and ease of access are most strongly associated with the constructs of assessing credibility and usefulness. Thus, assessing credibility and usefulness, when considered as key antecedents to trust formation, may be influenced by both information content and style and by peripheral factors such as ease of use. Previously, Sillence et al. [15] divided the factors contributing to the selection and trust of web sites into content factors, such as informative content and into design factors, such as good navigation aids. They also recognized the judgement to be dynamic, as do others such as Lucassen and Schragen [50], suggesting that trust formation is further influenced by user characteristics (such as domain expertise), which lead to different features of the information being used in trust judgements. Whilst this present study did not gather details on domain knowledge, a distinction was made on level of involvement in the information and, further analysis based on the model of factors influencing the judgements of usefulness and credibility indicated variation in the peripheral and the factors relating to information quality in assessing trustworthiness. Related research on the dependency role of the task context, for example user domain knowledge, has focused the impact on query attributes, search strategies and tactics, and search outcomes [51] and as determinants of the user’s ability to find useful information [52, 53]. For example, in relation to online health information, Keselman et al. [54], found that imprecise domain knowledge led consumers to search for information on irrelevant sites. Focusing more on the user’s evaluation of the information, Lucassen and Schraagen’s [50] study of Wikipedia use showed that those familiar with a topic focussed on the semantic features of the information, whilst those who were unfamiliar with the topic paid more attention to surface features. The ELM possibly goes some of the way towards explaining this in suggesting that user characteristics, such as motivation, explain the different uses of content and peripheral cues in credibility assessment. Both Cacioppo and Petty [32] and Sundar [31] have suggested that individuals may base their judgement of a message, not on its content, but on peripheral cues when the motivation to engage in the mental effort in analysing a message based on content is missing or lacking. In this present study it appears that, with a passing interest in the information, both factors relating to information quality and those which may be regarded as peripheral to content were used in the judgements of information usefulness and credibility, whereas when the investment or interest in the information was high, the factors relating to information quality had more influence on the user judgements.

The results of this study with the core criteria of trust, assessing usefulness and credibility, variously influenced by the content and peripheral factors suggests that, whilst the judgements are formed as antecedents to trust, their influence may vary with different factors used to form these judgements. In modelling trust formation in this way it is possible to allow for the fact that, regardless of the user context, trust will be formed based on an evaluation of the information usefulness and credibility; however the influence of various factors on these will vary according to the user context, in other words the dependency state in which trust is formed.

5.1. Limitations

The model is incomplete as the constructs of familiarity and triangulation/verification were removed from the model of factors in trust formation. The consequence is that important dimensions to the ‘trust model’ may have been omitted, but are accounted for in the explanation that they were not adequately observed in the questionnaire setting. In the propositions on trust these and other peripheral (i.e., not content based) clues could predict or influence trust formation. However our respondents were, more than likely, challenged to respond to these items in the survey. The proposition on the impact of the peripheral clues on assessing trustworthiness requires data collection during the users’ search for the information in question or immediately after. Whilst this hindered our ability to investigate the peripheral clues, it is notable that a ‘peripheral’ factor, Ease of Use − Access was found to hold a relation with the criteria of trust, especially Assessing Usefulness. To strengthen the model the data collection may be better carried out in situ, particularly to investigate the rating on the criterion of Assessing Usefulness and the influence of peripheral factors in making this judgement. Whilst the usefulness construct was measured on the two items, UF 1 and UF2 (the information tells me most of what I need to know and the information helps me to understand the issue better), when conducted in the immediate context of finding the information, some of the items relating to the informativeness of the information, such as UF 4 (I find the information interesting) or UF9 (The extent to which I felt that the information helped me) might converge in the Assessing Usefulness factor. In the limitations of the survey the decision was made to delete these items and the factors of verification/triangulation and familiarity.

5.2. Trust revisited

The aim of this study was to identify the core criteria and influencing factors in assessing the trustworthiness of digital information. The guiding framework was to define trust formation based on the assessment of trustworthiness in a dependency state and on key criteria employed in the critical evaluation of information and evidenced by a set of factors. The key propositions presented are that the judgements of usefulness and credibility are core to trust formation. Whilst trust itself may be intangible, these judgements of usefulness and credibility are measurable and can be treated as the dependent variables, that is, the judgements of the information that are essential in trust formation. This framework also allows the important distinction between the judgements in trust formation and the factors or constructs on which these judgements are based, and furthermore, the distinction is made between the influencing factors that are information attributes and those that are peripheral such as ease of access, brand and recommendation Whilst previous studies have speculated on the influence of all these factors in conceptualizing trust, adopting a psychometric procedure for the analysis of the questionnaire in this study enabled the modelling of the relations held among the constructs of trust being Assessing information Usefulness, Assessing information Credibility, Reliable Content, Style − Readable, Ease of Use − Access and Branded − Logo. There is scope, however, to collect further data on the assessment of trustworthiness when interacting with specific information to model possible further constructs of trust. Furthermore, in order to establish the extent to which the findings of this study are reflective of a generalized model of trust formation, further research would be required across domains other than health information. However, the contribution made in disentangling the factors influencing trust lies in the possible opening up of the investigation into trust formation in and across various contexts where the influences may have varying importance in the critical assessment of trustworthiness leading to intention to use.

References

Kohn

. Trust: Self interest and the common good. Oxford: Oxford University Press, 2008.

Chopra

Wallace

. Trust in electronic environments. In: Proceedings of the 36th IEEE annual Hawaii international conference on system sciences, 2003.

Shekorpour

Ketebi

. Modeling and evaluation of trust with an extension in semantic web. Web semantics: Science and agents on the World Wide Web 2010; 8(1): 20, 26–36.

Yoon

Davis

Lee

. Untangling the antecedents of initial trust in Web-based health information: The roles of argument quality, source expertise, and user perceptions of information quality and risk. Decision Support Systems 2013; 55(10): 284–295.

Ivanov

Vajda

Lee

Ebrahimi

. In tags we trust: Trust modeling in social tagging of multimedia content. IEEE Signal Processing Magazine 2012; 29(2): 98–107.

Kelton

Fleischman

Wallace

. Trust in digital information. Journal of the American Society for Information Science and Technology 2008; 59(3): 363–374.

Belanger

Carter

. Trust and risk in e-government adoption. The Journal of Strategic Information Systems 2008; 17(2): 165–176.

Rotter

. Interpersonal trust, trustworthiness, and gullibility. American Psychologist 1980; 35(1): 1–7.

Hardin

. Trust and trustworthiness 4. Russell Sage Foundation, 2004.

10.

Rowley

Johnson

. Understanding trust formation in digital information sources: The case of Wikipedia. Journal of Information Science 2013; 39(4): 494–508.

11.

Goudge

Gilson

. How can trust be investigated? Drawing lessons from past experience. Social Science & Medicine 2005; 61(7): 1439–1451.

12.

Corritore

Wiedenbeck

Kracher

Marble

. Online trust and health information websites. International Journal of Technology and Human Interaction 2012; 8(4): 92–115.

13.

. Correlates of consumer trust in online health information: Findings from the Health Information National Trends Survey. Journal of Health Communication 2011; 16(1): 34–49.

14.

Sillence

Briggs

Fishwick

Harris

. Trust and mistrust of online health sites. In: Proceedings of CHI 2004. New York: ACM Press, 2004, pp. 663–670.

15.

Sillence

Briggs

Harris

Fishwick

. How do patients evaluate and make use of online health information? Social Science & Medicine 2007; 64: 1853–1862.

16.

Mechanic

Meyer

. Concepts of trust among patients with serious illness. Social Science & Medicine 2000; 51(5): 657–668.

17.

Wang

Emurian

. An overview of online trust: Concepts, elements and implications. Computers in Human Behaviour 2005; 21: 105–125.

18.

Briggs

Burford

De Angeli

Lynch

. Trust in online advice. Social Science Computer Review 2002; 20(3): 321–332.

19.

O Neill

. A question of trust: The BBC Reith lectures. Cambridge: Cambridge University Press, 2002.

20.

Lucassen

Schraagen

. Factual accuracy and trust in information: The role of expertise. Journal of the American Society for Information Science and Technology 2011; 62(7): 1232–1242.

21.

Metzger

. Making sense of credibility on the Web: Models for evaluating online information and recommendations for future research. Journal of the American Society for Information Science and Technology 2007; 58(13): 2078–2091.

22.

Wathen

Burkell

. Believe it or not: Factors influencing credibility on the web. Journal of the American Society for Information Science and Technology 2002; 53(2): 134–144.

23.

Cugelman

Thelwall

Dawes

. The dimensions of web site credibility and their relation to active trust and behavioural impact. Communications of the ACM 2009; article 26: 455–472.

24.

Shen

X-L

Cheung

Lee

MKO

. What leads students to adopt information from Wikipedia? An empirical investigation into the role of trust and information usefulness. British Journal of Educational Technology 2012; 44(3): 502–517.

25.

Harris

Sillence

Briggs

. Perceived threat and corroboration: Key factors that improve a predictive model of trust in Internet-based health information and advice. Journal of Medical Internet Research 2011; 13(3): e51.

26.

Fergie

Hunt

Hilton

. What young people want from health-related online resources: A focus groups study. Journal of Youth Studies 2013; 16: 579–596.

27.

Sillence

Briggs

Harris

Fishwick

. Going online for health advice: Changes in usage and trust practices over the last five years. Interacting with Computers 2007; 19: 397–406.

28.

Saracevic

. Relevance: A review and a framework for the thinking on the notion in information science. Journal of the American Society for Information Science and Technology 1975; 26(6): 321–343.

29.

Borlund

. The concept of relevance in IR. Journal of the American Society for Information Science and Technology 2003; 54(10): 913–925.

30.

Rieh

Hilligoss

. College students’ credibility judgements in the information seeking process. In Metzger

(ed.) Digital media, youth and credibility. Cambridge, MA: MIT Press, 2008, pp. 49–71.

31.

Sundar

. The MAIN model: A heuristic approach to understanding technology effects on credibility. Digital Media, Youth and Credibility 2008, 73–100.

32.

Cacioppo

Petty

. The elaboration likelihood model of persuasion. Advances in Consumer Research 1984; 11(1).

33.

Tseng

Fogg

. Credibility and computing technology. Communications of the ACM 1999; 42(5): 39–44.

34.

Hargittai

Fullerton

Menchen-Trevino

Yates Thomas

. Trust online: Young adults’ evaluation of web content. International Journal of Communication 2010; 4: 468–494.

35.

Shankar

Urban

Sultan

. Online trust: A stakeholder perspective, concepts, implications, and future directions. Strategic Information Systems 2002; 11: 325–344.

36.

Lim

. How and why do college students use Wikipedia? Journal of the American Society for Information Science and Technology 2009l 60(11): 2189–2202.

37.

Hertzum

Andersen

HHK

Andersen

Hansen

. Trust in information sources: Seeking information from people, documents, and virtual agents. Interacting with Computers 2002; 14: 575–599.

38.

Pan

Hembrooke

Joachims

Lorigo

Gay

Granka

. In Google we trust: Users’ decisions on rank, position and relevance. Journal of Computer-mediated Communication 2007; 12: 801–823.

39.

Bailey

Thomas

Hawking

. Does brand name influence perceived search result quality? In: Yahoo!, Google, and Web. Proceedings of ADCS, 2007.

40.

Iding

Crosby

Auerheimer

Klemm

. Web site credibility: Why do people believe what they believe? Instructional Science 2009; 37(1): 43–63.

41.

Walraven

Brand-Gruwel

Boshuizen

HPA

. How students evaluate information and sources when searching the World Wide Web for information. Computers & Education 2009; 52(1): 234–246.

42.

Dobransky

Hargittai

. Inquiring minds acquiring wellness: Uses of online and offline sources for health information. Health Communication 2012; 27(4): 331–343.

43.

Gray

Klein

Noyce

Sesselberh

Cantrill

. Health information-seeking behaviour in adolescence: The place of the Internet. Social Science & Medicine 2005; 60(7): 1467–1678.

44.

Menchen-Trevino

Hargittai

. Young adults’ credibility assessment of Wikipedia. Information, Communication & Society 2011; 14(1): 24–51.

45.

DeVellis

. Scale development: Theory and applications, 2nd edn. Thousand Oaks, CA: Sage 2003.

46.

Fogg

Marshall

Laraki

Osipovich

Varma

Fang

. What makes web sites credible? A report on a large quantitative study. In: Proceedings of CHI’01: Human factors in computing systems, 2001, pp. 61–68.

47.

Hair

Tatham

Anderson

Black

. Multivariate data analysis, 5th edn. Englewood Cliffs, NJ: Prentice Hall, 1998.

48.

Segars

Grover

. Strategic information systems planning success: An investigation of the construct and its measurement. MIS Quarterly 1998; 22(2): 139–163.

49.

Fornell

Larcker

. Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research 1981; 18: 39–50.

50.

Lucassen

Schraagen

. Topic familiarity and information skills in online credibility evaluation. Journal of the American Society for Information Science and Technology 2013; 64: 254–264.

51.

White

Dumais

Teevan

. Characterizing the influence of domain expertise on web search behaviour. In: Proceedings of WSDM, 9–12 February, Barcelona, 2009, pp. 132–141.

52.

Wildemuth

. The effects of domain knowledge on search tactic formulation. Journal of the American Society for Information Science and Technology 2004; 55(3): 246–258.

53.

Hembrooke

Gay

Granka

. The effects of expertise and feedback on search term selection and subsequent learning. Journal of the American Society for Information Science 2005; 56(8): 861–871.

54.

Keselman

Browne

Kaufman

. Consumer health information seeking as hypothesis testing. Journal of the American Medical Informatics Association 2008; 15(4): 484–495.