Perceived Quality of Conversations in Online Communities: Conceptual Framework,Scale Development,and Empirical Validation

Abstract

Conversations among online community members as a form of everyday, unconstrained, flexible, and spontaneous communication are important for developing a sense of community, sustainability of online communities, construction of identity, public sphere, and for creating and maintaining offline taken-for-granted basic structures of everyday life. While the lack of qualities of online conversations in online communities is often discussed and researched, we argue that its positive sides should also be conceptualized and become the subject of quantitative empirical research. The purpose of this paper is to introduce the concept and scale of perceived quality of online conversations and inspect the psychometric properties of the proposed scale in terms of content and convergent, discriminant, and criterion validity and reliability. The paper presents a five-factor structure of the quality of conversation scale, which is tested for its quality on a sample of 1,276 online community participants. Confirmatory factor analysis supports the proposed multidimensional structure, while correlational and regression analyses demonstrate good levels of convergent, discriminant and criterion validity. The paper suggests several refinements of the measurement instrument, and concludes with the usefulness of the introduced scale for the research and understanding of the online community phenomenon.

Introduction

Online communities as computer-mediated places where people come together with others to converse, exchange information or other resources, learn, play, or just be with each other range from small close-knit groups to large and globally sparse networks with millions of participants.¹ They have become a ubiquitous new form of communication and social interaction, which research demonstrates may serve many different purposes: exchanging of social support,² medical help,³ sharing knowledge,⁴ education purposes,⁵ to get psychologically empowered,⁶ to name just some of them. Along with the popularity they have gained, it can be claimed that online communities have become new spatial-temporal zones of everyday life⁷ with important psychosocial functions in contemporary society.^8,9

Understanding communication processes in online communities is crucial, as conversation among online community members is among the main constitutional building blocks of each online community.^10–12 Furthermore, online conversation as a specific type of communication on everyday matters, which is spontaneous, unconstrained, and flexible,¹³ plays an important role in (re)creating the social glue that holds an online community together,¹⁴ enables its sustainability,¹⁵ helps build social capital,¹⁶ and stimulates development of online communities as part of public sphere.¹⁷ Moreover, with online communities becoming an important part of everyday life, it can be claimed that in online conversations about the weather, relationships, work, health, politics, and other everyday issues, we also create and maintain offline taken-for-granted basic structures of everyday life,⁷ friendships,¹⁷ and construct the concept of self and other and a sense of community.¹⁸ However, it is clear that not every conversation leads to such positive outcomes. Numerous concerns have been raised over the quality of online conversations with claims that failure to sustain a certain level of quality of conversations in the online community can even lead to the dissolution of the online community.¹⁹

There are extensive empirical accounts of an absence of quality of conversation in online communities, manifesting in hate speech, flaming, deception, and other distortions in communication.^20–22 While these are often discussed and empirically researched, the actual quality of conversations is rarely conceptualized, measured, or empirically investigated in the field of online community research. Moreover, while this field has already introduced several scales for measuring phenomena in online communities such as sense of virtual community,²³ online trust,²⁴ social support exchange,²⁵ there is a general lack of scales that would tap into the quality of communication processes in online communities. In line with Preece's call to acquire more knowledge on interactions in online communities,²⁶ we find it important to introduce the concept of and a scale for the perceived quality of conversations in online communities (PQOC). The goals of this study are thus (a) to define the content domain of the PQOC construct; (b) test its multidimensional structure; and (c) investigate the convergent, discriminant, and criterion validity of the proposed scale.

Defining Quality of Conversations in Online Communities

The first step in developing a valid measure of a concept is to find its theoretically grounded definition and identify the content domain of the concept in order to achieve content validity.²⁷ When authors emphasize communication as one of the main conditions for the existence of an online community, it is assumed that conversations between members are not uncivil, rude, or in any way distorted. Yet only a few authors discuss its positive qualities. Among these Preece,²⁶ for example, notes emphatic qualities of online interaction that are crucial for establishing and maintaining the social glue of online communities. Waks¹⁷ suggests that respect and mutual understanding in conversations are vital for the sustainability of online communities. Several authors mention the importance of perceptions of online communities' respectful, open, and reciprocal communication for the formation of group identity.^14,18 This list of positive attributes is helpful but does not offer a sound theoretical basis to identify the full content domain of the quality of conversations in online communities.

To define the quality of online conversations, we have to consider first what is meant by the quality of conversation is in general. A good starting point for this is, in our opinion, Habermas's work on discursive ethics,²⁸ where he elaborated an important claim that through informal and general conversations, which follow discursive ethics, mutual understanding and establishment of self and community is possible. Habermas related discursive ethics to conversations, which strive to agreement, and are based on reasoning and truthfulness.²⁸ Discursive qualities of conversation were more systematically conceptualized and also operationalized in the field of online deliberation research. Although this field is focused on political communication, recent work claims that online deliberation primarily occurs in nonpolitical online settings, suggesting that the criteria for what constitutes the quality of deliberative communication can be applied to everyday conversation.^29,30 This especially holds for social dimensions of deliberative communication,^18,31 which explicitly refer to qualities of conversations such as mutuality, reciprocity, rationality, openness, perspective taking, empathy, and sincerity. In our research, we defined the content dimensions of quality of conversation by relying on Dahlberg's criteria of deliberative communication^32,33 for two reasons: (a) he explicitly refers to Habermas's discursive ethics, and (b) upon these criteria, items were already developed and used in political communication research.

Quality of online conversations in online communities can then be defined as communication interchanges between members of an online community on everyday matters, which are characterized by reciprocity, justification, reflexivity, ideal role taking, and sincerity. The dimensions of PQOC are defined as follows.

Reciprocity refers to the quality of an online conversation as an interchange of listening, reading, and responding. In other words, it is the perceived degree to which a discussion among members of an online community is a real conversation and not just an exchange of monologues.³⁴ Justification refers to the perceived quality of conversation in terms of expectations that members would provide valid reasons for their statements if asked to do so.³³ Specifically, this refers to offering explanations, illuminating background knowledge of a certain opinion, and exchanging well-formed arguments.³⁵ Reflexivity grasps the perceived degree to which members of an online community are aware of their own biases and limitations of knowledge, and critically examine their values, assumptions, and interests.³³ It means that members are considering the arguments of others, reflecting upon their own claims, changing their positions, and considering what others write during the conversations. Ideal role taking refers to ongoing, respectful listening,³³ and includes treating others with respect, taking into account others' perspectives, and empathetic understanding of various positions.²⁸ Sincerity relates to members making an effort to express their true intentions, needs, and desires sincerely and revealing all relevant information.³³

It should be noted that we conceptualize perception of quality of online conversations and not the actual quality for several reasons. Various authors^14,18 claim that perceptions of the respectful, open, reciprocal, and mutual interpersonal communication are relevant for the development of a sense of belonging to the virtual community. This is further supported by the theory of relational signals,¹⁹ which claims that online community members actively monitor signals, which other members are sending with their behaviors, and form their own behaviors on the basis of these perceptions.

On the basis of the above presented content dimensions of PQOC, we propose the following hypothesis:

H1: The scale of perceived quality of conversations in online communities is a five-dimensional structure consisting of the above-identified dimensions.

To establish construct validity, we usually investigate whether the proposed measure “behaves” in relation to established measures of other constructs from the field in a way that it is supposed to behave.²⁷ More specifically, convergent validity refers to an empirical similarity between measures of theoretically related constructs, while discriminant validity pertains to the absence of correlation between measures of constructs that are theoretically unrelated.²⁷ Based on an extensive review of the literature on online communities, there exists a theoretical rationale for the association of PQOC with two relevant constructs in the field of online communities: online trust²⁴ and perceived credibility of exchanged information.²⁵ Online trust is directed to the whole online community instead of specific individuals and to a certain extent shows similarities with the PQOC construct. Specifically, online trust pertains to honesty,²⁴ sincerity of self-presentation and identity,³⁶ and empathic qualities of communication.²⁶ Online trust forms the basis for a belief that online community members will act cooperatively and will not take advantage of others.³⁷ Perceived credibility of exchanged information in online communities²⁵ is also theoretically closely related to the PQOC construct. The theoretical rationale lies in the fact that information processes in online communities are inherently linked to communication processes, as joining discussions among members, writing messages, tagging, posting and answering questions essentially produces information and knowledge in online communities.¹⁰ The credibility of online information is thus inherently linked to the quality of communication processes.

The rationale for investigating discriminant validity was to correlate PQOC with two dimensions of the sociopolitical control scale: self-efficacy and perceived leadership competence.³⁸ Both constructs pertain to an individual's own assessment of his/her competences, and are theoretically unrelated to an individual's assessment of communication processes in an online community of which he/she is a member. To test the convergent and discriminant validity, we thus propose the following hypothesis:

H2: Perceived quality of conversations in online communities correlates with online trust and credibility of exchanged information but does not correlate with self-efficacy and perceived leadership competence.

To get insight into the criterion validity of the PQOC construct, the latter was tested as an important antecedent of sense of virtual community (SOVC), which is one of the central concepts in quantitative studies of online communities. The concept of SOVC has already been empirically validated^15,24,39 and used as an antecedent of various important phenomena in online communities such as customer loyalty,⁴⁰ participation intention,⁴¹ and purchase intention.⁴² The literature suggests that various factors lead to the development of SOVC, among which empirical support has already been demonstrated for offline activities,^15,42 exchanging of social support,³⁹ and intensity of participation.⁴³ The role of PQOC in SOVC has not yet been analyzed, but there is strong rationale to claim that the former might be an important antecedent of the latter. For instance, Stromer-Galley⁴⁴ claims that the goal of casual conversations is often to socialize and to build social relationships. Kim and Kim¹⁸ further claim that dialogic qualities of communication help to construct a sense of community. The literature also suggests that perceptions of a community's conversations are important for the development of a sense of group belonging¹⁸ and emotional closeness,¹⁷ which are key components of the SOVC concept. We thus propose the following hypothesis:

H3: Perceived quality of conversations in online communities is an important antecedent of SOVC controlling for variables already empirically validated by existing research.

Methods

Sample

Online communities can nowadays be formed by groups of individuals who communicate with each other using various Internet communication channels, such as bulletin boards, e-mail lists, or social networking platforms.¹⁹ Our empirical study was limited to online forums, which are an early form of computer mediated communication, implying that they are already stabilized in people's everyday life routines while at the same time still being popular venues for Internet users' activities.^45,46 According to the data from the Slovenian statistical office, 27% of Internet users in Slovenia between the ages of 10 and 75 years participate in conversations by posting messages in online forums.⁴⁶ The data for our research were collected via a web survey, which was promoted on a sample of Slovenian online forums. First, a representation of the population of all online forums in Slovenia was identified by using various search engines and was reduced to those that had at least one message posted in the last month. This procedure resulted in a list of 191 online forums. Since a random selection of units was not possible, we decided upon a common approach in online community research.^3,47 Following methodological instructions for advertising surveys in discussion boards,⁴⁸ an invitation e-mail was sent to the administrators of the selected forums with a request to publish a link to the web survey. In total, the survey was published on 95 online forums for a minimum of 2 weeks in November 2010. The final sample size was 1,276 active participants in online forums (i.e., those who posted at least one message in the last 3 months) with a mean age 31.4 years (SD=9.9). Among the participants, 58.9% were female, which is significantly different (χ²=3.72, p=0.05) than the estimated population value of 44.1%.⁴⁶ The sample consisted of members of various online communities, such as online support groups, online communities of practice, and specific interest groups. The higher proportion of women in the sample is a consequence of the fact that online support groups, which are most represented in the sample, are primarily used by women.⁴⁹

Measures

Perceived quality of online conversations in online communities

Originally, 25 items were developed for the scale, five for each dimension. Suggestions for wording of some items were found in the online deliberation field, where items are used for measuring concepts in relation to reciprocity, justification, reflexivity, ideal role taking, and sincerity.^50,51 These were adapted to the context of everyday online conversations in online communities. Other items were originally developed upon the above presented conceptual definitions. The initial item pool was evaluated for face and content validity by three experts in the field of social science methodology and Internet research. Based on their recommendations, several items were changed and reformulated. Items were additionally pretested in a pilot study on a convenience sample of 270 online forums users, which confirmed the 5-factor structure of the PQOC concept. In the final study, 15 items (three per subscale) were included that performed best in terms of validity and reliability. Items were introduced in the questionnaire with the sentence: “Now we would like you to think about the conversations among forum members about different topics (economy, everyday life, politics, society, etc.). To what extent do you agree with the following statements about such conversations?” The scales were ordinal, ranging from 1=“completely disagree” to 5 “completely agree.” The wording of all items, including those that were excluded in the final study, can be found in Table 1.

Table 1.

Standardized Estimates of Factor Loadings for a Five-Factor Solution for Each Item ^a of the PQOC Measurement Instrument ^b–d

	Items	Recip	Justif	Refl	Ideal	Sincer	Mean (SD)
V1	Members of the forum put their heads together to find answers on posts from other forum members.	0.66					2.39 (1.08)
V2	Majority of forum members leave the conversation once they get the information they wanted. (r)	0.44					3.73 (0.95)
V3	Members of the forum join discussions only when they are irritated by the opinions of others. (r)	0.57					2.94 (1.00)
V4	Members of the community usually explain why they don't agree with someone or something.		0.84				3.15 (0.97)
V5	In the forum, the power of argument prevails.		0.57				3.83 (0.74)
V6	Participants in conversations do not provide arguments for their opinions, even if they are asked for them. (r)		0.40				3.47 (0.95)
V7	I sometimes notice in discussions that members admit that their knowledge on some issues is limited.			0.64			2.39 (0.91)
V8	Members of the forum do not change their opinions, even though others might have better arguments. (r)			0.83			2.94 (1.04)
V9	Participants in conversations try to understand different perspectives of the issue in question.			0.39			3.53 (1.03)
V10	Members of community often try to put themselves in the place of others while having a conversation.				0.84		2.76 (1.07)
V11	In conversations, members of the forum are often offended if others have different opinions. (r)				0.38		3.14 (0.95)
V12	I feel that members of the forum often try to understand the perspective of others while having a conversation.				0.73		2.72 (0.99)
V13	Members of the community often exaggerate so that their words appear more credible (r)					0.88	2.65 (1.04)
V14	I often feel that members of the forum are sincere about their intentions.					0.82	2.76 (1.10)
V15	I already noticed complaints in conversations that somebody did not tell the whole truth about some matter. (r)					0.61	2.90 (1.10)
	Mean	3.03	3.48	2.96	2.89	2.78
	SD	0.71	0.61	0.70	0.74	0.86
	Cronbach's alpha	0.55	0.60	0.62	0.57	0.81

Note. ^aIn the web questionnaire, the sequence of items was randomized. ^b(r) denotes a reversed item. ^cEmpty cells represent factor loadings that are fixed at 0. ^dThe following items were only used in the pilot and were excluded from final research. Reciprocity: (a) In conversations, the amount of members who only post questions is not dominant; (b) Lots of conversations in the forum lose their thread (r). Justification: (a) I feel that lots of members post opinions about matters on which they do not have much knowledge (r); (b) I would support a claim that the conversations in this forum are an exchange of well-formed arguments. Reflexivity: (a) Participants in conversations are not interested in opinions of others (r); (b) It can be inferred from forum discussions that members do not think about what other members write (r). Ideal role taking: (a) It is quite uncommon to see one member of a forum asking another to explain his statement (r); (b) Members of a forum put significant effort in trying to be understandable to others in their posts. Sincerity: (a) I am certain that people in the forum only write things that they sincerely believe in; (b) Forum conversations offer members a biased perspective of what people really think (r).

Online trust

An already validated 4-item scale of online group trust was used.²⁴ Responses ranged from 1=“strongly disagree” to 5=“strongly agree.” Cronbach's alpha was 0.78.

Credibility of exchanged information

A shortened 6-item scale of exchanging information, which was introduced as a dimension of empowerment processes in online communities, was used.²⁵ Cronbach's alpha was 0.79.

Self-efficacy

was assessed with four items measured on a 5-point Likert-type scale with values ranging from 1=“strongly disagree” to 5=“strongly agree.”³⁸ Cronbach's alpha was 0.71.

Perceived leadership competences

Three items measured perceived leadership competences,³⁸ where Cronbach's alpha was 0.64.

Sense of virtual community

An established scale for measuring SOVC was used,³⁹ where responses were collected on a dichotomous scale (0=“false,” 1=“true”) and summed in an index, demonstrating good reliability (α=0.76).

Social support exchange

A short, one-dimensional scale was used to assess the degree to which members of online community are involved in receiving and giving social support with other members. An established scale with six items was used²⁵ and demonstrated good reliability (0.84).

Offline activities

This scale was adapted from a 4-item scale¹⁵ and modified from an attitude scale into one that captures the attendance to various offline activities such as face-to-face meeting of community members and informal offline meetings, on a scale with values 1=“never,” 2=“once,” and 3=“more than once.”

Intensity of participation

A single item was used to measure participation as a frequency of posting messages in online forum discussion threads, on a scale ranging from 1=“less than once per month” to 5=“every day.”

Statistical analyses

We applied a confirmatory factor analysis framework to confirm the five-dimensional structure of the PQOC construct. To investigate whether these are indeed the dimensions of PQOC, a second-order confirmatory factor analysis was used, where five subscales were treated as indicators of PQOC as a second-order factor. Because the indicators are ordinal and some of them deviate from multivariate normality, a diagonal weighted least squares estimation method was used.⁵²

To test whether the proposed measurement model fits the data, several commonly used fit statistics are reported: chi-square, degrees of freedom, root mean squared error of approximation (RMSEA), root mean squared-residual (RMR), normed fit index (NFI), and adjusted goodness of fit index (AGFI). A RMSEA value below 0.05 indicates excellent fit, a value below 0.08 indicates adequate fit, and a value above 0.10 signifies poor fit.⁵³ A RMR value of 0.08 or less is indicative of an acceptable model.⁵⁴ The AGFI ranges between 0 and 1, with a cutoff value of 0.9 generally indicating acceptable model fit. NFI similarly ranges between 0 and 1, while the model fit is interpreted as good with values of 0.95 and above.⁵⁴ The internal consistency of the scales was estimated using Cronbach's alpha, where an alpha value of 0.6–0.7 indicates acceptable reliability and a value of 0.8 or higher indicates good reliability.⁵⁵

Results

Testing the measurement model

Table 1 indicates that responses on 15 items were quite variable, with item V2 having the highest mean (M=3.83, SD=0.74) and item V7 the lowest (M=2.39, SD=0.91). None of the items had any problematic psychometric properties, which makes them all suitable for further analyses. We computed the Kaiser–Meyer–Olkin (KMO) value and performed Bartlett's test of sphericity to see if the factor analysis was meaningful to perform. The KMO value was 0.921, exceeding the recommended value of 0.6 and the Bartlett's test of sphericity was significant, both coefficients supporting the factorability of the correlation matrix.

The proposed measurement model of PQOC demonstrates a moderate to good fit to the data (χ²=447.16, df=80, RMSEA=0.064, RMR=0.056, NFI=0.98, AGFI=0.98) according to various criteria,⁵³ altogether supporting the five-dimensional structure of the PQOC scale. Standardized estimates of factor loadings are presented in Table 1. The factor loadings range from 0.38 to 0.88, where the factor loading is just below the suggested margin of 0.40 for two items. The Cronbach's alpha values indicate that the measurement instruments for justification, reflexivity, and sincerity are reliable, while those for reciprocity and ideal role taking are just below the margin. Although some indicators have issues with quality, we nevertheless retained all indicators and subscales in the factor solution. Small deviations are tolerated due to the novelty of the measurement instruments and the low number of items per subscale.⁵⁵ Consequently, we created five subscales from the above items, with means and standard deviations presented in Table 1. It can be noted that the subscale of justification has the highest mean (M=3.48, SD=0.61), while sincerity has the lowest mean (M=2.78, SD=0.86).

In the next step, we tested a measurement model, where five subscales together form a higher-order latent variable, which corresponds to the PQOC concept. For this reason, a second-order factor analysis was used, which is presented in Figure 1. The fit indices of this model reveal adequate fit of the model to the data (χ²=481.92, df=85, RMSEA=0.070, RMR=0.059, NFI=0.98, AGFI=0.98). The factor loadings of the subscales are all higher than 0.79, supporting the idea that PQOC is indeed a single construct manifested through reciprocity, justification, reflexivity, ideal role taking, and sincerity. This is supported by a Cronbach's alpha of 0.88 for all items. In practical terms, this means that all items can be computed on a scale, which can be used as a single variable in statistical analyses.

FIG. 1.

Second-order confirmatory factor analysis of PQOC with factor loadings of subscales and their items (see wording of each item in Table 1).

Convergent and discriminant validity

The predicted relationships between the PQOC construct, online trust, perceived credibility of exchanged information, and the two dimensions of sociopolitical control scale were largely supported by correlational analysis on the sample of online forum members (Table 2). All PQOC subscales and the overall scale were significantly correlated with online trust and perceived credibility of online information, demonstrating convergent validity. On the other hand, the PQOC (sub)scales were not significantly correlated with self-efficacy and perceived leadership competences, demonstrating discriminant validity.

Table 2.

Bivariate Correlations Between PQOC and Theoretically (Un)related Measures

	Online trust	Credibility of information	Self-efficacy	Leadership competence
PQOC	0.420^*	0.458^*	0.002	0.040
Reciprocity	0.355^*	0.445^*	−0.006	0.033
Justification	0.339^*	0.371^*	0.027	0.039
Reflexivity	0.281^*	0.388^*	−0.031	0.028
Ideal role taking	0.381^*	0.392^*	−0.014	0.007
Sincerity	0.360^*	0.420^*	0.028	0.049

Note. ^*p<0.001.

Validation of PQOC as antecedent of SOVC

An OLS regression was performed to test the role of PQOC as antecedent of SOVC. We report the means and standard deviations of dependent and antecedent variables in Table 3. Two regression models were tested: one, in which distinct subscales of PQOC figure as separate independent variables (“subscales model”), and one in which a scale of an overall PQOC is used as an independent variable (“single model”). The results of both regressions are presented in Table 4. In both models, independent variables account for 44.8% of the variance in SOVC, and the explained variance is in both cases significant, F_subscales=82.4, p<0.001, and F_single=122.4, p<0.001. All antecedents that were taken from the literature were significantly associated with SOVC: exchanging social support (β=0.27, p<0.001), intensity of participation (β=0.22, p<0.001), and involvement in offline activities (β=0.16, p<0.001). In the subscales model, four subscales of PQOC were significantly associated with SOVC: reflexivity (β=0.14, p<0.001), reciprocity (β=0.11, p<0.001), sincerity (β=0.10, p<0.001), and justification (β=0.08, 0.05<p<0.01). In the single model, an overall PQOC scale demonstrated a moderate to strong association with SOVC (β=0.34, p<0.001). Among sociodemographic variables, only gender was associated with SOVC in both models (β=−0.20, p<0.001), indicating that females tend to express a lower sense of virtual community than males.

Table 3.

Descriptive Statistics of Variables in Regression Analysis

Variables	Mean	SD	Max
Reciprocity	3.03	0.71	5
Justification	3.48	0.61	5
Reflexivity	2.96	0.70	5
Ideal role taking	2.89	0.74	5
Sincerity	2.78	0.86	5
PQOC	3.03	0.58	5
Social support	2.98	0.85	5
Offline activities	1.45	0.67	3
Participation	3.60	1.14	5
SOVC	8.18	2.74	12
Membership (in months)	37.7	27.6	156

Table 4.

Multiple Regression with the Sense of Virtual Community as the Dependent Variable ^a

	Subscales model			Single model
Predictor variables	b	SE	β	b	SE	β
Reciprocity	0.41	0.12	0.11^***
Justification	0.36	0.13	0.08^**
Reflexivity	0.53	0.13	0.14^***
Ideal role taking	0.04	0.12	0.01
Sincerity	0.32	0.09	0.10^***
PQOC				0.32	0.02	0.34^***
Social support	0.87	0.09	0.27^***	0.87	0.08	0.27^***
Offline activities	0.61	0.09	0.16^***	0.61	0.09	0.16^***
Participation	0.52	0.06	0.22^***	0.52	0.06	0.22^***
Gender	−1.12	0.14	−0,20^***	−1.12	0.14	−0.20^***
Age	−0.01	0.01	−0.03	−0.01	0.01	−0.03
Education	−0.06	0.03	−0.04	−0.06	0.03	−0.04
Length of membership (log)	0.00	0.05	0.00	0.00	0.05	0.00
R² (adj)	0.45			0.45

Note. ^an=1276. ^**0.01<p<0.05; ^***p<0.01.

Discussion

The purpose of this study was to argue for the relevance of the PQOC concept, to develop its scale, and to inspect its psychometric properties in terms of validity and internal consistency. Although factor weights of some indicators were close to or below the margin, we managed to show that PQOC is indeed a five-dimensional scale. Furthermore, our study confirmed that PQOC is a second-order factor, suggesting that the 15-item scale can be used as a measure of an overall PQOC construct. The phrasing of the items could be (and should be) further optimized to achieve higher internal consistency and validity. It is also advisable to use more than three items per subscale. On the other hand, the content domain of the construct could also be further elaborated. The design of item pool was grounded mostly in the tradition of online deliberation research that stems from Habermas's discursive ethics. It might be profitable to affirm and extend the content domain by taking into account Grices's cooperative principles of communication.⁵⁶ These principles to some extent overlap with Habermas's discursive ethics,⁵⁷ as they similarly emphasize the importance of truthfulness and cooperativeness. On the other hand, PQOC does not consider principles of relevance and manner, which might prove important for understanding the quality of conversations in online communities.

Although the field of researching online communities is rather young and does not offer many established scales, we managed to provide some insight into the construct and criterion validity of PQOC. The results show good levels of convergent and discriminant validity, while the evidence for criterion validity is somewhat weaker. An overall measure of PQOC proved to be an important predictor of SOVC, while the influence of subscales is much weaker although still significant. This would suggest that it is more valid to use an overall PQOC scale as an explanatory variable than individual subscales.

The proposed scale is potentially useful on both a theoretical and practical level. We argued that it is important to research not only the deviant manifestations of the lack of qualities of online communication but also the qualities of normal communication. The number of web applications, whose attraction is to large extent dependent upon user activities and conversations among them, is rapidly growing, and the quality of interchanges between users can have an important impact on the social and psychological processes in those places. We already demonstrated the role of PQOC for establishing sense of community, which is an important mechanism for sustainability of online communities.²³ Furthermore, the roots of the PQOC concept lie in the idea of communication that provides the basis for a balance between individual interest and collective goals.⁵⁸ In this context, quality of online conversations taps into one of the crucial questions in online communities that refers to issues of collective action and loyalty.¹⁹ The proposed scale might thus be an important intermediary variable between participation in online communities and such resources as community knowledge, social capital, and empowerment. In addition, as research shows that initial feedback for newcomers is very important for their further participation in online community,¹ the proposed scale might play an important role for understanding the developing phases of online community.

On a practical level, the scale might prove useful as a diagnostic tool for online community managers. Monitoring PQOC would for instance allow them to detect problems with quality of conversations quickly, a symptom of deeper problems of the community, thus allowing them to take significant measures before more serious distortions in communication and social processes occur. In this context, we believe that the field of online community governance^1,59 should not only consider regulation of problematic communication, but it might be profitable to consider what sort of mechanisms would stimulate communication that is reciprocal, reflexive, sincere, and justified.

This study is not free of limitations. It has problems with representativeness, and the results of our procedures are not generalizable. Furthermore, the development of a valid and reliable measure is a lengthy process requiring multiple studies across periods of time, different cultures, and different online settings. This study was limited to one particular type of online community, but it is intended for adaptation to a wider spectrum of online domains, where the users are involved in public communication exchanges, whether they are small close-knit groups or sites with millions of users. This study is an initial step in developing such a measure, while further research should critically examine it and further develop its quality in an iteration process of scientific dialogue.

Footnotes

Acknowledgments

I wish to thank Andraž Petrovčič and Lea Lebar for intellectual support for this project. I would also like to thank the administrators of the web forums who published a link to the web survey.

Author Disclosure Statement

No competing financial interests exist.

References

Resnick

, Kraut

. (2011) Introduction. In Kraut

, Resnick

, eds. Building successful online communities: evidence-based social design. Cambridge: MIT Press, pp. 1–21.

Wellman

, Boase

, Chen

. The networked nature of community on and off the Internet. IT & Society, 2002; 1:151–165.

Tanis

. Health-related online forums: what's the big attraction?. Journal of Health Communication, 2008; 13:698–614.

Lin

, Chiu

, Tsai

. Modeling relationship quality and consumer loyalty in virtual communities. CyberPsychology & Behavior, 2007; 11:561–564.

Luck

, Norton

. Collaborative management education: an online community in Europe?. International Journal of Web Based Communities, 2006; 2:45–58.

Barak

, Boniel-Nissim

, Suler

. Fostering empowerment in online support groups. Computers in Human Behavior, 2008; 24:1867–1883.

Zhao

. The Internet and the transformation of the reality of everyday life: toward a new analytic stance of sociology. Sociological Inquiry, 2006; 76:458–474.

. Theorizing Web 2.0: a cultural perspective. Information, Communication & Society, 2010; 13:249–275.

Vivian

, Sudweeks

. (2003) Social networks in transnational and virtual communities. In Proceedings of the 2003 Informing Science and Information Technology Education Conference, Santa Rosa, CA: Informing Science Institute pp. 1431–1437.

10.

Preece

, Maloney-Krichmar

, Abras

. (2003) History and emergence of online communities. In Wellman

, ed. Encyclopedia of community. Thousand Oaks, CA: Berkshire Publishing Group, Sage, pp. 1024–1028.

11.

Rheingold

. (2000) The virtual community: homesteading on the electronic frontier. Cambridge: MIT Press.

12.

Whittaker

, Isaacs

, O'Day

. (1997) Widening the net: the theory and practice of physical and electronic communities. Proceedings of the ACM Conference on Computer Supported Cooperative Work, Boston, MA.

13.

Berger

, Luckmann

. (1966) The social construction of reality: a treatise in the sociology of knowledge. New York: Doubleday.

14.

Rollman

, Krug

, Parente

. The chat room phenomenon: reciprocal communication in cyberspace. CyberPsychology & Behavior, 2000; 3:161–166.

15.

Koh

, Kim

. Sense of virtual community: a conceptual framework and empirical validation. International Journal of Electronic Commerce, 2003; 8:75–93.

16.

Ellison

, Steinfield

, Lampe

. The benefits of Facebook friends: social capital and college students' use of online social network sites. Journal of Computer-Mediated Communication, 2007; 12:1143–1168.

17.

Waks

. John Dewey on listening and friendship in school and society. Educational Theory, 2011; 61:191–205.

18.

Kim

, Kim

. Theorizing dialogic deliberation: everyday political talk as communicative action and dialogue. Communication Theory, 2008; 18:51–70.

19.

Matzat

. A theory of relational signals in online groups. New Media & Society, 2009; 11:375–394.

20.

Caspi

, Gorsky

. Online deception: prevalence, motivation and emotion. CyberPsychology & Behavior, 2006; 9:54–59.

21.

Suler

, Phillips

. The bad boys of cyberspace: deviant behavior in multimedia chat communities. CyberPsychology & Behavior, 1998; 1:275–294.

22.

Wagner

, Ip

RKF

, Cheung

KSK

, et al. (2005) Deceptive communication in virtual communities. In Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS’05, Washington, DC: IEEE Computer Society pp. 21–23.

23.

Blanchard

, Markus

. The experienced “sense” of a virtual community: characteristics and processes. The Data Base for Advanced in Information Systems, 2004; 35:64–71.

24.

Blanchard

, Welbourne

, Boughton

. A model of online trust: the mediating role of norms and sense of virtual community. Information, Communication, & Society, 2011; 14:76–106.

25.

van Uden-Kraan

, Drossaert

CHC

, Taal

, et al. Participation in online patient support groups endorses patients' empowerment. Patient Education & Counseling, 2009; 74:61–69.

26.

Preece

. (2000) Online communities: designing usability, supporting sociability. Chichester: John Wiley.

27.

DeVellis

. (2012) Scale development: theory and applications. London: Sage.

28.

Habermas

. (1984) The theory of communicative action. Vol. 1: Reason and the rationalization of society. Boston: Beacon Press.

29.

Graham

, Hajru

. Reality TV as a trigger of everyday political talk in the net-based public sphere. European Journal of Communication, 2011; 26:18–32.

30.

Wright

. Politics as usual? Revolution, normalization and a new agenda for online deliberation. New Media & Society, 2011; 14:244–261.

31.

Gastil

, Black

. Public deliberation as the organizing principle of political communication research. Journal of Public Deliberation, 2008; 4:1–41.

32.

Dahlberg

. Computer-mediated communication and the public sphere: a critical analysis. Journal of Computer-Mediated Communications, 2001; 7(1).

33.

Dahlberg

. The Habermasian public sphere: a specification of the idealized conditions of democratic communication. Studies in Social & Political Thought, 2004; 10:2–18.

34.

Janssen

, Kies

. Online forums and deliberative democracy. Acta Politica, 2005; 40:317–335.

35.

Steenbergen

, Bächtiger

, Spröndli

, et al. Measuring political deliberation: a discourse quality index. Comparative European Politics, 2003; 1:21–48.

36.

Joinson

, Dietz-Uhler

. Explanations for the perpetration of and reactions to deception in a virtual community. Social Science Computer Review, 2002; 20:275–289.

37.

Burgoon

, Stoner

, Bonito

, et al. (2003) Trust and deception in mediated communication. Proceedings of the 36th Hawaii International Conference on Systems Sciences, Maui, HI.

38.

Zimmerman

, Rappaport

. Citizen participation, perceived control, and psychological empowerment. American Journal of Community Psychology 1988, 16:725–750.

39.

Blanchard

. Testing a model of sense of virtual community. Computers in Human Behavior, 2008; 24:2107–2123.

40.

Kim

, Lee

, Hiemstra

. Effects of an online virtual community on customer loyalty and travel product purchases. Tourism Management, 2004; 25:343–355.

41.

Lin

. The role of online and offline features in sustaining virtual communities: an empirical study. Internet Research, 2007; 17:119–138.

42.

Tsai

, Cheng

, Chen

. Understanding online group buying intention: the roles of sense of virtual community and technology acceptance factors. Total Quality Management & Business Excellence, 2011; 22:1091–1104.

43.

Tonteri

, Kosonen

, Ellonen

, et al. Antecedents of an experienced sense of virtual community. Computers in Human Behavior, 2011; 27:2215–2223.

44.

Stromer-Galley

. Measuring deliberation's content: a coding scheme. Journal of Public Deliberation, 2007; 3:1–37.

45.

Martchev

. Patterns of online behavior in the UK and Japan: insights based on online conversations. First Monday 2009; 14. http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2605/2304 (accessed Jul. 21, 2012).

46.

Statistical Office of the Republic of Slovenia (2011) Usage of information-communication technologies (ICTs) in households and by individuals, Slovenia, 1st quarter 2011—final data. www.stat.si (accessed Aug. 5, 2012).

47.

PKH

, Coulson

. Empowering processes in online support groups among people living with HIV/AIDS: a comparative analysis of “lurkers” and “posters.”. Computers in Human Behavior, 2010; 26:1183–1193.

48.

, Barnett

, Tenerowicz

, et al. The touro 12-step: a systematic guide to optimizing survey research with online discussion boards. Journal of Medical Internet Research, 2010; 12:e16. www.jmir.org/2010/2/e16/ (accessed Oct. 15, 2011).

49.

Coulson

, Malik

. (2012) Health-related online support communities. In Yan

, ed. Encyclopedia of cyber behavior. Hershey, PA: IGI Global, pp. 671–688.

50.

Gastil

, Burkhalter

, Black

. Do juries deliberate? A study of deliberation, individual difference, and group member satisfaction at a municipal courthouse. Small Group Research, 2007; 38:337–359.

51.

Janssen

, Kies

. Online forums and deliberative democracy. Acta Politica, 2005; 40:317–335.

52.

Schumacker

, Beyerlein

. Teacher's corner: confirmatory factor analysis with different correlation types and estimation methods. Structural Equation Modeling: A Multidisciplinary Journal, 2000; 7:7629–7636.

53.

Browne

, Cudeck

. (1993) Alternative ways of assessing model fit. In Bollen

, Long

, eds. Testing structural equation models. Beverly Hills, CA: Sage, pp. 136–162.

54.

, Bentler

. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 1999; 6:1–55.

55.

Brown

. (2006) Confirmatory factor analysis for applied research. New York: Guilford Press.

56.

Grice

. (1975) Logic and conversation. In Cole

, Morgan

, eds. Syntax and semantics. New York: Academic Press, pp. 41–58.

57.

Braaten

. (1991) Habermas's critical theory of society. Albany: State University of NY Press.

58.

Friedland

. Communication, community, and democracy: Toward a theory of the communicatively integrated community. Communication Research, 2001; 28:358–391.

59.

O'Keefe

. (2008) Managing online forums: everything you need to know to create and run successful community discussion boards. New York: AMACOM.