Confluence of social network,social question and answering community,and user reputation model for information seeking and experts generation

Abstract

Social question and answering (Q&A) is one of the most effective approaches to knowledge acquisition using information seeking and collaboration. Most modern social Q&A systems use a static points-based user reputation model, which has the effect of diminishing the value of experts. In order to overcome this issue, we have developed a dynamic points-based user reputation model that takes user rating and social network analysis as input. The impact weight of each relation and user ratings are not static but are dependent on the current level of asker and answerer and on the difficulty level of the question. We propose a novel social Q&A platform that is the confluence of different features of social network, social Q&A, and the dynamic points-based user reputation model. The beta version of the system was evaluated by conducting a clinical study for 4 months in different academic environments. The results show that the proposed social Q&A outperforms the available static points-based social Q&A systems in representing the actual user reputation with an increased user satisfaction.

Keywords

Expert modeling information seeking points-based user reputation social networking social question and answering (Q&A)user reputation

1. Introduction

Since its inception, the Web has become the first stop for information seekers. The two most common forms of information seeking on the Web are keyword searching and asking questions in natural language. The former uses search engines to map user queries with the available resources to find relevant information through ranking; however, the results are not always relevant and satisfactory. The latter uses social question and answering (Q&A) to answer natural language questions on a wide array of topics [1]. The life cycle of a question in social Q&A sites go through different algorithmic, economic, and social processing strategies such as answers, supports, and comments [2]. Users can find context-aware, concise, and meaningful answers to their questions rather than searching for relevant information in bulky documents and resources. This is why social Q&A systems have attracted millions of Web users. Well-known social Q&A systems include Yahoo! Answers,¹ Answer Bag,² and knowledge-iN³ [1]. In simple terms, social Q&A is a prominent means of getting help and helping others by answering questions that are otherwise difficult to get answered on the Web.

Each social Q&A system presents its own incentive model for ensuring long-term active participation from community members in order to generate quality information through collaborative efforts [3]. User generated content and active participation of community members are essential to the success of these systems [3 –5]. In order to measure user reputation and model expertise, most of the social Q&A systems use a points-based approach where users are rewarded with points upon participation in different activities [6]. This points-based approach not only identifies experts but also extrinsically motivates users for long-term participation [7]. Reputation engines calculate reputations using a combination of points and user ratings [6, 8, 9]. User reputation models in social Q&A systems do not distinguish between laymen and experts because they fail to take into account system-bound activities, such as asking, answering, and the level of difficulty of questions when awarding points. This is a very serious limitation that needs to be considered when designing a point-based user reputation model in any social Q&A system.

In this paper, we have contributed a novel social Q&A system that integrates modified features of social networking for building a community of likeminded users in different areas of interest, social Q&A features for asking and answering questions, and the proposed user reputation model for differentiating novices from experts. We have put forward a novel social community structure and dynamic points-based user reputation model that accounts for the difficulty level of the question and the expertise of asker and answerer while performing system bounded activities. The rest of the paper is organized as follows: Section 2 presents related work; Section 3 presents the proposed system, user interface and analysis; and Section 4 concludes our discussion and presents some future recommendations.

2. Related work

User reputation and expert finding have a profound role in social Q&A systems where user activities are used to compute user reputation. In deciding the level of expertise of a particular user, two approaches have been used, namely link analysis and points-based approach. Link analysis employs ranking algorithms, such as PageRank and Hyperlink-Induced Topic Search (HITS), along with their modified versions. Points-based approach assigns points to compute user reputation and perform expert modeling.

A number of approaches have been utilized in the application of HITS algorithm [10 –12] to calculate user reputation, which classifies users into hub and authority groups. The askers belong to a hub category whereas authority users are the answerers. Two values (hub and authority) are recursively calculated that converge upon several iterations [10 –12]. In order to determine user reputation, Jurczyk et al. [10, 11] used the HITS algorithm, which exploits the linked structure of Yahoo! Answers. They reported that HITS works well for some categories but not for all.

In addition to HITS, another alternative that uses link analysis is PageRank, in which user rank is based on the number of other users helped [13, 14]. Expertise is determined by user rank. The usefulness of PageRank is limited because the quantity of questions is not the only criteria that should be considered. It has been reported that one-third of the given answers are of poor quality, whereas 10% of the questions are badly answered [15]. Jiao et al. [13] used the modified version of the PageRank algorithm called ExpertRank for user reputation. They used a Microsoft discussion group to evaluate the algorithm. Expert profiles were used to calculate the relevance score in order to reflect dynamic relevance between candidate experts and user input query while considering the spam experts. Their results outperformed in finding user reputation and were successful in filtering spam experts. A comparative study of PageRank and HITS like schemes in modeling user reputation was carried out by Hong et al. [16] on different topics in Yahoo! Answers. They reported that PageRank-based approaches dominate HITS schemes.

To analyze the interaction between askers and answerers in Java community, Zhang et al. [14] used link analysis algorithms, including PageRank and HITS, and reported that 54.9% are askers, 13% are answerers, and 12.3% are both. Considering topic structure and topic similarity among users, Zhou et al. [17] experimented with a modified version of link analysis algorithms on Yahoo! Answers and reported that their solution is more efficient than previous solutions.

Yang et al. [18] proposed a multipurpose general model called Topic Expertise Model for expert modeling by combining expertise with topic interest. Chen et al. [19] proposed a bias-smoothed tensor model for user reputation in a comment-rating environment and reported a significant increase in performance. Liu et al. [20] considered user subject relevance, user reputation, and authority of a category in finding experts. Li et al. [21] proposed the Topic-level Expert Learning (TEL) model by combining graph-based link analysis and content semantic analysis for expert modeling in community Q&A. Song et al. [22] proposed a leading ability-detecting model in which they used multiple kernel learning algorithms on the basis of leading capacity and reported its effectiveness.

The points-based user reputation model allows users to ask and answer questions and rate each other. A reputation computation engine computes user reputation (reputation score) based on user ratings and earned points. Naver’s Knowledge-iN, Yahoo! Answers, Answerbag,⁴ and Stack Overflow⁵ are some of the best examples of an accumulative reputation display pattern that measures the reputation of community members using points. Reputation scores increase or decrease either monotonically or arbitrarily according to points. Naver’s Knowledge-iN (KiN) uses a points-based approach where 10 points are awarded for an answer, 25 points for a best answer, 1 point for voting, and 3 points for logging in every day [6]. In a similar fashion, Yahoo! Answers grants 100 points to users for creating an account, 1 point for each login, –5 points for asking a question, 10 points for selecting an answer as the best answer, and 3 points for selecting the best answer for his/her question [9]. Stack Overflow is another points-based social Q&A system for computer programmers, where reputation is achieved by convincing the community members that they have subject matter expertise. Stack Overflow grants privileges as the user’s reputation increases.

There are variations of the user reputation model. Wei Chen et al. [8] developed a user reputation model for CuteAid, which relies on a combination of social network analysis (SNA) and user ratings while considering different relationships include asking and answering questions, identifying correct answers, supporting and complaining about answers, and commenting on questions. McNally et al. [23] proposed a collaborative user reputation model for the social Web and considered the aggregate vote greater than zero by using three different social Q&A systems on Stock Exchange Network.⁶ Similarly, an incentive model for the IBM social networking site has been developed to encourage community member participation [1, 24]. It is noteworthy that points-based social Q&A systems are very popular with users because they encourage prompt responses [25]. Points-based social Q&A systems are more prevalent than link analysis approaches because of their accuracy, collaborative filtering, attraction to long-term community members, and active participation of users. All of these points-based social and community Q&A systems are static in assigning points where the current expertise level of the asker and answerer, as well as the difficulty level of the questions, are neither considered nor get boosted with the passage of time. Therefore, we aim to contribute a dynamic points-based user reputation model and to develop a social Q&A platform by integrating modified versions of social networks and social Q&A platforms with our dynamic points-based user reputation model. The next section, presents our solution.

3. The proposed solution

Figure 1 shows the proposed system consisting of three modules. The first module provides the basic social networking functionalities including profile management, mate recommendation, user notification, and updating mates’ status. The profile is used to display the network activities of mates and expertise statistics of the profile owner.

Figure 1.

Architecture of the proposed system.

The anatomy of social network and mate recommendation is shown in Figure 2. A user can register for different subjects and perform system-bounded activities. The user is presented with mate recommendations from the pool of subject experts. Recommendations are based on user profile and selected subject(s) information. The notification module keeps users updated about different activities of other users or mates in the network. The next module provides basic questioning and answering functionalities.

Figure 2.

Social network and mate recommendation system.

3.1 Social Q&A features

The proposed solution uses two customized features of social Q&A namely (1) Open Community Question and Answer and (2) Question Closing and Best Answer Selection. Open Community Q&A allows users to post questions and receive answers from other users. The community members play different roles in the life cycle of a question as shown in Figure 3. The early stages of the life cycle include posing a question to the community, boosting, or supporting a question if other community members have the same question, which increases difficulty level of the question being asked, and answering questions posed by other community members.

Figure 3.

Life cycle of a question and interaction of users in the proposed system.

The second feature is Question Closing and Best Answer Selection. It allows the asker to close the question after getting a satisfactory answer. It also allows closing the question after a specific interval of time. To avoid discussion, members are limited to providing one answer per question. At a later stage, the asker either selects a best answer or is prompted to select a best answer to his/her question. The resolved questions along with their best answers are added to the archive where they can be indexed and retrieved by Web search engines.

3.2. Dynamic points-based user reputation model

In order to increase trust in the content, build reputations, and differentiate novice from expert, we have developed a dynamic points-based user reputation model and integrated it into the proposed system. The reputation model is computed from user ratings and analysis of user social network relations. User rating is used to assess a user reputation based on other users’ experiences, whereas social network analysis evaluates the impact of relations of participating users. Tables 2 and 3 represent the impact weight of each relation and user rating, respectively. These are dynamically computed using the current level of asker ( $A_{L}$ ) and answerer ( $R_{L}$ ) and the difficulty level of the question ( $Q_{V}$ ). Note that when people start learning a subject, they are known as novices, but with the passage of time and learning, they become professionals or experts in that subject. Therefore, the difficulty level of the question is taken into account to reflect what happens in the real world.

In order to find the difficulty level of a question, we utilize a hierarchal structure, such as that of a book. A subject is viewed as a tree consisting of three levels as shown in Figure 4. Level 0 represents basic and general subject level questions with assigned difficulty value ( $Q_{V} = 0)$ . Level 1 represents chapters in a subject where a subject may consist of n chapters and each chapter is assigned an integer difficulty value in increasing order. Users can ask chapter level questions and the question gets a difficulty value of the corresponding chapter. Level 2 represents topic level question. Let us consider a chapter n consists of m number of topics; then the difficulty value of a topic is a real number with a lower bound $n - 1$ and upper bound $n + 1$ such that topic values monotonically converge to the upper bound with equal difference. For all T, there exists a lower bound $C_{n - 1}$ , and upper bound $C_{n + 1}$ , such that:

C_{n - 1} < T < C_{n + 1}

(1)

Figure 4.

Subject hierarchical structure.

Thus the question can either be subject level, chapter level, or topic level, and a difficulty value is associated with each question.

To differentiate novices from experts, the users are categorized into different expertise levels as shown in Table 1. These expertise levels are not fixed and may be changed by the administrator. System-bounded activities affect user reputation, and therefore, the expertise level is transformed after crossing a certain threshold value. The expertise levels of asker and answerer are denoted by $A_{L}$ and $R_{L}$ , respectively.

Table 1.

Points and level system.

Expertise level	Range
Level 1	0–19
Level 2	20–39
Level 3	40–59
Level 4	60–79
Level 5	80+

3.2.1. Social network and user relation analysis

The proposed system is a virtual community where users interact with each other by answering questions, selecting best answers, marking an answer as helpful and making complaints. Table 2 summarizes relationship types and their relative impact weights (IW). It is worth noting here that these weights can be modified by the administrator of the system.

Table 2.

Relations between community members and its impact weight (IW).

Relation type	IW
Answering Question $(P_{A})$	0–1
Best Answer Selection $(P_{BA})$	0–5
Answer Is Helpful $(P_{AS}$ )	0–2
Complains $(P_{C})$	−3

Community members are allowed to answer a question in specific subjects. The answerer is rewarded $P_{A}$ points on answering a question. The $P_{A}$ value ranges from 0 to 1 and depends on three elements including question difficulty value ( $Q_{V}$ ), asker current level ( $A_{L}$ ), and answerer current level ( $R_{L}$ ). Equation (2) calculates the $P_{A}$ value:

P_{A} = \frac{(A_{L} + R_{L} + Q_{V})}{Max (A_{L}) + Max (R_{L}) + Max (Q_{V})}

(2)

where $Q_{V}$ is the difficulty value of the asked question and $Max (Q_{V})$ is the maximum value of a subject. $A_{L}$ is the asker’s current level of expertise at time t, and Max( $A_{L}$ ) is the maximum expert value. $R_{L}$ is the answerer’s current level of expertise at time t, and $Max (R_{L})$ is the maximum expert value.

After receiving answers from the community members, the asker may select an answer as the best answer. If the asker selects an answer as best answer, then the answerer will be rewarded $P_{BA}$ points and is calculated using equation (3).

P_{BA} = (\frac{(A_{L} + R_{L} + Q_{V})}{Max (A_{L}) + Max (R_{L}) + Max (Q_{V})}) \times C

(3)

Where C is constant and is equal to 5 in the case of best answer selection.

If the asker supports an answer, then the answerer is rewarded $P_{AS}$ points ranging from 0 to 2, as can be calculated by using equation (4).

P_{AS} = (\frac{A_{L} + R_{L} + S_{L} + Q_{V}}{Max (A_{L}) + Max (R_{L}) + Max (S_{L}) + Max (Q_{V})}) \times C

(4)

Where $S_{L}$ is the current level of the person who supports the answer and C is a constant equal to 2 in case of answer support. Therefore, as knowledge is created with collaborative efforts, a mechanism is required to filter abusive material and to discourage negative activities. For this reason, complaints ( $P_{C})$ are used. The $P_{C}$ value is fixed ( $P_{C} = 3$ in our case) and can be changed by an administrator.

3.2.2. Social network analysis

The community of the proposed system can be viewed as a directed and weighted graph SN (N, L) called a sociogram [26] where $N = {n_{1}, n_{2}, n_{3} \dots \cdot \cdot n_{k}}$ is a set of nodes of graph SN. Each node in graph SN represents community members and $L = {l_{1}, l_{2}, l_{3} \dots \cdot \cdot l_{m}}$ is a set of edges corresponding to relations between community members.

The relation between two users can be of the following types:

Re lationType (e_{i}) \in R_{TYPE} = {\begin{matrix} AnswerQuestion (P_{A}), \\ BestAnswerSelection (P_{BS}), \\ AnswerSupport (P_{AS}), \\ Complains (P_{C}) \end{matrix}}

(5)

Where i=1,2, 3…n

$R_{TYPE}$ represents the relationship types between community members. These relationships are important and boost the node value that corresponds to user reputation. Each time a community member performs a system-bounded activity, a new edge of $R_{TYPE}$ is formed in the graph SN and the graph is updated from $S N_{i}$ to $S N_{i + 1}$ . The cost of each relation is not fixed but depends on the current levels of asker and answerer and the difficulty level of the question. User reputation is updated periodically in each subject and the computed reputation reflects the actual expertise of a user in a particular subject. Actual reputation of a user in a given subject is the combined effect of social network analysis and user rating. Each increment in the relation boosts user reputation.

Let us consider two nodes of the graph SN, $n_{k} and u_{} \circ$ , having a particular relation type ( $τ$ ) between them. This relation between $n_{k} and u_{} \circ$ can be represented as $(n_{k} \to u_{} \circ)$ . The node $n_{k}$ contributes to the reputation of user $u_{} \circ$ by establishing a new type $(τ)$ of relationship that occurs in time interval $t_{i}$ and $t_{i + 1}$ . Let us consider $Φ (u, τ)$ as a set of all relations $n_{k} \to u_{} \circ$ that occurs in time interval $t_{i}$ and $t_{i + 1}$ shown in equation (6).

Φ (u, τ) = {n_{k} \to u_{} \circ | \begin{matrix} n_{k} \in N \\ Λ (n_{k} \to u_{} \circ) \in L \\ Λ R_{TYPE} (n_{k} \to u_{} \circ) = τ \\ Λ t_{i} < TimeOf (n_{k} \to u_{} \circ) < t_{i + 1} \end{matrix}}

(6)

Let us consider that $n_{k}$ establishes a new relation of type $τ$ with $u_{} \circ$ , which boosts the value of $u_{} \circ$ . The total importance value of $u_{} \circ$ is the sum of all relations of type $τ$ and can be calculated using equation (7).

Δ I_{t_{i + 1}}^{τ \in ρ} (u_{\circ}^{ρ}) = \sum_{n_{k} \to u_{} \circ ϵ Φ (u, τ)} (n_{k} \to u_{} \circ)

(7)

In equation (7), $τ$ represents a particular type of relation between nodes of the graph SN. In social networks, the importance of a node is directly proportional to the number of nodes it connects. Let us consider a community member $n_{k}$ posts a question to the community in a specific subject $ρ$ and let in a subject $ρ$ user $u_{} \circ$ responds to the question in the form of answer. The relation from $n_{k}$ to $u_{} \circ$ is formed of type $AnswerQuestion (P_{A})$ in subject $ρ$ and can be represented as $n_{k} \to u_{} \circ$ . Therefore, the respective graph is updated and new edge is added to the graph SN of relation type $AnswerQuestion (P_{A})$ .

Equation (7) results in the total number of relations of type $(τ)$ in a particular subject $ρ$ . The total IWnof a specific relation of type $τ$ in a particular subject $ρ$ of user $u_{} \circ$ is the sum of respective link’s IW ( $I W_{τ}$ ) as shown in Table 2. The $I W_{τ}$ for each relation of type $τ$ is not fixed and depends on the current level of asker, answer and on the question difficulty level. Equation (8) gives the total IW for a particular relation of type ( $τ)$ in a specific subject $ρ$ of user $u_{} \circ$ .

Δ I_{t_{i + 1}}^{τ \in ρ} (u_{\circ}^{ρ}) = \sum_{n_{k} \to u_{} \circ ϵ Φ (u, τ)} [(n_{k} \to u_{} \circ) \times I W_{τ}]

(8)

Where $Δ I_{t_{i + 1}}^{τ \in ρ} (u_{\circ}^{ρ})$ represents user reputation for a particular type of relation $τ$ in subject $ρ$ and $I W_{τ}$ is the IW of each relation. Actual reputation is the sum of all types of relations from $t_{i}$ to $t_{i + 1}$ and can be calculated using equation (9).

Δ R_{t_{i + 1}}^{SN \in ρ} (u_{\circ}^{ρ}) = \sum_{\forall τ ϵ R_{TYPE}} (Δ I_{t_{i + 1}}^{τ \in ρ} (u_{\circ}^{ρ}))

(9)

Where $Δ R_{t_{i + 1}}^{SN \in ρ} (u_{\circ}^{ρ})$ represents actual reputation of user $u_{} \circ$ in subject $ρ$ that comes from social network analysis and $R_{TYPE}$ represents the set of all types of relations as mentioned in equation (5).

3.2.3. User rating

Along with user relation analysis, community members rate content generated by users. Two types of ratings are used: Answer Is Helpful ( $r P_{AS}$ ) and Best Answer Selection $(r P_{BA})$ . The values of both variables are in the range of 0–2 and depend on question difficulty level and levels of the answerer, asker, and rater. These behaviors are also addressed in social network analysis; however, social network analysis addresses the asker’s and answerer’s perspectives, whereas user rating considers the involvement of community members.

In the proposed system, community members can answer any question in any registered subject, which is visible to other community members. If another user finds an answer helpful, then that user can rate it as “Answer Is Helpful.” This rating has dual use as an increased rating for an answer results in an increased trust in the answer and also helps boost the asker’s reputation. Similarly, if an asker does not select the best answer then high-level experts in the relevant subject are allowed to select the best answer of a question. The impact weight of “Answer Is Helpful” and “Best Answer Selection” can be calculated using equation (10).

r P_{AS} / r P_{BA} = \frac{A_{L} + R_{L} + Rate r_{L} + Q_{V}}{Max (A_{L}) + Max (R_{L}) + Max (Rate r_{L}) + Max (Q_{V})} \times C

(10)

where C is constant and is equal to 2. The IWs of respective rating variables are mentioned in Table 3 and can be calculated using equation (11).

Δ {rP}_{t_{i + 1}}^{AS \in ρ} (u_{\circ}^{ρ}) = \sum_{t ϵ {t | t_{i} < t < t_{i + 1}}} (r P_{AS} \times I W_{t})

(11)

Table 3.

Rating types with respective weight.

Rating type	Weight of rating
Answer Is Helpful ( $r P_{AS}$ )	0–2
Best Answer Selection ( $r P_{BA}$ )	0–2

Equation (11) is used to sum all ratings of type “Answer Is Helpful” $(r P_{AS})$ for user $u_{} \circ$ in specific subject $ρ$ . Similarly, Best Answer Selection ratings $r P_{BA}$ for user $u_{} \circ$ in specific subject $ρ$ can be calculated using equation (12).

Δ {rP}_{t_{i + 1}}^{BA \in ρ} (u_{\circ}^{ρ}) = \sum_{t ϵ {t | t_{i} < t < t_{i + 1}}} (r P_{BA} \times I W_{t})

(12)

The total change in reputation of user $u_{} \circ$ in subject $ρ$ due to rating $(Δ R_{t_{i + 1}}^{R})$ from $t_{i}$ and $t_{i + 1}$ can be calculated using equation (13).

Δ R_{t_{i + 1}}^{R \in ρ} (u_{\circ}^{ρ}) = \sum_{t ϵ {t | t_{i} < t < t_{i + 1}}} [(r P_{AS} \times I W_{t}) + (r P_{BA} \times I W_{t})]

(13)

3.2.4. Reputation computation

Total reputation of user $u_{} \circ$ in a specific subject $ρ$ is the combined impact of social network analysis and user rating and can be calculated using the following formula.

Δ R_{t_{i + 1}}^{ρ} (u_{\circ}^{ρ}) = R_{t_{i}}^{ρ} (u_{\circ}^{ρ}) + Δ R_{t_{i + 1}}^{SN \in ρ} (u_{\circ}^{ρ}) + Δ R_{t_{i + 1}}^{R \in ρ} (u_{\circ}^{ρ})

(14)

where $Δ R_{(t_{(i + 1)})}^{ρ} (u_{\circ}^{ρ})$ represents the actual reputation of user $u_{} \circ$ in a specific subject $ρ$ at time $t_{i + 1}$ , $Δ R_{t_{i + 1}}^{SN \in ρ} (u_{\circ}^{ρ})$ comes from equation (9), i.e. user reputation is calculated by exploiting user relations of type $τ$ in a specific subject $ρ$ at time $t_{i + 1}$ , and $Δ R_{t_{i + 1}}^{R \in ρ} (u_{\circ}^{ρ})$ is the user reputation that can be computed by exploiting user ratings in a specific subject $ρ$ as shown in equation (13). $Δ R_{t_{i}}^{ρ} (u_{\circ}^{ρ})$ is the initial reputation of user $u_{} \circ$ in a specific subject $ρ$ at time $t_{i}$ if there is no initial reputation of user $u_{} \circ$ then it is set to 0. The main objective of user reputation is to project the expertise level of a user in a particular subject $ρ$ of interest in order to increase trust in content and in community members.

3.3 User interface and analysis

Our proposed system provides a Web-based interface and allows users to perform different system-bounded activities. The user interface of our system has been divided into a number of pages, e.g. Subject Pool and Mates Status Streaming. In the subject pool, all possible subjects are defined and community members select any number of subjects in which they want to build reputations on the Web and to perform social Q&A activities. The design of the subject pool is crafted on the basis of the real world subject selection principle. After the user has selected subjects from the subject pool, a user’s activities in the selected subject will affect the user’s reputation.

The mates status streaming interface enables users to see activities that occur within the user’s own mates network. The status streaming and subject activities pages are partitioned in to three panels. The right most panel is for user selected subject management. The middle panel displays Q&A activities and is dependent on the right panel selection, and the left panel shows user reputation in any registered subject being selected from the right most panel.

The left panel consists of two sub-panels as shown in Figure 5: the top sub-panel and the bottom sub-panel. The top sub-panel is used to display user-registered subjects, and bottom sub-panel shows the hierarchical structure of the subject being selected in the top sub-panel. In the subject hierarchical structure, numerical values show that there are questions from subject mates. Resolved questions and answers of a registered user are permanently archived under relevant topics in which either the questions were asked or the answers were made.

Figure 5.

Status streaming interface.

The middle panel provides space to ask questions in any selected subject or to answer questions from subject mates. The middle panel is further partitioned into two sub-panels including the top middle sub-panel, where a registered user posts a question in a selected topic. This top middle sub-panel consists of two text fields: one is for the question title, and the second for optional question details. The lower section in the middle panel shows streaming of questions asked by subject mates in the particular subject selected in the right panel. Each question consists of two buttons including “Me Too” to boost the question difficulty value and “Answer” to answer the question.

Upon clicking the “Answer” button, another window, a pop-up, opens as shown in Figure 6. It consists of two text fields: one is for providing an answer and the second is for providing references, if any. There are some restrictions on answering: askers are not allowed to answer his/her own question and only one answer per question can be placed per user. After receiving some answers, the asker can evaluate the answer quality using three actions: selecting the answer as helpful, marking the answer as abusive or selecting the answer as best answer. The right panel shows user profile and reputation in the selected subject as shown in Figure 5.

Figure 6.

Answering pop-up window.

3.4. Analysis and user studies

The beta version of the proposed system has already been uploaded and tested for about 4 months. Fifty students from three different universities, the University of Peshawar, Peshawar, Quid-e-Azam University, Islamabad, and Islamic International University, Islamabad, were contacted for testing the beta version. The students were contacted either through their teachers or department heads. Only 36 students showed their interest in the system and voluntarily created their profiles and agreed to share their personal information as well as their postings. Consent from each participant was obtained in written as well as in digital form using a registration form that contains all the terms and conditions. However, even after obtaining the informed written as well as digital consent from the users, their profile images and profile names were still kept blurred in Figures 5 and 6 so that they may not be identified by others.

In the subject pool, three different Computer Science subjects, namely Data Structure, Relational Databases, and C++, were included in order to test social Q&A activities and to test our dynamic user reputation model. All the participants were trained how to use the system and were encouraged to participate actively who use the system from 15 January 2014 up to 25 April 2014. The participants showed greater satisfaction where the statistics give a clear picture of the usage and performance of the system. During this interval, 400 questions were asked and 976 answers were received. Of these questions, 91.2% received at least 1 answer, and on average, 2.435 answers were received per question. Overall 65%, 22.2%, and 10% of the questions received 2, 3, and 4 answers, respectively, as shown in Figure 7(a). Statistics of further activities performed in each subject by the users are shown in Figure 7(b).

Figure 7.

Questions and answers statistics.

3.4.1. Question and answer quality

In order to show similar interest in a question, the Question Boost Button is used to boost the question so that community members pay special attention in answering it. The question difficulty is also proportional to the number of boosts. The use of the question boost was not very frequent and was used for 7.75% questions only. On average, 0.0825 boosts per question were recorded. The “Is it Helpful” button is provided for supporting answers and rating other users. The use of this functionality was quite satisfactory, 48% of answers were supported and an average support per answer was 0.569. Users were encouraged to add references to answers. The reference ratio was quite low and only 5.02% of answers were given with references. Furthermore, answers to the questions were comprehensive and concise. On the average, the answer length was 94.91 words, whereas the median answer length was found to be 72 words.

3.4.2. Social network analysis

The community of the proposed system can be viewed as directed and weighted graph SN (N, L) called the sociogram [26], where $N = {n_{1}, n_{2}, n_{3} \dots \cdot \cdot n_{k}}$ is a set of nodes of graph SN, each of which represents community members, and $L = {l_{1}, l_{2}, l_{3} \dots \cdot \cdot l_{m}}$ is a set of edges that correspond to relations between community members with respective weights $W = w_{1} + w_{2} + w_{3} \dots w_{j}$ .

There exist bi-directional weighted connections between two mates and boosting reputation of each other while performing system-bounded activities as shown in the Figure 8. Here, in Figure 8, $n_{1}$ and $n_{2}$ are two nodes of the graph SN and represent two community members where $W_{1}$ and $W_{2}$ are weights of $l_{1}$ and $l_{2}$ , respectively. $W_{1}$ and $W_{2}$ is the sum of costs of all contributions made by $n_{1}$ for $n_{1}$ and vice versa. In this social network analysis, we consider the global ranking of each node.

Figure 8.

Two nodes of graph SN corresponds to two mates of the community.

The social network is visualized using the social network visualization tool Gephi [27] where nodes and edges in Figure 9 correspond to users and relations among them, respectively. Node size in the graph corresponds to the global reputation of the user in the community, whereas link size corresponds to the contribution of source node to target node, i.e. a larger node size indicates a higher reputation while a smaller node size represents a lower reputation. The same pattern applied with links.

Figure 9.

Social network visualization.

3.4.3. Comparison with Yahoo! Answers

For comparison, we integrated the Yahoo! Answers’ point-based user reputation model. The point system of Yahoo! Answers is shown in the Table 4. We made two changes to Yahoo! Answers’ reputation model, i.e. no points were granted on the first time registration and no points for per day login instead of 100 points and 1 point, respectively. The reason is that Yahoo! Answers gives 100 points as bonus points for motivation and our proposed system gives no such points. Therefore, in order to make a valid comparison and evaluation, these changes were made. The level system as shown in Table 1 was used for both the reputation models. Table 5 shows a list of top five contributors in three different subjects with their respective reputation being computed by our proposed users’ reputation model and Yahoo! Answers point-based system, respectively.

Table 4.

Yahoo! Answers’ point-based reputation system.

Activity	Impact Weight
Asking question	−5
Asker choses a best answer	3
Answer a question	2
Self-deleting an answer	−2
Have your answer selected as the best answer	10
Thumbs-up/Rating	1
Complains	−10

Table 5.

Top five contributors and their respective reputation.

UID	Subject	Reputation computed by using the proposed reputation model	Reputation computed by using Yahoo! Answers’ point system
13	C++	138.314	66
22	Database	136.644	155
1	C++	130.149	29
41	C++	90.766	91
32	Database	69.974	32
1	Data Structure	60.200	17

Figures 10 and 11 show the reputation of users given in Table 5. In Figure 10, we see that the users’ reputation increases with the passage of time as they perform Q&A activities whereas in Figure 11 the users’ reputation remains same or is getting even negative after a significant number of contributions. In Figure 11, users with ID 1, 13, and 32 perform Q&A activities in two different subjects, i.e. C++ and Data Structures, but are unable to upgrade their expertise level according to the contributions they made. It is clear from the comparison that our proposed user reputation model projects the true reputation of the community members. It is also clear that in our proposed system, the Q&A activities performed by the users contribute dynamically to their reputation. For example, if a user answers a difficult question being asked by an expert, they receive high points, in contrast to Yahoo! Answers where almost everything is done statically.

Figure 10.

Users’ reputation computed by using the proposed dynamic point-based reputation model.

Figure 11.

Users’ reputation computed by using Yahoo! Answers' reputation model.

4. Conclusion and recommendations

The incorporation of social Q&A platforms into the social Web boosted its popularity and enabled community members to ask questions and get answers globally with little or no cost. However, the static nature of the points-based user reputation model presents a static situation where equal points are given to all users with different abilities and expertise and consequently do not account for question difficulty, asker level, and answerer level. With the development of the proposed system, which is the result of the confluence of social networking, social Q&A, and our proposed dynamic user reputation model, some of these problems have been solved resulting in an increase in user satisfaction. The novel and dominating factors of the proposed user reputation model are the adaptation of a dynamic points-based system and combination of two types of approaches for calculating user reputation: social network analysis and user ratings. Social network analysis is applied to evaluate the impact of participating user relations while user ratings are used to get a direct judgment of a user’s reputation based on the user’s other experience. The results show that the proposed system resulted in an increase in user satisfaction.

The beta release of the proposed system was tested by 36 students in three Computer Science subjects. However, further validation of the proposed system is required. To this end, we propose further analysis of user behavior by extending subjects to other fields as well. The proposed user reputation model is influenced by three basic elements, including question difficulty and current expertise levels of the asker and the answerer. Question difficulty value is based on the hierarchical structure of a subject. Further investigation is required in order to determine question difficulty with greater accuracy. Similarly, further research is required to integrate the proposed user reputation model with other Q&A communities to assess its behavior, e.g. the integration of the proposed user reputation model with tag-based Q&A communities.

Different social networks exploit different elements for recommending friends, such as context (location, time) and contents. We are planning to investigate and develop a powerful mate recommendation system. Similarly, in Q&A communities many questions are posted every day and community members face difficulty finding questions of interest. This problem may put cognitive overload on users searching for best answers as well. In order to overcome this problem, the recommendation of relevant questions, as well as answers, according to the expertise and interest of a user can play a significant role in the response time for answering a question.

Footnotes

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Notes

References

Farzan

DiMicco

Millen

Brownholtz

Geyer

Dugan

. When the experiment is over: Deploying an incentive system to all the users. In: Proceedings of the Symposium on Persuasive Technology, In conjunction with the AISB. 2008.

Gazan

. Social Q&A. Journal of the American Society for Information Science and Technology 2011; 62: 2301–2312.

Harper

Raban

Rafaeli

Konstan

. Predictors of answer quality in online Q&A sites. In: Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems. Florence, Italy: ACM, 2008, pp. 865–874.

Welser

Gleave

Fisher

Smith

. Visualizing the signatures of social roles in online discussion groups. Journal of Social Structure 2007; 8: 1–32.

Shah

. Exploring characteristics and effects of user participation in online social Q&A sites. First Monday 2008; 13.

Nam

Ackerman

Adamic

. Questions in, knowledge in?: a study of naver’s question answering community. In: Proceedings of the 27th international conference on Human factors in computing systems. Boston, MA: ACM, 2009, pp. 779–788.

Raban

Harper

. Motivations for answering questions online. In: Caspi

Samuel-Azran

(eds) New Media and Innovative Technologies. Beersheba, Israel: Ben Gurion University of the Negev Press, 2008, pp. 273–295.

Chen

Zeng

Wenyin

. A user reputation model for a user-interactive question answering system. Concurrency and Computation: Practice and Experience 2007; 19: 2091–2103.

Gyongyi

Koutrika

Pedersen

Garcia-Molina

. Questioning Yahoo! Answers. Stanford, CA: Stanford InfoLab, 2007.

10.

Jurczyk

Agichtein

. Hits on question answer portals: exploration of link analysis for author ranking. In: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval. Amsterdam, The Netherlands: ACM, 2007, pp. 845–846.

11.

Jurczyk

Agichtein

. Discovering authorities in question answer communities by using link analysis. In: Proceedings of the sixteenth ACM conference on Conference on information and knowledge management. Lisbon, Portugal: ACM, 2007, pp. 919–922.

12.

Chen

Nayak

. Expertise Analysis in a Question Answer Portal for Author Ranking. Web Intelligence and Intelligent Agent Technology. Sydney, Australia: IEEE Computer Society, 2008.

13.

Jiao

Yan

Zhao

Fan

. ExpertRank: An Expert User Ranking Algorithm in Online Communities. In: New Trends in Information and Service Science, 2009 NISS ‘09 International Conference on. Beijing, China: 2009, pp. 674–679.

14.

Zhang

Ackerman

Adamic

. Expertise networks in online communities: structure and algorithms. In: Proceedings of the 16th international conference on World Wide Web. Banff, Alberta, Canada: ACM, 2007, pp. 221–230.

15.

Jeon

Croft

Lee

Park

. A framework to predict the quality of answers with non-textual features. In: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval. Seattle, WA: ACM, 2006, pp. 228–235.

16.

Hong

Yang

Davison

. Incorporating participant reputation in community-driven question answering systems. In: Computational Science and Engineering, 2009 CSE’09 International Conference on. IEEE, 2009, pp. 475–480.

17.

Zhou

Lai

Liu

Zhao

. Topic-sensitive probabilistic model for expert finding in question answer communities. In: Proceedings of the 21st ACM international conference on Information and knowledge management. Maui, HI: ACM, 2012, pp. 1662–1666.

18.

Yang

Qiu

Gottipati

Chen

. CQArank: jointly model topics and expertise in community question answering. In: Proceedings of the 22nd ACM international conference on Conference on information & knowledge management. San Francisco, CA: ACM, 2013, pp. 99–108.

19.

Chen

B-C

Guo

Tseng

Yang

. User reputation in a comment rating environment. In: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. San Diego, CA: ACM, 2011, pp. 159–167.

20.

Liu

D-R

Chen

Y-H

Kao

W-C

Wang

H-W

. Integrating expert profile, reputation and link analysis for expert finding in question-answering websites. Information Processing & Management 2013; 49: 312–329.

21.

Zhao

. Topic-Level Expert Modeling in Community Question Answering. In: Proceedings of the 2013 SIAM International Conference on Data Mining. Texas, USA, pp. 776–784.

22.

Song

Wang

Que

Han

. Modeling leading users in professional social-network based community question answering services. Journal of Computational Information Systems 2014; 10: 3197–3204.

23.

McNally

O’Mahony

Smyth

. A model of collaboration-based reputation for the social Web. In: Seventh International AAAI Conference on Weblogs and Social Media. Boston, MA: 2013.

24.

Farzan

DiMicco

Millen

Dugan

Geyer

Brownholtz

. Results from deploying a participation incentive mechanism within the enterprise. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Florence, Italy: ACM, 2008, pp. 563–572.

25.

Vasilescu

Serebrenik

Devanbu

Filkov

. How social Q&A sites are changing knowledge sharing in open source software communities. In: Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. Baltimore, MD: ACM, 2014, pp. 342–354.

26.

Wasserman

. Social network analysis: Methods and applications. Cambridge, UK: Cambridge University Press, 1994.

27.

Bastian

Heymann

Jacomy

. Gephi: an open source software for exploring and manipulating networks. In: International AAAI Conference on Weblogs and Social Media. San Jose, CA: AAAI, 2009, pp. 361–362.