Abstract
Abstract
This study focuses on social feedback, namely on information on the outcome of users' online activity indirectly generated by other users, and investigates in a real setting whether it can affect subsequent activity and, if so, whether participants are aware of that. SkyPas, an application that calculates, transmits, and displays social feedback, was embedded in a common instant messaging service (Skype™) and used during a 7-week trial by 24 office workers at a large business organization. The trial followed an ABA scheme in which the B phase was the feedback provision phase. Results show that social feedback affects users' communication activity (participation, inward communication, outward communication, and reciprocity), sometimes even after the feedback provision phase. At the same time, users were poorly aware of this effect, showing a discrepancy between self-reported and observational measures. These results are then discussed in terms of design transparency and task compatibility.
Introduction
M
Computerized social feedback is an interesting case of persuasive technology 5 —computerized applications implementing social psychology models to change behavior and attitude. Feedback is persuasive because it makes accessible 6 the otherwise invisible social outcomes of an action. This type of information (a) allows actors to assess their distance from the desired outcome and motivates them to refine their subsequent activity to better approach the desired outcome (feedback intervention theory).1,7 In addition, social feedback (b) appeals to the tendency to compare oneself to others to boost self-esteem (social comparison theory8–10 ). In this way, it can nudge the user into a certain activity pattern without asking or forcing anything. 11 Since the persuasive strategy is embedded in the affordances of the medium, its persuasive effect might be overlooked by the user.5,12
Although there is some evidence that social media in organizations increase the ties between coworkers and facilitates knowledge sharing,13–23 no study has so far tested the persuasive effect of social feedback alone, without other social media features such as content sharing or profile creation. Existing studies on feedback effect are carried out in the laboratory24–27 or in class. 28 The present research aims to fill this gap by studying the persuasive effect of social feedback in a real work environment, and the users' acknowledgment of it, which is particularly important at work. 29
The next sections describe the application used in the study (SkyPas). They then present study hypotheses, method, and description of the results. A discussion of the conceptual and practical implications of the results will conclude.
SkyPas
The application used in the study is SkyPas, a modified version of Skype™. SkyPas consists of a plug-in that displays a variety of social feedback indices to the users, based on the Skype messages they exchanged in the previous 5 days and updated every 5 minutes. To calculate the indices, SkyPas records each message sent to a user through Skype as a tuple e = (x, y, t), where x corresponds to the sender, y corresponds to the receiver, and t corresponds to the time of the exchange. It builds a square matrix E, including the number of events exchanged between each pair of users (each cell exy contains the number of messages sent by x to y); the matrix is then dichotomized using a prefixed cut point C to obtain a directed matrix E′. A module in the server calculates the indices based on SNA algorithms 4 and sends them back to SkyPas.
When Skype is open, SkyPas shows as an additional window on the screen (Fig. 1), where all users are ranked according to one of four indices (Table 1). A number and an arrow in a tachymeter scale represent the index score. The scale is red near the lowest index values and green near highest index values. A brief description of the meaning of the index also appears on the right side of the screen. The index can be selected from the top menu bar.

SkyPas interface. The names of the indices appear in the menu bar at the top, and the users, ranked according to their scores in that index, are listed.
The Social Network Analysis index name 4 is given in brackets. Indices value ranges from 0 to 1.
Hypotheses
The two goals of the study are to investigate the effect of social feedback on a real work team and the team's awareness of such an effect. A precondition is that SkyPas is satisfactory to users and not perceived as biased or invasive; lack of accuracy and comprehension can compromise feedback effectiveness1,2,24 and concerns about privacy threats can undermine technology adoption.30,31 Thus, we checked preliminarily that:
H2–H3 and RQ1 address the core goals of our study. We investigated whether a real organization team showed the same effect found in other studies with SNA social feedback, namely an increase in the dimensions in which the feedback was provided.26,32 We expected an increase since, both in the prior studies and the current team, communication is instrumental to the users' activity (i.e., winning a team game based on treasure hunting and collaborating with colleagues, respectively). Also, this communication does not fall into the class of social media use that might be considered idle, and then negatively, in an organization.
33
So, we expect users want to improve in this activity. The effect is tested on the same dimensions of communication addressed by the feedback (participation, inward communication, outward communication, and reciprocity) since previous work suggests that feedback has an effect on the specific communication dimension about which it provides information,24–28,32 and the more specific is the feedback, the more restricted the task it will affect.
7
It was then considered more appropriate to analyze the feedback effects on the same aspects of activity that is addressed by the feedback itself. Thus:
To this hypothesis, we added a research question related to the duration of the feedback. Feedback facilitates learning,
7
so even after deactivation of feedback, the procedures able to obtain the same scores in the communication dimensions highlighted by the feedback should continue to operate in the group. On the other hand, the SkyPas index's association of red with low values made it clear that high values were desirable, as a type of injunctive norm.
34
Thus, after the deactivation of the feedback, only dimensions steadily considered relevant to the groups' activity might have maintained the improvement noticed during the feedback phase. Feedback literature shows increased effects in combination with explicit and preliminary goals,1,35 which this study lacked so as not to interfere with the spontaneous response to the feedback dimensions. Therefore, we assumed that the feedback effect might not endure the suspension of feedback in all communication dimensions observed, but since we had no reliable information on the perceived compatibility of the feedback with the task, we formulated this as a research question:
Third, we investigated participants' awareness of the feedback effect. Martino et al.
32
provided social feedback to players of an online treasure hunting game, while communicating through textual chat with other teammates. Results showed that feedback improved the exchange of messages, but players were largely unaware of this. The explanation might be that users do not realize the persuasive effect of feedback
5
; alternatively, they might be unwilling to acknowledge it because looking manipulated or manipulatory in their relationship to coworkers (as in Dirks and Ferrin
36
) might not be socially desirable.
37
Therefore, we articulate the awareness hypothesis into two subhypotheses, related to asking directly or not about the effectiveness of the feedback:
Materials and Method
Participants
The study involved a group of knowledge workers employed in the Italian headquarters of a large multinational organization. There were 24 participants, 7 women (29.2 percent) and 17 men (70.8 percent), aged 37.96 on average (SD = 6.44; min = 28, max = 54). Four were executives (16.7 percent), six were middle managers (25.0 percent), and 14 were clerks (58.3 percent). They used Skype on a regular basis to communicate within the organization and cooperated on shared projects and business objectives. Participation was completely voluntary and unpaid, and the use of SkyPas—installed on all participants' computers—was not mandatory. Participants' recruitment involved a series of preliminary explanatory meetings. Only the messages exchanged within the participants' group were monitored, according to the confidentiality commitment with the organization.
Design and procedure
We monitored the exchange of Skype messages between participants during their daily work activities for 7 weeks. We used an ABA design (Fig. 2). The first 2 weeks constituted phase A, during which we monitored the messages participants exchanged with other participants to create a baseline. The third week through the fifth week constituted phase B, during which social feedback was available in the SkyPas plugin. During the last 2 weeks of the trial, feedback was no longer available, but we continued to collect messages exchanged through Skype between participants to measure the duration of any feedback effect possibly observed in phase B. Participants received questionnaires right before and soon after phase B.

Schematic representation of the trial phases and data collection schedule.
We did not record message content. We anonymized user identities in our data by automatically replacing the names of all senders and receivers with alphanumeric codes.
Questionnaire
A questionnaire checked participants' awareness of the feedback effect and satisfaction with SkyPas (Table 2). Except for items 7–10 and 17–20, we phrased items as statements for participants to express their level of agreement on a 7-point Likert scale (−3 = “I strongly disagree,” +3 = “I strongly agree”). Sections III to V were only included in the questionnaire administered after phase B. In addition, at the start of the trial, we collected some demographic data (age, gender, education level, working role, and length of service).
All items were in Italian in our study.
Data analysis
A one-sample t-test defined whether the satisfaction scores' difference from the central value of the response scale was statistically significant (H1).
Using Ucinet software, we described users' position in the team communication structure according to four indices: participation (degree centrality), inward communication (indegree centrality), outward communication (outdegree centrality), and reciprocity (Table 1). 41 We calculated these indices twice for each user, once based on message traffic data and once based on the sociometric items of the questionnaire.
An ANOVA for repeated measures with “time” as factor was applied to the indices obtained from instant messages to test feedback effectiveness (H2) and effect duration (RQ1) (t1: phase A, no feedback; t2: phase B, feedback; t3: phase A, no feedback) on the four communication dimensions considered. A 95% confidence interval was adopted for all comparisons of the estimated marginal means. We also collected the frequency with which participants visualized each index, to check the correspondence between most effective and most visualized.
To test users' awareness of feedback effect (H3), we calculated correlation between indices obtained from messages and sociometric items. In addition, we compared the perceived communication structure in the team before and after phase B with an ANOVA for repeated measures. All analyses used SPSS v.19.0.
Results
Satisfaction
Table 3 shows the answers to the questionnaire. They suggest that the participants felt able to use SkyPas application, were at ease with it, and appreciated the feedback provided. The evaluation of SkyPas and the feedback was substantially positive, and the procedure we followed to ensure privacy protection was effective.
Significance level: *p < 0.05; **p < 0.01.
Feedback effectiveness and effect duration
Feedback proved able to influence group communication: the mean values of the examined indices increased significantly in conjunction with the display of the feedback in phase B of the trial (t2 in Fig. 3). With the exception of the reciprocity index, the values also decreased significantly in t3, when feedback was deactivated, returning the values to the level of t1 (Table 4).

The mean values of the social indices during t1 (no feedback), t2 (feedback), and t3 (no feedback).
Significance level: *p < 0.05, **p < 0.01.
Regarding user preferences, there is a significant difference in the frequency at which different indices were visualized, F(1.06, 24.44) = 95.92, p < 0.001, ηp2 = 0.81, the difference remaining when comparing each pair of index, p < 0.01 (Table 5).
Mauchly's test indicated that the assumption of sphericity had been violated, χ2(5) = 150.75, p < 0.001, therefore degrees of freedom were corrected using Greenhouse–Geisser estimates of sphericity (ɛ = 0.35).
A Bonferroni correction was applied.
Significance level: *p < 0.05, **p < 0.01.
Effect awareness
Participants declared that the feedback did not affect their communication activity (item 21: M = 0.087 on a scale ranging from −3 to +3, SD = 1.389) or the group communication activity (item 22: M = 0.364 on a scale ranging from −3 to +3, SD = 1.177). Their perception of the communication structure seems very limited. The answers to the communication structure items before and after the feedback phase (Fig. 4) differed significantly only for item 4 (“In this group, people help each other to accomplish the activities”: M (prefeedback) = 1.083, SD (prefeedback) = 1.742; M (postfeedback) = 1.625, SD (postfeedback) = 1.408; F(1, 23) = 6.760, p = 0.016).

Mean and error bars of the items related to the “communication structure” (1–6).
The indices obtained from sociometric items before and after feedback provision did not differ significantly, except reciprocity index derived from items 7, 9, and 10 (Table 6).
Discussion
SkyPas was considered satisfactory and accurate, confirming H1 and allowing a test for the core hypotheses of the study. H2 was confirmed: the four communication indices analyzed (participation, inward and outward communication, and reciprocity) increased during the feedback phase compared with phase A. Regarding RQ1, the feedback effect persisted after feedback provision in one dimension only, reciprocity. This index is also the one users visualized most often, if we exclude participation, which was displayed by default at the start of the application.
Finally, participants did not explicitly acknowledge any feedback effect, confirming H3a; also, supporting H3b, they had a very limited appraisal of specific changes in the communication structure during the feedback phase, and most of what they did acknowledge was in the reciprocity dimension.
We then proved that feedback based on social network indices can encourage the modification of the communication structure in real team work during its provision. These results are in line with previous laboratory studies on mediated communication,26,32 in which feedback on a specific communication dimension affected that dimension, and extend them for the first time to a real organizational setting. This provides strong evidence of persuasive power of a feedback1,6,7 because it is observed in a real work environment where users are not focused on social feedback alone, unlike a laboratory study, and uses several kinds of media besides the experimental one.42–44
Some remarks should be made on the role of the task. Since in our study the social feedback dimensions were compatible with the activity performed by participants, it would be interesting to continue this line of investigation by studying to what extent task incompatibility would affect social feedback effectiveness, especially if it works as a social norm. 34 We argue that the task type might also be responsible for the enduring effect of feedback on reciprocity: a future study could investigate whether a different activity, where even distribution of information is praised, would make also other indices (e.g., participation) remain high after feedback phase. The role of task has been studied regarding timing, complexity, and novelty,43,45 but not compatibility with the social norm inscribed in the feedback.
Finally, the limitation in participants' awareness of the feedback effect is in line with the indirect strategy of microsuasive strategies, 5 which work by providing affordances or information in technologies that do not have persuasion as the main goal, as is the case of Skype used in our study. This result might also depend on participants' unwillingness to admit persuasion when directly asked, but suggests nonetheless to resort to other measures than self-reported ones to investigate feedback effectiveness. It also suggests to negotiate the use of feedback with the user to avoid deception.5,12
Study limitations
Observation was related to one team; thus, some results might be specific to this team and its activity. The second limitation is the lack of qualitative data, which could have aided understanding the users' interpretation of the feedback and their position on being affected by it.
Conclusions
The article reports a field study on the effect of social feedback in the communication activity of a team in a work organization. The results are in line with the hypotheses showing that feedback can change the communication activity in a team according to the feedback dimensions, participants do not acknowledge this effect, and the effect persists only for some dimensions. This suggests some ethical provisions in feedback design and the possible role of task compatibility in mediating persistency of feedback effect.
Footnotes
Author Disclosure Statement
No competing financial interests exist. The study reported here is partially funded by the PASION project (Psychologically Augmented Social Interaction over Networks, reference number 27654 PASION, EU IST program), where the SkyPas application was developed.
