Methodological Issues in Using Data from Social Networking Sites

Introduction

Social networking sites (SNS) are defined by boyd & Ellison ¹ as web-based services that allow individuals to construct a public profile within a bounded system and articulate a list of other users with whom they share a connection. People join SNS to stay in touch with their friends and maintain existing social relations. As the members know each other, there is a high level of trust and regular communication between members of SNS.^2,3

The growing popularity of SNS has attracted the interest of researchers. With an increasing number of members, SNS allow researchers to access a large number of people efficiently. Researchers can invite members of SNS to participate in their research, or even distribute survey instruments via the SNS.

In particular, SNS allow researchers to access segments of the population that are difficult to reach or are empirically underrepresented. For example, homosexuals form a small proportion of the general population, and to reach this group using random sampling is generally difficult. As such, research with empirically underrepresented groups has largely used a snowballing sampling design where respondents refer the researcher to other prospective participants. ⁴ This process of obtaining access to prospective participants can be made more efficient through SNS. Members of SNS share similar interests and are known to each other. By selecting SNS where members share specific characteristics, a researcher can quickly gain access to prospective participants.

Other researchers see the regular communication between members of SNS as a rich source of naturalistic behavioral data that lend themselves well to cyber-ethnographic research.^1,4 Similar to research using diary entries as a means of collecting data, entries on SNS can potentially offer deep insights, as there is a regular stream of undirected data. This allows for temporal analysis of the data by examining how members of SNS construct meaning over time and how attitudes are changed through interaction with other members.^5–7 As entries in SNS are entered close to the respondent's experience, it is also subject to less distortion and is hence regarded as more valid.^8,9

However, the use of such data from SNS for research is not without its criticisms. As a medium leveraging on the Internet, SNS research shares some of the benefits associated with Internet-based research, including lower research costs and greater access to respondents.^10–12 At the same time, some of the limitations with Internet-based research also apply to research conducted on SNS. These include the criticism of whether findings from Internet users can be generalized to non-Internet users, the validity of responses due to anonymity and low accountability of Internet users, and the potential for unethical research practices. ¹⁰

The aim of this article is to review the methodological issues in using data from SNS for research and to address the criticisms in using such data for research purposes. Both the issue of generalizability of findings and the validity of responses as limitations will be discussed in this article.

Although it is not within the scope of this article to discuss the ethical issues in using data from SNS, researchers should be aware that conducting research on SNS can be potentially unethical. Online postings are generally considered information available in the public domain. As such, there is no need to obtain informed consent in using the data according to existing standards of most institutional ethics review boards. It is also debatable whether it is practically feasible to obtain informed consent from members of SNS in the first place. However, as information from SNS can be traced back to the individual, such information can be perceived to be personal. Hence, the use of information from SNS without the informed consent of individuals can be considered a violation of privacy.^10,12,13

This is not an issue that can be resolved easily. Researchers should review their research practice for potential violation of ethical norms when using data from SNS. Moreno et al. ¹³ have suggested an ethical framework that may be useful in guiding researchers in this aspect.

Generalizability of Research Findings

One of the main criticisms of research conducted on SNS is whether the findings can be generalizable to the general population. At issue is whether there is any difference between members and nonmembers of SNS.

Some studies have suggested that there are demographic differences between members and nonmembers of SNS. Members of SNS tend to be younger than the general population.^4,10,14–17 As such, they are not representative of the population, and the findings cannot be generalized to the larger population. ¹⁰ For example, a politician who is popular on SNS may not see his popularity translate into a victory at the polls, as the online community is made up of younger segment of the population, some of which may not even be eligible to vote. ¹⁸

Another difference between members and nonmembers of SNS lies in members being more likely to have access to the Internet and being more comfortable with technology.^15,19,20 With an increasing number of people with access to the Internet, it can be argued that this difference will be eroded with time. However, in countries where access to the Internet and SNS is limited to segments of the population with higher income or specific geographic locations, research findings are necessarily still limited in generalizability.

Studies on psychographic differences between members and nonmembers of SNS have not been conclusive. A study found that members of SNS have a higher need to belong to social groups. ¹⁵ However, this has not been replicated in other studies, and a recent review of the literature suggests that there is insufficient evidence to conclude that there are differences in personality traits between the two groups. ¹⁹

Low Levels of Activity

One of the attractions in analyzing data from SNS is the availability of a high level of regular communication between members. However, especially on SNS of commercial companies, the level of activity may be too low to allow for meaningful analysis. More importantly, when there is an absence of a large group of people actively communicating with one another in the account, the views cannot be representative. ²

Some commercial companies see SNS as an extension of their traditional marketing communication tool and use SNS to advertise their products. As such, the SNS for such companies is essentially a profile of the company, similar to a corporate Web page. SNS of this type do not encourage a conversation between members and, more importantly, fail to engage consumers to determine the cultural meaning of the brand.^19,21 The conversation is dominated by the corporation, and, as such, the views on such SNS cannot be representative.

Corporations who see their SNS as part of their traditional one-directional marketing tool will also be more likely to be uncomfortable with the interactive element in SNS. This is especially so when negative comments on their products or company are posted on their SNS. As more attention is given to negative reviews made online, such corporations may delete negative comments on their SNS. ²² While there have been few studies to examine the extent of this practice, it suggests that researchers should be aware of this possibility when analyzing data on SNS given that it can affect the validity of research findings. Hence, it is important to consider this issue especially when the level of activity is low and the conversation is dominated by the corporation.

It is not only the number of entries that matter, but also the quality of the posting on the SNS. Where members are unable to express an experience to sufficient depth, it will be difficult to provide a meaningful analysis of the data, especially when SNS entries tend to be short. ⁶ The value of SNS data lies in analyzing the postings over a period of time. This temporal analysis allows researchers to form a theory as to how attitudes or behavior may change with time or in response to other members in the social network. Hence, when members are unable to express themselves clearly over a period of time, researchers will not be able to analyze how consumers make cultural meaning of a brand or issue from the data on SNS.

Validity of Postings

As members of SNS know each other, there is a high level of trust between members, and the information shared is likely to be accurate and truthful. To a researcher, these data become valid, as they are a reflection of the true experience of the members in the SNS. However, there exists the potential for respondents to be less than honest about socially undesirable behaviors, as there is pressure to conform to group expectations and norms.^5,10,23 Members of SNS will be less willing to be honest about behaviors that may embarrass them.

The literature suggests that when a group is formed due to attachment between members, the group is defined as a common bond group. SNS with members who are friends with each other can be considered as common bond groups. In contrast, when a group is formed due to the attachment to the group as a whole, this group is defined as a common identity group. Such SNS include those formed by corporations and other organizations like schools, communities, and nonprofit organizations. This distinction is important, as there is greater pressure for members of common identity groups to adhere to group norms. ²⁴ Hence, researchers should be aware of the greater pressure among members of common identity groups to exhibit behaviors that are consistent with group expectations or norms.

In addition, where their security or safety is at risk, members may choose a response that is less than truthful when they can be identified. For example, an incumbent politician or a politician who is expected to have a high probability of winning in the elections may receive more positive feedback on SNS. This is because the members of the SNS may choose behaviors to avoid repercussions if the politician is voted in. ²⁵

While most SNS encourage users to construct accurate representations of themselves, members of SNS do this to varying degrees. ¹ As such, in analyzing data from SNS where most members do not know each other or do not have a high level of trust, there is some anonymity provided to members. This is especially so on corporate SNS where the number of members tends to be large. In such cases, using the Internet as a medium can encourage respondents to be more expressive of their thoughts and feelings because it does not involve face-to-face communications. ²⁶

Also, to preserve their anonymity, members of the online community may create fake accounts in order to post negative comments without repercussions. While the anonymity may encourage the responses to be more valid, the researcher will also need to be aware that such anonymity may invalidate the research findings, especially when fake accounts are used to specifically create a negative perception of the company, person, or issue without basis or out of malice.

More importantly, the researcher needs to be aware that the discussions on SNS do not necessarily translate into behaviors. This is because the nature of SNS favors a quick emotive response to postings rather than a prolonged, conscious, and cognitive response. Members of SNS can click on a “like” button or react to a posting with quick comments without much thought. However, when faced with the consumption opportunity, especially in high involvement decisions, the rational or cognitive process takes precedence and may result in a different behavior.

For example, SNS facilitate affective political alliance, as they allow the public to contribute comments and engage with the candidate easily. However, this affective political alliance may not translate into votes, as voting behavior remains a rational choice that requires a deeper level of information processing than the requirements of clicking a “like” button on Facebook.^18,27,28

Conclusion

With the popularity of SNS, the volume of data on SNS on how people form opinions and intended behaviors on products, companies, and issues has increased over the last few years. This presents an opportunity for researchers to mine the data and analyze them. However, as reviewed in this article, the researcher needs to be aware that analyzing data from SNS is not without its limitations.

First, as the members on SNS are self-selected, they may not be representative of the population. Those who are members of SNS tend to be younger and more at ease with Internet technology. Other variables may also work toward the bias in encouraging certain segments of the population in being members of SNS. As such, the researcher will need to be aware of the potential bias in findings resulting from the analysis of data from SNS.

Second, the researcher should determine if there are sufficient data in the SNS for analysis. Besides reviewing the quantity of postings, the researcher should also review the number of people posting on the site and the proportion of postings by each individual to ensure representativeness of the responses.

Third, the researcher should consider whether there is a risk for members not to be truthful with their responses and hence invalidate the findings. The risks will be higher where socially undesirable behaviors are investigated amongst members who know each other or where there is motivation to post untruthful responses. In addition, noting that the environment for SNS favors a quick emotive response as opposed to a cognitive response, the researcher will need to be aware of a possibly different behavior when members of a SNS are faced with a high involvement decision.

The review suggests that while there is potential in analyzing data from SNS, the researcher will need to be aware of the limitations. Perhaps it is not so much about which method is superior but rather that the researcher needs to be aware of the strengths and weaknesses of the different methods.^9,11,12