Abstract
Social network site (SNS) data provide scholars with a plethora of new opportunities for studying public opinion and forecasting electoral outcomes. While these are certainly among the most promising big data applications in political science research, a series of pioneering studies have started to uncover the vast potential of such data to estimate the policy positions of political actors. Adding to this emerging strand in the scholarly literature, the present article explores the validity of (individual) policy positions derived from the social network structure of the microblogging platform Twitter. At the aggregate party level, cross-validation with external data sources suggests that SNS data provide valid policy position estimates. In contrast, the empirical analysis reveals only a moderate connection between individual policy positions retrieved from the social network structure and those retrieved from members of parliament individual voting record. These results thus highlight the potential as well as important limitations of SNS data in indicating the policy positions of political parties and individual legislators.
Introduction
Social network site (SNS) data provide scholars with a plethora of opportunities to gain new insights into public opinion and political participation. Among the most prominent big data applications in political science research are the use of SNS data to predict electoral outcomes (e.g., Tumasjan, Sprenger, Sandner, & Welpe, 2010) and to gauge the popularity of political parties and individual candidates (e.g., Ceron, Curini, Iacus, & Porro, 2013; O’Connor, Balasubramanyan, Routledge, & Smith, 2010; see Gayo-Avello, 2013, for a critical review of this strand in the scholarly literature).
Yet, another strand of scholarly research exploits large-scale data sets of online political behavior to retrieve the policy positions or “political leaning” (Gayo-Avello, 2013, p. 652) of voters, candidates, and political parties alike. In fact, a series of pioneering studies highlight the vast potential of SNSs, most prominently Twitter (Barberá, 2015; Boutet, Kim, & Yoneki, 2013; Ceron & Cremonesi, 2013; Conover, Goncalves, Ratkiewicz, Flammini, & Menczer, 2011; King, Orlando, & Sparks, 2011) and Facebook (Bond & Messing, 2015), to retrieve such policy position estimates. As such, these studies extend the existing methodological spectrum, which includes inter alia surveys of experts and legislators, the content analysis of textual data including speeches and election manifestos, as well as the systematic analysis of behavioral patterns (see Laver, 2014, for an exhaustive overview of existing methods).
At the same time, these studies emphasize several key advantages of this social network approach. Most importantly, SNS data allow estimating individual policy positions for a fractional amount of the resources required by most alternative methods, while placing all relevant political actors on a common n-dimensional policy space (Barberá, 2015, p. 77; Bond & Messing, 2015, p. 63). These benefits may come at a price, however, as the estimated policy positions may be no valid approximation of actors’ actual policy positions. In fact, we still know comparatively little about the validity of policy positions retrieved via SNSs, in particular in the context of European multiparty systems. The present article thus adds to the emerging literature on SNSs by exploring the following two interrelated research questions: First, whether a social network approach provides valid policy position estimates at the aggregate party level. And second, whether and to what extent SNS data allow retrieving valid position estimates at the level of individual members of parliament (MPs).
In order to answer these research questions, the empirical analysis derives the policy positions of legislators of the Polish Sejm—the lower house of Poland’s bicameral legislature—based on the social network structure of the microblogging platform Twitter. I thereby exploit three characteristic features of the political landscape in Poland. First, members of the Sejm are among the most active European political elites on Twitter with an adoption rate above 50%. SNS data are thus available for a sufficiently large share of the population of interest. Second, the Polish multiparty system is still in the process of consolidation and institutionalization (Szczerbiak, 2013). Consequently, MPs often switch their partisan affiliation during the legislative term, and these defecting legislators provide us with considerable analytical leverage for validating individual policy position estimates. Third, the Polish Sejm is one of the few European legislatures, which employs electronic roll-call votes. As a result, the present article is among the first to validate policy positions obtained via the social network approach against the parliamentary voting behavior of individual legislators outside the U.S. context.
Overall, the results of the empirical analysis lend partial support to the validity of policy positions retrieved via SNSs. At the aggregate level of political parties, cross-validation with existing alternative measures indicates that SNS data provide valid estimates of party policy positions not only in Western Europe but also in the more complex party systems of Central and Eastern Europe (CEE). Consequently, the social network approach thus potentially allows scholars to obtain party policy positions across space and time more economically than most existing alternative methods. At the level of individual legislators, in contrast, the empirical results are less clear-cut. While the in-depth analysis of defecting MPs reveals a plausible empirical pattern, I find only a moderate connection between individual policy positions retrieved from the social network structure and those retrieved from MPs individual voting record. These results thus highlight both the potential and the important limitations of SNSs in indicating the policy positions of political parties and individual MPs.
SNSs, Political Elites, and Individual Policy Positions
Over the past few years, SNSs in general and Twitter in particular have become increasingly popular among citizens around the globe. While Facebook remains by far the most popular SNS, the microblogging platform Twitter now has more than 300 million monthly active users 1 with comparably high shares among the online population in the United States (23%) and in Europe (e.g., Poland: 18%, United Kingdom: 25%, and the Netherlands: 27%). 2 This rapid proliferation of SNSs among the general population has sparked considerable interest by scholars of political communication. In this context, a key question is whether SNSs have beneficial effects on social capital accumulation, civic engagement, and political participation. Recent empirical studies indicate that social media use has indeed a positive effect on these political outcomes (Valenzuela, Park, & Kee, 2009), in particular when geared toward political news consumption (Gil de Zúñiga, Jung, & Valenzuela, 2012; Valenzuela, Arriagada, & Scherman, 2012).
Given the growing popularity of SNSs among the electorate, political elites have quickly recognized the vast potential vested in direct communication with citizens and adapted their electoral strategies accordingly. As a result, a growing number of political parties, individual legislators, and electoral candidates today use Twitter in their day-to-day political work. In fact, its growing popularity is said to have fundamentally altered the patterns of contemporary citizens-elite communication in ways characterized as nothing short of a revolutionary process (Gainous & Wagner, 2014; Parmelee & Shannon, 2012).
However, the pace by which the supply side of the electoral market is adapting varies considerably between countries (adoption rates among legislators are 10% in Italy, over 50% in Poland, around 65% in the United Kingdom and over 90% in the United States). 3 A first line of research is thus concerned with the determinants of SNS adoption by political elites. Here, the empirical results suggest that party affiliation and individual characteristics, which are closely linked to visibility in and access to the mainstream media, play an important role (Gainous & Wagner, 2014; Larsson & Kalsnes, 2014; Skovsgaard & Van Dalen, 2013; Vergeer & Hermans, 2013). A second key concern is how political elites use SNSs. Do communication patterns follow the traditional top-down flow or do elites fully exploit SNSs interactive potential? Here, the empirical findings are less conclusive, although elites primarily seem to broadcast information and to mobilize electoral support via social networks, rather than to continuously interact with their electorate (Golbeck, Grimes, & Rogers, 2010; Graham, Broersma, Hazelhoff, & van ‘t Haar, 2013).
This research offers valuable insights on the adoption and use of SNSs. At the same time, SNSs provide scholars with highly valuable individual-level data on the content of political elite communication and the underlying social network structure. A series of pioneering studies exploit this publicly available information in order to estimate the policy positions of political actors (Barberá, 2015; Bond & Messing, 2015; Boutet et al., 2013; Ceron & Cremonesi, 2013; Conover et al., 2011; King et al., 2011). Based on the specific source of information used, we may differentiate three alternative approaches (Barberá, 2015, supplementary materials). A first approach focuses on the actual content of messages sent by key political actors. By identifying and collecting all relevant messages, researchers obtain characteristic word patterns used by political elites to communicate their policy preferences. Similar to existing studies analyzing political texts such as election manifestos, these textual data are then subjected to quantitative text analysis. Ceron and Cremonesi (2013) find this method to provide valid position estimates of factions within political parties in Italy.
A second approach is based on the communication patterns in SNSs. Characteristics such as the number of political messages sent by political elites, who replies to them and by whom their messages are forwarded, are expected to follow a distinct pattern. These patterns are then analyzed in order to place individuals on an ideological continuum. Using micro data of individual Tweets, this approach has been successfully applied to Twitter users during the U.S. midterm elections in 2010 (Conover et al., 2011) and the U.K. general elections in 2010 (Boutet et al., 2013).
According to Barberá (2015), however, both these approaches may suffer from substantial drawbacks. In particular, negative sentiment, ironic statements, and other latent content frequently characterize political messages in social networks. These characteristics often cast doubts on the validity of measures derived from the quantitative analysis of political texts. In a similar vein, the sole act of replying to or forwarding of a political message in SNSs often expresses criticism and discontent with the sender or may likewise be ironic. As a result, both these approaches may result in biased position estimates. In order to remedy this shortcoming, a third alternative approach exclusively focuses on the social network structure. In particular, researchers rely on the network structure between voters and political elites in SNSs to retrieve estimates of individual policy positions via a two-step process (Barberá, 2015; King et al., 2011). First, the social network is translated into a voter-to-elite adjacency matrix. In a second step, data reduction techniques are applied to the resulting social network structure in order to recover individual policy position estimates on a latent (ideological) dimension.
This social network approach also has a number of advantages vis-à-vis alternative measures of policy positions based on other data sources such as voting patterns or textual data. While a detailed discussion of each method’s positive and negative aspects is beyond the scope of this article, one of the arguably most authoritative data sources is the voting behavior of political elites. Existing research thus preferably relies on roll-call vote data to capture the policy positions of individual MPs. Unlike these measures based on parliamentary behavior, SNS data are less prone to the confounding effects of party discipline and strategic use by the legislative party leadership (Carrubba, Gabel, & Hug, 2008; Depauw & Martin, 2009). This is particularly true when examining the structure of elite-citizen networks on which political elites have only very limited influence. At the same time, the social network approach is applicable to countries, which do not regularly employ roll-call votes in the legislature. Finally, SNS data allow estimating the policy positions of individuals without voting record, most importantly legislative candidates.
In a similar vein, measures based on legislative speeches may likewise result in biased position estimates. Most importantly, Proksch and Slapin (2012) argue that dissenting preferences will be systematically underrepresented as the legislative party leadership exerts tight control over access to the floor. At the same time, MPs may purposely deemphasize divisive issues and conceal their actual policy preferences in order to maintain party cohesion. A final source used to estimate policy positions is documents and manifestos drafted by political parties and party factions. One immediately apparent drawback here is that these documents allow estimating aggregate policy positions of political parties or groups of MPs but not those of individual legislators. Also, only some party factions draft such programmatic documents on a regular basis (Ceron & Cremonesi, 2013, p. 6). As a result, these documents do not provide sufficient coverage for a meaningful comparative analysis beyond aggregate party policy positions.
A Social Network Approach to (Individual) Policy Positions
Overall, the above-mentioned discussion suggests that using the social network structure is beneficial both over alternative data sources and other recently developed SNS-based measures. Nonetheless, whether this method provides unbiased (individual) position estimates is ultimately an empirical question. In order to assess the validity of the social network approach, the subsequent empirical analysis is based upon policy position estimates of political parties and individual legislators of the Polish Sejm in 2014. To put these position estimates into broader context, the following section briefly discusses recent developments of the Polish party system and provides some general background information. The subsequent sections then describe in more detail how policy positions are retrieved via the social network structure of the microblogging platform Twitter. While the social network approach is in principle applicable to other SNSs, such as Facebook (Bond & Messing, 2015), there are two reasons for choosing Twitter in the present context. First, it is one of the most popular SNSs among politicians in Poland and generally used for professional rather than private purposes. And second, in contrast to other SNSs, such as Facebook, Twitter provides scholars with publicly available data on social network structures. In particular, it allows retrieving politicians’ entire follower network via its application-programming interface (API; see below for a more detailed description of the data collection process).
Even more than two decades after the country’s successful democratic transition, the Polish party system is still going through a consolidation and stabilization process (Szczerbiak, 2013). In fact, only recently has a division within the Post-Solidarity camp marginalized the postcommunist divide, which had dominated party competition in Poland for a long time. As a result, the centrist Civic Platform (PO) and the conservative Law and Justice (PiS) party now dominate the Polish party system and have been (jointly) at the head of the national government since the parliamentary elections in 2005 (Tworzecki, 2012). Yet, while both parties share common historical roots in the fight against the Communist rule, the former is generally characterized as a liberal and promarket-oriented party, whereas the latter has developed a distinct national-conservative profile (Szczerbiak, 2008).
Following the most recent parliamentary elections in 2011, the liberal Palikot Movement (RP; recently renamed “Your Move”) emerged as the third largest party in the Sejm, while both communist successor parties—the Democratic Left Alliance (SLD) and the Polish People’s Party (PSL)—lost part of their electoral support (Szczerbiak, 2013, p. 480). However, the RP has been continuously disintegrating ever since, losing 24 of the 40 seats it had initially secured in the election in 2011. As of December 2014, the PO-PSL coalition government jointly controls 233 of the 460 seats in the Sejm, while the PiS is the largest opposition party holding 135 seats. Table 1 briefly summarizes the parliamentary strength and general ideological profile of the five main parliamentary parties in the Polish Sejm.
Parliamentary Strength and Ideological Profile of Major Political Parties in the Polish Sejm as of December 30, 2014.
Note. This table shows the number of seats and the ideological profile of the five main parliamentary parties in Poland. Government parties are marked with an asterisk.
The empirical analysis is based on the individual policy positions of m = 259 cabinet members and MPs of the five main parliamentary parties in the Polish Sejm with an active Twitter account as of December 30, 2014. 4 Polish legislators are among the most active political elites in Europe with an adoption rate of over 50% (246 of all 460 members of the Sejm as of December 2014). All Twitter accounts are identified via the official website of the Sejm and/or MPs personal websites, and crosschecked for visual appearance. Inactive accounts are excluded based on the following three criteria: (i) less than 100 followers, (ii) less than 20 status updates (Tweets) in total, or (iii) no Tweet during the past 12 months (January 1, 2014–December 30, 2014). All subsequent analyses are thus based on the entire population of Polish legislators and cabinet members with an active Twitter account.
Next, all 253,256 individual Twitter users following at least 3 of the 259 relevant MPs and cabinet members are identified using Twitter’s API. 5 Again, inactive accounts are excluded from the analysis based on similar criteria (but lowering the follower threshold to 10 followers), substantially reducing the number of relevant users to n = 32,305. In contrast to studies based on API samples of individual Tweets, the subsequent analysis is thus based on the entire population of active followers of political elites. 6 The resulting m × n adjacency matrix thus depicts all active Twitter users’ binary decision to follow a particular politician. The underlying latent policy dimension is then retrieved via Poole and Rosenthal’s W-NOMINATE scaling procedure (1985; Poole, Lewis, Lo, & Carroll, 2011). Specifically, the individual policy positions of political elites are modeled as points in a multidimensional space, where users’ decisions to follow or not to follow a particular MP or cabinet member is a function of the distance between their policy positions and that of the politician. Similar to King, Orlando, and Sparks (2011), the first resulting dimension depicts the overall popularity of MPs on Twitter, 7 while the second dimension shows an ideological pattern where MPs positions cluster along party lines.
Empirical Analysis
The following empirical analysis assesses the validity of the social network approach via a stepwise procedure. Starting at the aggregate party level, I explore whether the party policy positions retrieved from the social network approach are consistent with descriptive accounts of the Polish multiparty system. Having examined their face validity, these aggregate-level estimates are then cross-validated with two established alternative measures of party policy positions. Specifically, I compare the median individual position estimate of each parliamentary party with position estimates retrieved from an expert survey and a survey among Polish MPs. Turning to the level of individual legislators, the empirical analysis initially explores the face validity by focusing on the estimated policy positions of prominent political actors and defecting individual MPs. In a subsequent step, I then cross-validate the obtained position estimates with individual policy positions retrieved via an analysis of roll-call votes in the Sejm. In a final step, the analysis then goes beyond simple cross-validation with existing measures by analyzing whether the position estimates are consistent with theoretically derived expectations. Specifically, I explore the well-established relationship between an MPs rank in the party hierarchy and the individual policy positions vis-à-vis those of the parliamentary party.
Validity at the Aggregate Party Level
The left panel in Figure 1 depicts the derived individual position estimates separately for each parliamentary party (as well as all independent legislators). The black marker highlights each party’s median legislator while the horizontal lines indicate the corresponding 95% confidence interval. Overall, we find that the policy positions of MPs with the same partisan affiliation cluster along the spatial continuum. At the same time, however, some individual position estimates are located far away from the party’s median legislator. Finally, the confidence intervals suggest that four of the five major parliamentary parties (excluding the Polish Righteous [SP] splinter group, see below) occupy unique and statistically distinguishable positions on this policy continuum.
Individual policy position estimates of members of the Polish Sejm in 2014. Source. Bakker et al. (2012; expert survey) and Deschouwer, Depauw, & Audrey (2014; elite survey). Note. This figure shows estimated (individual) policy positions based on the Twitter social network structure (left panel), an expert survey (central panel), and a survey among individual MPs (right panel). Hollow gray dots represent individual position estimates, while all median (mean) position estimates are indicated by black dots. The corresponding 95% confidence intervals (horizontal lines) are based on bootstrapped standard errors. Parties’ ordinal position on the y-axis follows their ranking by external data sources.
I continue the empirical analysis by confronting these estimates based on the social network structure with contemporary accounts of political parties in Poland. Overall, the left panel in Figure 1 nicely replicates the general structure of the Polish party system. As expected, the PO appears as a catch-all party, representing a broad ideological spectrum and positioning itself mainly in opposition to the conservative nationalism of the PiS (Szczerbiak, 2013, p. 496). At the same time, the RP occupies an ideological position more liberal than that of the PO mainstream deputies (Jasiewicz & Jasiewicz-Betkiewicz, 2013, p. 186). The PSL, on the other hand, is located in between the two Post-Solidarity parties and is the only party with the ability to cooperate with the PO to its left and the PiS to its right. Again, this finding corresponds with recent accounts on the development of the Polish party system (Antoszewski & Kozierska, 2014). Finally, the SP constitutes a parliamentary party group founded in late 2014 by defecting MPs from the PO and the PiS. As one would expect, these MPs hold highly divergent policy positions and are located between the two large Post-Solidarity parties.
The additional panels in Figure 1 depict the most recent position estimates derived from two external data sources and allow for cross-validation with existing measures of party policy positions. 8 Specifically, the central panel depicts estimates derived from expert evaluations in 2010 (Bakker et al., 2012). Here, each dot represents the average party position on a general left–right dimension with 95% confidence intervals marked by horizontal lines. The right panel depicts the self-positioning of Polish MPs on a general left–right scale in a survey administered in 2009 (Deschouwer, Depauw, & Audrey, 2014). Again, black markers designate each party’s median legislator, while the horizontal lines indicate the corresponding 95% confidence interval.
The ordinal alignment of political parties across data sources is similar though not identical. In particular, both expert and elite data suggest that the PO is located to the right of the PSL. In contrast, the estimates derived from the social network structure suggest that the PO is located on the center-left of the policy continuum. One potential explanation for this finding is the considerable time lag between measures. In particular, both external data sources capture the party’s position long before the 2011 election where the PO continued its considerable shift to align itself with the “liberal-left cultural and media establishment” (Szczerbiak, 2013, p. 496).
Concerning the estimated confidence intervals, one key advantage over both expert and elite data is the considerably smaller variance of the aggregate position estimates retrieved from SNS data. In this context, a main driving factor is that Twitter estimates are available for considerably more respondents (on average 41 active Twitter accounts per party) compared to the two alternative data sources (on average 14 experts and 10 MPs per party, respectively). Yet, another potential explanation is that SNS-based measures may systematically underestimate the extent to which MPs of the same party hold divergent or even conflicting policy positions. Here, the distribution of elite survey estimates provides a natural reference point as they likewise reflect the divergent policy positions held by MPs. According to these data, it is the two Post-Solidarity parties, PO and PiS, where individual policy positions are most heterogeneous. A similar pattern emerges for the estimates derived from the social network structure, where MPs from the PiS (SD = .24) are considerably more dispersed than those of the RP (SD = .16) and the SLD (SD = .15), for instance. In fact, both data sources suggest that the latter is the most homogenous party in terms of policy positions. The main discrepancy concerns the positional spread among MPs from the PSL. Here, elite data suggest that MPs locate themselves closely to the center of the policy continuum—an empirical pattern not observable for the Twitter data.
Cross-validating the SNS-based estimates with positions retrieved from other data sources comes with two important caveats. Most importantly, both external data sources capture party policy positions from 4 (expert survey) and 5 (elite survey) years ago. Thus, when comparing estimates based on different data sources, it is important to bear in mind potential structural changes of the party system as well as policy shifts, in particular in less consolidated party systems in CEE. Second, surveys among MPs generally suffer from low response rates and strategic response behavior. As a result, they provide mostly tentative evidence on policy positions at the level of political parties. Nonetheless, the above cross-validation with existing measures of party policy positions suggests that SNS data largely provide valid estimates at the aggregate party level.
Validity at the Individual Legislator Level
Having explored the validity of the social network approach at the aggregate level, we now turn to the results at the level of individual legislators. Figure 2 depicts the position estimates of two particularly interesting groups of political actors. The first group comprises five key figures in Polish politics including the former Prime Minister and now President of the European Council Donald Tusk. Similar to Ewa Kopacz—his successor as prime minister—the estimated policy position is located close to the PO’s median legislator. 9 Also, the position estimate for the SLD Chairman Leszek Miller closely corresponds to his party’s aggregate position estimate. In contrast, the estimated ideological positions of Janusz Piechocinski (PSL) and Jaroslaw Kaczynski (PiS) are somewhat at odds with popular accounts on the ideological stances of these important figures in Polish politics.
Individual policy position estimates of key political actors and defecting MPs in the Polish Sejm in 2014. Note. This figure shows estimated (individual) policy positions based on the Twitter social network structure. Gray dots represent individual position estimates of key political actors (black marker labels) and defecting MPs (gray marker labels). All median position estimates are indicated by black dots, while the corresponding 95% confidence intervals (horizontal lines) are based on bootstrapped standard errors. Position estimates of defecting MPs arranged according to their new party affiliation. Parties’ ordinal position on the y-axis follows their ranking by external data sources.
A second group of particularly relevant individual legislators are defecting MPs. Several Sejm members have changed their parliamentary party group affiliation after the elections in 2011, and these MPs provide us with considerable analytical leverage for further validating the individual position estimates. Figure 2 arranges them based on their new party affiliation while denoting their original affiliation via gray marker labels. Regarding these defecting legislators, their position estimates follow two consistent patterns. First, over half of the RP legislators have abandoned their original party, either joining a communist successor party or becoming independent legislators. In line with their original party affiliation, however, these MPs still hold “liberal” policy positions. At the same time, they constitute liberal factions within the SLD and the PSL, two parties whose median legislator is located considerably more to the right. In particular, for the PSL being the junior coalition partner, this pattern likely reflects individual MPs, who traded in policy goals for the spoils of government participation. The second pattern relates to the newly founded parliamentary party group of the Polish Righteous. Here, Figure 2 suggests that the SP mostly consists of liberal former PiS legislators (relative to their party) and “conservative” former PO MPs. Most SP legislators thus hold individual policy positions that are at odds with those of their original parliamentary party, suggesting that defection resulted from incompatible ideological stances.
An alternative method to assess the validity of the position estimates at the level of individual legislators is to cross-validate them with each MPs voting record in the legislature. Figure 3 plots the results of the social network approach (y-axis) against individual position estimates derived from analyzing 1,359 roll-call votes in the Sejm in 2014 (x-axis). 10 In the Polish context, the electronic voting system used for votes on all bills and amendments in the legislature mitigates concerns about strategic roll-call vote use by the party leadership. At the same time, any position estimate derived from parliamentary behavior is likely to reflect not only individual preferences but also government status and the extent to which political parties exercise a certain discipline upon their MPs.
Cross-validating the social network approach with roll-call vote data on all bills and amendments in the Polish Sejm in 2014. Note. This figure shows estimated individual policy positions based on the Twitter social network structure (y-axis) and an analysis of 1,359 roll-call votes in the Sejm in 2014 (x-axis). Each party abbreviation places an individual MP on the two-dimensional policy space based on the estimated policy positions.
Overall, Figure 3 suggests that both measures provide similar though not identical position estimates. Again, we observe that, with the exception of the SP, MPs of the same party cluster in the two-dimensional space. In fact, the correlation between the position estimates derived from the two different data sources is moderately high (ρ = .57). Yet, this empirical relationship is largely driven by the variance between parties, while the within-party correlation ranges from ρ = .08 to ρ = .75. Most importantly, Figure 3 suggests that roll-call vote data potentially underestimate the diversity of policy positions within parties. This is most apparent for the RP and the SLD whose voting record indicates that they consistently vote against government bills in the legislature.
Overall, assessing the validity at the level of individual legislators is an incomparably more complex task, since little alternative data are available in the first place. In terms of face validity, the individual position estimates of key political actors and defecting MPs largely follow the expected patterns. Cross-validation with roll-call vote data, on the other hand, results in a less conclusive empirical picture, mostly due to the considerably lower variance within parties’ legislative voting record. In this context, an alternative to cross-validation with existing measures is to explore whether a new measure’s empirical relationship with other characteristics is in line with well-founded and established theoretical expectations. This approach is also known as construct validity (Bollen, 1989).
A final validity test therefore explores the empirical relationship between the individual policy positions derived from the social network structure and a legislator’s rank in the party hierarchy. The scholarly literature on intraparty politics generally differentiates between backbenchers at the lower end of the hierarchy and MPs holding a leadership position within the general party, in parliament (e.g., chief whip) and/or in the executive (e.g., ministerial position). Here, the main theoretical expectation is that MPs incentives to adopt policy positions at odds with those of their party decrease as they climb the party hierarchy.
Several causal mechanisms support this theoretical assertion. First and foremost, leading MPs simply exert considerably more influence over their party’s policy position than backbenchers. For instance, the legislative party leadership controls access to the floor and may thus delay legislation in conflict with their own policy positions (Sieberer, 2010). A second mechanism is the restricted access to leadership positions. In fact, the party leadership, which has a strong gatekeeping function, closely scrutinizes all potential candidates. MPs with dissenting policy positions will thus be systematically underrepresented in leadership positions. Yet another aspect is political socialization (Kam, 2009). MPs holding leadership positions repeatedly interact with a wide range of legislators and have close ties to leading MPs and committee chairs from their party as well as from other parliamentary parties. This repeated process of social interaction is expected to result in less extreme policy positions. A final causal path highlights fundamental differences of the cost–benefit structure (Sieberer, 2010). Specifically, backbenchers adopting dissenting policy positions may ultimately lose their privileges as legislators if excluded by the party. High-ranking MPs, however, will also endanger the power vested in party, parliamentary and/or executive leadership.
In order to test this hypothesized relationship, I define the policy position of each MP relative to their party as the absolute distance between their individual position and that of the party’s median legislator. It ranges from 0 for the median legislator of each party to a maximum value of .79 with an average value of .14 and a standard deviation of .13. Next, I classify all MPs depending on whether they hold a leadership position in the party, the legislature or the executive or not. Overall, 24 of the 259 politicians with an active Twitter account (approximately 9.3%) hold such a leadership position.
Table 2 presents the difference in means between these two groups of MPs. As expected, holding a leadership position significantly reduces the ideological distance to the party’s median legislator (p < .10). Thus, MPs at the top of the party hierarchy, holding a leading position in parliament or being member of the executive, are ideologically closer to the median party position than backbenchers. In substantive terms, the bivariate analysis indicates that holding a leadership position reduces the gap by .05 scale points, a medium-sized effect given an average gap of .14 scale points. In terms of construct validity, this final analysis thus lends some additional support to the social network approach.
Mean Comparison t-Test of Absolute Distance From Median Legislator for MPs Holding Leadership Positions and Backbenchers.
Note. Mean comparison t-test with equal variances based on Levene’s robust test statistic for the equality of variances.
*p < .10.
Discussion
The increasing proliferation of SNSs both among the general public and political elites provides scholars with increasing volumes of individual-level data on their online political behavior. While SNS data thus provide unprecedented opportunities for analyzing various political phenomena such as electoral forecasting, recent studies highlight important limitations of these big data applications in political science research (e.g., Gayo-Avello, 2013). Adding to this line of scholarly research, the present article explores the potential as well as important limitations of SNS data in indicating the policy positions of political parties and individual MPs.
At the aggregate level of political parties, the empirical evidence in terms of cross-validation with descriptive accounts and external data sources suggest that Twitter data have considerable potential to generate such estimates. The present study thus adds to the encouraging findings on the validity of the social network approach in the United States (Barberá, 2015; Conover et al., 2011; King et al., 2011) and Western Europe (Barberá, 2015; Boutet et al., 2013; Ceron & Cremonesi, 2013) and is among the first to systematically analyze SNS data in the less consolidated and institutionalized party systems of CEE. At the level of individual legislators, the empirical results are rather mixed. Specifically, the detailed analysis of defecting MPs and leading politicians holding leadership positions within their party and/or the legislature provides tentative evidence on the validity of policy position estimates retrieved via SNS data at the individual level. In contrast, I find only a moderate connection between SNS-based policy position estimates and those retrieved from MPs individual voting record.
In a similar vein, the present analysis highlights an important limitation of all existing quantitative approaches to retrieve valid policy position estimates of political parties and individual MPs. Ultimately, their actual policy positions are unobservable quantities of interest, irrespective of how elaborate the methods used by scholars to tap onto them. Consequently, any quantitative approach by definition relies on observable manifestations of these policy positions, being it election manifestos, individual voting behavior, or Twitter users’ binary decision to follow a particular politician. To the extent, however, that researchers conceptualize (individual) policy positions as a quantifiable concept, the present study highlights the potential as well as important limitations of the social network approach.
Finally, the above analysis points to several promising venues for future research. Ultimately, validating the retrieved individual position estimates requires external data at the level of individual MPs. So far, such analyses have been done in the U.S. context, where roll-call data are readily available. Several other countries in CEE lend themselves to future research in this area, as electronic roll-call vote data are available not only in the Polish Sejm but also in other national legislatures in CEE (e.g., Slovakia). The present study also encourages further scholarly research on the determinants of individual MP positions. As indicated above, the differences in the policy positions of MPs holding leadership positions and party backbenchers are in line with well-established theoretical expectations. To the extent that SNS-based methods provide valid position estimates, this allows researchers to test more elaborate theoretical arguments at the level of individual legislators.
Footnotes
Author’s Note
The author gratefully acknowledges financial support from the Austrian Science Fund (FWF) (grant P25490) for the research project on “Coalition governance in Central Eastern Europe.” I also thank two anonymous reviewers, the Special Issue Editor Homero Gil de Zúñiga, Susumu Shikano, and the participants of the seminar “Internet and Democracy” and the seminar series at the Department of Government, University of Vienna, for helpful comments and suggestions.
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This research was supported by Austrian Science Fund under Grant Number P 25490.
