An ever-closer union? Measuring the expansion and ideological content of European Union policy-making through an expert survey

Introduction

The European Union (EU) has evolved into a complex multi-level governance system where supranational, national and sub-national actors interact (Hooghe and Marks, 2001, 2003). EU institutions have accumulated an expanding portfolio of powers, with regard to the range of policy areas and the degree of involvement in policymaking. Only a limited number of studies have measured the pace of expansion of EU policy-making competences. Moreover, these rely on information derived directly from EU treaties, producing measures that only vary over relatively long periods of time (Börzel, 2005; Schakel et al., 2015; Schimmelfennig et al., 2015). Decisions on the allocation of policy competences between supranational and national institutions are, however, taken on a regular basis through secondary legislation, thus varying across shorter periods of time.

This article seeks to contribute to studies of European integration by introducing a comprehensive dataset of the changing jurisdictional allocation of authority, an important aspect of Europeanisation. Firstly, by using an expert survey of 245 policy experts in a variety of academic fields, our measure of the pace of Europeanisation provides information on the distribution of authority between national and EU institutions across time and across nine broad policy areas. Unlike treaty-based measures of Europeanisation, the new data allow for a longitudinal assessment of the pace of Europeanisation at five-year intervals since the creation of the European Economic Community.

Secondly, the expert survey also provides data on the ideological leaning of EU legislation. Previous research has produced important findings about the positioning of political parties and public opinion along the left-right and the pro-anti integration dimensions. Even though EU legislation is ideologically charged (Follesdal and Hix, 2006; Pollack, 2000), no comprehensive data are available regarding the ideological trajectory of legislative acts at the supranational level. The new dataset will allow researchers to evaluate both the role of national actors in shaping the policy direction of European integration and the impact of European integration in shaping countries by promoting certain ideological policy orientations. The expert survey includes assessments of the ideological direction of both primary and secondary EU legislation, over time and across policy areas.

Thirdly, scholars have long observed that, due to both formal and informal processes, integration’s impact varies across member states (König and Ohr, 2013). To capture some of this variation, the new dataset also provides information on cross-national variation in Europeanisation, both for ‘top down’ and ‘bottom up’ processes. The experts were accordingly asked to rate EU member states in terms of levels of Europeanisation in their area of speciality as well as to point out any consistent agenda-setters and less effective member states. This information will allow for useful cross-country comparisons of the scope and effect of EU policy-making. ¹

The contributions of the expert survey to existing scholarly research

Europeanisation scholars have long sought to understand how European integration affects the policies, politics and polities of the member states. In recent years, the field has seen a growing number of quantitative, original datasets (e.g. Alexandrova et al., 2014; Häge, 2011), which have opened the way for the analysis of broader patterns of legislation, integration and decision-making in the EU over the past six decades. In this section, we highlight three key contributions of our expert survey towards Europeanisation research. Firstly, it traces the shifting of policy competences from the national level to the EU level, providing a more nuanced picture than previous research. Secondly, the dataset measures perceptions of the ideological orientation of EU policies and legislation. Thirdly, the survey also explores individual country effects linked to ‘top-down’ and ‘bottom-up’ processes of Europeanisation.

On the first dimension – the balance of policy authority between the national and EU levels – our work ties in with previous studies tracing the gradual integration of policies over time. So far, the balance of policy authority between the EU and national levels has primarily been captured through the analysis of qualitative case studies, focusing on individual treaty effects or the effects of secondary legislation within individual policy areas (Featherstone and Radaelli, 2003; Graziano and Vink, 2007; Saurugger and Radaelli, 2008). There are some notable exceptions which have provided a historical map of policy integration in the EU. Börzel (2005) studied the Europeanisation of national policies, considering the level and scope of integration. The first refers to the breadth of integration in each policy area, in terms of whether decision-making resides at the national or supranational level. The second describes the depth of integration, defined by the type of decision-making procedures that are involved. Schimmelfennig et al. (2015) adopted, simplified and expanded this measurement, by collapsing breadth and depth into one measure of vertical integration, whilst adding a second dimension on horizontal integration (territorial extension) focusing on variation in the territorial expansion of policy integration. Schakel et al. (2015), building upon estimates by Lindberg and Scheingold (1970: 67–71) and Schmitter (1996: 125), estimated the extent of EU involvement in 28 policy domains over time based upon existing treaties, combining level and scope.

Our Europeanisation index builds on these studies but uses a different methodology for data collection and includes secondary legislation produced in years between treaties. Thus, our first major contribution is in examining the micro-variation in integration patterns in the periods between treaties, as well as comparing the impact of secondary legislation (such as regulations, directives, decisions, opinions and recommendations) to that of the treaties. This will allow us to compare our results with previous findings on the unidirectionality and differentiation of policy integration, examining variation in the time periods between treaty reforms, whilst also linking with questions about the future trajectory of the integration process.

Our second main contribution is to a growing field of research that explores ideology in EU policies. The data from our survey allow us to assess the ideological content of primary and secondary legislation over time, for both the left-right and the authoritarian-libertarian direction of policies. The literature on ideological diversity in the EU has grown in recent years. Some scholars argue that EU policy is inevitably centrist, a result of the delicate compromises involved in the policy-making process (Follesdal and Hix, 2006) or advances neoliberal or regulated capitalism interests (Pollack, 2000). Opposing this view is a growing body of literature that contends that ideological competition is present and that it has a significant effect on EU processes. These studies have argued for the importance of an ideology effect on the European institutions, mainly (and perhaps most logically) in the European Parliament (EP) (Hix, 2001; Hix et al., 2006). Other institutions, such as the Council and the Commission, are not immune from ideological tendencies either. Studies have shown that the ideological diversity of actors’ policy positions is an important factor for evaluating the speed and outcome of decision-making processes (Klüver and Sagarzazu, 2013; König and Luig, 2012).

The expert survey links with this literature by presenting the first ever quantitative assessment of the ideological content of EU policies over time. The data will enable researchers to examine whether EU policy is centrist or ideologically charged when it comes to both treaties and secondary legislation. In addition, the data can indicate whether policy is ideologically ‘locked in’ (Scharpf, 1988), since it assesses the evolution of policy over time.

Lastly, the survey data allow for comparison across countries. The literature on Europeanisation has found a high level of disparity in European integration. This differentiation is not only restricted to the extent of integration between policy areas, but also between member states, a sort of ‘internal differentiation’ within the EU (Schimmelfennig et al., 2015). National differences in terms of economic integration levels are highlighted in König and Ohr’s (2013) ‘EU Index’, which shows heterogeneity between member states and indicates an increasing clustering of the members.

To understand the differentiation between countries, one can also look at studies of the agenda-setting process in the EU. One study has found that the agenda-setting powers in the European Council are driven by political power relationships between member states rather than by the presidency (Alexandrova and Timmermans, 2013). In this regard, we contribute to the literature by identifying who are the main agenda-setters, and the prominent policy-takers or downloaders, in each area. This would allow researchers to analyse whether there are clusters of member states in terms of Europeanisation in each policy area, and whether prominence in agenda-setting also correlates with higher levels of policy-taking. A positive finding in this regard would tie in with the lowest common denominator idea of integration, showing that differentiation in integration results from different ‘appetites’ for Europeanisation among member states.

Design and methodology of the expert survey

Expert surveys are an increasingly important tool for studying European integration. They have been shown to produce sufficiently valid and reliable results and they offer greater opportunities for researchers to study topics for which there is scarce information (e.g. Hooghe et al., 2010; Marks et al., 2007; Ray, 1999; Rohrschneider and Whitefield, 2009). More specifically, our survey approach offers the key advantage of overcoming the time-invariant element of existing Europeanisation measures for the periods between treaties. EU-level decisions are taken on a more regular basis through secondary legislation. By using this method, we sought to produce a more comprehensive picture of the progression of European integration in individual policy areas.

The expert survey was completed by 245 experts in nine policy areas, corresponding to the nine configurations of the Council of the EU. Within these policy areas, experts could pick a narrower sub-area to fit their field-specific knowledge or select the broad policy area. A total of 28 narrower subfields were included. The distribution of experts per item of the broad policy areas is in line with previous expert surveys in the field. For example, the 2014 Chapel Hill survey included 337 experts and 31 countries resulting in an average of 11 experts per item (country) (Bakker et al., 2015; Polk et al., 2017). For the Europeanisation index across the nine policy areas, there is an average of 27 experts.

Fieldwork was completed over the course of 10 months through repeated email reminders every four-six weeks. Experts were selected through a broad search for published scientific outputs in each policy area. In addition to published research, we utilized the list of Jean Monnet Chairs provided by the European Commission. The Jean Monnet Chairs are competitively awarded university teaching posts with a specialisation in EU-related subjects. In total, 629 Jean Monnet Chairs were contacted with an invitation to fill in the survey. This combined strategy of identifying experts through both publication outputs and teaching responsibilities related to European integration was needed to collect sufficient responses across policy areas and to increase the representativeness of the sample.

There was variation in terms of the nationality and academic field of the experts. ² National variation was needed in order to provide a degree of control for broader contextual or ideological differences in approaches to policy-making that may exist in some fields. Figure 1 summarizes the distribution of responses per policy area. As expected, broader policy areas such as employment, social policy, health and consumer affairs have garnered the most responses. However, this policy area also had a total of six sub-areas focusing on narrower fields. Alternatively, policies such as agriculture and fisheries, or environment have only two sub-areas (corresponding to the sub-departments of Council configurations again). OPEN IN VIEWER

A small proportion of survey respondents (11% of the total) were offered a monetary incentive for completing the survey. These respondents were asked to complete a more detailed version of the survey where they assessed the degree of Europeanisation and its ideological leaning (if any) for every year between 1957 and 2014. The remainder of the sample took part in the survey on a voluntary basis and completed both temporal assessments for five-year periods between 1957 and 2014. Qualtrics software was used to set up the questionnaire and collect responses Online.

The survey questionnaire had four sections. First, a general section included control questions and broad assessments. The three main sections corresponded to the areas of interest where the survey sought to generate new data, as outlined above: a temporal assessment of the pace of Europeanisation in each policy area; a temporal assessment of the ideological leaning of EU legislation; and a cross-national comparison of Europeanisation across member states. The full questionnaire is given in the Online Appendix.

Reliability checks

While expert surveys have many advantages, they have also been criticized for having difficulty controlling for differences in experts’ perceptions, ideological preferences (Curini, 2010) and general knowledge of the subject (Gemenis and Van Ham, 2014). Both Europeanisation and ideology, in the case of our survey, are widely accepted to be broad concepts that could be perceived differently across individuals and across national contexts. This drawback, stemming from the long-acknowledged observation that ‘individuals understand the “same” question in vastly different ways’ (Brady, 1985: 271), can be mitigated through clear phrasing of questions and providing explicit definitions of key concepts. While such standard survey strategies are implicitly the goal for any researcher aiming to reduce interpersonal incomparability, or differential item functioning (DIF), as described in the social psychology literature, recent studies have made improvements towards reducing DIF and increasing survey reliability. Using certain ‘anchors’ in the survey that allow comparison of inter-person variability before asking the substantive questions of interest has produced particularly good estimators of DIF in expert surveys in political science (Bakker et al., 2014; King et al., 2004). The long-standing Chapel Hill Expert Survey on political party positions, for example, used short ‘vignettes’ in its 2010 round – brief descriptions of hypothetical political parties that experts then rated on a left–right scale. This strategy allows for researchers to compare how experts understand left–right ideological positioning independent of their ratings of country-specific parties (Bakker et al., 2014).

In a similar strategy aiming to address the problem of inter-person comparability, we included three ‘benchmark’ questions to assess the extent to which respondents follow the definitions we provided for the concepts of Europeanisation, left-right ideology and libertarian-authoritarian ideology. In the first benchmark question, experts across all policy areas were asked to assess the degree of Europeanisation (at a single time point) for the same broad policy area (trade). In the second and third benchmark questions, we presented respondents with four EU directives and asked them to rate two of them on the left-right scale and the other two on the libertarian-authoritarian scale. For both questions, we included a short description of the directives in order to facilitate ratings. In the case of the left-right scale, we asked respondents to rate the Services Directive and a GMO Directive. For the libertarian-authoritarian benchmark, we asked them to rate a directive related to the free movement of people and a directive restricting tobacco sales. The benchmark questions allow researchers to identify any outliers and take those into consideration when undertaking further analysis. ³

We calculated the average standard deviations and agreement A scores (van der Eijk, 2001) for each policy area and for the entire sample. ⁴ Lower standard deviations and higher positive agreement A scores indicate greater agreement among experts. The standard deviations are rather small signifying that our respondents are thinking of Europeanisation and ideological placements in a similar manner. Similarly, all the agreement A coefficients are positive, indicating general agreement across the board, and all but one is higher than 0.5, indicating medium to strong agreement in responses. Interestingly, both the standard deviations and the agreement A coefficients show greater consistency when it comes to ideology, and vary to a greater extent when it comes to Europeanisation – though the difference is not substantial. A possible explanation for this could be that all respondents were asked to answer the questions on Europeanisation, but in the ideology section they could choose whether to answer questions on the left-right or the libertarian-authoritarian orientations of policy. Presumably, this allowed for participants to choose the categories that they were more comfortable with, thus generating greater consistency across responses. There is also some variation across policy areas – as expected, the higher the response rate, the more reliable the final scores per policy area. Fields with comparatively fewer responses (such as agriculture) have higher standard deviations and lower agreement A scores and hence higher disagreement among experts.

It is interesting to note that, overall, the responses for the five-year period measurements show more consistency than for the treaty measurements, indicating that experts are more likely to agree on the Europeanisation and ideological orientation of secondary legislation. This is important because one of the key innovations of our study is the added focus on secondary legislation, in addition to the treaties, which have also been examined elsewhere (Börzel, 2005; Schimmelfennig et al., 2015). A longitudinal assessment of European policy at five-year intervals allows for a more nuanced assessment of how secondary legislation has changed over the history of European integration, enabling the observation of micro-variation over time.

Data overview and potential uses

The core value added of the data is through new opportunities for longitudinal research. Previous measures have given us valuable insight about the trajectory of European integration based on the treaties. Our expert survey provides a novel approach to measuring Europeanisation and expands the empirical foundations that scholars can use to study the effect of European integration on policy and politics. Figure 2 presents the evolution of Europeanisation averaged across policy areas over the entire period of European integration based on secondary legislation. It also presents the ideological content of EU secondary legislation averaged across policy areas in the left-right and libertarian-authoritarian dimensions. ⁵ OPEN IN VIEWER

Across all policies there is an increase in Europeanisation and a shift towards the right as assessed by the secondary legislation in five-year intervals, ⁶ though there is less movement on the libertarian-authoritarian dimension. ⁷

We also checked the external validity of our measure by comparing our main Europeanisation index to pre-existing measures of European integration. Our measure performs well in terms of external validity – it is significantly correlated with all the other Europeanisation measures. ⁸ As is also shown in Figure 3, the experts’ assessment of the degree of Europeanisation paints a similar picture to the pace of Europeanisation as provided by content-coded measures. ⁹ OPEN IN VIEWER

We notice some interesting patterns in Figure 3. Overall, the expert survey scores moved along a similar trajectory as the content-coded measures. However, there is a more notable divergence in the post-2000 period where content-coded measures tend to consistently rate Europeanisation at higher levels than expert evaluations. The largest divergence between the ‘end points’ of Europeanisation (the last year we have data for) is between our index and the scores from Schmitter, which were published in 1996 and are partially based on projections about the expected outcomes of the Maastricht Treaty. Thus, one could argue that the expected pace of Europeanisation did not match the actual outcomes in the more politicized climate post-2000. The differences in ‘end-point’ scores between the expert evaluations and the more recently updated measures, such as the Schakel et al. (2015) and Schimmelfennig et al.’s (2015) Europeanisation indices, are much smaller. Finally, it is also possible that expert evaluations are more susceptible to thinking about informal processes in their policy area. The discrepancy in present-day levels of Europeanisation between survey-based and content-coded measures may be partially due to experts being more likely to think about political processes that restrain Europeanisation in their area even in the face of treaty legislation.

In terms of potential uses, the new index can contribute to a variety of salient research questions. What are the factors explaining variation in the pace of Europeanisation of policy over time? How do EU-level versus national-level political factors affect this variation? Alternatively, how does variation in the Europeanisation of policy affect political strategies, institutional arrangements, and MEP and MP behaviour? How does the Europeanisation of policy affect representation, such as the responsiveness of parties to citizens? These are some of the potential questions that can be studied in greater detail by using the expert survey data on Europeanisation.

The ideology scores can be used to advance a variety of research agendas. Combined with external sources, the ideology scores would be very useful for studies of representation at the EU level – comparing citizen preferences with legislative outputs in a certain area can shed light on the presence or absence of a ‘democratic deficit’ and unresponsiveness on the part of legislators. Additionally, the ideology scores can be combined with EP data on party groups, roll call votes and/or legislators’ preferences for an assessment of how variation in EP characteristics affects legislative outputs.

The index also provides valuable data on member states. Figure 4 shows the relationship between the level of Europeanisation and agenda-setting influence. Germany leads in both respects. Conversely, the UK displays relatively low levels of Europeanisation, but has been one of the main agenda setters. The dataset will enable comparative analyses of country effects linked to ‘top-down’ and ‘bottom-up’ processes of Europeanisation. OPEN IN VIEWER

Figure 5 shows how the ideology scores can point scholars towards exploring important ideological patterns. We used data from the Eurobarometer and European Social Survey to calculate the left-right position of mean voters per country per year. Higher values indicate movement to the right. The first three graphs in Figure 5 illustrate how the mean voter position compares to the mean left-right scores of EU legislation. The first graph shows mean EU legislation ideology scores averaged across all policy areas, while the second and third graphs focus on economic and social policy, respectively. We see that the mean voter position has shown much less variation overtime, while EU ideology, as assessed by the experts, has moved largely to the right, especially in the case of economic policy. On economic policy, we also see the largest distance between the mean voter position and the EU’s ideological leaning. The distance is very small in the early years of EU integration, but progressively widens from 0.5 to 2.3. ¹⁰ The ideological gap between citizens and EU economic policy shows no indication of subsiding and presents both scholars and policy-makers with a concerning puzzle that needs to be further studied and addressed by policy-makers. In comparison, there is a much smaller ideological gap between the mean voter position and the mean EU legislation scores on social policy over time. However, starting in the late 2000s this gap seems to be widening as well. Likely coinciding with the onset of the financial crisis, the discrepancy has grown wider, but driven primarily by a clear shift to the right in EU legislation, while the mean voter position has remained fairly consistent. OPEN IN VIEWER

Conclusion

We have presented a new dataset that can contribute to future research on European integration. The methodology and findings were described. The results section illustrated some general findings and suggested how the survey can be used to examine a variety of research questions. The descriptive findings suggested that European integration is perceived as having progressed in a broadly upwards direction across policy areas. Differences in the degree of Europeanisation across policies were reported based on the experts’ evaluations.

As with all survey-based measures, we acknowledge that the data are rooted in perceptions and susceptible to the usual biases present in survey research. In order to give readers more confidence in the reliability and validity of the data, we compared our scores with pre-existing content-coded measures of Europeanisation and conducted a number of reliability checks to ensure sufficient agreement and consistency across expert evaluations. The expert survey can be a useful empirical resource for scholars of European integration and its effect on policies, politics and citizens in the member states.