Peeping at the corpus – What is really going on behind the equality and welfare items of the Manifesto project?

Abstract

The Comparative Manifestos Project (CMP) data set quantifies how much parties emphasize certain topics and positions and is very popular in the study of political parties. The data set is also increasingly applied in comparative political economy and welfare state studies that use the welfare-specific items rather than the CMP’s left–right scale to test hypotheses on the impact of political parties on social policies, (in)equality and the welfare state. But do these items provide a valid basis for descriptive and causal inferences? What do the items precisely capture? To answer these questions on concept validity, we use the new manifesto corpus data for German parties 2002–2013 and, to provide a further test, for US parties 2004–2012. Corpus data are the digitalized, originally hand-annotated and coded texts of electoral programmes. We assess the validity of the codings directly at the level of quasi-sentences by re-categorizing and subcategorizing the originally coded statements on equality, social justice and welfare state expansion. Although concept validity concerns about the data seem exaggerated, we find that theoretically relevant and meaningful variation is ‘hidden’ behind the original categories. Hence, our approach allows researchers to assess the substantive meaning of the CMP data directly, and we offer an efficient new strategy for testing more specific hypotheses on the impact of political parties on policy.

Keywords

Comparative welfare state research manifesto corpus manifesto data text analysis validity

Introduction

The Comparative Manifestos Project data set, widely known as CMP (now MARPOR (Manifesto Research on Political Representation)), quantifies parties’ emphasis on certain topics and positions (Budge et al., 2001; Klingemann et al., 2006). It is among the most popular data sets for the study of political parties, but its construct validity, broadly defined as ‘the faithfulness of a research design to the theory’ (Gerring, 2012: 95), is often questioned (see Budge, 2013 for a summary and discussion of criticisms). Against the background of mounting criticisms of the use of party labels (Döring and Schwander, 2015), increasingly sophisticated applications of the data permeate substantive research in political economy and comparative welfare state studies (e.g. Finseraas and Vernby, 2011; Gingrich and Häusermann, 2015; Horn and Jensen, 2016; Jensen and Seeberg, 2015; Nygård, 2006; Schumacher, 2014). This research uses welfare-specific items (503, equality: positive; 504, welfare state expansion) rather than the left–right scale of the CMP or left–centre–right labels to test hypotheses on the impact of political parties on social policies and the welfare state. But do these items provide a valid basis for descriptive and causal inferences? What do the items actually capture? These are questions that concern the more specific issue of construct validity that is called concept validity: ‘the degree to which a concept, as defined, matches up with a set of empirical indicators (its operationalization)’ (Gerring, 2012: 442). In other words, we want to know whether the welfare-specific items really pick up what the researchers are theoretically after, namely, the extent to which parties are in favour of (further) welfare state expansion and (more) equality.

On the basis of the CMP corpus data – digitalized, originally hand-annotated and coded texts of electoral programmes (Lehmann et al., 2015), we assess concept validity at the level of quasi-sentences by re-categorizing and subcategorizing the original codings.¹ A quasi-sentence is the semantic unit the Manifesto team used for unitization and equals one statement or message, so a natural sentence can contain several quasi-sentences (see Appendices 1 and 2 for examples). The Manifesto coders then grouped each of these statements in one of 56 thematic categories. Because of data availability and because they are among the most complex party manifestos (and thus a tough test for the concept validity of the codings), we started by looking at German parties’ manifestos and their emphasis on welfare and equality in elections from 2002 to 2013. Both items have often been used by welfare state researchers and are among the most emphasized aspects in party manifestos. To see whether and to what extent our findings also hold in a very different welfare state- and party-system context, we repeated the analysis for the Democrats and the Republicans in the United States for the period 2004–2012.

We make three contributions:

Using subcategories, we show that the data are suitable for fine-grained analyses of welfare state and inequality issues. Miscodings are sparse, and codings are mostly in line with item descriptions. Yet, the results are also ambivalent as we demonstrate that standard codings miss theoretically crucial and systematic ideological variation. We make researchers aware of these hidden data characteristics and emphasize the great potential of subcategorization that the new corpus data offer. For instance, the three lines of research highlighted in Häusermann et al.’s (2013) authoritative review on the recent literature on party politics and the welfare state could benefit from subcategorization. The first line of research focuses on the electoral consequences of post-industrialization and how this affects parties’ positions on welfare policy. With our approach, it is possible to use subcategories of the welfare state items in the CMP corpus data to trace over time whether, how and to what extent parties have adjusted their policy message to this new environment. Similarly, the second strand of research, which focuses on how changing party competition affects welfare policy, could benefit from looking more closely at which dimensions of welfare state politics parties decide to compete on. The third strand of research, which criticizes traditional programmatic voter–party linkages in partisan politics, could use the new information available to test whether and to what extent even parties in developed democracies use social policy to construct particularistic or clientelistic appeals to specific voter groups.

Scholars can now address questions that are too specific to be addressed by the broad CMP categories. While the meta-discourse on the CMP data set has become a cottage industry of its own, applied researchers generally interested in construct validity or more specifically in concept validity of (particular) items can so far only draw indirect conclusions. Most articles focus on the best procedure for the construction of left–right indices, the comparability of scales over time and space and the sampling of the party documents that have been analysed (Benoit et al., 2009; Dinas and Gemenis, 2010; Franzmann and Kaiser, 2006; Jahn, 2011; König et al., 2013; Meyer and Jenny, 2013). Verdicts on validity are based on the discussion of the ‘face validity’ of party positions on the left–right scale in specific countries (Budge, 2001; Dinas and Gemenis, 2010; Pelizzo, 2003), intriguing, yet unsystematic examples of miscodings (Zulianello, 2013), or (coder) reliability tests (Mikhaylov et al., 2012). While these are important discussions, we complement these efforts by focusing on the concept validity at the level of individual codings and by demonstrating what is really at the heart of the items that is theoretically relevant for welfare state research.

We decipher systematic variations between German (and American) parties in the way they talk about the welfare state, equality and social justice. Some are not surprising but still theoretically important, for instance, the shift in emphasis from equality of outcomes to equality of opportunity along the left–right continuum. Other variations raise new questions or present old issues in a new light. Why do parties show less status quo bias and more future welfare expansionary orientation than expected by theory? What kinds of equality are parties referring to? This illustrates a final benefit of analysing the corpus data: re-categorizing and subcategorizing the data potentially generates interesting new puzzles.

What can we learn from ‘peeping at the corpus’?

The Manifesto data are both heavily used and heavily criticized in political science (Budge, 2013). However, their application in the welfare state and political economy research is relatively new. Here, the question of concept validity is pressing given the highly specific meaning assigned to two commonly used items: ‘Welfare State Expansion’ and ‘Equality: Positive/Social Justice’. While some studies focus on one of the items (e.g. Gingrich and Häusermann, 2015 use ‘Welfare State Expansion’ to quantify ‘high levels of welfare state support’), both items are more often used together as (part of) a measure for parties’ welfare state positions (Nygård, 2006), left–right shifts on welfare issues (Schumacher, 2014: 71) or welfare state emphasis (Jensen and Seeberg, 2015: 7). Tellingly, both items are also summarized in the Manifesto data set as ‘welfare’ (for instance in the version ‘MARPOR data 2014b’). Combined, both items take up on average 11.1 percent of parties’ post-war election manifestos.

How valid, however, is the operationalization of the welfare concept of interest? Poor validity would clearly question the descriptive and causal inferences based on these items. Those who use the right–left index ‘rile’ and are uncertain about its validity reassure themselves that most miscodings end up on the right side of the left–right dimension (but see Mikhaylov et al., 2012). However, users of the issue-specific items resort to subjective ‘face validity’ or cross-validation via other data sets, such as Jensen and Seeberg (2015), who argue that the scores they use are valid when ‘put against monthly data from the Danish parliament’ (p. 7). Re-categorizing and subcategorizing the corpus data allows us to reduce uncertainty about what the welfare and equality items really measure. We also see this as a chance to test if and how the subcategorization of the CMP categories could allow for questions for which the original categories were too broad. Whereas the original data capture what parties talk about, subcategories based on the corpus data provide an efficient way to find out how they do that. For instance, in the context of welfare research, it is of critical theoretical importance to distinguish whether parties merely want to maintain the status quo, criticize current deficiencies or call for further future expansion of the welfare state. Does the welfare state expansion item capture party programmatic statements that demand expansion? Or is a significant part of the pro-welfare state assertions simply calling for a defence of the status quo? In other words, including the extent to which parties wish to maintain existing arrangements (see the coding instruction in Box 1) would be valid if one aims to measure to what extent parties are generally in favour of the welfare state. However, if one wishes to gauge whether parties wish to expand the welfare state, caution is warranted and researchers would perhaps be on more certain ground if they use the more detailed information that actually exclusively indicates expansion. Anticipating our results below, we find that the welfare state expansion predominantly measures expansion. With respect to the equality item, distinguishing which concept of equality parties actually emphasize would be theoretically more instructive than just reporting how much parties talk about equality. Do they conceive of (in)equality in economic terms (redistribution and equality of outcomes), or do they emphasize chances for everyone, social inclusion and non-discrimination (equality of opportunity)?

Box 1.

The CMP welfare items.

503 (equality: positive)
Concept of social justice and the need for fair treatment of all people. This may include the following:
• Special protection for underprivileged social groups;
• Removal of class barriers;
• Need for fair distribution of resources;
• The end of discrimination (e.g. racial or sexual discrimination).
504 (welfare state expansion)
Favourable mentions of need to introduce, maintain or expand any public social service or social security scheme. This includes, for example, government funding of the following:
• Healthcare;
• Childcare;
• Elder care and pensions;
• Social housing.

CMP: Comparative Manifestos Project.

Our assessment of what the welfare and equality items of the CMP data really measure should also speak to the general debate about the properties of the data. Until now, we have only been able to assess indirectly whether the items that the ‘rile’ is composed of reflect the item descriptions and whether they can be used in indices or stand alone. Our procedure offers a direct approach.

The validity of the data can be induced from the alleged plausibility of party rankings on left–right indices, but conclusions regarding the data’s ‘face validity’ differ (positive: Budge, 2001 for the United States; sceptical: Dinas and Gemenis, 2010; Pelizzo, 2003 for Italy and Greece). In addition, there are exemplary discussions of problematic codings, for instance, as a consequence of the assignment of each quasi-sentence to only one code (Zulianello, 2013). However, the method of singling out and grouping some problematic codings does not provide guidance for testing of systematic problems of concept validity. In addition, there is the issue of (coder) reliability, on which Klingemann et al. (2006) reach favourable conclusions (pp. 86–103), whereas Mikhaylov et al.’s (2012) 20-item coding and re-coding experiment concludes that conventional levels of reliability are not achieved and that this also affects the left–right scale. They caution against using the data and call for a ‘much simplified coding scheme’ (p. 90). However, potential problems with the empirical basis for their sceptical verdict are not discussed. Should researchers interested in the meaningfulness of the data take this as evidence of the procedural mistakes in their instruction of coders, as an idiosyncratic product of the specific manifestos assessed or as proof that one well-trained and politically (more) literate coder can outperform several less suited ones?

The coder characteristics strike us as especially important in the coding of items such as ‘Welfare State Expansion’ and ‘Equality: Positive’. As we illustrate below, it is unrealistic to expect a meaningful coding of these items without awareness of the political context and the status quo. Surely, some would want to use them despite low reliability as long as the error is stable within items and across parties. But this only renders the question of what is really going on behind the CMP (welfare) measures all the more pressing. We provide expert codings to answer this question.

We focus on the five major German parties over the course of four elections, which means we look at 20 manifestos. The German case represents a tough test for the substantive meaningfulness of the codings because the manifestos are very complex in terms of length, substance and language (for instance, when compared to simpler Scandinavian party manifestos). Moreover, there is no other country for which we have data on four consecutive elections (the Manifesto team has kindly provided us with the corpus data ahead of publication). Re-categorizing and subcategorizing the quasi-sentences of German manifestos is also per se interesting, for instance, because it allows us to trace how parties have reacted to the entry of a consolidated party left of the Social Democratic Party of Germany (SPD) since the 2005 election when an SPD secession party merged with the thus far marginalized socialist party, Party of Democratic Socialism (PDS). Yet, to see whether our evaluations hold in a very different welfare state- and party-system context, we also include results for the (less comprehensive) corpus data that are available for the two major US parties.

Subcategories

The two items most relevant for comparative welfare state and political economy research are 503 (equality: positive) and 504 (welfare state expansion) (see Box 1).

One important question that we wished to explore was whether political parties that devote comparable proportions of their manifestos to the welfare state items are actually talking about the same things. Do they, for instance, stress income redistribution and equality of outcome rather than equality of opportunity or anti-discrimination? As can be seen in Box 1, various connotations of equality are grouped under ‘Equality: Positive’. To capture theoretically relevant variations within the ‘Equality’ item, we distinguish between an economic core (statements concerning the income and wealth distribution, for example, in favour of more progressive taxation), general mentions (statements without a specific connotation, for example, in favour of social justice), chances/social mobility (statements in favour of equal opportunity), anti-discrimination/inclusion and other specific causes (party-specific statements, for example, in favour of ‘climate justice’ for the Greens).

With respect to the ‘Welfare State Expansion’ item, we were inspired by debates over welfare state reform between the ‘power resources’ and the ‘new politics’ schools. As Paul Pierson (1994, 2001) and the researchers inspired by him (for an overview, see Van Kersbergen and Vis, 2014) argued, parties are likely to make different types of positive statements about the welfare state, some praising the status quo rather than demanding expansion, others criticizing the inadequate status quo, implying or explicitly requiring that action should be taken. In addition, we expected that some parties might voice negative expectations about societal developments that necessitate welfare state adaptation in the future (e.g. due to demographic ageing), for example, in an attempt to preclude policy drift. To capture such differences, we distinguished between status quo plus/minus and future plus/minus.

We first checked whether a quasi-sentence was classified in the correct item. If not, it ended up in our ‘miscoded’ subcategory. We only did so when we were convinced that the statements were misplaced (e.g. statements in favour of or against law and order measures or concerning the exodus from rural areas and its consequences). The correctly coded statements were then placed in one of the remaining subcategories of that item. Finally, we emphasize that two of us (social policy experts and native and quasi-native German speakers), not student assistants, did the coding. All alleged miscodings were discussed among the two coders, as were any ambivalent cases. A description and examples from Germany and the United States for each of the categories can be found in Appendices 1 and 2.

Equality

Figures 1 and 2 summarize what German parties talk about when they talk about equality in a positive manner. Figure 1 shows the distribution of the subcategories within the equality item for the five parties averaged over the four elections. Figure 2 tracks how the subcategories as a percentage share of the total manifestos have changed between 2002 and 2013. So while Figure 1 shows what parties talk about when they talk about equality, Figure 2 helps to put this into perspective and indicates changes in the share of our subcategories relative to the length of the entire manifestos. In both graphs, parties are sorted from left to right according to their CMP left–right scores (2013).

Figure 1.

The distribution of subcategories within the equality item for five German parties, averaged over elections, 2002–2013.

Figure 2.

Equality subcategories as a percentage share of total manifestos, 2002–2013.

On average, 33.6 percent of the statements emphasize economic equality, whereas 5.1 percent are general statements in favour of (social) equality. Emphasis on inclusion and anti-discrimination is the second largest subcategory with 24.3 percent. Talk about social mobility and chances for everyone makes up 10.4 percent of the manifestos. Other or specific mentions of equality occupy 11.7 percent. On average, 14.8 percent of the statements originally coded as item 503 are, in fact, unrelated to matters of equality, social justice or fairness. The distribution of subcategories across parties in Figure 1 indicates a clear left–right gradient. The further we move from the left to the right of the political spectrum, the more the balance between equality of outcome and equality of opportunity shifts towards the latter. The parties of the traditional Left – Die Linke, the SPD – emphasize economic equality most (SPD: 40%, Linke: 45.6%). The centre–right parties focus on anti-discrimination and call for greater inclusion and neglect economic equality (Union: 18.1%, Free Democratic Party (FDP): 7.8%). The new left party of the Greens is ambivalent. The overall scores suggest that they are more egalitarian than the SPD in economic terms, but most of these statements focus on inclusion and anti-discrimination (32.3%), as well as social mobility and equal chances (11.3%). Many of these quasi-sentences are devoted to non-discrimination of women and sexual and ethnic minorities. Only 27 percent of the quasi-sentences coded as ‘Equality: Positive’ in Green manifestos actually relate to economic equality or inequality.

Figure 2 shows that these differences are generally consistent throughout elections, but there are some important qualifications. For instance, the Greens’ emphasis on economic equality has declined persistently since the 2005 election. Another interesting fact is that the SPD and the Greens had exceptionally limited focus on economic equality compared to the other subcategories at the end of their first red–green government in 2002. Finally, Figure 1 suggests that the SPD puts most emphasis on promoting itself as the party of social justice (7.3%), and Figure 2 shows that it has done so especially when the focus on economic equality has been weaker than usual (2002, 2013).

Figure 1 shows that specific mentions of equality (‘other’) are more pronounced at the ideological fringes of the party spectrum (Linke: 15.2%, FDP: 15.1%). For Die Linke, statements in that category are disproportionately devoted to calls for better living conditions in the East/former German Democratic Republic (GDR). The FDP marches under the banner of generational justice for reform of the welfare state. The Greens explicitly emphasize ‘energy justice’, ‘climate justice’ and ‘internet justice’. We could not identify clear disproportionalities and unifying themes for the SPD and the Christian Democratic Union of Germany/Christian Social Union in Bavaria (CDU/CSU).

Generally, the reported 14.8 percent miscoded 503 sentences also include statements that actually belong in 504. Given our interest in the concept validity of commonly used ‘welfare indices’, our figure thus exaggerates the error. More consequential than the vague boundary between the equality and the welfare item is the distribution of miscodings across parties. The shares of miscodings are twice as high for the centre–right parties, Union and FDP, than for left parties. This is partly driven by the high number of miscodings for the Union in 2009, which is mostly the result of statements that should have been classified as welfare expansion. Figure 2 puts this discrepancy in perspective. The share of miscodings relative to the total number of statements is stable across parties (1.1% for FDP and Union, 1.2% on average), implying a baseline of miscodings independent of item frequencies.

Welfare state expansion

Figures 3 and 4 summarize what parties talk about when they talk about welfare state expansion. Figure 3 shows the distribution of the subcategories within the welfare state item for the five parties, averaged over elections. Figure 4 tracks how the subcategories as a percentage share of the entire manifestos have changed between 2002 and 2013. Again, the parties are sorted from left to right according to their CMP left–right scores (2013).

Figure 3.

The distribution of subcategories within the welfare state expansion item for five German parties, averaged over elections, 2002–2013.

Figure 4.

Welfare state expansion subcategories as a percentage share of total manifestos, 2002–2013.

Figure 3 shows that the 504 item for Germany actually almost exclusively measures statements in favour of welfare expansion. On average, for the four elections and five parties, the great majority of positive statements about the welfare state (76.1%) are indeed calls for expansion. We expected favourable mentions of the status quo to be much more prominent, but they only take up 7.7 percent on average. The negative assessment of the status quo is in fact more frequent in the manifestos (11.7%). However, this is primarily driven by Die Linke, a party that, on average, devotes no less than 20 percent of its statements on the welfare state to a negative description of the status quo to contextualize and underline its demands for future expansion. Die Linke tends to explicate the negative consequences of past governments’ welfare state reforms and the social costs of government inactivity in the face of pressing social problems before publishing its proposed expansionary remedy in a separate group of (quasi-)sentences. The future minus subcategory we distinguished on theoretical grounds does not seem to pick up much. We found only very few statements that belonged in this subcategory (1 for Germany, 10 for the United States), and their interpretation is less straightforward than for the other subcategories. This probably indicates that electoral manifestos are not a platform for more broad reflections on long-term scenarios.

We see some interesting differences over time. There is a pronounced status quo emphasis in the manifestos of FDP and Union, which made up the government coalition until 2013, and of the SPD–Green government in 2005. Still, the parties generally pay much more attention to future expansion.

Again, we find that the miscodings are on average somewhat tilted to the right. Some of the miscodings that we found in the FDP programmes probably came about because the coder misunderstood a retrenchment proposal as expansion because it was formulated as an introduction of a new type of benefit (e.g. the Bürgergeld-System in the 2002 manifesto). Similarly, some privatization proposals and policies to increase the role of the free market (e.g. in healthcare) were read as welfare state expansion. Note, however, that the overall share of miscodings is relatively modest (4.5%) and that it varies across the four elections.

Do the results hold for Democrats and Republicans in the United States?

Our results indicate that the welfare item and the equality item of the Manifesto project measure what they are supposed to, despite important and theoretically interesting variations underneath the original categories. To explore to which extent these conclusions depend on the German case, we also looked at the United States. Its welfare state context and party (system) characteristics differ markedly from the German case and US manifestos are not as detailed and complicated as German manifestos. Again, we provide examples for each of our subcategories in Appendices 1 and 2. We focus on the two major parties, Republicans and Democrats. Corpus data for both are available for the elections 2004, 2008 and 2012. Generally speaking, if we look at the distribution for the equality item in Figure 5 and the distribution of the welfare expansion item in Figure 6, we find reconfirmation that only a minority of statements (14%) are miscoded and that even in the US plans for future welfare expansion make up by far the biggest subcategory of the expansion item, as its name suggests. Again, the share of miscodings is higher on the Right.

Figure 5.

The distribution of subcategories within the equality item for US parties, averaged over elections, 2004–2012.

Figure 6.

The distribution of subcategories within the welfare state expansion item for US parties, averaged over elections, 2004–2012.

Looking at Figure 5, we clearly see that the Democrats, when talking about equality, devote more attention than the Republicans to economic equality and social mobility. The striking similarity between the parties is that ‘inclusion’ is the largest subcategory: both parties devote much attention to racial and other forms of discrimination and the policies to counter discrimination. The share of miscodings is much higher for the Republicans (24%) than for the Democrats (4%). Many of these misclassified quasi-sentences related to the rule of law or to physical security rather than equality. All in all, the exercise shows that also in the United States, the equality category picks up what it should, that there is a left–right difference between the parties in how important economic equality is, and that the specific American context of (race) discrimination is reflected in the programmes of both parties.

The main finding from Figure 6 on the welfare state expansion item is also that it clearly picks up the extent to which the parties promote the welfare state. Although we find a higher share of positive statements on the status quo relative to all welfare expansion statements in the United States than in Germany, we also see that most positive statements, like in Germany, actually call for welfare state expansion. Another finding, which cannot be read from the figures, is that in both 503 and 504 a large share of quasi-sentences is devoted to healthcare, which comes to the fore as one of the most contentious issues in American politics. The final thing to note, as in the German case, is that several quasi-sentences belonging to 503 ended up in 504 and vice versa, recoded by us as miscoded. But this should not affect the concept validity of the combined welfare state indices, as discussed above.

Conclusion: what have we learned from ‘peeping at the corpus’?

The (meta-)debates over the validity of the CMP items have been inconclusive because validity tests have necessarily been indirect and, hence, imperfect. Now that the CMP corpus becomes available, we can make real progress. We show how we can test directly, by re-categorizing and subcategorizing items, whether frequently used items to operationalize welfare state concepts are indeed valid in the sense that researchers using them claim them to be. We specifically looked at the welfare and equality items relevant for political economy and comparative welfare state studies of social policy and redistribution. Our main finding is encouraging for researchers who seek to apply or have applied these issue-specific items. The items do measure what they are supposed to measure: emphasis on equality and welfare state expansion. We also find that under the general label ‘equality’, parties offer different conceptions of equality. When parties write positively about the welfare state, they mostly call for expansion. We find few miscodings and little systematic error. Therefore, our main conclusion is that if we assume that items 503 and 504 do not behave radically differently in other countries, they can be used as valid indicators of parties’ positions on the welfare state issue. However, we also found that the 2009 German election marks an exception because much of what should have been coded as 504 was actually coded as 503. It is because these two items are usually combined in a welfare state index that such inconsistencies become irrelevant. The hidden variation in the kind of equality that parties emphasize, both in Germany and in the United States, could be of critical importance for current debates in the field. This article is an example of how re-categorizing and subcategorizing the corpus data can help researchers arrive at theoretically more meaningful indicators and results while still benefiting from the CMP data.

Let us finish with some qualifications, questions and points for further discussion. First, a limitation. We have not checked whether the coders have been exhaustive, that is, whether all quasi-sentences have been coded in the correct equality and welfare items; we have only checked whether the quasi-sentences that were placed in the two items were categorized correctly. Second, coders sometimes need considerable knowledge of the context to understand whether proposed measures must be categorized as pro-welfare or pro-equality or not. The proposal to introduce Bürgergeld can easily be misread as the introduction of a new benefit and, hence, as welfare state expansion, but in reality, it implied a harsh retrenchment. Third, while some CMP critics have called for a ‘much simplified coding scheme that would facilitate more reliable classification’ (Mikhaylov et al., 2012: 90), we point to subcategorization as an alternative. Our re-coding exercise shows that, especially for the German case, such simplification would be unrealistic because the quasi-sentences are complex and it takes an understanding of the context to categorize them correctly. From the perspective of users who are interested in what parties stand for in substantive terms, a simplification would further reduce the usefulness of the (already) broad categories. For those users, especially in the field of welfare research, our findings should be encouraging. We hope that the newly available corpus data will empower and stimulate them to work with more meaningful and detailed subcategories.

Footnotes

Appendix

Appendix 2

Re-categorization/subcategorization of the item ‘Welfare State Expansion’.

Subcategories, item 504	Description	Example (English translation in parentheses)
Status quo +	Praise of the status quo/what has been achieved	Unsere sozialen Sicherungssysteme stehen wieder auf festen Beinen. (FDP 2013) (Our social security systems are on solid ground again.) We reject the notion of the presumptive Republican nominee that Social Security is a disgrace; we believe that it is indispensable. (Democrats 2008)
Status quo −	Criticisms of the status quo, pointing to problems	Sie [die Rentenreform 2001] erhöht das Armutsrisiko im Alter vor allem für Niedrigverdienende, Teilzeitbeschäftigte, Arbeitslose und Versicherte mit Lücken in der Erwerbsbiografie. (PDS 2002) (It [the pension reform of 2001] increases old-age poverty risks, in particular for low earners, part time workers, the jobless and those with gaps in their occupational history.) [Families] should not be forced to bear the burden of skyrocketing premiums, unaffordable deductibles or benefit limits that leave them at financial risk when they become sick. (Democrats 2008)
Future +	Calls for expansion of programmes in the future	den Heizkostenzuschuss beim Wohngeld wieder einführen (SPD 2013) (reintroduce the heating subsidy for the housing benefit) We will strengthen Medicaid. (Democrats 2012)
Future −	Negative expectations make adaption necessary	Die Einnahmen werden damit [der demographischen Entwicklung] nicht Schritt halten können. (FDP 2002) (The revenues will thus not (be able to) keep pace with demographic developments/demographic ageing.) By the time young men and women who are entering the workforce today turn 65, there will be only two workers for each beneficiary. Doing nothing is not an option. (Republicans 2004)
Miscoded	Not related to the description of item 504	Das Familiengeld wird grundsätzlich nur für Kinder bezahlt werden, die in Deutschland leben. (CDU/CSU 2002) (Family allowance will, as a matter of principle, only be paid for children who live in Germany.) Die FDP tritt für mehr Markt und Wettbewerb in der Kinderbetreuung ein. (FDP 2002) (The FDP wants more market and competition in childcare.) The nation’s capital is a special responsibility of the federal government. (Republicans 2008)

Acknowledgements

Previous versions of this paper were presented at the Manifesto Data User Conference 2015 at WZB (Berlin) and at a workshop at the University of Southern Denmark, 2015. We thank the participants as well as the two anonymous reviewers for their helpful comments.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We have received generous financial support from Aarhus University Research Foundation’s AU IDEAS Programme.

Notes

References

Benoit

Laver

Mikhaylov

(2009) ‘Treating Words as Data with Error: Uncertainty in Text Statements of Policy Positions’, American Journal of Political Science 53(2): 495–513.

Budge

(2001) ‘Validating Party Policy Placements’, British Journal of Political Science 31(1): 210–23.

Budge

(2013) ‘The “Paradox of the Manifestos”: Satisfied Users, Critical Methodologists’, available at https://manifesto-project.wzb.eu/down/papers/budge_paradox.pdf.

Budge

Klingemann

H.-D.

Volkens

Bara

Tanenbaum

(2001) Mapping Policy Preferences: Estimates for Parties, Electors, and Governments 1945–1998. Oxford: Oxford University Press.

Dinas

Gemenis

(2010) ‘Measuring Parties’ Ideological Positions with Manifesto Data: A Critical Evaluation of the Competing Methods’, Party Politics 16(4): 427–50.

Döring

Schwander

(2015) ‘Revisiting the Left Cabinet Share: How to Measure the Partisan Profile of Governments in Welfare State Research’, Journal of European Social Policy 25(2): 175–93.

Finseraas

Vernby

(2011) ‘What Parties Are and What Parties Do: Partisanship and Welfare State Reform in an Era of Austerity’, Socio-Economic Review 9(4): 613–38.

Franzmann

Kaiser

(2006) ‘Locating Political Parties in Policy Space: A Reanalysis of Party Manifesto Data’, Party Politics 12(2): 163–88.

Gerring

(2012) Social Science Methodology: A Unified Framework, 2nd edn. Cambridge: Cambridge University Press.

10.

Gingrich

Häusermann

(2015) ‘The Decline of the Working-class Vote, the Reconfiguration of the Welfare Support Coalition and Consequences for the Welfare State’, Journal of European Social Policy 25(1): 50–75.

11.

Häusermann

Picot

Geering

(2013) ‘Review Article: Rethinking Party Politics and the Welfare State. Recent Advances in the Literature’, British Journal of Political Science 43(1): 221–40.

12.

Horn

Jensen

(2016) ‘When and Why Politicians Do Not Keep Their Welfare Promises’, European Journal of Political Research. Epub ahead of print 29 September. DOI: 10.1111/1475-6765.12173.

13.

Jahn

(2011) ‘Conceptualizing Left and Right in Comparative Politics: Towards a Deductive Approach’, Party Politics 17(6): 745–65.

14.

Jensen

Seeberg

H.B.

(2015) ‘The Power of Talk and the Welfare State: Evidence from 23 Countries on an Asymmetric Opposition–Government Response Mechanism’, Socio-Economic Review 13(2): 215–33.

15.

Klingemann

H.-D.

Budge

Volkens

Bara

McDonald

M.D.

(2006) Mapping Policy Preferences II: Estimates for Parties, Electors, and Governments in Eastern Europe, European Union, and OECD 1990–2003. Oxford: Oxford University Press.

16.

König

Marbach

Osnabrugge

(2013) ‘Estimating Party Positions across Countries and Time: A Dynamic Latent Variable Model for Manifesto Data’, Political Analysis 21(4): 468–91.

17.

Lehmann

Matthieß

Merz

Regel

Werner

(2015) Manifesto Corpus. Berlin: WZB Berlin Social Science Center.

18.

Meyer

T.M.

Jenny

(2013) ‘Measuring Error for Adjacent Policy Position Estimates: Dealing with Uncertainty Using CMP Data’, Electoral Studies 32(1): 174–85.

19.

Mikhaylov

Laver

Benoit

K.R.

(2012) ‘Coder Reliability and Misclassification in the Human Coding of Party Manifestos’, Political Analysis 20(1): 78–91.

20.

Nygård

(2006) ‘Welfare–Ideological Change in Scandinavia: A Comparative Analysis of Partisan Welfare State Positions in Four Nordic Countries, 1970–2003’, Scandinavian Political Studies 29(4): 356–85.

21.

Pelizzo

(2003) ‘Party Positions or Party Direction? An Analysis of Party Manifesto Data’, West European Politics 26(2): 67–89.

22.

Pierson

(1994) Dismantling the Welfare State? Reagan, Thatcher and the Politics of Retrenchment. Cambridge: Cambridge University Press.

23.

Pierson

(ed.) (2001) The New Politics of the Welfare State. Oxford: Oxford University Press.

24.

Schumacher

(2014) ‘When Does the Left Do the Right Thing? A Study of Party Position Change on Welfare Policies’, Party Politics 21(1): 68–79.

25.

Van Kersbergen

Vis

. (2014) Comparative Welfare State Politics: Development, Opportunities, and Reform. Cambridge: Cambridge University Press.

26.

Zulianello

(2013) ‘Analyzing Party Competition through the Comparative Manifesto Data: Some Theoretical and Methodological Considerations’, Quality & Quantity 48(3): 1723–37.