What does the general national pride item measure? Insights from web probing

Abstract

The general national pride question is a popular item in cross-national studies of national identity. Although it is a single-item indicator, researchers use it as a proxy for different aspects of national identity, such as national belonging, different forms of patriotism, and nationalism. This article evaluates the suitability of this item as a cross-national indicator for these aspects of national identity. It assesses which aspects of national identity respondents have in mind when providing a response to this item, whether these associations differ across countries, and how well the item works as a measure for the different aspects of national identity. By means of web probing results from a web survey conducted in five countries (Germany, Great Britain, Mexico, Spain, and the United States), we reveal that the general national pride is not a suitable proxy single-item indicator for specific aspects of national identity in a cross-national context because respondents associate different aspects of national identity with this item (issue of validity) and these associations differ across countries (issue of comparability). In addition, we uncovered several problem types that distorted the answer selection of respondents (issue of reliability). The vagueness of this item and the associated issues make the general national pride item a rather problematic cross-national indicator for specific elements of national identity. In addition, this study illustrates the potential of web probing for evaluating the measurement quality of single-item indicators.

Keywords

Cross-national comparability national pride single-item indicators web probing

Introduction

The study of national identity is an important research field that gained popularity in recent years (Latcheva, 2011). For example, the current bibliography of the International Social Survey Programme (ISSP) now lists 512 entries for the module on national identity (Smith and Schapiro, 2015). The recent resurgence of nationalist discourse in the United States and Europe (Bonikowski, 2016) will probably further increase interest in research on national identity. Since national identity is linked to attitudes and behavior toward immigrants and refugees worldwide (Grimm et al., 2016), the current migration crisis will probably further intensify research in this field. Despite the growing significance of this research area, criticism regarding studies of national identity persists. The criticism can roughly be distinguished as contestation over meaning and contestation over measurement.

Contestation over meaning

Several researchers complain about the lack of consensus on theoretical definitions regarding national identity (e.g. Davidov, 2009; Fleiß et al., 2009; Latcheva, 2011). National identity can mean various things to different researchers, and researchers address different aspects of national identity, such as patriotism and nationalism, national identification, and national pride.¹

Varieties of patriotism and nationalism

An important distinction in the field of national identity studies is between patriotism and nationalism. However, the distinction was criticized as being ‘fraught with analytical difficulty’ (Bonikowski, 2016: 430). Indeed, there exist various definitions of patriotism and nationalism.

One strand of research originated in Adorno et al.’s (1950) work ‘The Authoritarian Personality’ and researchers in this tradition usually define patriotism as ‘degree of love and pride in one’s nation’, whereas nationalism is for them a ‘perception of national superiority and an orientation towards national dominance’ (Kosterman and Feshbach, 1989: 271; see De Figueiredo and Elkins, 2003; Feshbach and Sakano, 1997; Sidanius et al., 1997, for further studies in this tradition).

A second strand of research studying patriotism and nationalism integrated the idea of collective goods (e.g. norms, values, history, and constitution) to connect the individual with the nation in the definition of both concepts (Latcheva, 2011). Researchers in this tradition usually distinguish between constructive patriotism and nationalism. For Blank and Schmidt (2003), the essential characteristics of nationalists are an idealization of their nation, feelings of national superiority, and an uncritical acceptance of national authorities. They reject any criticism of their nation, and their criteria for who is perceived as a member of the nation are based on descent, race, or heritage; they draw socially derogatory comparisons with groups that they do not consider part of their nation. Constructive patriots refuse an idealization of the nation and an uncritical acceptance of state authorities. They endorse its criticism, only support the nation if it is working according to humanistic and democratic principles, and cherish an advanced social system (Blank and Schmidt, 2003; Davidov, 2009; see Schatz et al., 1999, for a similar distinction²).

A further variation of patriotism is symbolic patriotism, which is the ‘affective attachment to the nation and its core values through symbols’ (Parker, 2010: 97) that are perceived as synonymous with the nation, such as flags, anthems, or national heroes (Bar-Tal and Staub, 1997).

It is important to note that the previously mentioned definitions of nationalism are distinct from perceptions of nationalism by researchers that are interested in the nation-building aspect of nationalism (Bonikowski, 2008; Dekker et al., 2003; Solt, 2011). Nationalists in this context ‘believe that their nation is unique or special, this conclusion does not necessarily mean that they think they are superior to other peoples, merely that they take pride in their own nation’ (Mearsheimer, 1990: 21).

National identification as national belonging

A further influential approach in national identity research is social identity theory. In this perspective, national identity constitutes one of multiple social identities and is the ‘positive, subjectively important emotional bond with a nation’ (Tajfel and Turner, 1986). It is ‘a subjective or internalized sense of belonging to the nation’ (Huddy and Khatib, 2007: 65). A strong national identity increases adherence to group norms and creates a mild in-group bias that is distinct from a love for the nation and does not necessarily translate to outgroup antipathy (Huddy, 2016). Given the overlap of terminology, we will discuss national identity in the framework of social identity theory as national belonging in the remainder of this article.³

National pride

An important aspect of national identity is national pride. In the tradition of social identity theory, national pride is perceived as an important component of social identity (Smith, 2007). It is both ‘the pride or general positive affect that a person has for one’s nation and the pride or positive general affect that a person derives from one’s national identity’ (Smith, 2009: 197f.), a definition that underlines the emotional connection to the nation. For other researchers, ‘national pride involves both admiration and stake holding – the feeling that one has some kind of share in the achievement or an admirable quality’ (Evans and Kelley, 2002: 303) or it is the ‘positive affective bond to specific national achievements and symbols’ (Müller-Peters, 1998: 702), which adds an evaluating element to national pride. Fabrykant and Magun (2015) differentiate between two types of national pride. Normative national pride is a very general feeling of national pride and is ‘instilled by the state and its agents … [It hinges] heavily on cognitive shortcuts, such as heuristics, and consists of a set of ready-made opinions that appear to their bearers as self-obvious and objective’ (Fabrykant and Magun, 2015: 4). The grounded kind of national pride is more a domain-specific national pride that is ‘calculated by the subject … [and] involves more or less explicit comparisons of a nation’s actual level of achievements providing specific ground for pride with the level of aspirations … regarding these achievements’ (Fabrykant and Magun, 2015: 4). The differentiation between a more abstract and general feeling of national pride and domain-specific national pride that is based on an evaluation of country’s achievements in various domains can be found in several research strands (e.g. Evans and Kelley, 2002). For example, Hjerm suggests further differentiating national pride into political and cultural national pride. Political national pride is connected to a ‘civic side of a society or the political institutions, the economy and social security systems’. However, cultural national pride is based on ‘people’s history, cultural practices and of achievements connected to the people’ (Hjerm, 2003: 416; see Ariely, 2011, for similar distinctions).

However, researchers disagree about how national pride relates to patriotism and nationalism. Some researchers equate national pride with patriotism (e.g. Rose, 1985). Following in the tradition of Kosterman and Feshbach (1989), national pride is defined as an ‘absolute measure of attachment to the nation as an in-group so that the term is roughly synonymous with patriotism, or love of country’ (Domm, 2004: 6). Others equate national pride with nationalism in the closer sense that reflects a belief in the nation’s uniqueness without notions of feelings of superiority (e.g. Solt, 2011). In addition, further researchers argue that national pride is a precondition for patriotism and nationalism: ‘National pride co-exists with patriotism and is a prerequisite of nationalism, but nationalism extends beyond national pride, and feeling national pride is not equivalent to being nationalistic’ (Smith and Jarkko, 2001: 1).

A further aspect to consider is that attitudes or emotions toward the nation can also be negative. Examples for negative national attitudes are ‘national shame (being ashamed of one’s people and country) or national disgust (being disgusted with one’s people and country)’ (Dekker et al., 2003: 347). This ambiguity in definition directly impacts the empirical measurement of national identity (Fleiß et al., 2009). As can be seen, researchers in the field of national identity studies conceptualize and define national identity and its elements differently.

Contestation over measurement

In addition to contestation over meaning, researchers also vary in their operationalization of the different aspects of national identity. On the one hand, different items are used to measure specific concepts, such as constructive patriotism (Davidov, 2009; Huddy and Khatib, 2007). This differential focus raises doubts about the comparability of research results that are based on a variety of indicators.

On the other hand, the same items are used to measure different and partly contradictory concepts. One of the most extreme examples of measuring various concepts with the same indicator is the use of the general national pride (GNatPr) item. The GNatPr item as it is implemented in the 2013 module on ‘national identity’ of the ISSP asks respondents ‘How proud are you of being [COUNTRY NATIONALITY]?’ The four-point scale runs from ‘very proud’ to ‘not proud at all’. The response options ‘not a citizen of country’ and ‘can’t choose’ are given, as well.

The majority of previous studies applied the GNatPr item as an indicator of one of the elements of national identity: national belonging, national pride, different forms of patriotism, and nationalism. It has been used to measure feelings of national belonging (e.g. national attachment or national identification). For example, Elkins and Sides (2006) measure national attachment defined as ‘an individual’s identification and loyalty to a nation-state in a very general sense’ with the GNatPr as single-item indicator. The item also served as an indicator for national identification both as a single-item indicator (e.g. Shayo, 2009; Tiryakian, 2006) or in combination with other items such as ‘How close do you feel to [country]?’ (e.g. Guinaudeau et al., 2009; Pehrson et al., 2009). Not surprisingly, several researchers used the GNatPr item as a single-item indicator to measure a very general feeling of national pride that is normative national pride (e.g. Dekker et al., 2003; Kavetsos, 2012; Miller and Sundas, 2014; Moaddel et al., 2008; Reeskens and Wright, 2013; Tilley and Heath, 2007). The GNatPr item is usually not used to assess grounded national pride, which researchers prefer to evaluate with domain-specific measures (e.g. ‘pride in the history’, see, for example, Evans and Kelley, 2002). In addition, researchers use the GNatPr item to study patriotism in the sense of ‘love of and pride in the country’ (e.g. single indicator: Ariely, 2012; Rose, 1985; multiple indicator: Balabanis et al., 2001; Kosterman and Feshbach, 1989; Li and Brewer, 2004) as well as part of multiple item measures of symbolic patriotism (e.g. Conover and Feldman, 1987; Kemmelmeier and Winter, 2008; Sidanius et al., 1997; Van der Toorn et al., 2014). Furthermore, two varieties of nationalism have been measured with the GNatPr. On the one hand, previous studies applied the GNatPr item to study nationalism as the counterpart of patriotism (e.g. multiple items: Blank and Schmidt, 2003; Fleiß et al., 2009; Wagner et al., 2012). On the other hand, researchers interested in the nation-building aspects of nationalism also used the GNatPr item to measure nationalism (e.g. Solt, 2011). However, it is unclear which aspects of national identity the GNatPr captures if it is used as a single-item indicator and whether it captures different aspects in different countries.

Given the popularity of the GNatPr item, this article aims to assess in an explanatory manner the suitability of the GNatPr item for cross-national studies of the different aspects of national identity: national belonging, normative and grounded national pride, love for the country, symbolic patriotism, constructive patriotism, and nationalism. The article will present reasons for choosing the GNatPr item and the critical issues involved in its use. We will introduce the method of web probing (WP). Using WP results, we will assess the validity and comparability of the GNatPr item. That is, we will reveal which aspects of national identity and which specific reasons respondents had in mind when answering the GNatPr item and whether the respondents’ associations differ across countries. In addition, we will assess how well the GNatPr item measures what it is supposed to measure. In this context, we will examine the prevalence of problematic issues and we will assess whether these issues have a distorting impact on the answer selection for the GNatPr item. Finally, we will conclude with a critical discussion of the study’s results and their practical implications. The results of this study indicate that the respondents in the different countries associate a variety of nation-related concepts with the GNatPr item. In addition, WP revealed several problematic issues that had a distorting impact on the answer selection of the GNatPr item. The vagueness of this item and the associated issues make the GNatPr item a rather problematic cross-national indicator for specific elements of national identity.

The GNatPr item and its usage

Several reasons exist why researchers might use this item as a single-item indicator for the different concepts of national identity. First, the implementation of this item in several cross-national studies – the ISSP (ISSP Research Group, 2015) or the Eurobarometer (European Commission, 2012) – provides longitudinal data (Muñoz, 2009; Tilley and Heath, 2007), whereas multiple indicator measures of the different constructs often are limited to specific surveys and therefore cover shorter time periods. Second, researchers can increase the number of countries in their study by combining different surveys, which might be interesting for studies conducting multilevel analysis. Third, several researchers justify the use of the GNatPr item as a single-item indicator (e.g. Moaddel et al., 2008; Muñoz, 2009) with a study by Tilley and Heath (2007), who found for Great Britain a high correlation between the GNatPr item and a more sophisticated measure of national pride from the ISSP. However, Tilley and Heath used for this correlation ISSP items that usually serve as indicators for nationalism (Davidov, 2009) or chauvinism (Coenders and Scheepers, 2003). This qualifies the GNatPr item more as an indicator for nationalism in Great Britain. In addition, it remains unclear whether the correlation between the GNatPr item and the ISSP items in this particular case also holds for other countries. Finally, Elkins and Sides (2006) assigned the GNatPr item face validity as a measure of national attachment. For them, the general scope of the GNatPr item makes it preferable to several nationalism and patriotism survey indicators that are too context specific (e.g. the pride in achievements in the arts and literature or history) and that are therefore problematic as indicators for comparative studies.

However, one methodological and several substantive reasons exist that caution against the use of the GNatPr item. The methodological criticism questions the very idea of a single-item indicator as a cross-national measure for complex constructs because it assumes a measurement free of error and a perfect relationship between the indicator and the theoretical construct (Ariely and Davidov, 2012: 273), two assumptions that are rarely met in empirical research. The use of the GNatPr item as a single-item indicator was previously criticized:

Thus, we must conclude that the use of one single item for a complex dimension like ‘national pride’ is strongly misleading and the analyses based on it can be seriously flawed. It is of utmost importance for international comparative research to ensure the validity and reliability of the indices and scales used. (Haller, 2002: 148)

In contrast to multiple-item measures, it is challenging to assess whether a single-item indicator is a valid, reliable, and sufficiently comparable indicator in cross-national research (Mohler and Johnson, 2010) because the usual quantitative approaches for assessing the cross-national comparability of items (e.g. multigroup confirmatory factor analysis, Jöreskog, 1971) presuppose that the concept is measured with multiple indicators (Bollen, 1989). Although it is possible to conduct basic data procedures such as screening the answer distributions and the percentage of item-nonresponse across countries, or comparing the correlations with benchmark items from a related concept across countries (Braun and Johnson, 2010), these procedures do not provide the same statistical insights that can be gained by using multiple indicators.

From a substantive point of view, two critical issues remain: missing control for effects of social desirability and missing control for respondents’ lack of national identification when they identify on a level that is distinct from the country level.

The concept of social desirability is based on the idea that social norms influence some behavior and attitudes and that respondents potentially misrepresent themselves to maintain a socially favorable self-presentation. Respondents are likely to underreport socially undesirable behavior (e.g. illicit drug use) and over-report socially desirable behavior (e.g. voting) (Kreuter et al., 2008; Krumpal, 2013). For the GNatPr item, the social desirability effect potentially works in two ways. On the one hand, respondents from certain countries might feel pressured to opt for a high level of national pride because they feel that they are supposed to be proud of their country. Previous research indicates that Mexican respondents especially may show this response behavior (Klesner, 2006). Moreover, this effect may be particularly problematic when the GNatPr item is interpreted as constructive patriotism. A previous study revealed that Mexican respondents see the different elements of constructive patriotism as problematic (e.g. pride in democracy or the social security system) (Meitinger, 2017) but rate their national pride higher when asked the GNatPr item (e.g. ISSP 2013 mean for ‘pride in democracy’ 3.2 and for GNatPr 1.7; both items are measured on a four-point scale running from, 1 ‘very proud’ to 4 ‘not proud at all’).

On the other hand, respondents might opt for a low level of national pride although they may feel a strong attachment to their country because it is not permissible in their country to overtly express national pride. A prime example of this scenario is Germany. In previous cross-national comparisons, Germany showed persistently low levels of national pride (Evans and Kelley, 2002; Smith and Jarkko, 2001) but a high level of national attachment (Miller-Idriss and Rothenberg, 2012). Smith and Jarkko (2001) explained this contradiction with the ‘war guilt effect’. In the aftermath of World War II (WWII) and the Nazi regime, a public narrative was established in West Germany that prohibited the open expression of national pride. The expression of national pride still triggers right-wing connotations to this day, as Germans often note that it is only possible to be proud of individual achievements and impossible to be proud of the country’s achievements (Miller-Idriss, 2009). The idea of the importance of individual achievements as a precondition of pride that many Germans provide as a reason for the impossibility to feel national pride can be traced back to 2001 when the social acceptance of national pride was fiercely discussed by the political elite in Germany (‘Nationalstolzdebatte’). This position follows the lead of the former German President Johannes Rau. Rau rejected the notion of German national pride:

You cannot be proud of something that you did not achieve yourself, but you should be glad or thankful to be German. However, you cannot be proud of it, to the best of my belief. You are proud of something that you accomplished yourself. (as cited in Häusler, 2002: 144; own translation)

However, the importance of this pride taboo and the meaning of ‘national pride’ are shifting. Whereas the older generations still perceive ‘national pride’ as equivalent to ‘rampant nationalism and racism’ (Miller-Idriss and Rothenberg, 2012: 90), the younger generation of Germans is yearning for a more open expression of national pride (Miller-Idriss, 2009). The question remains whether the GNatPr item is still affected by the ‘pride taboo’ or whether these social desirability effects are gradually disappearing in Germany.

The second substantive issue is the missing control whether the respondents identify themselves sufficiently with the country level. The GNatPr item presupposes that the respondents indeed identify with their country, although this may not always be the case. Respondents could reject the idea of a national identity altogether, or they could mostly identify on the global or regional level. Two countries – Spain and Great Britain – may especially be affected by the latter possibility. For example, Spaniards from Catalonia and the Basque Country show a lower pride in Spain than Spaniards living in other parts of the country. If the GNatPr item is used as a single-item indicator, it is impossible to distinguish whether these respondents identify as Spaniards and are alienated by the current Spanish nationalism or whether they identify on the regional level and the lower pride level expresses an absence of national identification on the country level (Muñoz, 2009). In a similar vein, in Great Britain, Scottish and Welsh nationalism may be a powerful source that could substitute a country level with a regional identification for some respondents. In particular, Scottish respondents show lower levels of pride in being British than their English and Welsh counterparts (Tilley and Heath, 2007).

WP as a method to uncover respondents’ thoughts

Several qualitative approaches exist that can help to assess whether certain items achieve cross-national comparability, such as cognitive interviewing (Willis, 2005) and WP (Braun et al., 2014; Behr et al., 2017). A major advantage of a qualitative approach is that it can reveal the reasons for missing comparability if quantitative approaches cannot establish measurement invariance or are inapplicable (Meitinger, 2017). The latter is the case when the GNatPr item is used as a single-item indicator, which makes measurement invariance tests unfeasible. In addition to an assessment of cross-national comparability, qualitative approaches can open the black box of respondents’ associations when they answer a survey question. These insights can be used to improve survey questions or develop new items that better match the respondents’ reality (Braun et al., 2014).

WP transfers traditional cognitive interviewing techniques from the laboratory context into web surveys. During WP, web survey respondents receive verbal probes while answering a questionnaire. Probing in the context of WP means that respondents first answer a closed item and then receive, on a second screen, follow-up questions – so-called probes – to gain further insight into the respondents’ cognitive processes when they answered the initial closed item (Behr et al., 2017). For example, a category selection probe asks the respondents why a certain answer category was chosen (Willis, 2005).

The implementation of WP in web surveys allows for large sample sizes and a comparison of several countries, which permits a quantitative data analysis of qualitative insights. This implementation enables researchers to judge the prevalence of themes or error types and to analyze specific subpopulations or response patterns. Since all respondents receive the same probe, the results are standardized (Braun et al., 2014), which can reduce data harmonization issues of cross-national cognitive interviewing (Lee, 2014). On the downside, the implementation in the web mode means that a motivating interviewer is missing, which potentially increases item nonresponse and does not allow a spontaneous probing if the respondent provides a response that is too short or unintelligible (Meitinger And Behr, 2016; Lenzner and Neuert, 2017). In addition, the analysis is restricted to population groups that can be reached online and that have sufficient reading and writing skills (Behr et al., 2017). Previous studies show the feasibility of probing in the web context (Behr et al., 2012), addressed methodological aspects such as design features (Behr et al., 2014; Meitinger et al., 2018) or nonresponse reduction (Kaczmirek et al., 2017), and provided insights into substantive topics such as gender roles (Behr et al., 2013), xenophobia (Braun et al., 2012), or nationalism and constructive patriotism (Meitinger, 2017; see Behr et al., 2017, and Braun et al., 2014, for a research synthesis on WP).

Research objectives

The main goal of this article is to assess with WP the suitability of the GNatPr item as a cross-national indicator for the different aspects of national identity, which is an assessment of validity and comparability. Therefore, the first question we want to answer with WP is ‘What does the GNatPr item measure?’ At the same time, we want to assess the prevalence and distorting impact of potentially problematic issues, such as effects of social desirability and the absence of national identification, which is related to a reliability assessment. The second question we want to address with WP is ‘How well does the GNatPr item measure what it is supposed to measure?’

From a methodological point of view, with this study, we want to provide an illustrative example of the potential of WP as an assessment tool of single-item indicators in cross-national research.

Methods and data

The WP results for this research came from a web survey conducted with 2,685 participants in May 2014. The respondents from Germany (N = 553), Great Britain (N = 535), Mexico (N = 533), Spain (N = 533), and the United States (N = 531) were drawn from a non-probability online panel with quotas for age (18–30, 31–50, and 51–65), gender, and education (lower and higher). The panel provider was Respondi (www.respondi.com) and its partners in respective countries. Quotas were met in each country (see Table 4 in Appendix 1 for the quota distribution).

The country selection was based on the goal to include a large variety of possible perspectives on the GNatPr item and the possibility to reveal potentially problematic issues. Therefore, we selected Germany due to its low level of GNatPr and the issue of the ‘pride taboo’. Spain and Great Britain serve as interesting cases due to the existence of strong regional national identities in both countries. In contrast, the United States and Mexico are two countries with an elevated level of GNatPr.

In this survey, we replicated questions from the ISSP module on national identity. At the beginning of the survey, respondents were encouraged to express their opinion with open-ended questions, and it was underlined that there are no right or wrong answers. After the respondents answered the closed GNatPr item, they received, on a separate screen, a category selection probe (see Figure 1) that inquired about why a certain answer category was chosen (Willis, 2005). The probe nonresponse for this item was comparatively low (Germany: 11.03%, Great Britain: 10.09%, the United States: 11.68%, Mexico: 1.88%, Spain: 7.88%). Respondents wrote on average 93 characters when answering this probe, with respondents from Mexico providing the longest (117 characters) and respondents from the United States providing the shortest (66 characters) responses. On average, respondents mentioned 1.9 reasons for selecting a specific answer value.

Figure 1.

Screenshot of category selection probe for the GNatPr item.

On the basis of the probe answers, we developed a coding scheme. All the responses were coded by a researcher, and a randomly chosen sample of responses (20%) were coded a second time by a student assistant. The intercoder reliability was 93%, and mismatched coding was discussed by the coding team and corrected accordingly.

Since this study intends to evaluate the validity and the cross-national comparability of the GNatPr item from the 2013 ISSP, we considered the replication of the ISSP answer distribution of this item in our web study as a precondition for drawing comparisons between the two data sets. Table 5 in Appendix 1 summarizes the mean values, the standard deviations, the percentage of non-response (NR), and the percentage of respondents that opted for the answer category ‘not a citizen of country’ at the closed item. In our analysis, we limited our sample to respondents with the countries’ citizenships. Our web survey approximately replicated the response pattern of the 2013 ISSP, although the Spanish respondents in our web survey chose lower pride values than the Spanish ISSP respondents.

It is important to note that the replication of the ISSP question in a web survey using a nonprobability sample has two implications. First, the use of a nonprobability sample has consequences on the generalizability of findings. However, the main focus of this study is on an assessment of validity, comparability, and reliability of the GNatPr item, which is a distinct research goal from that of achieving generalizability of findings (Willis, 2005). As a consequence, we refrain from a too detailed analysis of the regression results but focus on an assessment of whether the results of WP can reveal the different perspectives respondents adopt when answering the GNatPr item and whether the issues that were revealed during WP have a distorting impact on the answer selection at the GNatPr item. Second, the implementation in the web mode might lead to a reduced prevalence of social desirability effects in our data in comparison with face-to-face data collection in some of the ISSP countries since previous research indicates that social desirability effects are more prevalent in face-to-face situations than in web surveys (Kreuter et al., 2008).

Insights from WP

The WP results give insights into the cross-national variability of respondents’ associations regarding the GNatPr item.

The code schema

Since we wanted to answer the question whether the GNatPr item can serve as an indicator for different aspects of national identity, we chose a twofold strategy to develop our code schema. On the one hand, the coding schema contained all elements of national identity for which the GNatPr item served as an indicator or that were important dimensions of national identity in previous research. The code national belonging was assigned when respondents expressed national attachment or national identification (e.g. ‘I am British’, ‘I feel German’, ‘I identify as a Mexican’). We further distinguished between respondents mentioning their love of the country (e.g. ‘I love my country’) or expressed normative national pride (e.g. ‘I am proud to be an American’). Although some variations of patriotism combine a love of the country with national pride (e.g. Kosterman and Feshbach, 1989), we decided to code both concepts separately to receive a more finely granulated picture of the different elements of national identity. If respondents referred to national symbols (e.g. flags, national anthems), the code symbolic patriotism was assigned. We coded respondents as constructive patriots when they expressed their love or pride for the country and underlined the importance of democratic and humanistic values (e.g. ‘tolerance’, ‘freedom of speech’), based their answer selection on the perceived realization of democratic principles in their country (e.g. ‘state of the social security system’, ‘voting system’), or took a critical stance toward their nation, its history, or its national authorities. We coded respondents as nationalists when they took an uncritical stance toward their nation, its history, and its national authorities and when respondents perceived their nation as superior to other nations; made socially derogating comparisons with groups not considered to be part of their nation; or defined their own group by criteria of descent, race, or heritage. We also introduced a code for all responses that are probably nationalistic but that lack a clear nationalistic statement (pseudo nationalist, for example, ‘Britain is the best country in the world’; statement misses a clear derogatory comparison). Several respondents revealed negative national feelings, such as national shame or national disgust (e.g. ‘I am ashamed of my country’, ‘some things make me cringe’).

In addition, we report the different reasons the respondents used to justify their pride evaluation, such as the living conditions in their country, being born in the country, the importance of specific values, and general characteristics of their country’s citizens. Respondents further based their evaluation on the performance of the government, their culture and traditions, the country’s history, or its nature and landscape. Several respondents were concerned about the global reputation of their country and its worldwide influence. All specific reasons that were not mentioned by at least 5% of the respondents in any country were summarized as other specific reasons. It is important to note that some of the specific reasons also served as indicators for the coding of constructive patriotism and nationalism (e.g. values, history, performance of government, worldwide influence). At the same time, the different reasons mentioned also indicate distinct types of grounded national pride. The codes for living conditions and performance of government reflect political national pride and the codes culture and traditions, history, and general characteristics of their country’s citizens can serve as indicators for cultural national pride.

The code schema also captured different problems that appeared in the probe responses. We distinguished three main problem types. First, respondents had trouble answering this item because, for them, an individual achievement is a precondition to feeling proud of something. These respondents rejected the idea of being proud of a collective and more abstract achievement, which rendered impossible being proud of their country’s achievements. Second, respondents noted that it was either unacceptable to be proud of one’s country or they associated right-wing connotations with the term pride. Thus, the effects of social desirability affected their judgment of pride. Third, several respondents perceived national identity as an irrelevant component of their identity, or they felt more connected to other levels of identification, such as the local, regional, or global. The remaining issues that the probe uncovered are summarized in the code further problems. All remaining substantive answers that were not mentioned by at least 5% of the respondents in any country were coded as rest.

Descriptive results

The presentation of the WP results consists of two parts. We will first address the validity and comparability aspects of the GNatPr item (‘What does the GNatPr item measure?’). In the second part, we will address problematic issues regarding the measurement of the GNatPr item (‘How well does the GNatPr item measure what it is supposed to measure?’).

What does the GNatPr item measure?

The WP results reveal that respondents think about various aspects of national identity but also consider specific reasons for their pride evaluation when providing a response to the GNatPr item.

Aspects of national identity

This article started with the initial observation that the GNatPr item is used for various aspects of national identity. Therefore, we wanted to evaluate whether this item is a suitable single-item indicator for national belonging, love of the country, normative national pride, symbolic patriotism, constructive patriotism, or nationalism. If this were the case, the majority of respondents in all countries should have opted for one particular aspect of national identity. As Table 1 shows, respondents of all countries mentioned nearly all aspects of national identity. Approximately 8 percent of all respondents justified their pride evaluation with statements of national belonging, and several respondents expressed normative national pride (8.5%). The code national pride was most frequently assigned in Great Britain (12.5%) and the United States (12.8%). Respondents expressed less frequently just their love of the country (e.g. ‘I love my country’ [male, 38 years, the United States, ‘very proud’]; 4.4%). However, many respondents declared their love of the country and referred to democratic and humanistic values or the perceived realization of democratic principles or adopted a critical perspective toward the nation, history, or national authorities and were therefore coded as constructive patriots. Constructive patriotism was the most prevalent aspect of national identity in our sample (21.5%), with most of the constructive patriots being U.S. respondents (28.9%). In contrast, the GNatPr item does not seem to reflect symbolic patriotism; only a few respondents mentioned national symbols. Respondents also provide nationalistic reasons for their pride evaluation (9.0%), but countries vary regarding prevalence, with British respondents most frequently providing nationalistic responses (14.4%). However, several responses were coded as pseudo-nationalistic because they missed a clear nationalistic statement (e.g. ‘The best country in the world’ [British, male, 66 years, ‘very proud’]; response lacks a direct expression of national superiority).

Table 1.

Codes for the category selection probe for the GNatPr item in percent (substantive respondents).

Code	Germany (488)	Great Britain (464)	The United States (468)	Mexico (523)	Spain (486)	Total (2429)
Aspects of national identity
National belonging (identification)	10.5	6.7	4.5	8.8	8.9	7.9
Love of the country	3.2	4.1	6.6	2.7	5.4	4.4
National pride	6.6	12.5	12.8	5.4	6.0	8.5
Symbolic patriot	0	2.0	0.2	2.1	.8	1.0
Constructive patriot	14.3	17.9	28.9	23.5	22.8	21.5
Nationalist	7.4	14.4	5.3	10.5	7.2	9.0
Pseudo nationalist	5.9	5.2	14.7	5.5	3.7	7.0
Negative national feelings	6.2	6.3	3.4	2.9	14.6	6.6
Specific reasons
Living conditions	12.3	10.1	8.3	21.0	19.8	14.5
Born in the country	9.4	7.3	9.2	26.4	16.1	14.0
Specific values	12.1	18.5	26.5	11.9	11.9	16.0
General characteristics of country’s citizens	4.7	8.4	4.7	19.5	15.6	10.8
Performance of the government or national authorities	8.6	11.9	9.8	19.9	31.9	16.6
Culture and traditions	4.5	7.5	1.3	17.0	9.7	8.2
History	14.3	8.8	1.7	8.6	6.6	8.1
Nature and landscape	2.1	2.2	0.4	16.8	5.4	5.6
Global reputation	7.4	6.0	1.5	4.2	5.6	4.9
Worldwide influence	6.6	8.2	5.1	0.4	0.6	4.1
Other specific reasons	2.5	9.3	2.4	1.2	7.2	4.4
Problems with question
Missing individual achievement	10.7	1.1	0.6	0.2	2.1	2.9
Effects of social desirability	4.3	1.9	0	0	0.4	1.3
Missing relevance of national identity	10.3	11.6	1.1	1.0	8.6	6.4
Further problems	5.1	1.1	0.6	0.4	1.0	1.7
Rest
Rest	10.8	10.9	11.7	4.7	5.2	8.6

Notes: GNatPr: general national pride.

Respondents also expressed negative national feeling, in particular Spaniards (14.6%). Several Spanish respondents were dissatisfied with a political class they perceived as corrupt:

I cannot be proud to belong to a country that has an unbelievably corrupt political class. (Spaniard, male, 30 years, ‘not very proud’)

The heightened perception of corruption might also be due to the increased coverage of the corruption scandals by Spanish media in recent years (Marek, 2015).

In total, the probe revealed that respondents associate a variety of aspects of national identity with the GNatPr item. The findings question the usefulness of the GNatPr item as a single-item indicator for specific aspects of national identity.

Specific reasons

In addition to serving as indicators for aspects of national identity, the mentioned specific reasons also revealed that respondents evaluate different dimensions to arrive at their pride evaluation, which reflects grounded national pride. In addition, specific reasons provide insights on whether countries vary along the dimension of political and cultural national pride.

Many respondents mentioned specific reasons, but some respondents justified their pride selection without referring to any specific aspect of national identity. Although the respondents from all countries provided various specific reasons, some were country-specific, and the countries differed regarding the number of relevant factors. For example, the U.S. respondents were concerned mostly with specific values (e.g. freedom, liberty) but none of the other reasons was mentioned by more than 10 percent of U.S. respondents. The mentioned reasons predominantly belonged to the dimension of political national pride; U.S. respondents rarely referred to aspects of cultural national pride. American respondents tended to mention different aspects of national identity but infrequently provided reasons for their pride evaluation. This is an indication that normative national pride might be more prevalent in the United States than grounded national pride.

In contrast, respondents in the remaining four countries mentioned a variety of specific reasons that covered political and cultural national pride. However, the variety and prevalence of codes differed across countries. German and British respondents mentioned a smaller number of codes and less frequently than Spanish and especially Mexican respondents. Specific values also appeared in British (e.g. tolerance, multiculturalism) and German responses (e.g. freedom of speech, (in)equality, social security), but respondents also thought about the current living conditions in their country. The performance of the government and national authorities was also a relevant factor for the British respondents, whereas the German respondents more often explained their modest level of pride with reference to Germany’s history. Since these German respondents often refer to WWII and the Nazi regime in their probe responses, this is a strong indication that the ‘war guilt’ effect (Smith and Jarkko, 2001) still influences their pride evaluation.

In contrast, Spanish and Mexican respondents thought about a wider variety of specific reasons. Similar to other countries, the Spanish speaking respondents referred to living conditions and values (e.g. Mexico: freedom of choice, freedom of speech; Spain: (in)equality, family). A more detailed analysis of the probe responses revealed that they were also discontented with the current government and national authorities, especially the Spaniards. All Spanish probe responses that were assigned the code ‘performance of the government or national authorities’ took a negative stance, which showed that the Spaniards were affected by a high level of political disenchantment (31.9% of Spaniards). This result is in line with previous research indicating that the economic crisis in Spain has led to an increased level of political distrust and an increasing perception of political corruption (Torcal, 2014). These results question the assumption that the GNatPr item is superior to more specific pride items because it is less affected by context effects (Elkins and Sides, 2006). The pride level of the Spanish respondents with respect to the GNatPr item is affected by the economic crisis, which is a context effect. Furthermore, being born in the country was a response provided by about one-quarter of the Mexican respondents and approximately one-sixth of the Spanish. In addition, in both countries, the respondents pointed to the typical character traits of their fellow citizens. However, the Mexican respondents mentioned two reasons that appeared less frequently in the responses from respondents from other countries. First, the Mexican respondents underlined the importance of culture and tradition (17.0% of Mexicans), in particular, traditional Mexican cuisine. Second, the probe also revealed that the landscape, nature, and climate were central features of Mexican national pride (16.8% of Mexicans). This is a very relevant finding because these central features of Mexican national identity are usually not covered by measures of grounded national pride. For example, the ISSP item battery on domain-specific national pride asks respondents about topics such as the pride in how democracy works, the economy, the social security system, or the achievements in arts and literature, but it does not cover domains such as traditions or landscape. This means that core features that constitute Mexican national pride are not covered by this item battery. This is an example where WP could help to develop an item battery on domain-specific national pride that covers the full range of respondents’ associations.

How well does the GNatPr item measure what it is supposed to measure?

More troublesome than the previous issues is the percentage of respondents indicating one of three problem types. Since approximately one-quarter of the German respondents mentioned at least one problem type, the GNatPr item seems to be particularly troublesome in the German context.

Missing individual achievement

In particular, the problem code ‘missing individual achievement’ seems to be a country-specific issue restricted largely to the German respondents. More than one-tenth of all German respondents denied the possibility of being proud of their country. For them, being proud presupposes an individual achievement or contribution that is only possible on the personal, but not the national, level.

I cannot be proud of something that I did not work hard for. It is a coincidence that I am German. (German, female, 20 years, ‘not very proud’)

This has nothing to do with me, though. After all, I have not contributed anything. (German, female, 30 years, ‘can’t choose’)

In the other four countries, very few respondents mentioned this issue.

Effects of social desirability

In addition, effects of social desirability mostly appeared in British and German responses. A content analysis of the probe responses that were assigned the code ‘social desirability’ revealed that in both countries, ‘pride’ is associated with right-wing connotations:

It’s not that I don’t feel a sense of connectedness to my country or its people, I just see ‘Britishness’ or the ‘pride’ often as euphemisms for jingoism or racism. […] I don’t like the way that ‘pride’ in our country has become a way for right wing groups to exclude others. (British, female, 44 years, ‘not very proud’)

Whereas in this context, British respondents associated ‘pride’ with the right wing, several German respondents perceived the expression of national pride as something forbidden. The persisting effect of the ‘pride taboo’ (Miller-Idriss, 2009) still prevents Germans from freely expressing national pride and thus affects their answer selection the GNatPr item. As a German respondent noted,

The term pride does not fit any of these questions. […] Maybe one should substitute the term with to be glad or to be happy, then my answers would not all have been negative. The results of the survey will certainly be distorted through the term pride. (German, female, 65 years, ‘not proud at all’)

Missing relevance of national identity as source for pride

A third issue related to the GNatPr item is that some respondents rejected the national level as a primary source of pride and their identity. Although approximately one-tenth of German, British, and Spanish respondents mentioned this problem, the reasons differ across countries. The German respondents often maintained that the concept of national pride is irrelevant for their identity: ‘As for me, it does not matter which nationality someone has. It should have been possible to choose “I do not care” as an answer value’ (German, ‘can’t choose’).

A few German respondents also perceived themselves more as world citizens and observed the global level as a primary source of identification: ‘I do have my roots in Germany, but I am proud of being a world citizen’ (German, female, 65 years, ‘can’t choose’).

British and Spanish respondents also mentioned the irrelevance of national pride and identification as world citizens in their responses. However, the majority of British and Spanish respondents coded in this category emphasized that they identified more with a regional, rather than a national, level. British respondents characterized themselves more as English or Welsh, rather than British. However, Scottish respondents showed the strongest national identification on a regional level: ‘I am Scottish and resent the fact that Britain has become synonymous with England in the eyes of both the English and, by dint of the BBC, the wider world as well’ (British, male, 66 years, ‘not proud at all’).⁴

In addition, several Spaniards preferred Catalonia as a source of identification, rather than Spain: ‘I am proud of being Catalonian and I deeply regret to have been born with Spanish citizenship’ (Spanish, male, 39 years, ‘not proud at all’).⁵

Given the strong Scottish and Catalonian identities, it might not come as a surprise that several respondents rejected Spain or Great Britain as a source of national identity. This result is also in line with previous research on national pride regarding these countries (Muñoz, 2009; Tilley and Heath, 2007). Therefore, when using the GNatPr item for data analysis, it is important to be aware of these issues, and depending on the research question, it is necessary to control for these respondents. A feasible solution is to include questions that inquire about the importance of the different levels of identification (e.g. in the ISSP) or to use the Moreno scale (Moreno, 1988).

In total, WP revealed several problematic issues. However, the more pressing question is to what degree these issues have a distorting impact on the answer selection of the GNatPr item.

Which factors influence the answer selection of the GNatPr item?

To answer this question, we conducted an additional regression analysis of the GNatPr item on the probe codes with STATA 14 (see Table 2). Since the dependent variable (GNatPr item) is measured on a four-point scale, it is necessary to account for ordinality by using an ordered logit (OLOGIT) regression. Applying the proportional odds model, we assumed that the scale of the GNatPr item represents a rough measure of an underlying latent, continuous scale. Since we excluded all the respondents who gave a NR or ‘not a citizen of country’ response for the GNatPr item or a probe NR, the sample size was reduced to 2318 respondents. We also reverse coded the GNatPr item to facilitate interpretation. Now, the higher the answer value is, the higher the pride level. Since our primary goal was to evaluate the distorting impact of the three problem types, we started by including dummy variables for missing achievement, social desirability, and irrelevance for identity (Model 1). We further controlled for age, gender, education, and country (Model 2) and aspects of national identity (Model 3). In our final model, we also added specific reasons for the pride evaluation (Model 4). McKelvey and Zavoina’s R² increased with every model, which indicated a model improvement.⁶ Since we were calculating an OLOGIT regression, the interpretation of the regression was not as straightforward as an ordinary least squares (OLS) regression, but the effect direction and the significance level could be interpreted. Most of the variables have a significant or highly significant effect in Model 4 (except gender and several specific reasons). Most importantly, the three problem types in all four models have a highly significant negative effect on the pride evaluation.

Table 2.

Ordered logit regression analysis of the GNatPr item on problem types, background variables, aspects of national identity, and specific reasons (standard error in parentheses); N = 2.318.

Variables	Model 1Problem types	Model 2 + Background variables	Model 3 + Aspects of national identity	Model 4 + Specific reasons
Problem types (reference: not mentioned)
Problem: missing achievement	−2.490 (.242)***	−2.072 (.251)***	−2.033 (.260)***	−2.297 (.267)***
Problem: social desirability	−1.454 (.367)***	−1.071 (.376)***	−.941 (.392)**	−1.059 (.402)***
Problem: irrelevance for identity	−1.880 (.174)***	−1.717 (.176)***	−1.520 (.185)***	−1.623 (.198)***
Background variables:
Age		0.017 (.003)***	0.016 (.003)***	0.017 (.003)***
Men (reference: women)		0.020 (.079)	−0.030 (.081)	0.0146 (.082)
High education (reference: lower)		−0.189 (.080)**	−0.149 (.081)	−0.139 (.083)*
Country (reference: the United States)
Country: Germany		−1.484 (.133)***	−1.587 (.137)***	−1.575 (.142)***
Country: Great Britain		−0.859 (.130)***	−0.943 (.134)***	−0.864 (.136)***
Country: Mexico		−0.590 (.126)***	−0.598 (.132)***	−0.714 (.144)***
Country: Spain		−1.391 (.131)***	−1.312 (.134)***	−1.202 (.141)***
Aspects of national identity (reference: not mentioned)
National belonging (identification)			1.253 (.159)***	1.030 (.163)***
Love of the country			1.805 (.222)***	1.638 (.225)***
National pride			0.824 (.150)***	0.715 (.152)***
Constructive patriot			0.660 (.110)***	1.206 (.136)***
Nationalist			0.845 (.158)***	0.873 (.164)***
Negative national feelings			−1.902 (.175)***	−1.618 (.178)***
Specific reasons (reference: not mentioned)
Reason: living conditions				−0.004 (.119)
Reason: born/roots here				0.413 (.128)***
Reason: values				−0.304 (.132)**
Reason: characteristics citizens				−0.055 (.137)
Reason: national authorities				−1.071 (.126)***
Reason: culture/tradition				0.476 (.165)***
Reason: history				−0.240 (.161)
Reason: nature/landscape				0.377 (.202)*
Reason: outside perception				−0.261 (.187)
Reason: power/influence				−0.483 (.205)**
Reason: other specific reasons				−0.401 (.207)*
_cut 1	−3.136 (.097)	−3.476 (.180)	−3.544 (.196)	−3.702 (.200)
_cut 2	−1.597 (.057)	−1.900 (.161)	−1.740 (.174)	−1.832 (.177)
_cut 3	.391 (.044)	.233 (.156)	.652 (.171)	.666 (.173)
McKelvey & Zavoina’s R²	.103	.187	.314	.355
LR chi² (df)	244.64 (3)	457.12 (10)	800.69 (16)	919.88 (27)

Notes: GNatPr: general national pride; LR: likelihood ratio.

p < 0.1, **p < 0.05, ***p < 0.01.

To better assess the impact of the three problem types, we estimated the predicted probabilities for the countries in which each issue was most prevalent. The predicted probabilities provide the probability of choosing a certain answer value while holding the other variables constant at their mean values. As shown in Table 3, a German respondent’s mention of any of the three problem types increased the probability for choosing the answer value ‘not proud at all’ and decreased the probability for choosing the answer value ‘very proud’. In Great Britain and Spain, respondents referring to the problem type ‘irrelevance for identity’ also had a higher probability to opt for low pride values than respondents who did not mention this issue. Therefore, the three problem types distorted the answer selection at the GNatPr item, which further questions the cross-national comparability of this item.

Table 3.

Predicted probabilities of GNatPr values in different countries by problem type not mentioned versus mentioned.

Germany		Individual achievement			Social desirability
Germany		Not mentioned	Mentioned		Not mentioned	Mentioned
Not at all		.08	.48		.09	.22
Not very		.29	.38		.30	.43
Somewhat		.51	.13		.50	.31
Very		.12	.01		.12	.04
Irrelevance for identity	Germany		Great Britain		Spain
Irrelevance for identity	Not mentioned	Mentioned	Not mentioned	Mentioned	Not mentioned	Mentioned
Not at all	.08	.31	.04	.20	.06	.25
Not very	.29	.44	.20	.42	.23	.43
Somewhat	.51	.23	.55	.33	.54	.28
Very	.12	.03	.20	.05	.16	.04

Discussion

This study evaluated the suitability of the GNatPr item as a cross-national indicator for the different elements of national identity with WP. It assessed which aspects of national identity respondents had in mind when providing a response to the GNatPr item, whether these associations differed across countries, and how well the GNatPr item works as measure for the different aspects of national identity. The results suggest that the GNatPr item is not a suitable proxy single-item indicator for specific aspects of national identity in a cross-national context because respondents associate different aspects of national identity with this item (issue of validity) and these associations differ across countries (issue of comparability). In addition, we uncovered several problem types that distorted the answer selection of German, British, and Spanish respondents (issue of reliability).

But why does this matter? It matters because we need to know whether respondents from different countries indeed differ in their level of patriotism or nationalism or whether this difference is artificial and due to variations in the associations as well as problems with the GNatPr item. This is particularly important because empirical research detected different consequences of patriotism and nationalism that lead to different normative evaluations of both forms of national attachment, as Huddy (2016) explains: ‘Nationalism, but not patriotism, is what gives national attachment their bad name because unlike patriotism it leads to heightened xenophobia and greater perceived external threat […] Nationalism thus constitutes the dark side of national attachment’ (p. 10) Previous research shows that nationalism can lead, for example, to xenophobia and anti-Semitism (Blank and Schmidt, 2003; De Figueiredo and Elkins, 2003; Sidanius et al., 1997). Therefore, it is very important to assess the prevalence of nationalism correctly in the different countries. As we have seen, the answer selection was distorted in some countries (Germany, Great Britain, and Spain) but some countries were not affected by these issues (the United States and Mexico). These findings reveal that differences across countries might be artificial and do not necessarily reflect the ‘true’ percentage of nationalists.

In contrast to nationalism, patriotism is usually described as the ‘benign’ (Huddy, 2016: 10) side of national attachments, but findings regarding its consequences are inconsistent (Ariely, 2017). Patriotism is shown to not only increase solidarity (De Rooij et al., 2012) but also to reduce egalitarian attitudes (Solt, 2011; see also Ariely, 2017). This inconsistency in findings might be due to its measures (Huddy, 2016). The use of the GNatPr as an indicator of patriotism might be the reason for some surprising findings regarding the consequences of patriotism. For example, in his study on the relation of patriotism and globalization, Ariely (2012) notes that he cannot find a clear pattern between the different dimensions of national identity and the globalization level of the different countries. In this study, he used the GNatPr item as an indicator for patriotism and finds that only 26 percent of the respondents in the Netherlands can be classified as patriots, whereas the percentage in Ireland is 74 percent (based on the selection of the answer value ‘very proud’). Given the findings of our study, it might be possible that the issues related to GNatPr might have had an obscuring impact on the relation between globalization and patriotism.

Conclusion

This study set out to evaluate the suitability of the GNatPr item as a measurement of national identity and its different aspects in cross-national studies. We used qualitative insights from WP to assess the validity, comparability (‘What does the GNatPr item measure?’), and reliability (‘How well does the GNatPr item measure what it is supposed to measure?’) of the GNatPr item.

Our results clearly question the assumption that the GNatPr item can serve as a valid and comparable proxy for specific concepts, such as national belonging, nationalism, or patriotism in cross-national studies, since our respondents associated various concepts with this item and these associations differ across countries. Many respondents provided specific reasons for their pride evaluation, which reflects grounded national pride and not normative national pride. WP revealed a country difference regarding normative and grounded national pride. Normative national pride is more frequent in the United States than in any of the other countries. In addition, U.S. respondents mentioned mostly aspects belonging to the dimension of political national pride, whereas respondents in the four other countries referred to both dimensions of grounded national pride (political and cultural). These findings suggest that the GNatPr item could serve, at the very best, as a very abstract indicator of national identity.

However, this would not solve the different problems associated with this item. We uncovered several problem types that distorted the answer selection of our German, British, and Spanish respondents. In addition, the WP results revealed context effects in the respondents’ pride evaluations, which questions the assumption of the temporal stability of this indicator. These finding suggests that the GNatPr does not perform equally well across countries as a measure of national identity and its aspects. These results caution against the use of the GNatPr item as a single-item indicator to measure specific aspects of national identity in a cross-national context.

Practical recommendations and future research

Instead of relying on this vague indicator, researchers should define which aspect of national identity they want to study and choose their measures accordingly. For example, if researchers aim to analyze constructive patriotism and nationalism, they could use Davidov’s (2009) multiple indicator measures for these constructs that showed metric invariance for 34 countries. The factor scores of these measures could, for example, be used in a regression analysis. Hjerm and Schnabel (2010) also propose a multiple-indicator measure of national sentiments that achieved metric invariance for European countries. Although these multiple-indicators measures might also be imperfect (e.g. lack of scalar measurement invariance), they provide more statistical insights about the different aspects of measurement quality than single-item indicators. The challenge for researchers is to ensure that their indicators indeed capture what they are supposed to measure and that respondents in all countries understand and associate the same concepts with these indicators. The indicators, thus, have to be valid, comparable, and reliable. Given the complexity of cross-national data, validity and equivalence should never be assumed but must be tested. Using multiple indicators instead of single-item indicators allows for a quantitative assessment of different aspects of measurement quality.

However, multiple item measures are not always available. The current study provides valuable insights for researchers who consider using the GNatPr item in their research. It revealed which aspects of national identity the GNatPr item captures in the different countries. Therefore, researchers planning a study on national identity can make a more informed decision whether the GNatPr item can serve as a useful indicator in their research. In addition, the WP results pointed to issues such as potential social desirability effects or missing national identification. WP revealed which countries are particularly affected by these issues (Germany, Spain, and Great Britain). At the same time, the qualitative data also exposed which issues should be taken into account when analyzing data from these countries. Researchers can draw on this information for the decision which control variables to include in a statistical model that uses the GNatPr item as indicator. For example, the WP results showed that in Great Britain and Spain, it might be useful to include measures of national identification below the country level (e.g. Basque, Catalonian, or Scottish).

In addition, the WP results also could be applied to reevaluate previous studies using the GNatPr item that found inconsistent findings regarding specific aspects of national identity such as patriotism or nationalism. The qualitative insights could potentially be used to shed light on and explain these inconsistencies in research.

The WP results also could help to expand existing measures of national identity or develop new ones. For example, WP provided information on the cross-national variability of sources for grounded national pride (e.g. Mexico: nature and landscape). Based on these findings, the existing measures of grounded national pride could be extended or new measures could be developed.

Finally, future research could use WP to assess other relevant measures of national identity. Due to the large sample sizes and the qualitative data, WP is work- and time intensive and should specifically target potentially problematic items. The selection of national identity items to include in WP studies can be based on two approaches. On the one hand, quantitative measurement invariance tests can be used to detect items that lack cross-national comparability. This approach also identifies the countries in which the item might not work. However, this approach is based on multiple-indicator measures (Meitinger, 2017). On the other hand, previous studies in the field of national identity can be used to find problematic items for a WP study. For example, Heath et al. (2009) reported about issues of cross-national comparability of further measures in the ISSP module on national identity. WP certainly could be a valuable tool to analyze the validity, comparability, and reliability of other measures of national identity.⁷

Footnotes

Appendix 1

Table 5.

Comparison of the mean, proportion of non-response, and proportion of the respondents who choose ‘no citizen of country’ of the 2013 ISSP and the web survey for the GNatPr item.

	2013 ISSP			Web survey
	Mean (SD)	NR (%)	No Citizen of Country (%)	Mean (SD)	NR (%)	No Citizen of Country (%)
Germany	2.1 (.7)	12.2	0	2.2 (.8)	14.1	0.7
Great Britain	1.8 (.7)	2.4	1.3	1.9 (.8)	3.9	3.2
Mexico	1.7 (.8)	3.0	0	1.7 (.8)	2.1	0
Spain	1.7 (.9)	2.3	2.3	2.1 (1.0)	3.8	0.7
The United States	1.3 (.5)	1.4	.1	1.5 (.7)	2.6	0.2

Notes: Scale of the GNatPr item: four-point scale running from 1 ‘very proud’ to 4 ‘not proud at all’. ISSP: International Social Survey Programme; GNatPr: general national pride; NR: non-response.

Funding

This work was supported by the German Research Foundation (DFG) as part of the project ‘Optimizing Probing Procedures for Cross-National Web Surveys’ [BR 908/5-1].

Notes

References

Adorno

Frenkel-Brunswik

Levinson

et al . (1950) The Authoritarian Personality. Oxford: Harpers.

Ariely

(2011) Constitutional patriotism, liberal nationalism and membership in the nation: An empirical assessment. Acta Politica 46(3): 294–319.

Ariely

(2012) Globalisation and the decline of national identity? An exploration across sixty-three countries. Nations and Nationalism 18(3): 461–482.

Ariely

(2018) Evaluations of patriotism across countries, groups, and policy domains. Journal of Ethnic and Migration Studies 44: 462–481.

Ariely

Davidov

(2012) Assessment of measurement equivalence with cross-national and longitudinal surveys in political science. European Political Science 11(3): 363–377.

Balabanis

Diamantopoulos

Mueller

et al . (2001) The impact of nationalism, patriotism and internationalism on consumer ethnocentric tendencies. Journal of International Business Studies 32(1): 157–175.

Bar-Tal

Staub

(1997) Patriotism in the Lives of Individuals and Nations. Chicago, IL: Nelson-Hall Publishers.

Behr

Braun

Kaczmirek

et al . (2013) Testing the validity of gender ideology items by implementing probing questions in web surveys. Field Methods 25(2): 124–141.

Behr

Braun

Kaczmirek

et al . (2014) Item comparability in cross-national surveys: Results from asking probing questions in cross-national web surveys about attitudes towards civil disobedience. Quality & Quantity 48(1): 127–148.

10.

Behr

Kaczmirek

Bandilla

et al . (2012) Asking probing questions in web surveys: Which factors have an impact on the quality of responses? Social Science Computer Review 30(4): 487–498.

11.

Behr

Meitinger

Braun

Kaczmirek

(2017) Web probing - implementing probing techniques from cognitive interviewing in web surveys with the goal to assess the validity of survey questions. GESIS Survey Guidelines. Mannheim: GESIS - Leibniz-Institut für Sozialwissenschaften. https://doi.org/10.15465/gesis-sg_en_023

12.

Blank

Schmidt

(2003) National identity in a united Germany: Nationalism or patriotism? An empirical test with representative data. Political Psychology 24: 289–312.

13.

Bollen

(1989) Structural Equations with Latent Variables. New York: John Wiley & Sons.

14.

Bonikowski

(2008) Research on American nationalism: Review of literature, annotated bibliography, and directory of publicly available data sets. RSF Working Paper, Russell Sage Foundation, New York, May. Available at: http://bit.ly/29rLVco (accessed 11 July 2016).

15.

Bonikowski

(2016) Nationalism in settled times. Annual Review of Sociology 42: 427–449.

16.

Braun

Johnson

(2010) An illustrative review of techniques for detecting inequivalences. In: Harkness

Braun

Edwards

et al . (eds) Survey Methods in Multinational, Multiregional, and Multicultural Contexts. Hoboken, NJ: Wiley-Blackwell, pp. 373–393.

17.

Braun

Behr

Kaczmirek

(2012) Assessing cross-national equivalence of measures of xenophobia: Evidence from probing in web surveys. International Journal of Public Opinion Research 25(3): 383–395.

18.

Braun

Behr

Kaczmirek

et al . (2014) Evaluating cross-national item equivalence with probing questions in web surveys. In: Engel

Jann

Lynn

et al . (eds) Improving Survey Methods: Lessons from Recent Research. New York: Routledge, pp. 184–200.

19.

Coenders

Scheepers

(2003) The effect of education on nationalism and ethnic exclusionism: An international comparison. Political Psychology 24(2): 313–343.

20.

Conover

Feldman

(1987) Memo to NES board of overseers regarding ‘measuring patriotism and nationalism’. Available at: www.icpsr.umich.edu

21.

Davidov

(2009) Measurement equivalence of nationalism and constructive patriotism in the ISSP: 34 countries in a comparative perspective. Political Analysis 17(1): 64–82.

22.

De Figueiredo

Elkins

(2003) Are patriots bigots? An inquiry into the vices of in-group pride. American Journal of Political Science 47(1): 171–188.

23.

De Rooij

Reeskens

Wright

(2012) Does pride in one’s nation foster participation? On the causal linkage between national pride and political. Available at: https://lirias.kuleuven.be/bitstream/123456789/346963/1/MPSA_NatIdPolPart.pdf

24.

Dekker

Malova

Hoogendoorn

(2003) Nationalism and its explanations. Political Psychology 24(2): 345–376.

25.

Domm

(2004, December) Explaining public support for European integration in eight member states: The role of national pride, European identity, nationalism and xenophobia. EUI Working Paper SIS No. 2204/19. Florence: European University Institute.

26.

Elkins

Sides

(2006) In search of the unified nation-state: National attachment among distinctive citizens. UC Irvine Center for the Study of Democracy, University of California, Irvine, CA, April. Available at: https://escholarship.org/uc/item/20f203bx (accessed 10 July 2016).

27.

European Commission (2012) Eurobarometer 62.2 (Nov-Dec 2004). TNS OPINION & SOCIAL, Brussels [Producer]. GESIS Data Archive, Cologne. ZA4231 Data file Version 1.1.0. DOI: 10.4232/1.10964.

28.

Evans

Kelley

(2002) National pride in the developed world: Survey data from 24 nations. International Journal of Public Opinion Research 14(3): 303–338.

29.

Fabrykant

Magun

(2015) Grounded and normative dimensions of national pride in comparative perspective. Higher School of Economics Research Paper No. WP BRP /62SOC/2015. DOI: 10.2139/ssrn.2599967.

30.

Feshbach

Sakano

(1997) The structure and correlates of attitudes toward one’s nation in samples of United States and Japanese college students: A comparative study. In: Bar-Tal

Staub

(eds) Patriotism in the Lives of Individuals and Nations. Chicago, IL: Nelson-Hall Publishers, pp. 91–107.

31.

Fleiß

Höllinger

Kuzmics

(2009) Nationalstolz zwischen Patriotismus und Nationalismus? Berliner Journal für Soziologie 19(3): 409–434.

32.

Grimm

Huddy

Schmidt

et al . (2016) Introduction. In: Grimm

Schmidt

Huddy

et al . (eds) Dynamics of National Identity: Media and Societal Factors of What We Are. London: Routledge, pp. 1–7.

33.

Guinaudeau

Fuchs

Schubert

(2009) National identity, European identity and euroscepticism. In: Dieter

Raul

Antoine

(eds) Euroscepticism: Images of Europe among Mass Publics and Political Elites. Farmington Hills, MI: Barbara Budrich Publishers, pp. 91–112.

34.

Haller

(2002) Theory and method in the comparative study of values: Critique and alternative to Inglehart. European sociological review 18(2): 139–158.

35.

Häusler

(2002) Die ‘Nationalstolz’-Debatte als Markstein einer Rechtsentwicklung der Bürgerlichen Mitte. In: Butterwegge

Cremer

Häusler

et al . (eds) Themen der Rechten – Themen der Mitte: Zuwanderung, Demografischer Wandel und Nationalbewusstsein. Opladen: Leske + Budrich, pp. 123–146.

36.

Heath

Martin

Spreckelsen

(2009) Cross-national comparability of survey attitude measures. International Journal of Public Opinion Research 21(3): 293–315.

37.

Hjerm

(2003) National sentiments in eastern and western Europe. Nationalities Papers 31(4): 413–429.

38.

Hjerm

Schnabel

(2010) Mobilizing nationalist sentiments: Which factors affect nationalist sentiments in Europe? Social Science Research 39(4): 527–539.

39.

Huddy

(2016) Unifying national identity research. In: Grimm

Schmidt

Huddy

et al . (eds) Dynamics of National Identity: Media and Societal Factors of What We Are. London: Routledge, pp. 9–21.

40.

Huddy

Khatib

(2007) American patriotism, national identity, and political involvement. American Journal of Political Science 51(1): 63–77.

41.

ISSP Research Group (2015) International social survey programme: National Identity III – ISSP 2013. GESIS Data Archive, Cologne. ZA5950 Data file Version 2.0.0. DOI: 10.4232/1.12312.

42.

Jöreskog

(1971) Simultaneous factor analysis in several populations. Psychometrika 36(4): 409–426.

43.

Kaczmirek

Meitinger

Behr

(2017) Higher data quality in web probing with EvalAnswer: A tool for identifying and reducing nonresponse in open-ended questions. GESIS Papers 2017/01. Köln: GESIS - Leibniz-Institut für Sozialwissenschaften. http://nbn-resolving.de/urn:nbn:de:0168-ssoar-51100-0

44.

Kavetsos

(2012) National pride: War minus the shooting. Social Indicators Research 106(1): 173–185.

45.

Kemmelmeier

Winter

(2008) Sowing patriotism, but reaping nationalism? Consequences of exposure to the American flag. Political Psychology 29(6): 859–879.

46.

Klesner

(2006) Economic integration and national identity in Mexico. Nationalism and Ethnic Politics 12(3–4): 481–507.

47.

Kosterman

Feshbach

(1989) Toward a measure of patriotic and nationalistic attitudes. Political Psychology 10(2): 257–274.

48.

Kreuter

Presser

Tourangeau

(2008) Social desirability bias in CATI, IVR, and web surveys: The effects of mode and question sensitivity. Public Opinion Quarterly 72(5): 847–865.

49.

Krumpal

(2013) Determinants of social desirability bias in sensitive surveys: A literature review. Quality & Quantity 47(4): 2025–2047.

50.

Latcheva

(2011) Cognitive interviewing and factor-analytic techniques: A mixed method approach to validity of survey items measuring national identity. Quality & Quantity 45(6): 1175–1199.

51.

Lee

(2014) Conducting cognitive interviews in cross-national settings. Assessment 21: 227–240.

52.

Lenzner

Neuert

(2017) Pretesting survey questions via web probing-does it produce similar results to face-to-face cognitive interviewing? Survey Practice 10(4): 2768.

53.

Brewer

(2004) What does it mean to be an American? Patriotism, nationalism, and American identity after 9/11. Political Psychology 25(5): 727–739.

54.

Long

Freese

(2014) Regression Models for Categorical Dependent Variables Using Stata, 3rd edn. College Station, TX: Stata Press.

55.

Marek

(2015) The effects of scandals on perceived corruption in Spain. Global Journal on Humanities and Social Sciences 2(1): 1–8.

56.

Mearsheimer

(1990) Back to the future: Instability in Europe after the Cold War. International Security 15(1): 5–56.

57.

Meitinger

Behr

(2016) Comparing cognitive interviewing and online probing: Do they find similar results? Field Methods 28(4): 363–380. doi: dx-doi-org.web.bisu.edu.cn/10.1177/1525822X15625866.

58.

Meitinger

(2017) Necessary but insufficient: Why measurement invariance tests need online probing as a complementary tool. Public Opinion Quarterly 81(2): 447–472. doi: dx-doi-org.web.bisu.edu.cn/10.1093/poq/nfx009.

59.

Meitinger

Braun

Behr

(2018) Sequence matters in online probing: The impact of the order of probes on response quality, motivation of respondents, and answer content.” Survey Research Methods 12(2): 103–120. doi: dx-doi-org.web.bisu.edu.cn/10.18148/srm/2018.v12i2.7219.

60.

Miller

Sundas

(2014) Testing the national identity argument. European Political Science Review 6(2): 237–259.

61.

Miller-Idriss

(2009) Blood and Culture: Youth, Right-Wing Extremism, and National Belonging in Contemporary Germany. Durham, NC: Duke University Press.

62.

Miller-Idriss

Rothenberg

(2012) Ambivalence, pride and shame: Conceptualisations of German nationhood. Nations and Nationalism 18(1): 132–155.

63.

Moaddel

Tessler

Inglehart

(2008) Foreign occupation and national pride the case of Iraq. Public Opinion Quarterly 72(4): 677–705.

64.

Mohler

Johnson

(2010) Equivalence, comparability, and methodological progress. In: Harkness

Braun

Edwards

et al . (eds) Survey Methods in Multinational, Multiregional, and Multicultural Contexts. Hoboken, NJ: Wiley-Blackwell, pp. 17–29.

65.

Moreno

(1988) Scotland and Catalonia: The path to home rule. In: McCrone

Brown

(eds) The Scottish Government Yearbook. Edinburgh: Unit for the Study of Government in Scotland, University of Edinburgh, pp. 166–182.

66.

Müller-Peters

(1998) The significance of national pride and national identity to the attitude toward the single European currency: A Europe-wide comparison. Journal of Economic Psychology 19(6): 701–719.

67.

Muñoz

(2009) From national-Catholicism to democratic patriotism? Democratization and reconstruction of national pride: The case of Spain (1981–2000). Ethnic and Racial Studies 32(4): 616–639.

68.

Parker

(2010) Symbolic versus blind patriotism: Distinction without difference? Political Research Quarterly 63(1): 97–114.

69.

Pehrson

Vignoles

Brown

(2009) National identification and anti-immigrant prejudice: Individual and contextual effects of national definitions. Social Psychology Quarterly 72(1): 24–38.

70.

Reeskens

Wright

(2013) Nationalism and the cohesive society: A multilevel analysis of the interplay among diversity, national identity, and social capital across 27 European societies. Comparative Political Studies 46(2): 153–181.

71.

Roccas

Berlin

(2016) Identification with groups and national identity. Applying multidimensional models of group identification to national identification. In: Grimm

Schmidt

Huddy

et al . (eds) Dynamics of National Identity: Media and Societal Factors of What We Are. London: Routledge, pp. 22–43.

72.

Rose

(1985) National pride in cross-national perspective. International Social Science Journal 37(1): 85–96.

73.

Schatz

Staub

Lavine

(1999) On the varieties of national attachment: Blind versus constructive patriotism. Political Psychology 20(1): 151–174.

74.

Shayo

(2009) A model of social identity with an application to political economy: Nation, class, and redistribution. American Political Science Review 103(2): 147–174.

75.

Sidanius

Feshbach

Levin

et al . (1997) The interface between ethnic and national attachment: Ethnic pluralism or ethnic dominance? Public Opinion Quarterly 61(1): 102–133.

76.

Smith

(1991) National Identity. London: Penguin Books.

77.

Smith

(2007) Social identity and socio-demographic structure. International Journal of Public Opinion Research 19(3): 380–390.

78.

Smith

(2009) National pride in comparative perspective. In: Haller

Jowell

Smith

(eds) The International Social Survey Programme 1984–2009: Charting the Globe. London: Routledge, pp. 197–221.

79.

Smith

Jarkko

(2001) National pride in cross-national perspective. Unpublished manuscript.

80.

Smith

Schapiro

(2015) A compilation of documents used to develop the national identity III questionnaire for the International Social Survey Programme in 2014. NORC Report, National Opinion Research Center, Chicago, IL. Available at: http://www.gesis.org/issp/modules/issp-modules-by-topic/national-identity/2013/(accessed 10 May 2016).

81.

Solt

(2011) Diversionary nationalism: Economic inequality and the formation of national pride. The Journal of Politics 73(3): 821–830.

82.

Tajfel

Turner

(1986) The social identity theory of intergroup behavior. In: Austin

Worchel

(eds) Psychology of Intergroup Relations. Chicago, IL: Nelson-Hall Publishers, pp. 7–24.

83.

Tilley

Heath

(2007) The decline of British national pride. British Journal of Sociology 58(4): 661–678.

84.

Tiryakian

(2006) Coping with collective stigma: The case of Germany. In: Rothbart

Korostelina

(eds) Identity, Morality, and Threat: Studies in Violent Conflict. Lanham, MD: Lexington/Rowman and Littlefield, pp. 359–398.

85.

Torcal

(2014) The decline of political trust in Spain and Portugal: Economic performance or political responsiveness? American Behavioral Scientist 58(12): 1542–1567.

86.

Van der Toorn

Nail

Liviatan

et al . (2014) My country, right or wrong: Does activating system justification motivation eliminate the liberal-conservative gap in patriotism? Journal of Experimental Social Psychology 54: 50–60.

87.

Wagner

Becker

Christ

et al . (2012) A longitudinal test of the relation between German nationalism, patriotism, and outgroup derogation. European Sociological Review 28(3): 319–332.

88.

Willis

(2005) Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks, CA: SAGE.