The inclusion of quantitative techniques and diversity in the mainstream of feminist research

It is bad

As Gerring (2007: 11) observes, ‘the purpose of a statistical sample is to reveal elements of a broader population’. In the 1970s and 1980s it was exactly this practice of generalization that was heavily criticized by feminists: assuming homogeneous societies, ignoring important differences between the position of men and women, and assuming that results concerning one half of the population (men) also hold for the other half (women) (Bernard, 1975; Harding, 1986; Jayaratna, 1983; Millman and Kanter, 1975; Oakley and Oakley, 1979). This idea is part of a larger ‘war’ between ‘the two cultures’ (qualitative and quantitative) in the social sciences (King et al., 1994: 3; Mahoney and Goertz, 2006), and steers feminist empirical research away from deriving general statements and using quantitative techniques.

However, in the current mainstream of gender studies, many take the position that it is possible (and even desirable) to also make generalizations about (groups of) persons. This resonates with (feminist) scholars who have argued convincingly that ‘bad’ is not synonymous to quantitative (see Andermahr et al., 1997; Harding, 1997; McCall, 2005; Sprague, 2008). The problem of bad quantitative research is not that it is ‘quantitative’, but that it is ‘bad’, implying that good quantitative research is possible as well, and some authors have indeed formulated ways to treat quantitative results carefully (e.g. McCloskey and Ziliak, 1996; Miller and Van der Meulen Rodgers, 2008). Having said this, the observation remains that quantitative techniques mainly feature in the mainstream of gender studies as an object of critique, not as a practice. As a consequence, I suggest that ‘badness’ is not inherent to quantitative research, but the ‘badness’ label still sticks.

It is practically impossible

A cluster of more practical arguments forms the second reason for the absence. Quantitative research models that include high levels of social complexity (as a good gender studies study would) require large and rich datasets (see also Warner, 2008), highly technical models, and (top peer-reviewed) journals that accept lengthy explanations (cf. Sprague, 2008). These difficulties might be particularly problematic for gender studies scholars who focus on many different identities and their intersection (see McCall, 2005: 1787).

This argument is problematic in two ways. First, good research is equated to ideal type research. However, no matter what data or methods are used, no quantitative or qualitative study will be perfect. Knowledge will grow step by step. Second, regarding the journals, it is assumed they are ruled by ‘outsiders’ with their own norms (as suggested by McCall, 2005). True or not, this does not explain the observation that opened this article. As Sprague stresses, ‘very high quality’ gender studies journals ‘with a broad readership’ have been created (Sprague, 2008: 702), and these are – according to their aims and scopes – open to complex, lengthy and quantitative articles. I therefore conclude that practical impossibilities are not the problem, but that either the supply of quantitative studies is low or the review hurdle is disproportionally high. Both might be attributed to an aversion to quantitative work.

Thinking ‘it is bad’

I suggested above that in the culture of mainstream gender studies quantitative research still tends to be regarded as bad. In this section, I argue that this is due to thinking in binary pairs instead of scientific progress. McCall’s (2005) recent and influential attempt to build a bridge between different methods is my starting point, because it frames the problem aptly around categorization, but it does not resolve the core problem: positional thinking. McCall discusses three approaches to intersectionality in terms of categories (in other words generalization or diversity), and argues that quantitative research can contribute significantly to research on intersectionality.

The first approach – anticategoricalism – is highly critical of categorization because dividing people into groups is the fundament of power relations. Categorization entails inclusion and exclusion based on non-reality, because all categories are an artifice of language, and it denies the multiple identities each individual has (McCall, 2005: 1777–1779). Anticategoricalists set out to attack existing stereotypes and deconstruct categories. Consequently, large restrictive essentialist identities are broken down and multiple and fluid identities emphasized. The second group – intracategorical research – does not reject categorization, but complicates existing boundaries. In particular, seeing women as a single unitary group has been heavily criticized (McCall, 2005: 1778–1780). Scholars have refined this category, by constantly asking which identities are important for a particular social structure and the related power relations. People in different positions have different experiences, which should not be ignored or overgeneralized. This view focuses on diversity within a group and the role of multiple categories. The last approach – intercategoricalism – starts from the notion that in terms of power relations social groups do exist, so the research is category-based. These power relations are studied and explicated, which ‘requires the provisional use of categories’ (McCall, 2005: 1785). This includes the study of how categories change and interact. In the intercategorical approach in particular all sorts of statistical techniques and models can be applied (McCall, 2005: 1784–1788).

Together these three traditions are the main building blocks of McCall’s ‘continuum of complexity management’ and it seems that by showing how the different approaches try to manage complexity, McCall tries to create a common understanding (e.g. McCall, 2005: 1775). However, the continuum is built around three (archetypical) groups of research. This means that it is not a continuum, but a set containing three positions which resonate strongly with the epistemologies Harding (1986, 1997) distinguishes: postmodernism, standpoint feminism and feminist empiricism (see Figure 1).

OPEN IN VIEWER

Figure 1.

A continuum of complexity management and epistemological positions

McCall herself acknowledges this and also hints at its consequences: ‘in the anticategorical and even intracategorical camps’ ‘advanced quantitative techniques and large data sets . . . [are] associate[d] negatively with the legacies of positivism, or empiricism’ (McCall, 2005: 1791) (see Figure 1). However, the classification of intersectionality research strengthens this binary thinking. This is especially so because McCall mostly stresses the distinction between intercategorical research on the one hand and the intra- and anticategorical approach on the other. From the dominant position in the mainstream of gender studies (the standpoint view and postmodernism) the negative association with quantitative research then remains (see Figure 2).

OPEN IN VIEWER

Figure 2.

Judging quantitative studies

In sum, McCall’s classification of research in different groups enables her to show how different types of research can contribute to research on diversity, but it does not transcend the positional thinking and the negative attitudes towards quantitative research. Because research methods are tied up with epistemological positions it becomes very difficult to change these attitudes. Reconceptualizing the debate in terms of diversity and judging relatively might shed new light on the issue.

Judging whether it is better

Thinking about feminist research in terms of diversity might provide us with a conceptual tool for creating a true continuum. One of the two extremes on this continuum is infinite diversity: each and every individual is unique in every aspect. The other end of the continuum is no diversity at all. Each and every individual is a specimen of one single larger group: what holds for one of them holds for all. Between these two poles, the level of diversity differs and thereby the degree of generalization differs as well.

The concept of diversity in this approach has two dimensions: the basis of diversity and the type of diversity. The basis of diversity is the dimension on which (groups of) people are distinguished: the identities that are considered relevant, such as gender, sex, class, ethnicity, race, age, sexual orientation, religion and able-bodiedness (see e.g. Lorber, 2006; Phoenix and Pattynama, 2006; Verloo, 2006; Yuval-Davis, 2006). On the continuum these are not a priori hierarchically ordered: taking one into account is not by definition more diverse than taking another into account. However, distinguishing between three or more sexes is more diverse than distinguishing between two, and distinguishing between three sexes and several ethnicities is even more diverse (ceteris paribus).

For the second dimension – the type of diversity – a similar argument holds, but there is a technical hierarchy. First there are differences in the position of (groups of) people based on their identities, and second, (groups of) people are differently affected by phenomena because of those identities. A prerequisite for understanding such an effect is establishing the position of different groups of people. Whether education has a different effect on women’s and men’s employment status implies knowing their employment status.

The usage of the concept of diversity as described above leads to a two-dimensional continuum (Figure 3a), on which feminist as well as androcentric research can be placed. Each study occupies a unique position – has its own degree of diversity – and hardly any study is in an extreme corner of the concept.

OPEN IN VIEWER

Figure 3.

(a) A two-dimensional concept of diversity (b) Relative positioning on the diversity continuum

Placing studies on such a continuum shows how a study relates to other studies – the state of the art in the particular disciplinary field (indicated by black circle in Figure 3b). By adding one new distinction that leads to an understanding of inequality, progress is made in that field (black square in Figure 3b) if judged relatively. The focus now shifts from judging something as being ‘bad’ or ‘good’ to judging it as being ‘better’ or ‘worse’. In that sense, differentiating between women and men in medical research might be far more progressive or feminist (cf. Hankivsky and Christoffersen, 2008: 272; Nieuwenhoven and Klinge, 2010) than studying political representation differences between straight and gay men and women in the US Black and Latino communities.

Applying the diversity continuum shows that quantitative techniques offer gender scholars a means to increase our knowledge of differences, make scientific progress and simultaneously enable scholars to relate to existing knowledge outside gender studies in a way that makes communication between the different fields possible.

Differences in positions

This section focuses on the options for distinguishing groups, describing these groups and drawing inferences for these groups from a background of existing data and basic quantitative techniques (Table 2). Some caveats are discussed at the end.

OPEN IN VIEWER

Table 2.

Techniques and different diversities in quantitative methods

	Classifying	Describing	(Cor)relating and explaining
Position	Distinguish between sexes create a nominal sex variable Use the most detailed classification possible interval > categorical > nominal AND n+1 > n categories Check for differences on other identities create variables for other identity variables Use intersectional identities cross the identity variables so that all subgroups are uniquely classified	Give descriptives per group and its subgroups calculate the measures of centrality – mean – for all distinguished groups Describe not only centrality but also variability per (sub)group include the standard deviation and minimum & maximum or interval(like) variables and the frequencies on nominal and categorical data	Test for structural differences in position include identity variables in multiple regression models Let the different identities intersect include interaction terms of ‘crossed dummies’ so that each group has its own regression coefficient Study whether characteristics that are identity-related are responsible for the found differences include additional control variables that are for instance gender informed Include societal explanations when studying individual positions use multilevel models that include individual and societal characteristics
Effects			Test whether effects differ for different (sub)groups of people, at least by sex include interactions terms of each explanatory variable with the identity variable Test for ‘intersectional’ differences in effect include three-way interactions or ‘crossed dummies’ Test whether societal characteristics affect different individuals differently according to their identity include cross-level interactions

To understand the different positions of different groups of people it is necessary to use a classification. Although large comparative datasets such as the European Social Surveys, World Values Surveys and Programme for International Student Assessment data often do not include the fine-grained sets of questions that are, for example, discussed by Browne (2008) regarding sexuality, or Völke et al. (2010) on ethnicity/race/skin colour/ancestry, current techniques and data still provide several opportunities. First, virtually all surveys provide binary information about the sex of the respondent. This information can be used to create a variable that compares women and men. Second, most surveys provide data on some other identities. These can be used to understand differences between other identities – preferably in as many categories as possible – and subsequently a combination of identity variables can be used to create an ‘intersectional’ identity matrix. If I, for example, want to know the position of a black woman, I should realize that this position is probably more than the sum of the two parts: the difference between being black and white might be different for men and women (e.g. Hancock, 2007; Wekker and Lutz, 2001). Consequently, I have to distinguish four groups.

When describing the characteristics of groups, researchers should not only provide figures on the population as a whole, but also for each of the subgroups. Furthermore, besides measures of centrality/central tendency (such as the mean, median and modus), the differences within a group can be provided by including measures of variability/dispersion (such as the standard deviation and the minimum and maximum) for interval variables. These latter measures show the heterogeneity within a group: how precise the mean is and how many people are close to it.

In order to draw inferences about the relationship between sex, gender and other identities and outcomes such as income, participation and health, intersectional positions can be included relatively easily in regression models. First, all relevant and available identity variables such as sex, gender and ethnicity can be included as a regular explanatory variable or dummy (Hardy, 1993; Pollock, 2009). The advantage of this over the techniques discussed above is that it is possible to take into account differences that correlate with these identities (see Allison, 1999). For instance, the wage gap between men and women will then be a net difference because differences caused by occupational segregation and working experience are filtered out. Second, intersectionality can be incorporated by crossing identities. This can be done by either using specific dummies in the case of ordinal or nominal variables, or by interaction terms for any type of variable (see Jaccard and Turrisi, 2003; Pollock, 2009). For instance, Almeida-Filho et al. (2004: 1347) study how the interaction of gender, ethnicity and social class influences the prevalence of depressive disorders in Bahia (Brazil) and find that among upper-middle-class respondents no notable gender differences were found, regardless of ethnicity. Among poor and working-class respondents, women were found to have a higher prevalence of depressive disorders, but not among whites.

Third, non-identity factors that are socially attached to an identity can be included in statistical models. For instance, besides sex, variables tapping into gender norms and structures – such as having children in the case of labour market participation – can be included. If after inclusion of this variable the effect of being male/female decreases, the researcher could conclude that the differences between women and men is partly due to the caretaker role women have in society. Fourth, when studying individuals, researchers can use multilevel models to see whether societal factors have an important role in causing differences (see Jones and Duncan, 1998; Painter, n.d.). For example, besides education levels, having children and marital status, societal factors such as gender equality policies and gendered labour market structures can be very relevant for women’s labour market decisions and position (e.g. Spierings et al., 2010).

While these basic and available techniques can increase our understanding of diversity enormously, there are some caveats that should be considered. First, when looking at the difference between women and men, essentialist conclusions lurk around the corner. A statistical technique only shows a correlation between factors; it says nothing about the origin of the difference or the causal mechanism at work. The tendency in, for instance, mainstream sociology to invoke biology when sex differences are found, as observed by Lorber (2006: 449), serves as a reminder that statistical packages do not resolve the nature–nurture debate. Second, drawing an inference means that the researcher generalizes the results to the population as a whole. This (theoretical) population should be clearly demarcated (see Goertz and Hewitt, 2006). Ordinarily, general conclusions can only be drawn for groups included in the sample that is analysed. Within that sample, one can test for differences in position (see above) or effect (see below). Third, creating subgroups does indeed mean deciding where to draw the line; this is both arbitrary and can create power relations. However, in line with McCall and others, I argue that the use of new and finer categories – for a particular field – is also a way to contest existing ideas and power relations.

Differences in effects

Many factors have a different impact on people with different identities. An obvious example is the effect of having children on men’s and women’s labour market participation. In order to study these kinds of different effects and draw inferences about them, each explanatory variable in a regression model can interact statistically with (an) identity variable(s). This will show whether the coefficients differ significantly (statistically and substantially) across sex, ethnicity, sexual orientation and so on (see Jaccard and Turrisi, 2003; Pollock, 2009). In other words, it shows whether an effect is gendered or equivalently related to other identities.

For instance, in the case of a dichotomous sex variable and a two-group ethnicity variable, the researcher can calculate interaction effects between the explanatory variable and dummies for each group of people (e.g. black women, black men, white men, white women). In the case of interval variables, three-way interaction can be used (Jaccard and Turrisi, 2003: 43–60). By using these complex structures, intersectionality is transformed from a positional approach into a causal approach: the researcher not only discusses how the positions of the different groups deviate from each other, but also tests whether and how people at different crossroads are affected differently by similar processes. The example given above concerning gender, ethnicity and social class in the prevalence of depressive disorders is basically a three-way interaction.

In multilevel models, the inclusion of interactions allows for testing the different effects of norms, policies and structures on individuals. For instance, Huisman and Smits (2009) show that the number of schools in an area influences the educational participation of girls but not boys. Multilevel modelling might then also create opportunities for understanding intersectionality in terms of the structure/agency or the institutions/individuals (see Ferree, 2009: 87).

The major drawback of including interactions is that it leads to more complex models and requires richer datasets. However, as argued above, given the state of the field, even with existing data and relatively simple models, we can make progress in our analyses of diversity. Furthermore, it is of course possible to train (new generations of) gender scholars to use more advanced techniques. Similarly, it is of course possible to collect new and better data (e.g. Browne, 2008), among other things, to avoid the risk of ‘too many variables, too few cases’ (see Landman, 2008: 303).

Example: Differences in political interest

For several decades, an important research focus has been the determinants of individual interest in politics. Research shows that the political interest of women is lower than that of men, and that education increases it (see Inglehart and Norris, 2003; Koch, 1997; Verba et al., 1997). From this starting point, steps in the two diversifying directions (see Figure 3) can be made to increase our knowledge of political interest and sex differences: how do the effects of factors such as education differ by sex (studied by Morgan et al. [2008] for the Dominican Republic) and what are the differences among women (or men)? Let us start with the latter.

Morgan et al. (2008: 42) find that the political interest of men in 2007 is about 2.9 on a scale running from 0 to 6. Women score 2.3. For both groups, the median is 2. Further descriptives are not given, but the Latinobarometer 2007 data show a similar difference – the average political interest in the Dominican Republic is 2.2 (on a scale from 1 to 4), ⁶ with men scoring 2.3 and women 2.0 – and they permit further investigation in this study. The standard deviation is just over 1, indicating quite a wide spread, which is also shown by the frequencies of political interest (see Figure 4 for women). In addition, Figure 5 shows that Catholic women’s political interest is 2.0 and that of Protestant women 2.3; a difference equalling the variation among the sexes. The descriptives for self-reported ethnicity show that the differences between mulato and white people are almost negligible, but that mestizo women have a political interest (1.94) that is about 0.2 lower on average than that of indigenous women (2.16). These results illustrate that very simple statistics (means and standard deviations) show how a category differs internally and that those differences can be just as important as differences across the sexes (Figure 5). Including this form of diversity provides a better view of the phenomenon of political interest, and it constitutes progress in terms of the diversity continuum (Figure 3).

OPEN IN VIEWER

Figure 4.

Political interest among women

OPEN IN VIEWER

Figure 5.

Differences in political interest

A second way to increase diversity entails modelling differences in effects as Morgan et al. have done. They include an interaction term for education and gender (see Table 3), and show that women with no education have 0.63 lower political interest than men with no education. On a seven-point scale, political interest rises by 0.33 for men once they are educated; for women this is 0.15 (i.e. 0.33 – 0.18). In other words, education has more effect on men’s political interest than on women’s and thus increases the political interest gap between men and women. The authors conclude that ‘This sizeable difference in the effect of education suggests that while education promotes political engagement overall, there may be something about the education process . . . that encourage[s] men to take more interest in politics than women’ (Morgan et al., 2008: 47–48). Their study illustrates how knowledge of the genderedness of effects on political interest is increased by a step in the upward direction on the diversity continuum (Figure 3).

OPEN IN VIEWER

Table 3.

Results from Morgan et al. (2008: 45): ‘OLS regression: Political interest in the Dominican Republic 2004’

Variable	B-coefficient	Standard error
Female	−0.63**	0.09
Education	0.33**	0.03
Education*Female	–0.18**	0.06

**

p < .01.

Note: Also included were: evaluation of services, pocketbook evaluation, employed, age groups, interactions between age and gender, socioeconomic status, casada(o)/unida(o), rural.

In summary, applying existing data and techniques allows three important steps to be taken in the field of political interest and diversity. First, it was found that women are generally less interested. Second, differences among women are charted to contextualize the sex difference. Third, differences in the effects of other factors are found for different sex categories. Further progression could be found by including (more) gender(ed) characteristics instead of sex in order to explain why there is a difference, and to interact those aspects with, for example, education. Also, the interaction between identities could be included, such as whether religious differences are different for men and women. ⁷