Abstract
This conversation explores emerging debates concerning teaching to and about marginalized populations often left out of “representative” data sets. Based on our experiences studying, teaching, and belonging to some of these unrepresented populations, we outline some strategies sociologists may use to transform the limitations of data sets traditionally labeled as representative into tools for delivering core sociological concepts. In so doing, we argue that sociologists may respond to increasing critiques of “representative” data by using these critiques to facilitate critical thinking skills and methodological awareness among students. In closing, we encourage sociologists to consider the challenges and opportunities presented by increasing awareness of unrepresented populations within our classrooms and the broader social world.
Keywords
After class one day last year, a colleague pulled me aside to ask a question. Like many sociologists at present (the authors of this piece included), my colleague regularly uses statistical findings from “representative” surveys in his classes to show students the usefulness of quantitative methods, patterns in oppression and privilege, and variations in the outcomes people face in relation to many social phenomena. 1 Earlier in the semester, however, my colleague had begun working with me on scholarship concerning transgender and intersex people, which involved the erasure of these people from mainstream sources of information and education (Cragun and Sumerau 2015; Sumerau and Cragun 2015; Sumerau, Cragun, and Mathers 2015). As a result, my colleague had begun to notice that the “representative” samples he had used for years in classes were the exact types of knowledge that facilitated the erasure of non-cisgender people.
As we sat on a bench outside of his classroom, he asked me, “So what do I do? None of the data captures an actual representation of the nation, but I need the data to show students how to use statistics. I don’t want to contribute to the erasure of minority groups, but I also know it’s important for our students to learn about statistics and how sociologists use them.” As I increasingly do in recent years when someone—usually a cisgender scholar—asks me questions like this, I responded by noting that “This is an emerging issue in the academy as a result of decades of ignoring various populations that are increasingly gaining mainstream attention and recognition” and sharing strategies I have seen and used for managing such conflict.
In this article, we seek to facilitate discussion of these issues among sociology instructors. 2 Specifi-cally, we draw on our own experiences both navigating this issue in the use of “representative” data sets as researchers and teachers and navigating our existence as people generally “not represented” (or explicitly measured) in these data sets to outline strategies we have used and seen for handling such conflicts. Rather than suggesting such strategies are in any way ideal or complete, however, we use them as starting points for conversations. In this way, we seek to encourage our fellow instructors to reflect on the ways we go about representing the world and utilizing existing data sets.
Such a conversation is especially important in the current sociological and broader social climate (see also Wentling et al. 2008). Within the past year, for example, the practice of calling surveys “nationally” or otherwise representative when they do not measure transgender and other gender and sexually fluid populations has been criticized in mainstream media by stars like Laverne Cox; in academic journals like Gender & Society (Westbrook and Saperstein 2015), Social Currents (Sumerau et al. 2015), Sociological Forum (Miller and Grollman 2015), and Journal of Sex Research (Cragun and Sumerau 2015); and on academic blog sites like Conditionally Accepted (Sumerau 2015). At the same time, the past two years have witnessed a concerted effort in the health sciences to begin incorporating measurements of bisexual, intersex, transgender, and other identities in major survey efforts (Ivankovich, Leichliter, and Douglas 2013) and the launch of major interdisciplinary surveys targeted at such communities (Harrison, Grant, and Herman 2012). Sociological instructors and students increasingly walk into educational environments wherein traditional notions of “representative” are increasingly being met by growing social awareness of phenomena they do not explicitly represent or leave unrepresented.
These dilemmas may become increasingly salient with growing recognition of the size and experience of transgender populations in contemporary American society. 3 In terms of size, for example, conservative population estimates suggest between 0.3 percent and 2 percent of Americans are transgender, which suggests this population is larger than some groups (e.g., Jewish, Muslim, and Mormon Americans) regularly “counted” in current surveys (see Grant et al. 2011 for review). Further, emerging studies reveal significant differences in, for example, educational, economic, health, and religious outcomes and experiences between both transgender and cisgender people and between transition-interested and nonbinary transgender people (see e.g., Grant et al. 2011; Harrison et al. 2012; Sumerau et al. 2015). As understanding of the scope and experience of transgender Americans grows, teachers may increasingly be required to integrate such issues into their classroom examples and discussions.
While these issues may be new to many cisgender instructors and students, the three of us have been wrestling with these questions since we entered academic and educational practice. From the first use of the term representative in our undergraduate social science courses, the three of us were well aware that people like us were absent from the representation offered by the measurements contained in most surveys. Specifically, the first author is an agender person while the second and third authors are both genderqueer and bisexual. 4 While people like us might be hidden somewhere in “representative” data, we do not generally exist in the actual elements measured in such data.
This is, perhaps, the sociological equivalent of the philosophical question about trees falling in the forest. If people with certain characteristics might be present in a data set but invisible in its codebook and absent from measurement and analyses, are they truly represented? Echoing the published work on transgender people noted previously, we suggest they are not, which presents both challenges and opportunities for sociology instructors. Here we offer strategies instructors may utilize in hopes of facilitating conversation about the meaning of “representative” data, the experiences of populations hidden by or within such data sets, and ways students may critically engage with these important sociological questions.
To this end, we offer a brief summary of traditional interpretations of “representative” data 5 and then offer strategies we have seen and used to engage students on these topics. In conclusion, we suggest further avenues for discussing, debating, and reflecting on these issues both among ourselves and within our classrooms. In so doing, however, we recognize that in many cases background elements of institutional norms and decision-making processes create the current forms of survey research. Rather than engaging in methodological debates about the construction of surveys, here we focus on the ways sociologists may manage limitations within existing surveys in the classroom. As such, we offer the following discussion in hopes of contributing to and spurring conversation about these issues.
What does “Representative” Mean
Over the past few decades, “representative” survey instruments have become standard in sociology. Specifically, sociological researchers and instructors often rely heavily on the findings and generalizations created by the use of data sets (1) collected in an attempt to represent the elements of a given population numerically, (2) weighted to match the population demographics provided by the United States census or other official data repositories, and/or (3) sampled, oversampled, or undersampled on specific measures in an attempt to map respondent characteristics onto hypothesized models of what an actual population looks like in the empirical world (for further elaboration, see Westbrook and Saperstein 2015). Sociology instructors then use the insights gleaned from these data sources to demonstrate and discuss broad patterns in social experience, outcomes, and disparities in the world our students inhabit. Further, many scholars and students alike interpret these data sets as accurate or factual representations of the world at large and thus rely on them to make sense of their place within existing social relationships and structures.
While the use of such data sets can be incredibly rewarding, the emergence of increasing numbers of students and researchers from diverse backgrounds increasingly reveals the limitations of these “representations” (see also Sumerau et al. 2015). Rather than representing society, for example, most representative surveys mirror the officially recognized portions of society contained within census measurements. These instruments thus often leave out measurements relevant to some of the most marginalized populations within a given society and rely on the willingness and ability of a given government to collect and recognize the existence of marginalized communities. Not surprisingly, some populations are often unrepresented due to the lack of official government numbers on these populations. As a result, representative data sets typically represent only the segments of a given society allowed by existing governmental counts and collections rather than an actual representation of society.
This observation is not all that surprising when we consider the construction of contemporary survey instruments. 6 Simply put, surveys—like other socially constructed representations (McCabe et al. 2011)—are created by people. As such, people make decisions about what can or should be measured explicitly, what aspects or groups within a given society are necessary for understanding social phenomena and accessible to a given set of researchers, and how various types of information should be collected and categorized. As a result, we only know the contents of surveys or the characteristics of respondents that survey designers decide to explicitly measure in the first place. A data set may not have a variable wherein respondents identify their gender or may only allow respondents to identify as one of two genders, but this does not necessarily mean the data set does not contain respondents who claim a wide variety of gender identities. Rather, this means the data set in question can only “represent” the types of gender deemed important enough to measure no matter the actual gender diversity (or lack thereof) contained within the data. Regardless of the reasons for measuring one group and not measuring another group, the end result is the same—the survey represents a piece of a given society or population built from specific decision-making processes concerning measurement and sampling rather than the actual society or population in question.
However, this would not necessarily be problematic if teachers and researchers explicitly noted this fact in their efforts. If, for example, we called such data sets representative of “insert specific collected portions of the population here” only, then other groups would not be erased in the process and students could be encouraged to critically consider why variables identifying certain groups are missing from official data. Likewise, if we noted that all “representative” data sets rely on the convenience of existing official numbers or the decisions made by their creators about “what counts” (i.e., referred to them as limited or convenience samples as we do with many other surveys), then we would not be teaching students to adopt a representation of society that does not match the actual society in question. These tactics, however, are not the standard approach in sociological teaching and research at present. Rather, researchers and teachers are typically taught to interpret (and thus teach others to interpret) these sets as actually representative of a whole population despite the fact that any survey design will ultimately be unable to fully represent a given population 7 (Westbrook and Saperstein 2015).
This problem becomes especially apparent in relation to gender (Cragun and Sumerau 2015). Whereas transgender, intersex, and other nonbinary populations have existed as long as we have records, these groups typically disappear in “representative” survey instruments. As Westbrook and Saperstein (2015) note, these “oversights” in the data arise as a result of the tendency for representative surveys to either not ask about gender at all or only allow people to identify as male or female on such surveys (for examples, see Ivankovich et al. 2013). Especially as gender-fluid communities gain more social recognition, these patterns in current survey practice may leave sociology instructors with little to say in regards to major public debates, student concerns, and social experiences.
As noted in the previous section, we can already see such conflicts arising within and beyond sociology classrooms as more and more students and faculty are at least aware of transgender, intersex, and otherwise nonbinary people and experience (Sumerau 2015) and more sociology instructors wrestle with the limitations of existing data sets in their classrooms. We thus use the next section of this article to outline some ways we and other instructors have managed these conflicts within our own classrooms in hopes of providing options to other instructors. Then, we conclude by calling for conversation, debate, and reflection on the meanings, usefulness, limitations, and pitfalls of teaching sociology via “representative” survey instruments within a world increasingly composed of people from diverse backgrounds and experiences that often go unrepresented, unidentified, or unmeasured in such data sets.
Overcoming Representative Limitations
As students and fellow instructors have become more aware of populations generally left unidentified in representative samples in recent years, we have embraced the opportunity to revise traditional approaches to such data within classroom settings. As Wentling and associates (2008) note, such efforts necessitate moving beyond established conventions to develop more sensitive and nuanced approaches to scholarly materials and traditionally marginalized (or unrepresented) populations. To this end, we offer the following strategies we have developed over the past few years for simultaneously utilizing large-scale surveys in sociology classes and remaining sensitive to the limitations of these data sources. Rather than an exhaustive protocol, however, we offer strategies that have worked well in our classrooms in hopes of spurring conversation concerning the use of such data sources in sociological education.
Demonstrating Social Change
A hallmark of sociological education involves demonstrating the ways social structures—like government, education, and religion—change over time (see e.g., Durkheim [1897] 2002; Marx [1848] 1998; Weber 1930). As such, one way instructors may overcome limitations in existing data is by turning these limitations into evidence of the continuously shifting “realities” (Berger and Luckmann 1966) promoted by social authorities. Specifically, this strategy relies on explaining to students the origin of “representative” as an attempt to match officially produced representations of the world offered by governmental authorities. After such an introduction, an instructor may then present other historical examples (i.e., shifting measurements of race, sex, gender, sexualities, and religion) contained in data archives or via comparison between surveys to demonstrate how what “counts” or gets “represented” in surveys over time shifts in relation to what “counts” or gets “represented” by shifting governmental actions and social movement endeavors.
In zir 8 introduction to sociology discussion sections, for example, the third author encountered students curious about the absence of transgender people in current survey instruments and the broader sociological literature. In response, the third author noted the absence of sexual identity in the General Social Survey until 2008 and asked students why this variable (one incredibly important in recent politics) might have escaped attention until the past decade. 9 Not surprisingly, the students quickly noted that the timeline corresponded to increased mainstream awareness of sexual identity issues and conflicts and suggested—as recent scholarship has (Westbrook and Saperstein 2015)—that the same might likely happen in relation to transgender populations as our government and broader society became increasingly aware of these groups. In so doing, the third author used the limitation in existing data to contextualize the creation of “official” facts. The limitation became a strength in delivering the broader lesson that governments and “official” realities shift over time in relation to social changes.
Instructors may utilize limitations in contemporary “representations” of society to help students make sense of the fluid nature of reality and established truth (Berger and Luckmann 1966). In so doing, instructors may utilize insights concerning marginalized communities left out of “representations” from earlier time periods to demonstrate the ever-evolving nature of society, government, and sociology. Such efforts can show students how our own data lend empirical weight to traditional sociological theories concerning the malleability of the social world, the influence of social movements, and the practices whereby governments and broader populations make sense of change (Mills 1940). In this way, existing data limitations may become fuel for developing sociological imaginations.
Teaching Research Design
One of the core course offerings in most sociology programs involves the construction and accomplishment of research projects (Wagenaar 2004). One way instructors may use existing limitations in data collection is by using these limitations to teach students the subjective decision-making processes embedded within any research design (Kleinman 2007). Specifically, instructors may utilize these limitations to teach students to interrogate their own blind spots, biases, and assumptions about the social world. Likewise, instructors may use these examples to demonstrate the importance of ongoing funding and education for research by noting that all studies—no matter how well planned or executed—will have limitations that require replication, comparison, debate, discussion, and further study. Finally, instructors may use these limitations to encourage students to come up with questions and measurement strategies currently missing from scientific endeavors and contribute to the ongoing development of better knowledge and understanding of the world around us.
In their courses and independent studies with undergraduate and graduate students, for example, the first author provides students with a collection of survey data sets and requires them to examine these data sets for missing information that could improve the social world. Students then critique and examine these knowledge sources based on their own understandings of the world and training (however much or little) in scientific methods. The first author then guides the students to note the things they found that other students missed and the things they missed that other students found. In so doing, the first author points out that what one believes to be relevant or important to a given subject is dramatically influenced by one’s existing knowledge base and experience (Kleinman 2007). The first author then walks students through practices of comparison and collaboration that allow them to build stronger research protocols while also noting the importance of multiple research designs and perspectives in the attempt to understand any given social phenomena.
The first author thus begins lessons on research design by incorporating limitations in all data sets in order to demonstrate the importance of ongoing research, critical thinking, and debate. To this end, the first author primes students to expect and find limitations in data sets and approach such realizations as opportunities for further study and better understanding. Rather than simply delivering “facts,” the first author encourages students to become fact checkers and knowledge producers by critically evaluating what is deemed to be true by themselves and others active in the current social landscape. In this way, the limitations within existing data become springboards for emerging sociological careers, research agendas, and research protocols students may carry with them into the next phase of their educational and occupational endeavors.
Revealing the Power of Language
A longstanding element of sociological education involves the many ways inequalities are created and reproduced in varied social situations and contexts (Schwalbe et al. 2000). One of the primary ways this is accomplished is via the use of language that presents one set of norms as valuable, taken for granted, and good while downplaying, erasing, or otherwise marginalizing other sets of norms (Kleinman 2007). Central to this tradition is the observation that all socially constructed documentation—intentionally or otherwise—challenges and/or reproduces existing patterns of structural and interpersonal inequality within a given society (Kleinman 2002). As a result, one way instructors may use limitations within existing data sets is by using these limitations to reveal the power of language to shape the social world, mask social inequalities, and replicate dominant assumptions about the way the world is or should be.
In zir introduction to sociology courses, for example, the second author explains the origins, uses, and elements of “representative” surveys. In so doing, the second author pulls up example population variables common in these surveys and asks students to come up with missing populations that are not explicitly represented in the surveys. Students never have much trouble doing this and generally offer groups including but not limited to transgender, nonbinary, multiracial, bisexual, pansexual, and intersex people after having learned of these groups in their social lives, via media, or via other courses they have already taken. Borrowing a question a transgender student asked zir years ago in a classroom, the second author then asks the students “if there are missing parts of the national population in these surveys, why do you think we tend to call them representative?”
The second author uses the limitations of existing data sources to encourage students to question what is considered an adequate representation of a population and what such language says about the communities not measured or identified by such data sets. Students quickly note that it gives the impression that “missing” and “unidentified” groups are not important or not really part of the nation, and many of them become angry at this realization. After discussing these dynamics and the importance of how we talk about populations, students are then required to offer other names we could use to describe these data sets. In response, they often offer suggestions including but not limited to government surveys, Christian surveys, selective surveys, Republican surveys, national surveys, and convenience surveys. 10 Once they are exposed to the amount of people not measured or identified in these data sets, students do away with the notion of such data being “representative” of the concrete world they inhabit. While the second author initially worried that this strategy might lead students to reject survey results, ze was surprised to learn—via their written responses—that students actually expressed greater faith and trust in surveys as a result of understanding how they were created, the limitations they contained, and the ways they could always be improved and revised over time to better capture the world. 11
Illustrating Gender Inequalities
While the aforementioned elements (i.e., social change, research methods, and inequalities) represent common issues taught throughout sociology curriculums, instructors may also utilize existing data limitations to illustrate specific concepts and content areas. While we have already witnessed instructors doing this in relation to various concepts, here we demonstrate how the three of us have done so to educate students about gender. To this end, we illustrate two gender concepts limitations within “representative” surveys can help instructors illustrate for their students. We focus on gender here both because we each regularly publish and teach in various areas of gender scholarship and because, as others have noted (see e.g., Berkowitz, Manohar, and Tinkler 2010; Kleinman 2002; Wentling et al. 2008), sociological commentary on teaching gender (within and beyond Teaching Sociology) remains limited to date.
Doing gender via representation
The first way we utilize notions of “representative” data to teach students about gender involves the ways that the construction of such data relies on researchers “doing gender,” or locating themselves and others in distinctive gender categories based on appearance or other social cues defined as masculine or feminine (West and Zimmerman 1987). When constructing the General Social Survey, for example, researchers do not ask anyone to identify their gender. Rather, as Westbrook and Saperstein (2015) note, investigators place respondents into sex categories (male and female only) based on their own assumptions and beliefs about what it means to be and look like a man or woman in society. In this way, the General Social Survey represents a way researchers (intentionally or otherwise) “do gender” rather than explore or question gender explicitly.
Following this lesson, we then have students discuss “how” the investigators “decide if a respondent is female or male.” Since they do not inspect the respondents’ genitals that we know of (this is often the first suggestion offered by some students), they must be relying on some understanding or taken-for-granted assumption (Ridgeway 2011) to gender respondents. Students then offer examples (i.e., appearance, hair length, voice, etc), and we show them exceptions to these assumptions (i.e., people who appear one way based on such assumptions but are not in terms of identification). After a few rounds of this process, we introduce students to the concept of “doing gender” and the ways our social world is built on the assumptions people use to sort themselves and others into these categories whether on a survey or in daily life. With this background in mind, students—not surprisingly—can then think of many ways this process plays out on other survey platforms, social media protocols, and in their interactions with others every single day. The initial data set becomes a lever for demonstrating processes of doing gender embedded throughout our students’ realities and experiences.
Cisgendering reality via representation
A second way we utilize notions of “representative” data to teach students about gender involves the ways that the construction of such data reproduces wider societal processes of “cisgendering reality,” or the process whereby authorities create and promote representations of the world predicated on the existence of two and only two sexes and genders (Sumerau et al. 2015). When constructing the General Social Survey, for example, researchers automatically sort people into two and only two sexes and offer no option at all for intersex, transgender, or otherwise nonbinary people in society. Further, if a respondent’s gender changes at any point, the General Social Survey treats this change as a “data error” that must be corrected or done away with for analysis (for discussion of this practice in the General Social Survey, see Westbrook and Saperstein 2015). Like religious mythologies popular in Judeo-Christian-Islamic traditions (Sumerau and Cragun 2015) and governmental norms popular in many current Western nations (Westbrook and Schilt 2014), such surveys cisgender the world for researchers, students, and other viewers by automatically limiting their representations to cisgender options and defining “other” options as “errors.” In this way, the General Social Survey represents a way researchers (intentionally or otherwise) “cisgender reality” rather than exploring our concrete, empirical world.
Following this lesson, we then have students discuss “why” the investigators “decide to change natural realities into a cisgender representation.” Since there is no evidence our world has ever only contained two and only two sexes and genders, there must be some social processes underlying such decision making on the part of scholars. Students then offer possible explanations (i.e., maybe it comes from religious influence, maybe they never met non-cisgender people so they did not realize they were missing these people, etc.), and we walk them through the historical marginalization and dis-enfranchisement of non-cisgender people in America and other countries. Not surprisingly, students quickly begin to offer a myriad of other ways their religions, schools, media, families, and other social relations engage in processes of cisgendering reality. The initial data set becomes an epiphany whereby students begin to notice the ways cisgender norms and assumptions populate their lives regardless of their intentions or opinions about non-cisgender people and communities.
While we could provide many more examples wherein limitations in data called “representative” can be used to illuminate gender concepts that are often hard for students to understand at first (see also Berkowitz et al. 2010), the take-home point remains the same—the surveys we call representative typically illustrate existing gender inequalities embedded throughout the contemporary social world. Rather than tossing these data sources out the window, however, we echo Wentling and associates’ (2008) discussion about ways to incorporate transgender experience into existing sociological educational forms. Specifically, we suggest that instructors may use the problems with such data to provide illustrative cases for teaching students important gender concepts.
Considering the Uses of “Representative” Data in Sociology Classrooms
While increasing social recognition of the diversity and complexity of contemporary social relations and identities presents challenges for traditional models based on numerical representation (Westbrook and Saperstein 2014), the proceeding discussion highlights some ways instructors can use these challenges to benefit sociological education. 12 While a pessimist may see shifting understandings of social realities as a problem and an optimist might see the same as purely an opportunity, our experiences reveal a middle ground wherein the problems with existing “representative” data sets may create opportunities for critical thinking, the refinement of research design and methods, and the illustration of important sociological concepts our students wrestle with in daily life. We encourage sociologists to consider, discuss, and debate the possibilities and pitfalls of data we call representative within our ongoing educational efforts.
While our own discussion has centered on “missing, unmeasured, and unidentified” elements of gender in contemporary survey-based data sets, these observations could also facilitate important discussions concerning other marginalized or underrepresented social groups and activities. When sociological educators seek to engage students in discussions of adopted people, people produced through artificial insemination, people with different functional abilities (mentally and physically), and homeless populations, for example, contemporary “representative” surveys are not likely to provide many answers. Similar problems may emerge in relation to, for example, multi- and interracial groups, emerging social movements, and new or less well known religious traditions. In all such cases, the strategies we outline in relation to gender may be useful teaching strategies or sources for class debate and discussion for contemporary educators. Further, such populations represent opportunities to engage in discussions about the meaning, content, and use of current data sets throughout the discipline.
To this end, we have outlined problems with traditional approaches to such data as well as some ways we have transformed these problems into resources for educating our students about core sociological concepts and issues. While we do not claim that our own strategies are in any way exhaustive, we believe they may provide examples for debating the benefits and limitations of “representative” data use in varied classroom settings and for considering the multitude of ways such data sources may maintain educational relevance even as data collection efforts change and students become increasingly aware of their limitations. We thus ask other educators to consider what we call these data sets in educational practice, how we use them in our delivery of content, and how we approach the limitations they possess. These questions may well allow us to more easily transition alongside shifting methodological conditions brought about via increased social awareness of many marginalized groups.
These endeavors may be especially important for at least two reasons. First, sociology instructors have already noted the need for more discussion concerning the teaching of gender in society as well as transgender experience specifically in relation to emerging social and political debates in mainstream society (see e.g., Berkowitz et al. 2010; Nowakowski and Sumerau 2015; Wenling et al. 2008). Second, an exploration of Teaching Sociology—as well as other educational resources—quickly reveals that most of the content involving surveys consists of the presentation of survey results rather than concrete tips and strategies for using surveys in classroom settings or methods for handling issues with survey data that may arise in classrooms (see e.g., Blouin and Moss 2015; Lauer and Yadonis 2004; Royce 2007). The combination of these patterns alongside increasing social commentary—within and beyond classrooms—about gender fluidity and complexity in our world suggest many instructors will increasingly face such questions in their educational efforts. As a result, it may be the optimal time to collectively consider the ways we go about representing the world in our data and classrooms in hopes of maintaining our ability to adequately respond to the needs, concerns, and experiences of sociology students and the broader society.
Footnotes
Editor’s Note
Reviewers for this manuscript were, in alphabetical order, David Blouin, Royce Singleton, and Theodore Wagenaar.