Abstract
The authors provide a brief case study of a three-strategy approach for teaching undergraduate research methods that (1) incorporates active learning assignments and discussion-based learning, (2) integrates a cross-discipline and cross-method faculty guest discussion facilitators series, and (3) focuses on the challenges and rewards of conducting research. The authors propose that opportunities for faculty teaching and research collaborations may result from the implementation of these strategies in the context of dwindling institutional resources and increasing professional demands. Finally, the authors consider how involving students in active learning projects focusing on research, and encouraging more open and honest dialogue about the challenges, struggles, and failures faculty members experience when conducting their own research, generates a reciprocal learning environment that is enriching for both students and faculty members.
The American Sociological Association recommends that completion of a research methods course be a requirement for all sociology majors (McKinney et al. 2004). Schutt, Blalock, and Wagenaar (1984) offered eight specific goals for teaching such research methods courses. These eight goals include helping students to understand: (1) the necessity for research in light of differences between mainstream perceptions and presentations of social issues and social realities; (2) differences between inductive and deductive reasoning and the methods that follow from each; (3) researcher ethics and their critical importance; (4) gray areas and challenges associated with research and data conceptualization, collection, analysis, interpretation, and dissemination; (5) the limits of research in providing unequivocal “facts” and the importance of considering researcher assumptions, biases, and conflicts of interest; (6) critiques of measures, themselves, as concrete indicators in the social world and consideration of reliability and validity; (7) the difference between correlation and causation and the methods required to make claims about either; and (8) how to make, interpret, and critique claims about research generalizability. These goals are also consonant with McKinney et al.’s (2004) recommendations for effective pedagogies, which focus on the critical importance of empirical research, engagement with cross-cultural and cross-national social contexts, consideration of the overlaps and distinctions between sociology and other related social science disciplines, active learning techniques that enable student engagement, and provocation of students’ critical thinking skills.
Yet despite clearly outlined goals for effective sociology pedagogies, and research methods courses in particular, students tend to hold persistent unfavorable attitudes toward research methods (Macheski et al. 2008) that, paradoxically, may become even more negative upon taking a research methods course (Manning et al. 2006; Sizemore and Lewandowski 2009). One explanation for this drop in attitudes may be the lecture-style presentation of information when a more experiential and hands-on approach might be more effective in increasing student interest (Manning et al. 2006; see Sobal et al. 1981 for an exception). In a study of leaders in the discipline of sociology, the factor that most distinguished these individuals as teachers was their focus on actively involving students in research and exposing them to empirical research in class (Persell, Pfeiffer, and Syed 2008). Student lack of interest toward (and perceived lack of utility of) research methods courses may also be due to misunderstanding the goals and applications of research itself (Sizemore and Lewandowski 2009). So what are some other strategies for increasing student learning and engagement, particularly in courses that focus on research methods?
Although empirical research on specific strategies for teaching research methods to undergraduates exists (e.g., Schutt et al. 1984; Sizemore and Lewandowski 2009), relatively few discuss strategies for increasing student interest and learning in these courses (for exceptions, see Shostak et al. 2010; Takata and Leiting 1987; Waltermaurer and Obach 2007). The majority of studies published on teaching research methods courses focus on class exercises (e.g., Tan and Ko 2004; Taylor and McConnell 2001). When trying to explain why students hold negative attitudes toward research methods, psychologists have focused their attention primarily on individual students’ attributes and attitudes (e.g., Manning et al. 2006; Sizemore and Lewandowski 2009), paying less attention to the ways in which material is presented. Another way to think about students’ reactions to courses on research methods, however, is to focus on the learning environment and teaching and/or learning methods.
Scholarship on critical pedagogies, for example, posits that educational methods often tend to reproduce existing social inequalities, hierarchies, and norms (Braa and Callero 2006; Fobes and Kaufman 2008). One example of the reproduction of social hegemony is teaching practices that situate instructors and researchers as infallible experts on the subject matter under study and students as blank-slate repositories of knowledge (Fobes and Kaufman 2008). In research methods courses, adopting more critical pedagogical approaches holds the potential to directly engage students as active learners and reconceptualize researchers and teachers not as infallible experts, but also as learners: real people who sometimes struggle, fail, or do the wrong thing. Team teaching–type approaches focusing on particular research topics may allow more fruitful exchanges between and among students and faculty members (Frey and Nowaczyk 1982). Shostak et al. (2010) considered a multilevel team approach by integrating both graduate and undergraduate students in research methods seminars involving active collaboration in a research project.
Importantly, divisions between teaching and research are often structurally rewarded within the academy, with limited opportunities for developing collegiality (Baker 1986). Mounting evidence demonstrates intensifying limits to collegiality over the past 25 years as work demands across the areas of teaching, research, and service increase in a context of dwindling resources (American Association of University Professors 1994; June 2009; Miller 2010; Wright et al. 2004). Despite these strains on collegiality, in the case study that follows, we present strategies for developing more collegial and engaged communities of learners-scholars-teachers (Baker 1986; Macheski et al. 2008). Our case study also demonstrates the development of a course that attends to McKinney et al.’s (2004) strategies for effective sociology pedagogies and Schutt et al.’s (1984) goals for teaching research methods.
A Case Study on Teaching Research Methods
Despite their inherent limitations, case studies provide in-depth exploration that may synergize the formulation and development of more systematic, large-scale, and rigorous examinations of effectiveness (Searle 1999). In this article, we use a case-study approach to share our strategies for teaching research methods. These strategies allow bridging across disciplines and methods, reframing learning and teaching interactions between and among students, researchers, teachers, and colleagues, and may contribute toward the development of more active, engaging, and effective research methods pedagogies, as well as interdisciplinary and mixed-methods research insights and collaborations between colleagues. These opportunities are sorely needed, particularly at teaching-intensive universities and colleges. Our approach consists of a focus on: (1) active learning assignments and discussion-based learning, (2) cross-discipline and cross-method guest discussion facilitators, and (3) doing research in the real world. 1
We discuss the potential of this approach to foster cross-disciplinary faculty teaching and research collaborations.
To provide a bit of author and institutional context for this case study, we hold tenure-track assistant professor of sociology positions with 4/4 teaching loads at a regional branch of a large, research-intensive, public university. We completed doctoral training at research-intensive institutions and continue to pursue active research agendas despite daunting teaching loads. Our university is focused primarily on granting baccalaureate degrees to first-generation college students in a rural region of the Midwest on a commuter campus. The student body at our university currently numbers 5,551, with 500 to 750 students taking sociology courses each semester. The research methods sequence consists of a 300-level (junior) course in statistics and another in research methods. Although the only required prerequisite for taking the methods course is statistics, most students taking the methods course have also completed courses in introductory sociology and social problems. The course meets twice weekly for 75 minutes per session and is a three-credit class offered once per year; enrollments are generally between 7 and 12 students each semester. Although we both use survey research methods, one of us (Carla) uses in-depth interviews and ethnography, while the other (Christabel) uses experimental methods. Our social sciences department is methodologically and disciplinarily diverse, composed of an anthropologist, historians, a philosopher, a political scientist, psychologists, social workers, and sociologists whose methods span across those presented in methods texts.
Strategy 1: Incorporation of Active Learning Assignments
In planning our research methods courses, we chose an accessible text that provided a solid overview of both quantitative and qualitative methods and allowed us to engage with each of Schutt et al.’s (1984) goals for teaching research methods courses and McKinney et al.’s (2004) strategies for effective sociology pedagogies. Importantly, class periods were designed using the textbook as a foundation for building common ground on which learning was developed using a primarily discussion-based rather than lecture-based class format. We also created “action learning” or “learning by doing” (see Takata and Leiting 1987) assignments to bring students into more active researcher roles. As Sobal et al. (1981:403) noted, “Many students see sociology as abstract and theoretical, and involvement in experiential activities can help it seem more ‘real.’” Students began, for example, by reading about research ethics, discussing and debating examples of potential ethical breaches and dilemmas in research in class, and completing actual online institutional review board training for ethical research with human participants. All students received official Collaborative Institutional Training Initiative certification for social science research involving human participants upon completion of this first assignment. Completing this training not only provided hands-on engagement with research ethics but also resulted in the receipt of a certificate that may make students more employable to local social service agencies and researchers in the area. Students also developed research proposals, conducted peer-reviewed literature searches and reviews, designed and piloted interview protocols and survey instruments (which included operationalizing and developing measures for variables and attributes), and performed content analyses, focus group interviews, and ethnographic site observations as key active learning assignments in the course.
For one of these active learning assignments, for example, we adapted Stalp and Grant’s (2001) method for teaching qualitative coding to undergraduates using personal ads. Because the (working and commuting) circumstances of our students’ lives make group work outside of class exceedingly difficult, we adapted this learning tool into an in-class methods workshop. Prior to the in-class workshop, students were assigned a chapter reading on qualitative coding methods and completed homework that involved the creation of coding schema as well as some preliminary coding. During the in-class workshop, copies of actual personal ads written by men and by women from the local area were collected and distributed to pairs of students (all pairs received the same groups of ads). The student pairs read through all of the ads and then co-developed an abbreviated list of various themes they observed across the ads (e.g., self-promotion as employed, description of one’s own body type, use of humor). They next used the coding schema they had co-developed to independently code the ads, then compared their coding to establish inter-rater reliability estimates, working through discrepancies in coding to settle on their final codes. Once the smaller group work was completed, the class came back together to discuss the challenges they faced in their coding task as well as their process for implementing a resolution to any discrepancies.
Each group presented its coding schema and the general trends they observed by gender of personal ad author. The larger group discussion highlighted the common themes that most class members noted but also revealed less common coding categories that not all groups had discovered or chosen. Class members discussed how some of their findings seemed to contradict popular wisdom, such as men’s relatively frequent mention of active efforts toward weight management or reduction. As such, these findings prompted discussions of the importance of replication and the potential of research to transform or challenge popular conceptions, helping students better understand why they might see studies on topics or that seem like they could be answered through appeals to “common sense” alone. We are able to integrate material from across the sociological curriculum into the methods sequence by discussing: (1) structural inequalities and their connection to ethical breaches with human subjects, (2) the importance of developing survey measures and interview protocols that are inclusive of participants’ various identities, and (3) how to attend to power, influence, and coercion in dynamics between researchers and participants (see Waltermaurer and Obach 2007 on the importance of cross-course collaboration). In addition to developing active learning assignments for our courses, we decided to draw on the existing disciplinary and methodological diversity of our faculty to bring research methods to life for students.
Strategy 2: Incorporation of Cross-Method and Cross-Discipline Guest Discussion Facilitators
After covering methods as outlined in the text, a four-week faculty guest discussion facilitator series on “research in the real world” was developed. We have termed this strategy a “guest discussion facilitator series” rather than “guest speaker series” to highlight the discussion-based format of these visits, which encouraged open dialogue throughout the session. Faculty guest discussion facilitators included an anthropologist, a historian, a psychologist, and a sociologist (Christabel), whose diverse methodological approaches exemplified all of those covered in the text. Inviting colleagues from across the social sciences allowed for dialogue about research methods across a broader diversity of methodological and disciplinary approaches.
For this guest discussion facilitator series, an anthropologist colleague spoke about conducting in-depth interviews and ethnographic participant-observation research on resettlement processes among Bosnian Muslim women war refugees living in the United States. A psychologist colleague discussed laboratory behavioral observation and survey research with infants and their mothers to assess the effects of maternal sleep deprivation on infants and maternal-infant interaction. A historian colleague focused on archival cross-cultural research in France across such wide-ranging topics as rhabdomancy (the use of divining rods), hot-air ballooning, and spontaneous combustion. And the sociologist discussed laboratory experiments and survey research on the differential effects of women’s leadership on group task performance among women and men. Students prepared for each guest discussion facilitator by reading a chapter, article, and/or excerpt (often from the discussion facilitator’s own published work) as background information and then preparing three substantive questions about his or her research methods and/or the challenges and rewards of conducting research. Preparing for each guest discussion facilitator’s visit helped keep students actively engaged, as suggested by Macheski et al. (2008).
Strategy 3: Focus on Research Methods in the “Real World”
Because we wanted students to understand that neither researchers nor research is perfect (and that imperfection may result in research insights and innovations), we asked guest discussion facilitators to consider sharing background information about not only their research methods but their research motivations, processes, rewards, and challenges as well. To this end, guest discussion facilitators discussed modifications to one’s research intent or plans upon assessing the needs and desires of one’s participant population, the challenges of operationalization of variables in the research process, recording equipment placement to simultaneously capture behaviors of interest while reducing observational effects, and access to adequate research equipment (particularly among underresourced graduate student researchers).
For example, the anthropologist described her intention to learn Bosnian by traveling to Croatia yet shifting her research priorities once she determined that her displaced refugee participants’ requests that she teach them English as a second language were more critical (to participants, personally, and to establishing rapport with participants) than her original plan to travel abroad for her own language-learning experience. She also described lasting connections that were made with research participants and the ways in which research boundaries may become more permeable through ethnographic methods. She prompted students to consider what they might do if required to engage in these complicated personal and professional balancing acts and how research ethics are implicated across these tightropes researchers sometimes walk.
The historian challenged students to come up with historical “facts,” urging them to reconsider and more critically analyze ontological and epistemological evidence in piecing together narratives about historical events. He also provoked students to further consider their assumptions about cause-effect relationships, illusory correlations, and dialoguing about the importance of situating events within their social, historical, and cultural contexts. The psychologist presented video footage from her research with mothers and infants and introduced students to a behavioral coding exercise wherein they were asked to first operationalize what “counts” as an infant smile and to subsequently code three minutes of video for instances of infant smiling as a mother interacted with her infant in the laboratory. Once students completed individual coding, they then paired off to assess interrater reliability. Assessments demonstrated that the students could establish virtually zero interrater reliability, and she shared that the same had been true for her and research assistants during the coding process. This “failure” led to a very productive class conversation about research challenges. Some of these challenges included recording equipment placement, budget and access to adequate research equipment (particularly among graduate student researchers), operationalizing an infant “smile,” artificiality of laboratory environments, the Hawthorne effect, and logistical challenges to coding timed-interval video sequences.
The sociologist discussed the purpose of experiments—to test theory, not to generalize research findings—and how common misperceptions about the purpose of this research method (even among other researchers) can be challenging for those engaged in experimental research, particularly within sociology. Students viewed the “training videos” that participants had been shown in the experiment, and the class discussion brought to light issues connected with deception in research, the role of research confederates, and “failure” to find evidence in support of one’s research hypotheses. The sociologist argued that failing to find evidence in support of a research hypothesis (or an experimental manipulation not working as intended) is not necessarily a failure per se; rather, it is a new opportunity to take research in new, unexpected directions.
Fostering Faculty Teaching and Research Collaborations
Each of these guest discussion facilitators offered research methods students multiple opportunities to consider and dialogue about how research unfolds in actual practice, but these class visits did more than that. They also provided a rare opportunity for faculty members at a teaching-intensive university to discuss and engage with one another’s research, current and past, in critical and productive ways. Through one faculty member’s (Christabel’s) visit as a discussion facilitator in another’s (Carla’s) class, we each began to realize potential overlaps in our own research trajectories and initiated some collaborative projects (e.g., Rogalin and Pfeffer 2012). Although the insights enabled through these class guest discussion facilitator sessions marked the beginnings of our research collaboration, we suggest that engaging with one’s cross-disciplinary and/or cross-methods colleagues in this sort of pedagogical activity may hold institutionalized (rather than merely serendipitous) potential to foster interdisciplinary and multiple-method research collaborations or insights for others as well. We consider this cross-discipline and cross-method discussion facilitator series a critical pedagogy tool for both students and faculty members (particularly faculty members at institutions with relatively few structural opportunities or encouragement for colleagues to dialogue about their research). Furthermore, we believe this tool holds generative potential for pushing beyond professional productions and posturing to engage in more open and earnest conversations with colleagues on the challenges, disappointments, and failures we experience in our research.
A key aspect of our approach that we wish to stress is that teaching and research collaboration should be just that: fair and equitable collaboration between colleagues who are cognizant of the potential for exploitation in the context of dwindling higher education resources and increased work demands. As such, we have emphasized reciprocity in both the teaching and research aspects of our faculty collaborations, and each of us has given, and/or offered to give, guest lectures in our colleagues’ courses, as well as provided additional forms of teaching and research support such as proctoring exams and consulting on teaching and research questions and dilemmas. Offering to guest lecture in others’ courses further provides opportunities for sharing our research-based topical knowledge beyond research methods courses and across disciplinary boundaries as well. With regard to research collaboration, we trade lead author status on collaborations, alternating primary responsibilities on the basis of topic knowledge, availability, as well as a sense of fairness and equitability. We would strongly encourage attention to fairness and the potential for exploitation when structuring faculty teaching and research exchanges and collaborations.
Preliminary Feedback and Next Steps
All of the students (n = 10) who attended the first day of class completed and passed the course. Ninety percent of students (n = 9) completed anonymous online end-of-semester course evaluations, a remarkable, nonincentivized completion rate for a non-online course. Student evaluations of the guest discussion facilitator series were largely positive in both anonymous qualitative and quantitative evaluation measures. As one student noted, “I really enjoyed the guest lectures, Not only did I get to understand different areas of research, but I also got to meet different teachers that I will probably have a class with.” All students selected “agree” or “strongly agree” (with nearly all selecting “strongly agree”) for the question “The guest discussion facilitators contribute significantly to this course.” One student wrote: “Guest lectures were amazing and helped to bring the semester together in ‘real’ examples.” Other quantitative measures of students’ levels of active engagement with the course were also assessed, with most items scoring significantly higher than prior semester medians for this research methods course, despite all students selecting “strongly agree” on the measures: “This course has been challenging” and “This course has effectively challenged me to think.” All students selected “agree” or “strongly agree” (with most selecting “strongly agree”) with the statements “One strength of this course is the classroom discussion,” “My instructor encourages questions and expression of ideas,” and “I actively participate in class activities and discussions.” Encouraged by this initial feedback, we have received institutional review board approval to conduct a five-year longitudinal study involving broader and more systematic evaluation of this course.
In this article, we have presented three strategies for teaching research methods: (1) active learning assignments and discussion-based learning, (2) a cross-method and cross-discipline discussion facilitator series, and (3) focus on the challenges and rewards of conducting research in the “real world.” We further suggest that engagement with these three strategies may generate enhanced opportunities for faculty teaching and research collaborations. Although guest speakers may not improve student engagement outcomes across all student levels or every type of course (e.g., Mooney 1998), the pedagogical methods described herein, all of which emphasize discussion- based over lecture-based forms of instruction, may hold promise for increasing the notoriously low engagement of advanced students taking a required research methods course. The active learning assignments and guest discussion facilitator series we discuss in this case study also reflect McKinney et al.’s (2004) recommendations for effective pedagogies and Schutt et al.’s (1984) goals for teaching research methods, as previously outlined.
A “research in the real world” guest discussion facilitator series requires us, as faculty members, to shed the persona of infallible experts—and to acknowledge that we make mistakes, that we do not know everything, and that sometimes our research simply does not “work.” Furthermore, in the guest discussion facilitators series format we have discussed, this risky shedding takes place publicly, in front of both students (who may show up in our classes in the future) and our colleagues (including those who may make decisions about our tenure and promotion). Nevertheless, we believe that situating ourselves as real people doing real research has many positive benefits, especially for our students. The guest discussion facilitator series method outlined in our case study provides one tool for combining teaching and research to increase overall faculty productivity. Given increasing demands on faculty members, and heightened feelings of being pulled in many contradictory directions at once, we suggest that strategies for reciprocal teaching arrangements with colleagues, such as the guest discussion facilitators series outlined in this case study, may provide additional avenues for more efficiently combining teaching and research obligations in ways that still provide enriching and engaging learning opportunities for students.
Next steps in this project include continuing data collection and assessment focusing on student outcomes. This study includes repeated measures of student learning, engagement, and comfort across their tenure in the research methods sequence over a study period of five years. Assessment measures 2 have been adapted from Sizemore and Lewandowski (2009), as well as co-developed by ourselves, to be used in these efforts. Furthermore, we have added a faculty interview component to assess faculty members’ perceptions and experiences with regard to their engagement with the guest discussion facilitator series as well as any independent or collaborative research projects that are initiated or supported through their involvement with the guest discussion facilitator series.
We would suggest that future implementation and assessment of the methods described in this case study adopt an interrupted time-series measurement of students’ research methods knowledge, comfort, and engagement using a pretest-pretest-posttest design: (1) on the first day of class, (2) after completion of the standard “textbook” portion of the course, and (3) again after completion of the guest discussion facilitator series component of the course. Further replication might also include a comparison group without the guest discussion facilitator series and/or inversion of the sequencing of strategy 1 (learning about research methods using active learning assignments) and strategies 2 and 3 (guest discussion facilitator series with a focus on doing research in the “real world”) to determine order effects as a potential threat to internal validity. Another option for implementation would be to integrate guest discussion facilitators with the method being presented in the text, though doing so may make independent assessment of the strategies more difficult or require additional data collection points in the time series.
Assessing student outcomes using a time-series approach may allow greater understanding of the sequence and process of student learning and engagement, as well as provide evidence in support (or not) of the guest discussion facilitator series on research in the “real world” strategies in particular. It is our intention that the collection of student outcome data using this time-series approach may allow us to develop a future model for teaching research methods through isolating particular components of the model (currently conceptualized as “strategies”) that evince positive student outcome change scores. Our intention herein is to compel consideration of a particular set of strategies for teaching research methods so that others might also consider implementing and evaluating them across a broader and more diverse group of students, faculty members, courses, and institutions than is possible through our lone efforts. We are hopeful that the creation of more active, dialogic, and cross-disciplinary research methods courses holds the potential to generate reciprocal learning environments that are engaging and enriching for both students and faculty members.
Footnotes
Acknowledgements
We wish to thank Kathleen S. Lowney, anonymous reviewers at Teaching Sociology, Cheryl DeLeón, and Laura Hirshfield for their helpful feedback on earlier drafts of this work.