Abstract
We present our approach to a qualitative research methods course to prepare practitioner-scholars for their dissertation and independent research. We explain how an instructor’s guide provides consistency and rigor, and in-class activities to scaffold learning, and helps faculty connect the content to students’ out-of-school lives. We explain how reflection is used to develop students’ reflective practice and enable us to improve the course in both the short and long term. We argue the necessity of having tools that ensure our teaching philosophy translates into a consistent curriculum and instructional approach, maximizing the chances that faculty facilitate our students’ development into practitioner-scholars.
Keywords
In the current educational climate, educational leaders are expected to be able to use existing research and data derived by their own organizations to make informed decisions in their districts and at their school sites (Barnett & Muth, 2008; Coburn & Talbert, 2006; Coburn, Toure, & Yamashita, 2009; Marsh, 2012). More specifically, they are encouraged to become practitioner-scholars who are positioned to engage in inquiry as practice, to see scholarship and practice as inseparable and to be situated in critically oriented inquiry (Jenlink, 2014). To do this, they are expected to regularly use research strategies in their own settings to solve problems of practice (Barnett & Muth, 2008), connect theory to practice (Horn, 2002), gain insight into their own practices to facilitate growth inside of their organizations (Hebert, 2010), and increase equity and access for students who consistently experience inequitable and often unjust conditions (Jenlink, 2014). Yet there are unique challenges that come along with preparing practitioners to become practitioner-scholars who are able to use existing research, combine their experience with theory, learn to investigate and reflect upon the ideologies that shape their own beliefs and practices, and/or conduct independent research to solve problems of practice that are embedded in the culture of their institutions. Some of these challenges, in our experience, come from the fact that some portion of the doctoral students with whom we work aspire to careers in practice but do not see the utility of research methods in their work. This may be due to the proverbial research–practice gap in which research is often seen as something directed toward other researchers, those who live in the Ivy White Tower, rather than those who work in “the real world” (Duke & Martin, 2011; Robinson, 1998). Or, this may be due to the fact, as Wagner (2007) argues, that educators have not been trained to analyze and solve problems. Wagner suggests that aspiring leaders are taught more about management than about how to develop and use analytic skills to solve problems. Others point to the fact that aspiring leaders are more likely to be taught to handle technical challenges, those with known answers, rather than swamp problems, those without clear points of origin or obvious solutions (Murphy, Moorman, & McCarthy, 2008; Schall, 1995).
Others have suggested that the challenges that exist in equipping graduate students with the knowledge and skills they need to use and engage in research to solve problems of practice are resident in the approaches taken by graduate schools (Barnett & Muth, 2008; Bernauer, Semich, Klentzin, & Holdan, 2013; Brown-Ferrigno & Muth, 2012; Gutherie, 2009; Pemberton & Akkary, 2010). Gutherie (2009) asserts that many programs serving practitioners have focused too much on developing researchers rather than developing practitioners with a sophisticated skill set that allows them to navigate their practice-based world. Bernauer et al. (2013) offer that the ongoing debate in the methods community over the best way to teach methods has left doctoral faculty to decide for themselves how to “facilitate student learning of research methodologies” without any empirical evidence as to “best practices” (p. 177). Thus, doctoral programs, among other things, approach the teaching of research methods using cohort models (Barnett & Muth, 2008), learner-centered instructional strategies (Browne-Ferrigno & Muth, 2012), group-constructed or individually driven pilot studies, and action research approaches (Bernauer et al., 2013).
We locate our work in this broader discussion over the best way to teach doctoral students research methods and, by extension, the knowledge and skills we believe they need to use research, draw on theory, and engage in independent research to solve problems of practice in ways that actively acknowledge and allow them to address their own biases as well as the ideologies that shape the culture and practices within their institutions to the detriment of their students. We do not pretend to solve the debate over the best way to equip doctoral students. Instead, we tackle another aspect of teaching methods: ensuring that all students are afforded access to high quality instruction and the same content knowledge and skills irrespective of when they take their methods courses and who happens to be their professor. In a way, we build on Browne-Ferrigno and Muth’s (2012) work in which they offer their thoughts on learner-centered instructional strategies by suggesting it is not enough to have a shared teaching philosophy among the students and faculty. We argue that it is essential to have mechanisms in place that ensure that the teaching philosophy espoused by the program and course consistently translates into the curriculum and instructional approaches used by each person teaching the course and experienced by each student taking the course.
In this article, we discuss the approach we have adopted for a qualitative inquiry course that is intended to support doctoral students who are practitioners as they develop into practitioner-scholars. Our approach is focused on ensuring that the instructional approaches used each time the course is taught are consistent with our teaching philosophy, irrespective of the faculty member teaching the course, and that the curriculum experienced by the students is consistent across classrooms and over time. In addition, we discuss the ways in which we have incorporated what we have learned about the needs of our students into the course design and delivery.
Thus, in the remainder of this article we describe our approach. We begin by providing an overview of the context within which our work takes place, our teaching philosophy, and the structure of the course. We then turn our attention to the approach we take to ensuring that the faculty teaching the course enacts the teaching philosophy underlying the course and the expected curriculum, through the use of an instructor’s guide (IG). Next we discuss the way we use an ongoing reflective process to create coherence within the faculty teaching the course, build a critical reflective practice with our students, and improve the course both in the short and long term. We conclude by discussing some of our thoughts about next steps for improving our efforts.
Overview of the Program
To understand the particular context within which we work, and thus our rationale for arguing for a consistent curriculum and instructional approach, it is important to provide an overview of the program and the research methods curriculum students are expected to complete to develop into practitioner-scholars. Our EdD program enrolls between 100 and 200 students each fall semester in a 3-year program of study. Our students are teachers, counselors, principals, and other educational and non-educational leaders and aspiring leaders. In their second year, students are asked to select a topic they are interested in pursuing for their dissertations. They do this by attending a summer conference in which they are introduced to members of the faculty who are serving as “thematic dissertation chairs.” Faculty members propose themes or topics that ultimately guide students’ dissertation foci, and students then select the thematic group they wish to join. Students are then brought together in “thematic groups” where they work with their chairs throughout their dissertation experience.
To support students through the dissertation process and to help them acquire the knowledge and skills to use research, combine theory and practice, and engage in independent research that will position them to address the challenges their students face in their educational contexts, we provide our students with two research methods courses in the curriculum. All students take the first methods course in the spring semester of their first year. This course is designed to introduce qualitative, quantitative, and mixed-methods approaches, as well as how independent research is initiated through a search of the literature, identification of a research problem, and development of research questions. Students then take their second methods course in the fall of their second year. Given the length of the program and the fact that our students will use either qualitative or quantitative methods for their dissertations, we offer students a choice between a qualitative and a quantitative class. As it is assumed that they will be using what they learn in this class in their dissertation process, students make their selection based on the guidance of their dissertation chairs. In this article, we focus on the second research methods course, and specifically the qualitative methods class.
This second methods course is offered both as a 15-week course and as weekend versions on two branch campuses. These weekend versions mirror the content of the 15-week class but take place during 4 weekends spread across the 15 weeks of the term. Given the number of students enrolled in the course, we typically have between six and eight sections of the course being offered at one time. The large number of sections causes us to have a combination of full-time tenure track, full-time clinical, and part-time clinical faculty teaching the course. In addition, it is often the case that the faculty teaching the course changes from year to year.
As the two members of the faculty primarily responsible for ensuring the methods courses in the program are positioned to prepare our students to use research, combine theory and practice, and engage in individually directed research projects to address questions of equity and access, we have responsibility for determining the teaching philosophy that will be enacted across all class sections in a given year and over time. We have adopted a teaching philosophy heavily grounded in sociocultural and constructivist learning theories. What this means is that we want students to construct meaning together while we serve as their guides, anticipating their zones of proximal development (ZPDs) and providing appropriate scaffolds such as feedback and other cognitive tools. These choices align with Tharp and Gallimore’s (1988) concepts of assisting performance. We also demand that students engage in meaningful learning, being intellectually active in their pursuit of the skills and knowledge associated with research methods. Here we draw from Anthony (1996) for her concepts of both constructivist theory and the cognitive strategies necessary to be intellectually active learners. It is important to us that this philosophy translates into very specific pedagogical acts on the part of the faculty teaching the course. In addition, we are also responsible for determining the curriculum that will best support our students’ ability to acquire the research related knowledge and skills they are expected to master. Given that this is only intensive research methods course that prepares students for their dissertations and independent research, there is little room to experiment with the content or sequence of the topics covered. It is essential that our curricular choices are adopted and enacted by each person teaching the course. To ensure coherence and consistency in both teaching and curriculum, we have developed an IG for the multiple faculty who teach this class. In the next sections, we explain what the IG is and how it is used as well as the way we use ongoing reflection to build our students’ capacity to critically reflect and to engage in short-term and long-term course improvement.
Instructor’s GuideAs described above, the IG, which is saved in a shared cloud-based folder for this course, was developed to enable multiple faculty to teach this class. It is an electronic document comprised of the syllabus for this course and week-by-week recommended approaches to the class. These week-by-week recommendations are interspersed through the syllabus so that individual faculty can track the progression of the course in a way that is consistent with the way the students experience the course flow. We supplement the document with written comments that capture our thoughts, reminders, and warnings of the potential mistakes that one can make and recommendations for how to avoid those mistakes. These comments are usually located in the weekly activity section of the IG and are derived from our experiences teaching the course in the past. They allow for the IG to be a living, constantly evolving tool for faculty. We are using this IG to ensure that all students, irrespective of their instructors, are experiencing high quality instruction and a coherent curriculum. The guide does this by providing (a) the syllabus topics and learning objectives with suggested week-by-week activities and expected in-class resources, (b) consistent scaffolds for faculty to use with students to practice the skills in incremental and developmentally appropriate ways, (c) suggestions for faculty regarding how students can make connections to their out-of-school lives and work within and beyond the class, and (d) coherence in the curriculum the students experience. The IG also ensures the continuity of the learning experience over time although, as discussed in the next section, we reflect on and make changes to the course to best meet the students’ needs. In the remainder of this section, we outline how the IG is used so as to provide practitioner-scholars a solid foundation as they learn qualitative research methods.
Course Coherence and Quality
Although we make every effort to hire faculty who have experience as practitioner-scholars and can relate to our students, the people who are best positioned and have the methodological training and experience to teach this course tend to have been or currently are professional researchers or evaluators as opposed to practitioner-scholars. The faculty members live and breathe research methodology and are regularly conducting research as a part of their everyday practice. This sometimes means that they do not understand that practitioner-scholars need to develop (a) research methods skill sets and (b) the habit of mind that inquiry can be instrumental in solving complex equity-based problems of practice. The IG is intended to ensure that no matter who the faculty members are, we arm them with the tools and resources to help students develop the skill set necessary to be practitioner-scholars. As the lead faculty who have worked in this program for 7 and 3 years, respectively, we know our students well and understand the context within which they are learning research methods. As such, we have compiled resources so that new or part-time faculty are not left to create the course from scratch or left to guess our students’ ZPDs (Tharp & Gallimore, 1998) or learning needs. While the IG does not dictate that faculty must implement the course in a lockstep fashion, it provides them with the tools and resources to support practitioner-scholars as they take on this new learning and, in so doing, ensure high quality in the teaching approach we expect to see in our course.
One of the ways we have tried to ensure that the faculty follow the syllabus, and by extension our philosophy for this course, is by providing them with a week-by-week summary of what can and should be covered in each class session. Each week has a list of learning objectives we expect the students to walk away having addressed. For example, Figure 1 is a list of learning objectives for one class session on conducting interviews.
Example of learning objectives.
The IG also includes discussion questions faculty can use or adapt to help students meet the learning objectives. Here is one example of a set of discussion questions on observational protocol design (Figure 2).
Example of discussion questions.
As the reader will notice, it is suggested that students first engage in an independent write, drawing on the readings for the week, then to engage in small group discussion with their peers, and finally to come together for a whole group discussion on this topic. This suggested structure is intended to help our students work both independently and collaboratively to construct their knowledge, which is aligned with our constructivist (Anthony, 1996) and sociocultural perspective (Tharp & Gallimore, 1998) to teaching and learning.
In addition to providing a week-by-week summary of the content to be covered, the IG also includes activities intended to support students’ learning. In comment boxes, we provide suggestions for faculty (Figure 3).
Example 1 of suggestions for lesson implementation.
The reader will notice that this comment box also foreshadows for the instructors what students will be expected to do in a later class session.
Figure 4 is an example of an analysis exercise suggested in the IG. As the reader will notice, this particular week’s exercise is building on work the small groups did in a previous class session.
Example 2 of suggestions for lesson implementation.
The IG is also a good place to discuss how much time faculty should plan to spend on any given activity. These suggestions help to ensure consistency across sections and to communicate to faculty what content or in-class activity we deem more important. See Figure 5 for an example.
Example 3 of suggestions for lesson implementation.
Finally, the IG serves as a place where faculty members can note the need for changes to the course. As issues or challenges arise in our respective sections, we type suggestions for revisions in this communal document so as to inform future revisions. As will be further discussed below, these additions to the IG help us to improve the course on a continual basis. Because this will be discussed in more detail below, in this section we focus only on the process used to make suggestions in the IG. Figure 6 offers a few examples of faculty making mention of issues to address in future iterations of the course.
Example of instructional thinking.
In short, the IG allows us to have a document in which to communicate our collective ideas of what can and should be done to deliver the content. The use of this guide is particularly important when working with practitioner-scholars because it allows us to be specific when recommending activities to faculty, who may not have a clear understanding of our students’ learning needs and ZPDs. The recommendations in the IG come from our clinical knowledge of the kinds of activities that seem to have worked or did not work with our students in years past. We believe that every subsequent year’s IG is better aligned to the needs of our students than the previous version because it builds on our experiences teaching this class to our students.
Scaffolding
As previously stated, the suggestions and resources in the IG are focused on how to scaffold the learning for our students who are novice researchers. By scaffolding, we are referring to Tharp and Gallimore’s (1988) use of the analogy for how teachers provide selective assistance to students to move them through their ZPDs so that they are ultimately able to acquire new knowledge and/or skills. Three important means of assistance are modeling, questioning, and feeding-back (Tharp & Gallimore, 1998). The IG, including the suggested in-class activities and discussions, helps to ensure that all three means of assistance are incorporated into the faculty’s practice. The major assignment in this class is a pilot study, which requires students to move through the research process from beginning to end over the course of 15 weeks. Much like other programs, our students identify a problem, conduct a brief review of the relevant literature, develop research questions, sample, develop instruments, and collect and analyze data. Along the way, we have incorporated in-class activities and out of class assignments that scaffold and support students’ skill and knowledge development all of which are clearly outlined in the IG course schedule (Figure 7).
Course schedule.
As the reader will notice, we move progressively through the research process with assignments due throughout the course. Prior to entering into the field we address interviewing and interview protocol design. Once their protocol has been developed and vetted by instructors, they are given the opportunity to begin conducting interviews. During this same period of time, we teach them the basics of conducting observations and explore the types of documents and artifacts that they might collect in relation to their research questions. The students’ window of opportunity for data collection is between Weeks 7 and 10. We provide them with this 4-week window so that they can determine the best approach to and timing for collecting their data.
Our goals for the pilot study are to ensure that students conduct a study that is rigorous, and feasible, allowing for the realities of our students’ professional lives (most of our students work full-time so taking off work time to collect data is often a challenge). In terms of rigor, our goal is to ensure that they fully enact the skills associated with interviewing, observing, and analyzing data (but do not require the collection and analysis of documents or artifacts). By requiring that they complete three interviews and three observations, each of which must last at least 1 hour, we are enabling them to create a corpus of evidence that lends itself to triangulation of these two methods. The data corpus is also sufficient to allow for the emergence of empirical and analytic codes and initial themes and findings. While an in-depth study would require more qualitative data, including artifacts, the data they do collect is rich and descriptive enough to provide them with a legitimate experience in data collection and analysis. Thus, we seek to balance rigor with feasibility.
To further ensure that students’ pilot studies are feasible, we impose certain constraints on the study they will conduct. For example, we require that their research questions are conducive to both data collection methods. We limit the number of research questions to one or two (no more). We also expect that students will engage in convenience sampling. They may choose to collect their data at their own work site, for example. By allowing them to do this, we simultaneously improve their ability to complete data collection—because they do not need to leave their place of employment during the work day to complete assignments—and create other challenges for students (e.g., they may not know how to “leave the field” and they face ethical considerations that they might not face if they were collecting data in “neutral sites”). For example, as Brooks and Normore (2015) contend when referring to studies of educational leadership, our students’ pilot studies must attend to “relational, power and gatekeeper dynamics that influence the study” (p. 800). Our students’ involvement in their workplaces brings to the fore many of the issues related to the roles of the researcher, power dynamics, access and ethics, and credibility/trustworthiness of the findings. These issues allow us to pose questions to our students about how they can reconcile their roles as both practitioners and as scholars/researchers, and the importance of reflecting carefully on how their roles might pose ethical or validity issues. Through this, our students learn through experience what is meant by “the qualitative researcher is the primary instrument of data collection” (Merriam, 2009, p. 14), the importance of considering entry and access (Bogdan & Biklen, 2007), and ethics (Glesne, 2011) in this very humanistic work. Students also learn how and why reflexivity becomes of critical importance when conducting qualitative research (Creswell, 2014).
Given that much of the work (i.e., data collection and analysis) is to be done outside the confines of the classroom, the IG suggests in-class activities and discussion questions that are intended to support this large amount of independent work with appropriate scaffolds. Some of our scaffolding considerations have been the need to (a) vary the complexity of the readings, (b) show examples from an actual study, and (c) space out the assignments in such a way that scaffolds students’ learning through the provision of faculty feedback. An understanding that our students are working professionals who are simultaneously novice researchers drives all of these decisions.
Varying the complexity of the readings
First, we recognize that our students take up readings in the field of research methodology in very different ways. For many, complex texts are overwhelming and irrelevant to them as practitioner-scholars. For some, such texts are a welcome addition to their doctoral studies. Given the varying degrees of interest in grappling with the texts and different ZPDs, we have purposefully assigned course readings with varying levels of difficulty. For example, our main texts are Maxwell’s (2013) Qualitative Research Design: An Interactive Approach and Merriam’s (2009) Qualitative Research: A Guide to Design and Implementation. Whereas Maxwell is a harder read, Merriam’s text is more student-friendly and accessible. Students are expected to read all the reading, including the more complex, but the inclusion of the more student-friendly texts ensures that they will understand the content. Students have provided feedback that they enjoy Merriam’s writing style, while appreciating Maxwell’s more theoretical perspective.
In addition, the texts we use to teach data analysis also address varying levels of student interest in and tolerance for complex texts. For a step-by-step, more procedural treatment of data analysis, we use selections from Harding’s (2013) text, Qualitative Data Analysis From Start to Finish. To supplement, we use Miles, Huberman, and Saldaña’s (2014) text, Qualitative Data Analysis: A Methods Sourcebook, to be both comprehensive and complex. We have found that some students both desire and find the more complex texts necessary for their education. Other students, however, appreciate more user-friendly texts that take into account their level as novice researchers who appreciate a procedure to follow, even while they are exposed to the complex, overarching process. In short, we have tried to be thoughtful about our choice of required readings, recognizing that our students come to us with varying levels of knowledge and comfort in research methodology. Providing students with these different texts allows students to gravitate to the ones they find most useful while also providing them with a breadth of perspectives in qualitative methodology. By not shying away from the more complex texts, we believe we are upholding the rigor of the course, something necessary for practitioner-scholars.
Modeling: Showing examples from an actual study
Another consideration that serves to scaffold students’ learning is the use of a single example of a qualitative study. We have found that using one common “class” example enables faculty to model the different steps of the research process. Currently, we are using a large-scale qualitative evaluation managed and conducted by the first author. Although we recognize that an evaluation is not comparable with all types of qualitative research, including problem-solving approaches (i.e., action research) that many of our students will engage in, this study was chosen because we have examples from each stage of the research process—everything from raw, handwritten notes to different steps in the analysis process including memos and the researcher’s reflections on the data. As such, it serves as a good way to show how each piece fits together to create the whole. Such a detailed resource is hard to come by, and yet necessary, if the aim is to model the steps for students. And, consistent with the urging of qualitative methods scholars (Fontes & Piercy, 2000; Onwuegbuzie et al., 2012), we believe it is important to use real data to teach qualitative methods. Using one example allows students to have a common study to discuss and with which to explore the methodological concepts they are collectively learning in class and in the course readings. 1
We introduce the common study during the third week of our 15-week course, 2 presenting a brief, three-page background document that includes the context of the study, the research questions, the conceptual framework, and a brief summary of the data collection and analysis methods. In our subsequent in-class activities, we provide examples relevant to each unit’s topic, asking students to engage in discussions or practice techniques using this common study. For example, when we cover interview protocol design, we provide students with one interview protocol from the study to use as a starting point for their in-class discussions. Students work together to identify the information the protocol might contain (i.e., an introduction, verbal consent to be audio recorded, a set of questions, transitions from topic to topic) and what good questions look like (i.e., how to word questions that are aligned to a conceptual framework, are open-ended, single-barreled, and not leading). After small group discussions on the common protocol, students are asked to create their own interview protocols for their pilot studies as a course assignment. It is only after we have scaffolded this skill through the use of the common example that we expect students to create their own.
In addition to providing examples of each step, the use of a single common study throughout the course allows students to understand how all the research design elements are connected to each other; that all research design decisions are driven by the research questions, which are framed in the context of the research problem under examination. For example, the research questions focus on the role of leadership in improving instruction. This in turn is reflected in the conceptual framework, which includes different aspects of leadership including vision (Is there a collective vision and a commitment to that vision?), distributive roles (Is the vision forwarded by different people?) and facilitation (Are people receiving the help/support they need?). Faculty members ask students to identify how the conceptual framework reveals itself in the interview protocols. Consequently, we impress upon our students the need to consider the key concepts the researcher has identified—usually through a thorough review of the literature—when developing data collection protocols. In a later part of the course, students practice using the researcher’s conceptual framework as a “start list” of a priori codes when they begin to learn how to code and analyze qualitative data. For example, students might create the codes Leadership:Vision, Leadership:Distributive, and Leadership:Facilitation as a way to begin analysis. It is important to stress that the conceptual framework is just a way to begin analysis. We do spend instructional time modeling how a priori and emergent codes (Harding, 2013; Merriam, 2009) are often used in concert during data analysis. We have found that for practitioner-scholars who have less time to complete a dissertation and even less time when they are conducting research in their workplace contexts, having a clearly defined and detailed conceptual framework facilitates the research process, even while they stay open to emergent codes.
Using a common example of a study, then, enables us to provide a common set of tools and products for students to examine and discuss in small groups during class. Only after working through the collective examples, discussing the decisions made by the authors, and reflecting on how these decisions are aligned with qualitative methodology tenets, such as the need to explore meaning-making, process, and a holistic orientation, do we ask students to apply this knowledge and create their own tools and products for their pilot studies.
The provision of feedback
As novice researchers, our students are, for the most part, learning all aspects of a systematic research process for the first time. We believe faculty feedback on work products is critical to our students’ learning and serves as yet another scaffolding mechanism. We see feedback as a critical component of our instructional approach and see the course assignments as opportunities to provide detailed, helpful feedback to facilitate students’ learning. Therefore, when designing the course, we strategically spaced out the “graded” assignments (those bolded in Figure 7) in such a way as to allow faculty time to read carefully, provide useful feedback, and return assignments in a timely manner to influence students’ subsequent work products. As the assignments build on each other, each seeking to scaffold the next, it is incredibly important to us that we structure the provision of feedback to students. The list of assignments and their order are found in Figure 8.
Assignments table.
As an example, students’ first assignment is to write a paper that outlines the context of the research problem, a brief literature review, and proposed research questions. This assignment is due 2 weeks before students are asked to complete the research design matrix, which outlines their sampling and data collection choices. This assignment spacing allows faculty a 1-week window to carefully read and provide feedback to all students, thus leaving time for students to incorporate their professor’s feedback in their research design matrix assignment prior to submitting it. Recognizing the amount of time necessary for faculty to do a careful read to provide constructive feedback is an important part of structuring the course and spacing assignments accordingly. All of these decisions are mentioned in the IG to ensure that faculty understand the value and need to provide feedback to students to help inform their next assignment.
Connections to Their Out-of-School Lives
To ensure the pilot studies are meaningful and useful to students, we make every effort to connect them to both their lives as students and as practitioner-scholars. As students, they are interested in beginning work on their dissertations, even though they will not have formally started the dissertation process when they are enrolled in the second methods course. Faculty members encourage students to explore the general topics they are considering for their dissertation, but they may not seek to answer the specific question they may take up for their actual dissertation. We do this so that they can begin to explore the area they are interested in investigating without requiring that they go through the institutional review board (IRB) process and without compromising their dissertation process by collecting data from people who might turn out to be ideal participants for their actual dissertation projects. Students are able to use the course assignments as a test or trial run of what they might ultimately end up doing for their dissertations. We have heard from many students how the pilot study process has enabled them to better focus the study, to refine their protocols, and begin to think of ways the data can ultimately be analyzed.
The Role of Reflection
An essential part of our approach to this course has been our use of a continual cycle of improvement. We use a reflective process that both enables us to support our students as they develop their own reflective process and allows us to improve the way we are teaching and the content we provide to our students within a given semester and across time. Here we discuss each of these ideas. First, we discuss the way we set out to build reflective practitioner-scholars who are positioned to engage in critically oriented inquiry (Jenlink, 2014) and investigate their own practices (Merriam, 2009) so that they are able to facilitate learning (Rodgers, 2002) and growth inside of their organizations (Hebert, 2010). Then, we turn our attention to our approach to improving the course during a given semester and finally, from one semester to the next.
Building Reflective Practitioner-Scholars
As mentioned above, qualitative research, by its very nature, recognizes that the researcher is the instrument for data collection and analysis (Maxwell, 2013; Merriam & Tisdell, 2016). Thus, the method demands that researchers grapple with their biases, their assumptions, and beliefs, and the ways in which those biases, assumptions, and beliefs will shape every aspect of the research process—from the research question(s) they pose to the conclusions they draw from their data. Thus, in addition to the content they read about the ways they are expected to address the “more subjective side of [their research] journey” (Bogdan & Biklen, 2007, p. 122) by writing reflective field notes, memos, and observer’s comments during data collection and analysis, and the responsibility they have to address their ways of ensuring the credibility and trustworthiness of their pilot study findings, we have built additional reflective activities into the course. As part of their major assignments, they are asked to reflect on what they are learning about the research process, what they still need to learn, and what the implications are for their dissertation work once the course is over. In our feedback to them, we ask them to explore their thinking further, moving from description to critical analysis of what they are learning (Jay & Johnson, 2002). We believe this process will also enable them to be positioned to engage in critical inquiry (Jenlink, 2014) regarding the way their own beliefs and assumptions influence their success in improving the organizations they lead and the ways in which larger societal ideologies influence the policies, programs, and people that support or impede learning for the children with whom they work. In addition, we ask them to use descriptive feedback (Rodgers, 2006) at the end of class so that they are able to gauge what they have learned during a given class time, what they believe impeded their learning, and what supported their learning. This activity is intended to support the students’ ability to be metacognitive about their learning process, drawing attention to the ways they learn and what they and their instructor do that impedes and supports their learning. These reflections include discussions of power and position in the classroom. These activities allow students to practice the act of critical reflection, a necessary skill for practitioner-scholars.
Within Term Reflection
In addition to providing the students with reflective and critical reflection opportunities, we engage in weekly faculty meetings. These meetings are mandatory. We make it clear when faculty members agree to teach the course that they will be expected to attend. We use this time to discuss our own understandings of the content and skills we are teaching, review the approaches we have taken in previous terms to teaching the particular week’s content, and to contemplate alternative approaches given the experiences we have had and the current composition of the faculty. This time is essential for us as we are able to examine our own thinking and our own biases, ensure that we have agreement among ourselves about the concepts and skills, and examine our own practice and the way our own choices support or impede learning. We are also able to explore new and different ways to approach the same content as we bring new faculty on board. For example, during a recent term, we grappled with a challenge that we continually face in helping our students understand the concept and role of a conceptual framework. 3 To introduce the concept in previous terms, we shared examples of previous doctoral students’ conceptual frameworks. We walked students through these examples, telling them what the strengths and weaknesses were. In this term, one member of the faculty experimented by using a conceptual framework graphic she had designed. She provided it to students. She did not explain it to them. Instead, she asked them to examine it. Unlike her previous experiences with this lesson, she found students were able to unpack the graphic and discover how it served as a conceptual frame that could drive a study design, data collection, and analysis. At the next faculty meeting, she brought the idea to the rest of the faculty. The faculty discussed the high level of complexity of the conceptual framework and, although they saw the benefit of the approach, thought a less complex framework would be more appropriate for the students. Together, the faculty settled on the decision to undertake the same approach, giving students a conceptual framework graphic to unpack, but using one that was created by another doctoral student. In addition to using the meetings to consider whether previous approaches should remain the approach we take in a given week, we also document problems that arise so that we can collectively consider ways to respond to those problems. We capture this information in a draft version of the syllabus and IGs (discussed above) that we plan on using in the next cycle of teaching the course.
In addition to the face-to-face faculty meetings, we regularly use email to communicate specific planning choices we make during our final preparation for class time and choices we make during class time. We offer each other thoughts on the success of our choices, providing evidence that we think demonstrates why a choice was successful or where that choice led us in an unintended and problematic direction. The combination of weekly meetings and ongoing email communication increases our instructional and curricular alignment throughout the semester and ultimately enables reflection on the part of faculty.
Longer Term Reflection to Foster a Cycle of Improvement
In addition to engaging in this cycle throughout the term, we also hold a series of retreats after the end of the term and before the next time the course will be offered. All faculty who taught the class during the term are invited to attend. During these retreats, we return to the notes we have taken in both the draft syllabus and IG and methodically work through each of these notes. In some cases, these notes lead us to revise assignments. In other cases, the notes lead us to reconsider the readings we are using or the activities we use in a given week. Through this process, we ensure that we have a way of learning from what has helped our students learn and what has impeded or failed to support their learning. For example, for the first several years of teaching this course, we asked students to master qualitative skills and knowledge and demonstrate mastery of academic writing. We spent hours providing feedback on the structure of their writing in addition to providing feedback on the content. During one of our retreats, we decided to create templates for each assignment. These templates, with properly formatted section-level headings, allow students to focus on the content we want them to learn and allow us to provide them feedback on those elements without spending inordinate amounts of time on what we have come to decide are important but not essential skills to be developed in this particular class. We also use student feedback gathered during the course and in course evaluations to make changes to the course. For example, when students told us that they believed that some of the readings were repetitive, we reviewed the readings we had assigned with their comments in mind and reduced the number of readings, excluding those we did not think added enough value to students’ learning. We have also been working on increasing the ways we ask students to engage in critical reflection in their roles as novice researchers and educational leaders. Critical reflection is a sophisticated act that demands one to be able to engage in a deep level of self-examination and an awareness of the role and power of hegemony in society. It is not a skill that students acquire easily or rapidly. We have expanded our focus on ethics and credibility and trustworthiness and the role of reflexivity and reflective memoing as vehicles for this discussion. These are just a few of the many ways we have used our improvement process to guide us in making informed, evidence based decisions to strengthen the course for our students so that they will be practitioner-scholars.
Conclusion
When designing and continually improving this course, we have been very cognizant of our audience. Our students are obtaining an educational doctorate with the expectation that they will be able to acquire knowledge and skills in our program that they will use to complete their dissertations. Our primary goal is to teach them about research methods so as to support their development as practitioner-scholars. In addition, as the two faculty members responsible for ensuring that the two research methods courses prepare our students to be practitioner-scholars, we have, over time, reflected on how the various aspects of the course, both instructional strategies and curriculum, have supported and/or hindered our students’ learning. Because research is both new to our students and evidence based decision making is something they will be expected to be engaged in systematically upon completion of our program, we have thought long and hard about how to afford students access to high quality instruction as well as make the course content meaningful. Students have told us that this course and the assignments they are required to complete enable them to understand the research process, what challenges they might face, and to think of corrections they might make when it comes time to conduct their dissertation research. They have also told us that they have greater insight into the way their biases shape their views of the world and have implications for the approaches they take to solving problems.
To continue to make this course a rigorous and meaningful one for our doctoral students, we have presented, in this article, how we use an IG to ensure consistently high quality teaching and a rigorous curriculum as well as the role of reflection in building critically reflective practitioner-scholars and simultaneously helping us improve the course both in the short and long term. And while we are proud of the progress we have made up to this point, there remain ways that we can improve the course and the experiences for our students.
One additional way to ensure the course is fulfilling its intended purpose is to hold faculty members accountable to following the IG. Some faculty members deviate from the IG, and we sometimes hear about these deviations from our students. Given that the IG is intended to ensure consistency and high quality instruction for all of our students, it is important that we hold faculty accountable to their decisions. And while we do not expect the course to be taught in a lockstep fashion, we struggle with finding the balance between flexibility to allow faculty to be the teachers they believe they should be and our desire for alignment to the curriculum and instructional approach we have adopted. We have not always been able to hold faculty accountable to using the guide. Part of what has impeded us is our own hesitation to mandate the use of the guide, especially when the person deviating is a member of the full-time faculty. Thus, we continue to look for ways to encourage and sometimes cajole faculty into seeing its value and abiding by it even if it is not their natural tendency, including making hiring and re-hiring decisions based on part-time faculty’s willingness to abide by the use of the guide.
We believe the use of an instructor’s guide is essential whether the context is a large program with multiple sections or a small program with only one section. Even in a smaller program, program coherence is important, and when faculty make significant changes to a syllabus and thus the content being taught on a yearly basis, it is critical to record these changes and ensure the course remains aligned to the initial objectives as set out by the program. In terms of course integrity, changing the course every year may be beneficial for students, but only if the instructor is ensuring that the way the course was being taught before revealed weaknesses that need to be addressed. That can only be done with some form of tool to delineate what the course looked like and what the strengths and weaknesses of that approach were when implemented. An IG provides for such a tool and enables evidence based decision making.
Another desired next step to improve our instructional coherence is to spend the time observing one another and the other faculty who teach this course. While we often “observe” online recorded class sessions in our other programs, this particular program is face-to-face and on campus. As such, we have not systematically used peer observation to help improve our practice as a team of faculty. This would be a welcome addition to our work because we currently only rely on our weekly faculty meetings and email exchanges to discuss any deviations from the IG and any reflections on the course’s efficacy. Observing one another’s practice promises to take our collaboration to the next level and enable us to continue reflecting and making improvements to our instruction and the course. Ultimately, the goal is to have an approach to delivering our methods courses in such a way as to ensure that all of our students have access to the instruction and curriculum that they need to develop into practitioner-scholars, well-versed in research methods, able to use research and theory in their applied settings, and ready to take on their independent research endeavors that help them address issues of access and equity both within their doctoral studies and beyond.
One tension not addressed in this article is whether one methods course is sufficient to develop students’ abilities to engage in the data collection and analysis associated with the dissertation process or independent research. While this is the second methods course in the students’ program of study, we recognize that it is also the only intensive course designed to build their capacity and is therefore limited in what it can accomplish. Where we have found opportunities to increase students’ exposure to this content and these skills, we have done so. For example, we have developed a website called Navigating the Dissertation that is specifically geared toward reinforcing students’ knowledge and skills after they leave this course. In addition, thematic dissertation chairs are expected to provide support to their students throughout their dissertation process. Finally, where possible, such as in new programming, we have increased the number of research methods courses available to students by breaking the course into two components: (a) data collection and (b) data analysis and reporting. Consequently, given the fact that we only have one semester in which we intentionally set out to build students’ capacity to enact the skills associated with being practitioner-scholars, we are maximizing this time through the use of an IG that contains the instructional approaches and content we expect our students to experience and a reflective process that ensures continual course improvement over time. This article is about our approach to accomplishing this end and not to advocate for the use of a single class to equip practitioner-scholars in qualitative research methods.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.