Accepting Rotation in the Method Prism

One does not need to attend but a few conferences at which scholars from several disciplines present their research to identify differing philosophical, theoretical, and methodological assumptions. These effects are heightened when the conference is intended and planned to be an interdisciplinary event. At these events, philosophical and theoretical differences are often more tolerated than methodological differences. But when it comes to methodology our differences are more open to criticism. This criticism can be identified when we ask one another, “What data did you collect?,” “How did you collect that data?,” and “What findings can you develop from your results?” Although we often accept multiple philosophical and theoretical frames for the study of group phenomena, it seems to me that we are less accepting of methodological differences.

Obviously, we are products of our training and exposure, and those foundational educational experiences (i.e., courses we chose to take or not) create a frame for evaluating one’s own and others’ scholarship. At a conference such as INGRoup, we can agree that groups and teams are the focal point of scholarship. But a group or team provides many methodological entry points. We can study

Thus, the study of groups and teams is prismatic—it is wonderfully rich and complex and can be approached from a variety of levels (see Kerr & Tindale, in press) and methods (see Runkle & McGrath, 1972): experiments in all of their forms, questionnaires, interaction analysis, content analysis, textual/discourse analysis, field observation, interviews and focus groups, ethnography, case studies, and more (see Hollingshead & Poole, 2012, for one list of group research methods). As delightful as I find this state of affairs, the study of groups and teams remains methodologically contested. Philosophical and theoretical differences are more tolerated than our methodological differences. Why is that?

Research is a personal process—and a process that supports and serves our professional identity and success. I believe this is what makes us hold onto methodological preferences and discourages us from learning from others when their methodology is different from our own.

Although differences can be enlightening, we often focus on differences as being—well different. And when something is different we tend to evaluate it negatively. As coeditor of Small Group Research that draws submissions from group and team scholars across disciplines and nations, I often get this response: “I can’t review this, I don’t use [fill in your favorite methodology].” This fascinates me, as I have selected the reviewer for his or her expertise in the content of the manuscript with the hope that his or her expertise in leadership, for example, would trump any methodological difference. Certainly, I don’t expect that potential reviewer to learn a new method, but I do want that reviewer to contribute to the conversation about the content and to give his or her professional evaluation on the basis of his or her knowledge of the method (regardless of how limited).

This type of methodological difference is also evident at conferences. Some questioners (and, at some times, respondents) ask questions in such a way as to negatively evaluate how the presenter has collected data. (I am presuming that I’m not the only conference presenter who has felt this type of public stinging criticism.)

What can we do to bridge methodological differences in the more public venue of conferences and the more critical venue of publishing? First, let’s ask how the theory and methods used by an author are congruent? By using this metric, minimally, we may discover something new about the theory, or we may learn about a method with which we have little training or exposure. Second, rather than asking, “Why did you do it that way?,” let’s try questions along the line of, for example, “How do you see that methodology as illuminating [a particular concept/construct]?” Third, if a method is unfamiliar, let’s ask where we can find expert sources to learn more about it. As you can see, I advocate a learn rather than evaluate mode when approaching the work of other scholars. This does not mean that we avoid questioning one another, but our questions should be formed in such a way as to interrogate our own methodological preferences and privileges as much as they interrogate the methodological choices of others.

Beck’s questions ask us to view methodological differences in even broader ways. First, how can scholars use comments outside their discipline to enhance their discipline-specific research? There are standards by which each methodology is evaluated, and those standards focus on two different aspects of research reporting, such as that done at conferences. First, standards illuminate how well the research study was designed and then conducted. Second, standards can illuminate the validity, reliability, and/or credibility of the data analysis. I separate these out, as an expertly designed study can be poorly conducted or data from a well-executed method can be poorly analyzed.

One way group and team scholars could help one another and advance the study of groups and teams is to ask questions specifically targeted toward design, execution, and analysis. If you see a flaw in my design, believe you have spotted a flaw in the execution, or identify lack of logic in the analysis, I want to know. How others react to my scholarship is important because presenting at a conference is a test for the potential publication of the manuscript. Rather than deriding all aspects of my chosen method, guide me toward the specific weakness you observed. Engage me in a dialogue about my choices. When group and team scholars from different disciplines have open dialogue about method and analysis, we all gain. I get a preview of what might happen if I submit the manuscript for review for publication. You might learn something new about a method with which you are less familiar. This is how we could help one another and advance the study of groups and teams.

Beck’s second question asked, “How can scholars from different disciplines integrate methodological differences to enhance group research?” One way to accomplish this is to engage group scholars from other disciplines in your research. Recently I was on an interdisciplinary research team (communication, management, and psychology). As a team we were preparing an interview guide for field interviews. The guide was long, as it was the first set of interviews to be conducted and we were uncertain of where this research might take us. Something interesting happened as we independently reviewed the interview guide. Team Member A thought that the interview guide was overdirective, as very specific prompts and secondary questions were listed in the margin next to each major question. In other words, she believed that the interview guide would restrict her from following the interviewee’s lead. Team Member B believed the interview was not structured enough, as she was concerned how interviewees’ responses to different secondary questions could be analyzed. And, of course, Team Member C thought the interview guide was fine as written. Exposing our different viewpoints to one another caused us to have a conversation about the design and execution of the interview study—a good thing to happen before any data were collected. Thus, we integrated our disciplinary differences on what constitutes good field interviewing, which we hope will result in publications in disciplinary and interdisciplinary outlets.

Hollingshead and Poole (2012) credit Joe McGrath with saying, “Every research method is flawed” (p. 4). They go on to cite McGrath, Martin, and Kulka’s (1982) book in which these authors describe research design as consisting of three dilemmas: precision, realism, and generalizability. They also note that it is impossible to achieve all three in a single study. I agree with this evaluation by both sets of scholars. But rather than saying every research method is flawed, I would rephrase this as every research method is partial. Regardless of our methodological and analytical choices, one set of choices cannot describe, predict, and explain a group or team phenomenon in totality. A group or team scholar’s methodological differences should not be evaluated simply because they are different than a research design I would conduct. Rather, our differences should be part of a conversation, not the basis of methodological one-upmanship.

Let’s rotate the prism to learn rather than rotating the prism to focus on what we already know.