Abstract
Many business schools continue to use contribution in teaching, research, and service as measures of faculty performance. There has been a long tradition of thinking of faculty as making their research contribution within a specific subdiscipline. We call these teaching and discipline scholars (TDS). However a growing number of faculty who, although they teach in a subdiscipline, are choosing to make their research contribution in the teaching and learning area. We call these persons teaching and learning scholars (TLS). A major hurdle facing TLS candidates is a promotion and tenure (P&T) system primarily designed for teaching and discipline scholars. This article takes a granular look at the typical P&T system within business schools. It proposes a way of thinking about what is typically meant by teaching ability and how it might be measured. It then discusses what is meant by research, how this definition might be applied to measure the output of TLSs and the special challenges for TLSs in having their research accepted as part of their P&T portfolio. Suggestions are provided for how the TLS may navigate the P&T process in light of these challenges.
Keywords
Candidates for promotion and tenure (P&T) must demonstrate their abilities in research, teaching and service to the community; but how to assess these abilities has been a moving target and subject to interpretation. Within this environment a major challenge to career advancement is faced by tenure track teaching and learning scholars (TLS). By TLS, we mean a person who aspires to pursue a career focused on both teaching in a subdiscipline such as marketing and conducting research on teaching and learning, likely within the context of the subdiscipline. This is in contrast to the teaching and discipline scholar (TDS) who pursues a career teaching in a subdiscipline and conducting research aimed at advancing knowledge within the subdiscipline. The attention devoted to TLS at universities appears to be growing (Asarta et al., 2017) and hope has been expressed that it will spread (Asarta et al., 2018; Gallos, 2008). But as pointed out by Arbaugh (2016) and confirmed by the experience of editors of management teaching and learning journals (Bacon, 2016; Bridgman & Bell, 2016), there are relatively few scholars in business schools that exclusively focus their research on teaching. As will be demonstrated, one of the factors contributing to the paucity of committed TLS scholars is the P&T system, which tends to be organized around the research activities of discipline scholars. The goals of this article are to provide a granular discussion of selected aspects of the P&T process within business schools, describe how it differs for TLS versus TDS scholars, identify the resulting P&T challenges facing the TLS scholars and provide helpful suggestions regarding how some of these challenges in navigating the P&T process may be addressed. We clearly have a “point of view” that is intended to stimulate scholarly debate in such areas as the role of institutional strategy in the P&T process, how research may be defined and implemented and what constitutes a contribution to our knowledge of teaching and learning.
Due to the large number of business schools and the variety of their strategic goals, we have found it necessary to narrow the scope of our article. First, while we are aware that a large number of instructors are full-time or part-time nontenure track faculty and that proportion is growing (V. Anderson, 2007; Crimmins et al., 2017; Vander Kloet et al., 2017), our focus is on full time tenure track faculty. Second, we focus on those institutions for which research constitutes at least a part of the faculty member’s mandate. Third, while we know that service is an important component of faculty activities (Jaeger & Thornton, 2006; Mamiseishvili et al., 2016), we focus only on the teaching and research aspects of the P&T process. Although our discussion is primarily limited to business schools in North America, we believe that our approach provides insights that are applicable in other disciplines and geographies as well.
Background Considerations
The Importance of Institutional Strategy and the Role of Subdisciplines
A university or college may be thought of as an amorphous polity that represents a wide range of social and economic interests. Its stakeholders may include such groups as the public at large, government, donors, faculty, staff, alumni, unions, and students. Since a tertiary educational institution tends to be quite democratic in nature, there is a constant push and pull between these interest groups. Nonetheless, it is the task of the senior administrative staff to pull together the goals of these diverse groups into an institutional strategy and to devise a means of implementing the strategy. The particular strategy chosen is affected by such factors as the institution’s history, geographic location, financial resources, competitive conditions, and the anticipated political, economic, social, and technological environment in which it expects to operate. Three of the key strategic choices to be made are the level of designation to be offered (such as PhD, masters, or undergraduate degrees), the specific programs to be offered and the perceived learning needs of the intended audience. The programs and methods of delivery tend to primarily be the responsibility of the individual departments. It is within this environment that the university must make hiring decisions and provide rewards for performance. For faculty this is operationalized through the design and implementation of a P&T system. This view of the institutional environment is particularly important in this article because it means that, because each institution is likely to have a different strategy and tactics for implementing it, there is no single P&T system that is appropriate for all institutions, although some practices are more widespread than others.
In this article, we employ the word discipline when referring to a broad field of study such as business while a specialty within a discipline such as marketing, finance, or accounting is called a subdiscipline. Most faculty identify with a subdiscipline (McKinney, 2013). This identification begins early in the career of academics since PhD programs are typically subdiscipline based and at the time of their first academic appointment individuals are attached to a subdiscipline. Administrative departments within Business Schools are also typically organized by subdisciplines that have the primary responsibility for activities such as recruiting faculty and assigning teaching obligations. This identification with a subdiscipline extends to the P&T process in which candidates are usually expected to demonstrate that they have made a contribution to research in their subdiscipline. Confronted with the reality of the importance of subdisciplines the faculty member who wants to be a TLS faces a formidable challenge; namely how to have a home in a subdiscipline while at the same time making a significant research contribution to what is known about teaching and learning.
Teaching, Research, and the P&T System
As universities have evolved the importance attached to research relative to teaching has increased. Boyer captured this change by noting “ . . . the reality is that, on far too many campuses, teaching is not well rewarded and faculty who spend too much time counseling and advising students may diminish their prospects for tenure and promotion” (Boyer, 1990, p. xii). The observed emphasis on subdiscipline research in P&T decisions has become even greater in intervening years; especially in research oriented and comprehensive universities (Chalmers, 2011; Gioia & Corley, 2002; Youn & Price, 2009). This trend toward an emphasis on research has resulted in resistance from students, public funding bodies, employers, and the faculty themselves; many of whom want to see more attention paid to teaching excellence. Led by a landmark report by Boyer (1990), a movement evolved to champion teaching by broadening the definition of scholarship of teaching that was later called the Scholarship of Teaching and Learning (SoTL). Subsequently many authors attempted to redefine the definition of scholarship and design P&T processes to reward such activities (Fanghanel et al., 2015; Miller-Young et al., 2018; Tight, 2018b). However, many schools continue to use the teaching, research, and service categories in their P&T systems. Given these categories, navigating the P&T process for the TLS remains a somewhat challenging process, at least partly because of an unclear distinction in the literature between what constitutes teaching and what constitutes research.
Universities and colleges provide broad guidelines for P&T policies and procedures; leaving it up to the Business School to implement them in light of the unique characteristics of each subdiscipline. The normal steps of progression in many countries are from assistant professor to associate professor, tenured associate professor, and finally full professor. At each step, the candidate is expected to submit a portfolio to the P&T committee in the Business School indicating their achievements in teaching, research, and service (Jonker & Hicks, 2014). The P&T Committee usually supplements this portfolio with solicited comments from scholars who are members of the subdiscipline located at comparable institutions. Although the descriptors vary from one institution to another it is common to ask the candidate for tenure to achieve “superior” performance in either teaching or research and “excellence” in the other, plus a reasonable service contribution (see, e.g., University of South Carolina, 2019). For promotion from associate to full professor, the candidate may be asked to be superior in one of the three categories and to demonstrate excellence in the other two. We employ this distinction between superior and excellent performance throughout the article although we recognize that it may not apply to all universities and regions of the world. In fact, a review of the literature suggests that there may be significant regional differences in P&T practices between such countries as the United States (Hardré & Cox, 2009), Canada (Gravestock & Gregor-Greenleaf, 2008), Australia (Vardi & Quin, 2011), South Africa (Vithal, 2018), and the United Kingdom (Parker, 2008).
In light of our discussion about the wide possible range of institutional strategies, it makes sense that superior or excellent performance should be uniquely defined by the School; often with input from the members of the subdiscipline. But, to the frustration of academics, the criteria for P&T continually change as schools modify their strategic thrust in response to a variety of market forces.
The Challenge for the TLS Begins in Graduate School
Academic development of scholars begins during graduate training. Students are taught the latest concepts, theories, empirical findings, and practical applications within the subdiscipline, how to conduct research including how to critique the work of others, the current unaddressed issues in the field, the latest in appropriate research methodologies and how to pursue a publishing strategy. Capabilities in each of these areas are enhanced by the writing of a doctoral thesis.
Since the candidate will almost certainly take on teaching responsibilities in future, they are sometimes provided with knowledge of best practices in teaching and learning, especially as they apply to the subdiscipline. For many programs, training takes the form of a single seminar on teaching and learning although some provide a more extensive introduction to this topic. In addition, many graduate students undertake some teaching duties that may take place under the mentorship of senior faculty. Most PhD programs in North America devote only modest focused effort to teaching relative to research (Arbaugh, 2008; Barney, 2019; Brightman & Nargundkar, 2013; Lewicki & Bailey, 2016; Marx et al., 2016; Rynes & Brown, 2011).
Consider the several challenges this minimal level training on teaching and learning presents to an aspiring TLS. The literature on teaching and learning is not studied in depth with the result that students are only superficially acquainted with the latest learning and teaching theory and empirical research. Since the research methods learned and practiced in the program relate to the subdiscipline they may not be the methods that will ultimately be required in order to conduct teaching and learning research (Divan et al., 2017). And the candidate does not become familiar with the journals in the teaching and learning field or how to conduct a publication strategy. The result is a scholar in North America who is underprepared for a career that has a TLS focus. Outside of North America, there is a trend toward ensuring that teachers are well trained so one should be cautious about the geographic reach of this generalization (Gibbs, 2016). And some have argued that the solution to this training deficiency should be, and often is, addressed at the School where the academic is first appointed (Le, 2016).
Assessing Teaching Capability for TLSs and TDSs
Since both TLSs and TDSs teach within a subdiscipline, for P&T assessment purposes there should be no difference between them in how teaching capability is measured. Thus, for example, superior performance in teaching for the TLS and TDS should be defined and measured in the same way. However, it should be noted that if the TLS proposes to demonstrate excellence in teaching, whereas the TDS prefers to demonstrate superior performance in teaching, the evidence provided by the TLS will be more extensive.
For our purposes an accomplished teacher in a Business School is one who provides students with the opportunity to acquire appropriate knowledge, skills, and cognitive and affective abilities to be a successful practicing manager both now and in future (Devlin & Samarawickrema, 2010). While there may be broad acceptance of the foregoing definition, a problem arises in converting this statement into specific P&T criteria. As a condition of their appointment all faculty are expected to engage in certain basic teaching activities such as; specifying course goals, showing that the course sequence will achieve these goals, designing rubrics that measure whether these learning goals are being achieved, providing a course outline, being well prepared for classes, treating students with respect, being available for consultation, providing accurate and timely feedback, and so on. However, these activities are basically “table stakes” and are not enough to demonstrate a “superior” or “excellent” level of teaching performance.
In addition to these basic requirements, in order to progress through the P&T process, the instructor may be expected to demonstrate abilities in four areas. The first is that the instructor must show that their courses are evidence based; that is, they teach how to manage based on the latest scientific research in the discipline (Rousseau, 2012). Second, the instructor should show that they use best practices in teaching that are appropriate for the achievement of course objectives (Charlier et al., 2011). Third, the candidate must demonstrate an ability to transform the knowledge of their field into actionable management practices (Rousseau, 2012). And finally, the instructor should demonstrate skill at course delivery.
Since the members of the P&T committee may not be members of the applicant’s subdiscipline, it is important that all teaching-related activities should be documented and critically reviewed by peers. The documentation should go beyond the submission of course outlines and other materials to include explanations of how these materials relate to the four abilities discussed above. Critical review is important because it provides evidence that the teaching approach is considered up to the standards of thoughtful members of the academy. This exposure to critical review can range from, the sharing of course materials and teaching processes with colleagues at the home school or beyond, to wider publication of such items in text books and education related journals.
Cases are widely used in business schools and often come in for special attention. Well-written cases and teaching notes provide a number of benefits to the teacher including; establishing credibility with students, identifying the major current issues facing practicing managers and enhancing the candidate’s ability to apply recent theory and empirical research to the practice of management. Teaching cases should be reviewed by colleagues and external reviewers and where possible external recognition of their contribution should be sought. Teaching notes should be based on the most recent theoretical and empirical literature and should indicate how the cases could be taught. The relative merit of teaching cases that are produced can be assessed by P&T committees by looking at lists of best-selling cases provided by organizations such as the Ivey Business School and the Case Centre, by success in open case writing competitions put on by business schools such as the Molson Business School and CEIBS or by publication in a blind reviewed case journal.
Measuring skill at course delivery is a challenging and controversial area. Although teaching evaluations provided by students have come in for severe criticism (Boring et al., 2016; Boysen et al., 2014; Clayson, 2009; Emery et al., 2003; Spooren et al., 2013; Stark & Freishtat, 2014), such evaluations continue to be used when assessing teaching effectiveness due to their ease of use and because a better method has yet to be devised (University of Texas, 2018; University of Michigan, 2018). Peer review in the classroom has been suggested to supplement student evaluations as a measure of teaching effectiveness (D. J. Bernstein, 2008). Thoughtful letters from former students are also commonly used. Finally, the research output of doctoral students or postdoctorate fellows is a useful indication of the teaching success of a faculty member.
Assessing Faculty Research Capability
We now turn to a general discussion of the nature of research and how research output is evaluated for P&T purposes. A perplexing question facing the P&T committee when evaluating both TDSs and TLSs is, “what is research”? At its most general level research is an activity that is conducted in order to understand some observed phenomenon and thereby contribute to knowledge in a field of study. The more generalizable the understanding of the phenomenon the more important it is likely to be.
Theoretical and Empirical Research
Research is typically divided into two types; conceptual and empirical. Conceptual research typically is aimed at generating theories or conceptual frameworks that are intended to rigorously describe some phenomenon. Such research is often grounded in descriptive observations and is inductive in nature (Wright, 2017). Empirical research is data based and is aimed at testing a theory or conceptual framework (Wright, 2015). To ensure that both theoretical and empirical research is valid it must be conducted employing acceptable scientific principles. Once a phenomenon is understood academics can undertake the task of sharing that knowledge with students and demonstrating how to convert that knowledge into action, thus improving managerial decision making.
Basic and Applied Research
Basic research is typically aimed at understanding phenomena that are broadly generalizable to many different situations. For example, research into the Law of Gravity was, for many years, considered generalizable to phenomena both on the earth and beyond and would be considered basic research. Basic research is commonly found in such university departments as physics, chemistry, economics, neuroscience, psychology, and sociology. On the other hand, applied research usually involves the application of basic research findings to narrower fields of study found in professional schools such as engineering, medicine, and business. In this sense, most research in business schools is applied in nature.
Is a Candidate’s Research a Contribution to Knowledge?
Who judges whether research done by an academic constitutes a contribution to knowledge? This judgment is usually made by an academy made up of scholars who are trained in scientific methods and familiar with extant knowledge in a field. They are thus able to assess if the research by a candidate is valid and is a new contribution to a field of study. In passing, it should be noted that this focus is the reason why most research papers include a section on research methods and a review of the literature in the field. Since all members of a P&T committee are not likely to be experts in an individual candidate’s area of interest, the portfolio is typically sent out for review by specialized members of the academy.
Whether particular types of applied research should be considered a contribution to knowledge for P&T purposes can be controversial. Some applied research focuses on a specific problem in a particular situation such as how to design a compensation system for employees of Company X. To the extent that the results are not generalizable beyond the target company, this tends to not be considered a very important research contribution. It is more in the realm of consulting. However, if the research can be generalized, for example, to all companies in a particular industry, one might consider this research. As mentioned earlier, if the research can be further generalized to all companies in all geographic areas and in all cultures, it would clearly qualify as research. Of course, even the narrowest applied research can lead to new theories in which case it may qualify as a research contribution.
What Kinds of Publications Are Acceptable?
To assess the importance of the research and to confirm that it is a valid contribution to knowledge, the school is likely to require publication in blind peer reviewed journals. A number of lists of such journals are available. The broadest published list is the journals included in the Social Sciences Citation index (SSCI) operated by Clarivate Analytics. For a journal to be included in this list, a number of criteria must be met including; regularity of publishing, availability in the English language, evidence of peer review, and citation by other authors. SSCI produces a Journal Impact Factor (JIF) for each of its journals which measures the frequency that an article in a journal is cited in other journals over a particular period of time. Clarivate also publishes an Emerging Sources Citation index which essentially covers newer journals awaiting consideration for the more senior SSCI. A number of competing lists of journals and related impact factors have also evolved. For example, Scopus which was initiated by Elsevier in 2004 is an abstract and citation database that includes a large number of journals. They provide a CiteScore which competes with the JIF. A highly correlated rank index for journals is the SJR (SCImago journal rank) produced by SCImago based on the Scopus database. It is presumed that the frequency of citation is an indicator of the scholarly importance of a journal although this view has come in for some criticism (Sangster, 2015).
Within the list of acceptable journals, a distinction is often made between the “A level” journals and all the rest. The A level journals typically are well known within a particular discipline, are considered to have the most prestigious editorial boards, are presumed to have a very rigorous review process and have low acceptance rates. For example, Dugan and Kellaris (2015) note that the four “A level” journals in marketing (called the Big 4) are the Journal of Marketing, Journal of Marketing Research, Journal of Consumer Research, and Marketing Science. Publishing in “A level” journals is very difficult and arguably represents an unreasonable hurdle to be met by most faculty members, particularly those not located at research universities (Dugan & Kellaris, 2015). With this in mind many schools consider publication in other refereed journals as perfectly acceptable. Nonetheless, many P&T committees still tend to rely on JIF’s and CiteScores to assess the research quality of individual journals.
Business schools pay close attention to the ranking of their programs relative to other schools. Such rankings are often provided by business magazines such as Forbes, Financial Times, Business Week, and The Economist. One of the best known school ranking systems is the FT index. This index employs several criteria to measure university quality. Although research output (having a 10% of weight in the FT ranking) has only a modest impact on the rank it has a significant impact on faculty P&T assessment since some schools especially reward faculty who publish in the 50 business-related journals used in the ranking process.
Since there are many hundreds of business schools in the United States and thousands around the world, it is unreasonable to expect all schools to be near the top or even on the FT list. The great majority of schools have strategic goals that are much more modest and use different performance metrics by which to assess their relative position in their competitive universe. The performance metric chosen by a School should be expected to impact on its P&T criteria.
How Much Research Is Enough?
Armed with a sense of the most important journals in a field the candidate still has to answer the question; how much and what kinds of output are enough to be promoted? Perhaps the simplest method is to set as a target a particular number of papers published within a selected group of refereed journals. Some schools also count articles published in practitioner journals but the relative importance of these depends on the strategic thrust of the school. To the extent that a school wants to improve their ranking through publications, the school may compare itself with other schools that have higher rankings and try to achieve output results that are comparable to the target schools by setting “stretch” output requirements for their faculty through the P&T process (Chen et al., 2015). Using a target number of “acceptable” publications as a requirement for promotion has been criticized by authors who argue that due to the number of available journals, relative to the number of faculty who are members of a particular discipline, there may be bias in favor of certain disciplines (Dennis et al., 2006; Hitt & Greer, 2012).
Although the number of articles published may be a useful indicator of productivity it does not measure the impact of the scholar on their field of study. Impact is usually measured by the number of times a scholar’s work is cited by others. A number of such impact factors have been created by data suppliers such as Web of Science and Scopus. In general, business schools prefer to have their faculty publish in high-impact journals and publish articles that have high impact.
Some contend that the measurement of the impact of a scholar’s work should not be restricted to its uptake by other members of the academy. They suggest that a broader impact measure should be used that takes into account other stakeholders such as practitioners, students, and the public in general (Aguinis et al., 2014). Aguinis et al. (2012) compared the rank of academic scholars using standard citation indexes relative with their rank based on the number of pages found in the Google not-.edu domain. They concluded that high impact within the academy did not necessarily imply high impact outside the academy. Furthermore, some have argued that the existing reward system does not favor scholars who bridge the gap between academic research and practice; often one of the key strategic thrusts of a business school. Pearce and Huang (2012) investigated whether the research published in selected leading management journals was actionable by managers. They concluded that there was a steady decline in actionable research published over the 1960 to 2010 period. As data availability and computing power have grown, more attention is being paid to the general use of altmetrics (Priem et al., 2010; Sugimoto et al., 2017), which are a variety of alternative measures of impact such as number of views, mentions in social media and comments in the popular press. However, we were unable to locate any studies of the use of this approach to facilitate P&T decisions. The conclusion we might take away from this discussion is that it may be perfectly appropriate to utilize broader measures of impact in P&T decisions as long as the measures are aligned with the strategic goals of the school.
What About Text Books and Cases?
To this point, we have discussed journal articles as a form of research output. But there are two other types of publications that often come in for scrutiny; cases and text books. As discussed earlier, we take the point of view that most research in business is applied in nature and a contribution has been made if there is a development of a new theory or conceptual framework or an empirical test of a theory has been conducted. How do these criteria apply to cases? First, we make a distinction between research cases and teaching cases. Employing the case research method is a legitimate way of conducting grounded research aimed at generating new or expanded theory (Eisenhardt & Graebner, 2007) and unambiguously should qualify as research. On the other hand, teaching cases tend to be much less rich due to the need to shorten their length and to focus on one or more learning issues. In our view, if the instructor wants to contend that their teaching case makes a contribution to theory or conceptual frameworks, the teaching note can make a convincing argument along those lines by including a review of the literature highlighting the contribution to knowledge. But our experience has been that only under exceptional circumstances does a teaching case lead to the generation of a new theory or constitute empirical research (Lapoule & Lynch, 2018).
Turning to text books, we would employ similar criteria. A text book makes a contribution to the learning of students and serves as a guide to teachers. A P&T committee can assess the teaching contribution of a text book by its rate of adoption and comments by adopters. To the extent that a text provides a conceptual framework for approaching the understanding of a subject or an original synthesis of existing knowledge, it may qualify as research in much the same way as a published review of the literature may constitute research. In the P&T package, the onus is on the author to make a case for its contribution to knowledge. This may be done by noting citations of the book and reviews of the book in academic journals.
Assessing Research Capability of TLSs
Basic Versus Applied Research
Having reviewed how traditional research output is measured for P&T purposes, we now turn to the challenges of conducting and measuring the research output of TLSs. The research output of teacher scholars may be usefully divided into two types; basic educational research and applied educational research. Basic research is aimed at advancing generalizable knowledge of what is known about learning and teaching from all fields including psychology, sociology, and education. It includes both conceptual and empirical research. In contrast, applied educational research is intended to advance knowledge of teaching and learning as it relates to a particular business subdiscipline. This may also include both conceptual and empirical research, but the results are not as broadly generalizable as those from what we have called basic research. These two types of research may overlap as basic research knowledge may be advanced as a result of applied empirical studies.
Advancing basic knowledge of teaching and learning can be very challenging for an academic located in a business school. There is already a long history of basic research in disciplines such as psychology and education (Tight, 2018a) and to make a true contribution to the field of education and learning, the teaching scholar must be familiar with that literature and build on it. As noted by Kanuka (2011), Similar to other disciplines, scholarship in the field of education also includes original research inclusive of building on prior research, using a methodology falling within the traditions of education and framed within a learning theory (exclusive perhaps of grounded theory research and phenomenography). (p. 3)
The incoming editor of the Academy of Management Learning & Education (AMLE) journal put the problem facing teaching based scholars succinctly when he observed, “Becoming familiar with new literature streams, operationalizing new variables, and considering different research methods can present significant obstacles for business school faculty that typically have no prior training in educational research” (Arbaugh, 2008, p. 5). The TLS located in a Business School is at a severe disadvantage in publishing basic research as they do not have the doctoral level qualifications to meet the demands of basic research journals outside of the business field.
Given the challenges of doing basic research, Kanuka (2011) notes that most of the research on teaching and learning by business faculty relates to specific subdisciplines where the expertise of the faculty lays. Dolan et al (2018) assert that subdiscipline-based educational research represents the intersection between the content of a subdiscipline, social science research methods, and the task of advancing teaching and learning. Furthermore, they note that applied educational research “pursues research questions and hypotheses about teaching, learning, and ways of thinking in a discipline that extend beyond single classrooms and programs in order to yield original, generalizable, and mechanistic insights into educational processes and their effects” (Dolan et al, 2018, p. 33).
Target Publications
There has been a significant increase in publications devoted to research on teaching and learning in recent years (Abernethy & Padgett, 2011), and there is now a lengthy list of available journals in which the teacher/scholar can publish. In 2008, the AMLE journal published a preliminary list of 88 established business management education journals based on consultations with scholars from several business disciplines (AMLE Journal, 2008; Whetten, 2008). In an update of the AMLE list, Currie and Pandher (2013) identified 84 available business education-related journals. Later, Arbaugh and Bielinska-Kwapisz (2016) produced a sample of 123 management education journals and used bibliometric analysis to create an index indicating the percentage of articles a journal publishes on Business and Management Education (BME) research. These results supplemented the work of Currie and Pandher by adding relevant educational research journals.
Just as with the subdisciplines, a selection of journals have been identified as the consensus top journals on management teaching and learning. Arbaugh (2008, p. 8) observed that the “big four” management and learning journals were AMLE, Journal of Management Education (JME), Management Learning (ML), and the Decision Sciences Journal of Innovative Education (DSJIE). Based on four field interviews and the precedence of prior publications, Beatty and Leigh (2010) narrowed down the list to a top three; AMLE, JME, and ML. As indicated earlier, given this short list of journals, it is unlikely that most schools will insist on having their faculty publish in these journals for P&T purposes; instead relying on the broader lists of journals discussed above.
The Quality of Teaching Research Has Been Questioned
A major challenge facing TLSs is that articles written on teaching and learning have been devalued by other scholars. Arbaugh (2008) points out that there is a lack of clarity of the field of management education journals and they suffer from a lack of legitimacy. Dehler et al. (2010) assert that it is difficult to establish management education and learning (MEL) as a core research area because the quality of scholarship is often not considered obvious. They note that, The MEL field itself may contribute to its contested status by the kinds of scholarship it publishes (e.g., exercises and cases) and the methods it employs (e.g., first-person reflection, action research) that may be inconsistent with more accepted traditional, positivist research. (2010, p. 107)
Kanuka (2011) sets out several criteria that must be met for scholarly research into teaching and learning. She contends that it must be theory based, rigorously apply appropriate research methods, build on the literature, and be generalizable. In her view, much of the research being done in this area by TLSs does not meet these criteria. Beatty and Leigh (2010) note that “Such debate about standards highlights the legitimacy problems SoTL experiences across many disciplines” (p. 368).
In an attempt to evaluate the legitimacy of TLS research Rynes and Brown (2011) defined several parameters of legitimacy for a journal. Based on an empirical study of the papers dealing with classroom research published in four journals (AMLE, ME, MLE, and DSJIE) over a 6-year period they concluded that while much progress had been made there could be improvements in the published papers in such areas as having a clear theory underlying the research, utilizing objective measures of learning, doing pretest and posttest measures of the variables of interest, improved research design, and providing more focus on moderator variables. The editor of AMLE recently observed that “The submissions to AMLE that struggle with research design, statistical analysis, and appropriate reporting often suffer from a lack of understanding of the current best practices regarding the methods they employ.” (AMLE, 2017, p. 173). In summary, the prevailing view appears to be that TLS research is suspect and by inference does not represent adequate evidence of research scholarship for P&T purposes.
However, all is not bleak. Other research in which we are engaged has detected that over time there has been a steady improvement in the use of theoretical constructs and more rigorous empirical research methods. We have also noted, based on reviews received from journals of our article submissions, that the journals at all levels appear to be making a substantial effort to make positive suggestions for how the submissions might be improved.
Lack of SSCI Indexed Journals
Another challenge facing the TLS is that there may be fewer SSCI outlets for them than for discipline scholars and to the extent that a school’s P&T committee uses SSCI publication as a criterion for assessing contribution TLSs may be at a disadvantage. A study by Leigh and Beatty (2008) identified 80 peer-reviewed journals in business topics that had a sole focus on pedagogy. Of these, only five journals were included in the 2007 SSCI. ML and AMLE were in the SSCI list but the widely respected JME and most of the other journals were not. In their study of 84 management education journals, Currie and Pandher (2013) noted that . . . MLE journals are notably missing from the SSCI, the largest and best-known index for ranking journals in the social sciences; citation-based rankings are only available for 11 of the 84 journals evaluated in this study and for only two of the four core MLE journals. (p. 195)
Some authors have approached the problem of legitimacy by identifying journals that are perceived to have impact on the field. For example, Currie and Pandher (2013) produced a ranking of 84 available business education related journals. Peer assessment by active scholars in the field was employed to assess the journals by quality, awareness, and importance. They assessed the importance of each journal using the product of the journal’s relative quality times, its’ percentage level of awareness as provided by respondents. Arbaugh and Bielinska-Kwapisz (2016) utilizing their sample of 123 journals and citation counts from Google Scholar used bibliometric analysis to create indices for both the percentage of articles a journal publishes on BME research and the relative scholarly affect an article is likely to have after being published in that journal. Based on their results, they reached a number of conclusions. First, AMLE dominates all other journals. Second, journals published by disciplinary professional associations rank highly on the list. Third, publishing educational research in a (usually more highly regarded) subdiscipline-based journal seemed to garner only modest attention relative to publishing in an educational journal in the same subdiscipline. But these latter journals are not typically included in the SSCI index or the Financial Times index and therefore may not have much importance for P&T purposes.
In spite of the favoritism accorded to SSCI journals, it has been pointed out that articles published in non-SSCI journals can have a significant impact on the field. For example Arbaugh and Hwang (2015) in their study of the 100 most influential articles in BME research point out that, “In spite of not being listed on the Social Science Citation Index, the JME had the fourth highest number of articles” (p. 171).
What Can the Aspiring TLS Do?
In spite of the challenges, we have documented we concur with Gallos (2008, p. 539) who saw the great potential for management education scholarship. Most TLSs evolve into this focus after they have been TDSs for a number of years. The result is that many TLSs must make a major transition in their career. Assuming that a TDS wants to evolve exclusively into a TLS by conducting and publishing credible research in the learning area; what steps should they pursue?
Become Familiar With Basic Learning Theories
The first step should be to become familiar with the major relevant learning theories derived from the philosophy, psychology, and education literature. Of these, the most directly applicable to the business domain are constructivism (Fosnot, 2005) and social constructivism (Pritchard & Woolard, 2010) which deal with student focused learning and learning within a social context, respectively. The experiential approach to learning as advocated by Kolb (2015) is especially appropriate for understanding the conceptual framework underlying the case method and outside of classroom learning projects commonly found in business schools. A very useful and widely used conceptual framework for understanding and formulating teaching objectives is provided by Bloom (L. W. Anderson & Krathwohl, 2001). This work focuses on the development of higher order thinking skills which are targeted by business schools. As the TLSs conceptual understanding grows and their specific interests become more focused, the works of scholars who have made key contributions to the field of learning such as Dewey (1933), Bruner (1961), Piaget (1970), Vygotsky (1978), Schön (1987), and Argyris (2002) can be consulted.
Master Appropriate Research Methods
The second consideration is that the empirical research should be methodologically sound. This means that there should be a clearly stated hypothesis, the anticipated relationship between independent and dependent variables as well as the role of intervening variables should be specified and the components of the research should be valid and reliable (J. L. Bernstein, 2018). As discussed earlier, much of the published research in the SoTL area does not meet these basic requirements. To brush up on social science research methods, a useful starting point is the online book offered by Trochim (2006). The book is conversational in tone so it is easy to read but covers many of the basics that can then be studied in further detail once a particular research project and methodology is chosen. Another book that we have found to be useful is by Ling and Ling (2020). They explain the nature of scholarship in education research and provide a number of useful paradigms for conducting such research.
Ensure Research Is Relevant and Makes a Contribution
The third consideration is ensuring that any research conducted is of current relevance and makes a contribution to the field. The TLS should make a habit of reading key business related journals on teaching and learning that have a broad scope such as AMLE, ME, and JME or are more narrowly focused on teaching and learning in a subdiscipline. Most journals outline their scope (see, e.g., Dean & Forray, 2018) and these statements should be read carefully to choose the appropriate outlet for the research. Of course, the candidate will want their publications to be widely cited and guidance is available on this pursuit (Hwang et al., 2019).
Establish a Network
A fourth consideration is building a network of like-minded individuals who are willing and able to critique ones’ work. An important step in this process is to acquire the assistance of a mentor if one is available. Development of the network can begin with like-minded faculty within the School and expand to other departments within the university. Many universities now have centers for teaching and learning to assist TLSs. To network with persons outside of the university, attendance at teacher/learning conferences is essential. Subdiscipline conferences often have a track dealing with teaching but there are also more generic conferences on teaching. The conferences serve to build a scholar’s reputation both nationally and internationally, especially if papers are presented or the candidate is an active participant in discussing the work of others. In all of these activities, it is important to aspire to a leadership role such as acting as a reviewer for journals or organizing a track at a conference; both of which act as evidence of a contribution to the field. The end result of all of these activities is that letters of support can be solicited from well-known TLSs for inclusion in the P&T portfolio.
Understand the P&T Process
Fifth, the candidate should become intimately familiar with the P&T criteria within the Business School and, if possible, ensure that some representatives on the committee are noted TLSs. Early and continuous discussions with senior administrators regarding an appropriate career path are essential. These discussions play a number of roles. The guidance is useful for career planning and paves the way for the submission of the candidate’s P&T portfolio. It also alerts senior administrators as to the candidates’ interest in the area which can lead to the direction of opportunities for funding and other activities to the candidate.
Careful Preparation of the Portfolio
Finally, the submitted portfolio should be accompanied by a clear statement of both the candidate’s teaching and research philosophy. Any claims to achievements should be supported by as much relevant data as possible. There should be a clear research stream indicating overall goals, anticipated impact on the field, progress to date and planned future initiatives. It is important to be mindful that to achieve tenure the focus tends to be on potential while to achieve a full professor status the focus is more on demonstrated achievements.
In Summary
This article describes a number of considerations that a P&T committee addresses when evaluating a candidate’s portfolio. In particular, we focus on the challenges of defining and measuring a TLSs teaching and research contribution. Throughout the article, we emphasize the need for a P&T committee to align their criteria with the school strategy. Our view is that, due to varying school strategies and tactics, the result is that a variety of P&T criteria may be reasonably implemented across schools. We have outlined the special P&T challenges facing what we have called the TLS. And for those academics who wish to transition into such a career focus, we have provided a number of suggestions as to how they might proceed. While this article has put forward our own thoughts on the P&T process our hope is that the article will stimulate a productive debate on these issues among members of the academy.
Footnotes
Acknowledgements
The authors would like to acknowledge the valuable contributions of the reviewers of the article and of Ann Frost, Dale Laird, Simon Parker, Michael Pearce, and Mark Speechley in reviewing earlier versions of this article.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
