An Evaluation Framework for the Dual Contribution of Action Research: Opportunities and Challenges in the Field of Operations Management

Dual Purpose in AR Studies

A crucial aspect of rigorous research design is a proper description of the unit of analysis. One aspect that differentiates AR studies to that extent is that the unit of analysis is treated as an active agent that is part of the research group. The outcome of an AR study is, therefore, both action and research (Coughlan & Coghlan, 2002), where research is used to inform practice, and practice is used to inform research (Näslund, 2002). Consequently, as emphasized by Coughlan and Coghlan (2016), before selecting AR as a research approach, the researcher needs to carefully evaluate the needs of practitioners, the research field and the research issue. In this way, we can properly frame the dual contribution of AR based studies.

Besides, AR projects are cyclical by nature, and actually go through a spiral of phases (cycles) of planning, action, observing, as well as overall analysis and reflection as a basis for new planning and action phases (Ballantyne, 2004). The researcher is involved in the research project, and then needs to step aside to meticulously reflect on and analyze what happens in the organization (Daudelin, 1996). Thorough understanding and analysis are key requirements for taking new action.

The collaborative nature of AR is also mentioned extensively in the literature. It is another key aspect of AR and directly related to its dual purpose (Coughlan & Coghlan, 2016). In the research process, researchers and practitioners collaborate in all the AR cycles by sharing ideas and reflections. The involvement of the organization studied in the cyclic research process will increase the authenticity and reliability of the research findings. Thus, research rigor is achieved in a collaborative manner.

Due to the requirements of the collaboration between researchers and practitioners, the features of the relationship between the two will affect the success or failure of an AR study.

First, the team should naturally consist of both academic researchers and members from the participating organization(s). Such a collaborative team-based approach can enhance both the theoretical and practical knowledge for all members, as it explores the tacit knowledge of employees, their work experience, skills and know-how (Raelin & Coghlan, 2006).

Second, from a research point of view, a team-based approach with more than one researcher and with different profiles working on the same phenomenon is also recommended (Baskerville & Wood-Harper, 1996), as this can increase the rigor of data collection and analysis, as well as reduce the risk of bias (researcher triangulation).

Third, it is crucial that both researchers and practitioners have specific skills that facilitate their relationship during the project. As Coughlan and Coghlan (2002, p. 233) emphasized: “AR is a challenging approach to research because it requires confident and experienced researchers to cope with the uncertainty of the unfolding story and to be able to work as researchers exposed to the reality of organizational change in real-time.”

Figure 2 summarizes the main requirements that an AR study should consider in order to achieve the dual contribution that this kind of research approach can produce.

OPEN IN VIEWER

Figure 2.

Main features of AR that assures dual contributions.

Evaluation Framework Proposed (RQ1)

This section develops a framework (Figure 3) for the analysis and classification of the dual contributions of AR studies (i.e., research and practitioners’ contributions), addressing RQ1. So, we explain how we have defined the dimensions of research and practitioner contributions.

OPEN IN VIEWER

Figure 3.

Framework of the paper.

Systematic Literature Review Stages

We followed the systematic literature review (SLR) methodology, as described by Tranfield et al. (2003, p. 209) since it is a replicable, scientific, and transparent process.

Figure 4 illustrates the different steps we followed to select the final 62 papers for analysis: (1) to define the objective and scope of the literature review in terms of relevant journals and time period, (2) to identify papers based on title, abstract and keywords, (3) to filter the identified papers in order to select those that represent AR empirical studies, (4) to codify the selected papers and to build the database, and (5) to classify the selected papers in a systematic way in terms of our evaluation framework.

OPEN IN VIEWER

Figure 4.

Selection of papers.

As it can be seen in Figure 4, the first step of our search strategy consisted on defining the journals where to look for AR papers in OM field. In this sense, even most SLR searches were carried out on databases, there are SLR studies whose search was based on specific journals such as Chatha and Butt (2015) and Danese et al. (2018). The selection of journals was based, first, on studies that carried out literature reviews in the field of OM based on the analysis of specific OM journals, such as Chase (1980), Swamidass (1991), Dangayach and Deshmukh (2001) or Pilkington and Meredith (2009). Second, we also considered those studies that have analyzed the relative importance of OM journals or have ranked OM journals according to their perceived quality and relevance for OM research, such as Barman et al. (2001) and Olson (2005). This allowed us to finally identify 15 reputable journals (listed in Table 1).

OPEN IN VIEWER

Table 1.

Number of Papers in Terms of the Journal.

Journal	#	%
IJOPM	23	37.1
IJPE	9	14.5
PPC	9	14.5
IJPDLM	5	8.1
JPSM	5	8.1
IJPR	4	6.5
SCMIJ	4	6.5
JOM	2	3.2
JBL	1	1.6
TOTAL	62	100.0

The literature search was carried out for the years 2000 to 2017. This process started in 2000 as AR was not cited as an influential methodology in OM until mid-90s and early-00s (Coughlan & Coghlan, 2002; Westbrook, 1995). Therefore, we cover a research period that allows us a representative analysis of the dual contributions of AR papers.

Our initial search in the 15 selected journals was for articles that included the word “action research” in the title, abstract or keywords. We discarded: (a) seven papers because they do not describe the use of AR methodology, (b) five papers on AR that were purely theoretical or descriptive in nature, and (c) one about teaching OM based on AR.

Approach of Analysis

The analysis of the 62 AR studies was carried out as follows. First, each author analyzed the same set of five articles to determine their primary academic and practical contributions according to the framework described. As the three categories of each type of contribution are not mutually exclusive, the results were then compared, and if there were disagreements between the authors about the classification, there was a joint discussion to refine the criteria. As a result, a template was created with detailed criteria that allowed the authors to classify the papers in a distributed manner (i.e., one paper-one “reviewer”). This procedure is detailed in Online Appendix 1. When there were questions about the classification of an article, the other co-authors were consulted to agree about the primary category for it.

The reference and the classification of each of the 62 analyzed papers is available in Online Appendix 2. It provides a detailed description of the research and practitioners’ contributions for each paper, and their classification in terms of both research contribution and practitioner contribution, according to our dual framework.

Descriptive Analysis by Year and Journal

This section presents the findings related to (1) trends in the number of published papers, and (2) the distribution of the analyzed papers in terms of the journals in which they were published.

Figure 5 shows the chronological appearance of the papers from 2000 to 2017. More than 50% of analyzed papers were published in the last 7 years (2011–2017). As such, the number of papers is low in the whole period, and the weight of AR in the field of OM can therefore be considered as residual.

OPEN IN VIEWER

Figure 5.

Chronological summary of papers analyzed (2000–2017).

Table 1 details the number of analyzed papers for each journal where at least one AR study was found. It shows that five OM journals include 82% of the built database. IJOPM papers represent 37% of the published AR studies, and IJPE, PPC, IJPDLM and JPSM together cover another 45%.

RQ2 (I): What Kinds of Research Approaches Are Used in Published AR Articles in OM?

Figure 6 shows the number of papers classified according to the three types of research contributions during the period of analysis (2000–2017). Seventeen of the 62 papers used a theory testing approach, 33 studies were defined as theory building, and 12 were classified as theory elaboration. Four of the papers classified as theory testing were published before 2009, and 13 were published in the period 2010–2017. While it is true that theory building is the predominant research approach, the number of AR papers in OM journals that use both theory testing and theory elaboration approaches is significant, which leads to the conclusion that the AR approach is applicable to all research contributions from our framework.

OPEN IN VIEWER

Figure 6.

Trend of AR papers in terms of research approach.

Being conscious that linking theory testing with AR may be considered controversial (as mentioned in second section), the analysis focuses on how AR is used for this purpose. In this sense, Table 3 shows 9 papers that have been classified as theory testing in terms of their main contribution to research. It can be observed that most of these papers focus on determining factors that influence an operational decision/issue and to refine an existing framework in a specific context.

OPEN IN VIEWER

Table 3.

Examples of Theory Testing Papers.

Theory testing paper	Main Contribution to Research	Why Theory Testing?
Bamford et al. (2015)	To study factors influencing lean implementation.	“The research questions were tested using evidence from field-based, action research, within a food manufacturer and an NHS organization hybrid process (construction/ transactional).” (p. 708)
Berasategi et al. (2011)	To determine the main factors that influence the efficiency of innovation networks on the proper deployment of collaborative networked innovation processes.	“in order to validate and refine the developed methodology, it was tested in a real-world context.” (p. 582)
Bittici et al. (2006)	To analyze cultural variables influencing performance measurement.	A framework was defined through literature review, and it was tested and refined with five case studies.
Boardman and Clegg (2001)	To refine a framework for structuring and synchronizing phases and stage-gates within an extended enterprise.	The remainder of this paper describes work-in-progress on a collaborative research project. The project attempts to provide a means to answer some of the questions posed by the aforementioned perspectives. (p .810)
Braz et al. (2011)	To show a better understanding of the process of reviewing and updating performance measurement systems based on internal and external environmental changes.	Literature review defines a set of relevant questions to be answered during the performance system evaluation.
Dey (2012)	To develop an integrated analytical framework for effective management of project risks using combined multiple criteria decision-making technique and decision tree analysis.	“First, a conceptual risk management model was developed through thorough literature review. The model was then applied through action research on a petroleum oil refinery construction project in the Central part of India in order to demonstrate its effectiveness.” (p. 903)
Dey et al. (2015)	To analyze how supplier performance impact on operational and business performance of client organization through an integrated analytical model that combines QFD and the analytic hierarchy process method.	A previously developed framework is tested in practice and its insights are analyzed.
Eltantawy et al. (2015)	To adopt concepts within the theory of swift and even flow, the authors integrate the physical (material) and information flow among these supply partners to ensure higher productivity through timely production and distribution.	“Empirical investigations reporting supply chain coordination using a three-level supply chain are very limited. In fact, numerous operations management scholars point out the dearth of theory testing studies of complex OM dynamics at the supply-chain level (i.e. beyond a dyad). To address this significant gap in literature, our study relies on a three-echelon supply chain consisting of three independent organizations with the main unit of analysis being the order process initiated by the focal company.” (p. 896)
Eriksson (2010)	To increase the understanding of how various aspects of lean thinking can be implemented in a construction project and how they affect supply chain actors and their performance.	This paper utilizes the scientific literature on lean construction to develop a frame of reference on which the analysis of case study findings is based. (p. 394) The empirical pattern was then compared to the theoretical predictions in order to investigate differences and similarities between the empirical data and theory. (p. 398)

RQ2 (II): What Kinds of Practitioners’ Benefits Predominate in AR Articles in OM Research?

Figure 7 shows the number of studies classified according to the three types of practitioner contributions during the period of analysis (2000–2017). The majority (51) of the papers used an implementation approach, 3 studies were proposals, and 8 were classified as diagnosis.

It is important to emphasize that authors often had to classify practitioner’s contribution based on a few scattered sentences in the article, in which the authors summarized their contribution to practice, as there typically were no specific sections reporting practitioner’s contribution in a more detailed manner.

OPEN IN VIEWER

Figure 7.

Trends of practitioner’s approach in terms of chronological order.

The analysis of practitioners’ contributions confirmed that our framework based on diagnosis, proposals and implementation is justified. Papers for each approach were found, although the first two were much less common. This dominance has even increased in recent years, often with detailed reporting about very specific contributions of AR studies for the case companies. For example, Baker and Jayaraman (2012) reported a 27% inventory reduction thanks to an AR project. Similarly, Cagliano et al. (2005) reported a 5% lead-time reduction at the company they collaborated with. Visintin et al. (2017) reported the results of developing and implementing a surgery room scheduler at a hospital; the study compared the performance before and after implementation in terms of resource utilization, complexity of surgeries and bed utilization.

In some other cases, for example, Adebanjo et al. (2013), the description of the implementation is more general, such as the reorganization of the supplier selection process, and no specific results are reported.

In the proposal category, the main contribution for practitioners in the three identified papers were recommendations about specific improvements to be carried out in the company. In Neely et al. (2000), for example, the recommendation was a workbook that provided guidelines for the process-based design of a performance measurement system. Shaw et al. (2001) developed a framework that aimed to become a model of good practice for new product (or process) development (NPD) in the fine chemicals industry. Finally, Braz et al. (2011) focused on presenting a proposal to improve an energy company’s performance measurement system.

Many of the diagnosis papers appear to follow a similar conceptual approach. Based on the analysis of one or more specific case studies, the articles then developed a more general methodology. A representative example of this is Dey et al. (2015), as this research proposed both leading and lagging factors for supplier evaluation and developed a model based on the diagnosis of the strategic evaluation of suppliers in a company. In this study, the contribution for practitioners is not about the implementation but rather the framework to diagnose how to implement a model combining quality function deployment and the analytic hierarchy process. Hales and Chakravorty (2006) studied the critical factors that influence how Deming’s style of quality is implemented in a plastics company. This study is especially significant because the diagnosis outcome was produced after the implementation stage, but the main report is not about how the action was carried out but rather to describe a diagnosis of a specific issue related to the implementation, and this is also a significant outcome of the AR project.

Rigor as the Key Requirement to Enrich Research Contribution of AR

As McNiff (2016, p. 51) stated, “demonstrating methodological rigor in action research implies following the action steps of a project: (a) identify issues and raise research questions, (b) monitor practice systematically to generate data, (c) describe the action (gather data), (d) generate evidence to explain the data and make knowledge claims, (e) test the validity of knowledge claims, (f) make your AR methodology public and (g) disseminate findings and communicate the significance of knowledge claims.”

As shown in Figure 9, in the context of our interest, two aspects are crucial to enrich the research contribution of AR: (a) to exploit/to explode access and data collection and (b) to use AR as a meta-methodology.

Need for Relevance as the Basis to Enrich Practitioners’ Contributions

Meredith et al. (1989) defined AR as an ongoing cycle of research stages. Similarly, Susman and Evered (1978) referred to the “cyclical process of action research.” Thus, during the life cycle of an AR study, many cycles of diagnosis, planning, actions, results and diffusion occur. In this way, relevant outputs for both researchers and practitioners may be found. In this vein, the intermediate results form part of the AR spiral or cycle, although AR studies do not complete (or do not include) the complete cycle and manifest themselves in diagnosis or proposals.

The first identified opportunity is a better learning during and after each of the AR cycles illustrated, i.e. to really exploit the whole AR cycle. The goal should be to generate different outputs from the different AR stages, that is, diagnosis, proposals and implementation. This means that an AR study do not have always to imply “action” outputs (implementation) as long as the main requirements of AR hold: dual objectives, collaborative relationship in all the stages, rigor in methodology and relevant results for both academics and practitioners.

However, there is a relevant challenge associated to this opportunity: the need that AR projects are defined, managed, monitored and funded in a specific context. As Coughlan and Coghlan (2002) emphasized, monitoring and managing the interactions between researchers and practitioners along this cyclical process of an AR study might be critical to achieve the dual objective of such studies. In this way, it is crucial to develop “support infrastructures” that allow establishing a collaborative framework that lets the researchers develop the project from inside the organization. These “support infrastructures” permits practitioners to participate, interact and be more proactive and they can get continuous feedback during the process. Likewise, managers can use their experiential knowledge to reframe their understanding of situations they are close to. They need to consider the impact of organizational politics on the process of inquiry, who the major players are, and how they can be engaged in the process (Coghlan, 2001).

As it is shown in Figure 9, three types of support infrastructures for the development of long-term AR projects are proposed: (1) long-term agreements with firms or business associations, (2) national and international research projects funded by public organizations, and (3) educational programs.

With respect to the first option, Avella and Alfaro (2014, p. 195) explained how a business chair”—an agreement between a university and an external organization (firm, association of firms, public institutions, other kinds of organization) to manage and finance activities usually related to teaching and research”—can be a useful infrastructure for developing AR research projects in the field of OM. The role of such chairs is very important in terms of their dual contribution, because they are very much interested in getting both types of contributions—research and practice—in order to justify their existence. As such, a committee of business chairs encourages both practitioners and researchers to report all the contributions, including the intermediate outputs, which is a way of enriching the dual contributions. In addition, this infrastructure usually has an objective to develop research projects with dual contributions, which is the right scope for developing the AR methodology.

Secondly, one example of the development of long-term AR projects within the framework of national or international publicly funded projects is the study by Coghlan and Coughlan (2006), which forms part of the results of CO-IMPROVE, a European Union (EU) funded project. This study provided a design and implementation framework for a combined action learning and action research program (ALAR, as coined by Zuber-Skerrit, 2003), which aims to address collaborative improvement in the extended manufacturing enterprise. Moreover, this paper forms part of wider research, which has also received financial support from the European Commission. It describes an AR initiative with two objectives: (a) to facilitate continuous improvement of operations practice and performance through collaborative action learning in a small number of firms, and (b) to develop a contingent approach to address this need, which could be replicable both in Ireland and throughout Europe.

Thirdly, there are several studies about the benefits of using AR methodology within educational programs that promote AR projects based on doctoral theses or final course projects in master’s programs. Zuber-Skerritt and Fletcher (2007) highlighted the quality points for a doctoral thesis under this methodology and Coghlan (2007) presented the characteristics of a doctoral thesis using AR, where the PhD student is a manager who is both researcher and practitioner at the same time. A doctoral thesis is a rigorous piece of long-term learning and research in which there are at least two researchers, the doctoral student and the thesis advisor, who must request periodic reports and results.

Along these lines, the AR literature has looked at the role of “insider action research” (Coghlan, 2001, 2007; Coghlan & Brannick, 2001) when managers are engaged in action research in their own organization. This can also be applied to master’s programs that promote the development of projects based on a dual contribution and that involve both researchers and practitioners as supervisors of the project carried out by the master’s student.

Implications for the Field of Management

Operations management is the field of management in which AR is mostly used (Erro-Garcés & Alfaro-Tanco, 2020). But as there are studies that highlight the relevance of AR in other fields of management, such as marketing (Kates & Robertson, 2004), human resources (Githens, 2015), information systems (Baskerville & Wood-Harper, 1996) or project management (Sankaran et al., 2021), some of the findings of this study can hopefully be valuable for other fields of management. More specifically, the evaluation framework developed represents a tool that may help to assess the dual contribution of any research study based on AR and not only in the field of OM. Furthermore, the identified challenges can be extrapolated to other areas of management in order to reinforce the role of AR as a meta-methodology that helps to link better the interests of the university with the needs of companies.

Further Research

This paper opens different avenues for further research. First, regarding the database, a more extensive and detailed analysis of the descriptive information could allow us to elaborate an article about the state-of-art of AR in OM. In this way, the identification of research groups and to determine the main topics, countries and industries would help to analyze AR in OM in a deeper way and it could offer valuable insights. Second, to analyze not only the practitioners’ contribution if not taking into account the managerial implication of AR studies, that is to say, in which way the results can be generalized to other firms and sectors; to emphasize this point would help to enrich the contributions of AR studies. Third, traditionally, AR studies have contributed to “both science and practice” but, as explained with RQ2 (II) the practical contributions have mainly been focused on “implementations.” Thus, if further AR studies in the field of management actually use our framework, then the framework will be able to be validated. Finally, another possible line of research could be to test the framework in other fields of management, such as marketing or human resource management.