Abstract
Recent years have witnessed a surge of interest in what has been described as Evidence Based Management (henceforth, EBM). Its proponents assert that management practice should be based on a better appreciation of ‘what works’, as determined by the research evidence. This has attracted a varied critique, which suggests that the evidence base on most management issues is uncertain, and constrained by the interests of powerful organizational actors. EBM is depicted as an effort to marginalize more critical perspectives within the field. This article adopts a different approach. Drawing on a critical realist epistemology, it suggests that while research evidence is rarely conclusive it can aspire towards the progressive displacement of ignorance by knowledge. While some advocates of EBM would accept this, it is argued that such acceptance is more rhetorical than substantive, and is undermined by a standpoint which systematically downplays the power saturated organizational contexts in which evidence is assembled and employed in decision making. The key tenets of critical realism as applied to EBM are therefore considered. Critical realism acknowledges epistemological relativism, yet also accepts the need to construct robust causal explanations for social phenomena. Recognizing that organizations are co-created and co-defined by multiple actors, rather than only by managers, it is argued that we should replace the concept of ‘Evidence Based Management’ with the notion of ‘Evidence Oriented Organizing’.
EBM has been defined by its key proponents as the systematic use of the best available evidence to improve management practice (e.g. Briner et al., 2009; Latham, 2007; Rousseau, 2006; Rousseau et al., 2008). Much of the writing on the subject invokes the precedent of ‘evidence based medicine’, an approach ‘which draws on “science” and all its modernist trappings of truth and progress and is presented as rational and politically neutral’ (Hewison, 2004: 336). The last decade has witnessed an explosive growth of interest in the management version of this approach. Reay and colleagaues’ (2009) review identified 144 papers dealing with EBM since 1948, but only eight of these had appeared before 1995. Beyond the journal literature, several books from key figures have also been produced, aimed at both management and academic audiences (e.g. Latham, 2009; Locke, 2009; Pfeffer and Sutton, 2006). 1 Their overall intention is to have a transformative impact on both the theory and practice of management.
This article explores the strengths and weaknesses of the concept, and of the critiques that it has attracted. What counts as ‘evidence’ in the field has long been a contested issue, with scholars differing substantially in their assessments of the ontological status of organizations, and the consequent implications for management theory and practice (e.g. Clegg et al., 2006; Pfeffer; 1993; Van Maanen, 1995). There have also been intense controversies over what qualifies as ‘rigorous’ research, and the extent to which anecdotes, case studies, journalistic accounts and qualitative work should be deployed when developing hypotheses, or testing and refining key theories, particularly in relation to research on goal setting (Guest, 2007; Latham and Locke, 2009; Locke and Latham, 2009; Ordonez et al., 2009a, 20099b). A critique of EBM has thus emerged that challenges its alleged privileging of a management voice above that of other organizational actors, and questions its commitment to the establishment of a coherent knowledge base from which allegedly ‘best’ practices can be identified. From this perspective, the search for EBM is at best misguided, and at worst possibly part of an attempt to strengthen managerial power at the expense of other organizational stakeholders, while also enforcing a positivist and functionalist paradigm within the field of organization studies (e.g. Learmonth, 2008, 2009).
This article argues that a critical realist perspective can facilitate a deeper exchange on these issues. In particular, critical realism proposes a series of insights by which we can seek to hypothesize the causal or retroductive mechanisms underlying social processes, and by which we can at least aspire towards the gradual reduction of what is not known (Bhaskar, 1978, 1998a). This takes a more subtle account of the variegated interests which populate organizations than mainstream writers on EBM have yet recognized. Utilizing a critical realist frame of reference, I explore the core notions of the EBM movement, while illustrating how critical realism can offer a richer epistemological insight into the search for evidence, meaning, and theory in organization studies. This article therefore outlines the key propositions of the EBM movement, problematizes them, and outlines a critical realist framework that offers an alternative account of causality and theory building in organization studies. In so doing, I also suggest that we move on from the notion of Evidence Based Management, and reposition the discussion in terms of ‘Evidence Oriented Organizing’ (henceforth, EOO).
Evidence Based Management: its claims and intentions
The core argument in favour of EBM is stated succinctly by Rousseau et al. (2008: 476) as follows: ‘Effective use of evidence … means to assemble and interpret a body of primary studies relevant to a question of fact, and then take appropriate action based upon the conclusions drawn’. A sense of urgency pervades this literature, driven by a recognition that many organizational interventions occur despite a significant body of evidence suggesting that they will have either no positive benefits, or prove harmful. While many practitioners are devoted to what they see as ‘best practice’, Lawler (2007: 1033) has noted that ‘In some cases, there is simply no evidence that validates what are thought to be best practices, while in other cases there is evidence to suggest that what are thought to be best practices are inferior practices’. Notoriously, downsizing remains a popular palliative for organizational ailments despite abundant empirical evidence that it reduces trust, lowers morale, damages productivity and profitability and often results in the loss of knowledge that is vital for future organizational survival (e.g. Campion et al., 2011; Cascio, 2002; Morris et al., 1999; Tourish et al., 2004).
The solution, as Rousseau (2007: 1038) envisages it, is that scholars should ‘…provide, maintain, and update online, user-friendly, plain language summaries of the practice principles that the best available evidence supports, while sharing information regarding their effective use as well as their limitations’. Many have noted (e.g. Starkey and Madan, 2001) that a great deal of management practice is oriented around fads that are often derived from a proliferation of popular texts which owe little to research, rigorous or otherwise. However, they gain credence because management research is often perceived by managers as indifferent and/or irrelevant to their needs (Starkey et al., 2009). As envisaged by advocates of EBM, summaries of best practice that are derived from research can ease this problem, by providing managers with an empirically validated basis for more thoughtful, rational and effective action (Rousseau and McCarthy, 2007). It follows that management syllabi within Universities should take account of the key principles guiding the EBM movement, and thereby ensure that managers receive greater encouragement to ‘embrace scientific evidence’ (Charlier et al., 2011: 223).
The case for EBM problematized
However, the advocates of integrative literature reviews and best practice guides themselves acknowledge that all is not well with the task of producing them. Rousseau et al. (2008: 476) noted that the evidence on many management issues is ‘seldom assembled or interpreted in the systematic fashion needed to permit their confident use’. There is a lack of coherent, systematic and agreed knowledge that can be transmitted to managers, either during their formal education in business schools or afterwards. Moreover, many attempted literature reviews have been criticized as being overly descriptive, for including writers based on the implicit biases of those compiling the reviews, and for a lack of critical evaluation (Tranfield et al., 2003). The consequence is that research is often misused or misunderstood, and that limited and inconclusive findings are over generalized. Rousseau et al. (2008) view the reluctance of academics to address these issues by producing the literature reviews that are deemed necessary as a quirk of the academy, suggesting that organizational theorists have come to value novelty over convergence (see also Pfeffer, 2005). This compels them to engage in ever more abstruse theoretical squabbles, in an effort to differentiate themselves within what has become an increasingly crowded marketplace for ideas. The result is a neglect of practice that should be corrected, through a collective effort of will. This radical reorientation is a key goal of the EBM movement.
The assumption underpinning this is that the production of integrated studies is generally feasible. Even from within a purely EBM paradigm this is questionable. Rousseau et al (2008), in an otherwise strong advocacy of EBM, provide a key example of the difficulty, on the issue of incentive pay. Based on his survey of the evidence Pfeffer (1998) suggested that it does not motivate improved individual job performance. In contrast, an overview by Rynes et al. (2005) concluded the opposite, and proposed several mechanisms whereby it is claimed that improved job performance will result. While enthusiasts for EBM seem to believe that accumulating and summarizing more and more studies will produce ‘convergence’, despite the fact that many of them reach contradictory conclusions, this example suggest that the problem may be more deeply rooted than is imagined. There are inherent difficulties of study design, conflicting theoretical paradigms, sample sizes, and multiple variables which means that on many important issues there is no research convergence merely awaiting discovery, and to be then reported in practitioner friendly summaries. Rather, there is discord, conflict and deeply rooted vested interests that seem programmed to resist reconciliation. The urge to produce convergent studies fails to account for why some issues are studied and attract funding rather than others, or to offer a clear view of how research priorities should be determined. Managers, for example, might well be interested in studies into how downsizing can be ‘improved’, such as that of Campion et al. (2011). They are unlikely to be equally fascinated by studies into how resistance to downsizing can be made more effective, or inclined to offer access for case studies.
Accordingly, the research process which produces the evidence on which EBM bases itself cannot be value free and objective. As Weiner (2003) has noted, researchers sometimes report data selectively, choose informants for interviews that they believe will be particularly sympathetic to a given point of view and become cheerleaders for various interventions rather than dispassionate researchers into their effects. Ioannidis (2005) has argued that, for these and other associated reasons, most published research findings in most fields are in fact false. Proponents of EBM have yet to credibly indicate how studies which offer divergent findings, often caused by such factors, can be integrated so that they offer definitive (or close-to-definitive) conclusions on heavily contested issues.
Rather, the problem is evaded by rooting the case for EBM within an overtly positivist paradigm, despite occasional claims to a more pluralistic theoretical orientation. Thus, it is claimed that Evidence is the essence of human knowledge. It deals with the regularities our senses and measuring tools can detect. Scientific evidence is knowledge derived through controlled test and observation … evidence-based management is the complimentary (sic) use of scientific evidence and local business evidence. (Rousseau et al., 2008: 481)
Knowledge is depicted here as objective, value free and weighted in favour of methods such as experimentation that are overwhelmingly quantitative in nature. Certain forms of knowledge and research methods are favoured over others (Morrell, 2008). Similar problems have been identified in debates over the role of systematic review in educational research, with those advocating its use (e.g. Evans and Benefield, 2001) allegedly prioritizing both positivist research for inclusion in such reviews, and also applying positivist models to the job of producing the reviews themselves (Hammersley, 2001). I noted, at the outset of this article, that the EBM movement draws explicitly on the precedent of evidence based medicine. But even within healthcare there have been numerous controversies on such issues as the quality of various types of evidence, and the relative weight that should be attached to some forms of evidence over others; the problems that have been found in attempts to change practice in the light of evidence; and how various professional groups respond, or ought to respond, to different forms of evidence (Hewison, 2004). While similar issues arise in the case of EBM, the challenges are mostly side-stepped by its advocates. An approach is adopted in which the world of management practice is simply ‘out there’, awaiting discovery. The issue of how to adjudicate between different truth claims is thus purely technical—we need to do more studies, until the issue in question is resolved. However, as one leading exponent of critical realism within management studies has expressed it,
‘There is no theory-neutral observation, description, interpretation, theorization, explanation, or whatever. There is, in other words, no unmediated access to the world: access is always mediated. Whenever we reflect upon an entity, our sense data is always mediated by a pre-existing stock of conceptual resources which we use to interpret, make sense of and understand what is, and take appropriate action’ (Fleetwood, 2004: 30).
An additional problem, from the standpoint of determining what evidence should be used and of assessing its objectivity, is that precisely what is studied, and how it is studied, itself reflects powerful sectional interests. Managers are not merely waiting for research to facilitate subtle changes in course direction. Rather, they are committed to many of their practices, which reflect strongly held views of the world and are perceived as being in their own interests. Rousseau and McCarthy (2007) have themselves anticipated that managers may be resistant to possible research findings—such as any that suggests they should de-centralize decision making, and place less emphasis on powerful, charismatic visionary models of leadership.
This creates a twofold problem for EBM. Firstly, it places limits on the kind of research that is likely to be conducted. Funding increasingly follows lines of inquiry deemed practical, applied and ‘relevant’ to current rather than hypothetical future business needs. Secondly, the assumption that summaries of research findings will be welcomed by managers, and that insight alone will be sufficient to reorient their practice, may be in error.
The evidence base for EBM
A key paradox within the debate is that the evidence base on which EBM rests is itself questionable. Reay and colleagues’ (2009) overview noted that 53.6% of the papers on EBM that they identified argued for or against it purely on the basis of the author’s opinion, rather than offering any empirical evidence in either direction. This may arise, in part, from the nature of the academic publication process, which militates against the steady accumulation of empirically informed wisdom on management practice. A study of papers in the Academy of Management Journal over five decades highlights a growing focus on theory development, and hence construct proliferation, even when such constructs differ little from what existed before (Colquitt and Zapata-Phelan, 2007). Such practices hinder the accumulation of empirical information about issues for which no theory yet exists, however pressing a problem may be in the real world of management practice (Hambrick, 2007). In consequence, scholars enter into ever higher states of specialization. Few are therefore equipped to deal with rival truth claims, or are impartial enough to do so (Donaldson, 2009)—a challenge for those attempting to produce integrative literature reviews.
Consistent with this, as Rousseau et al (2008) noted, major academic journals are generally not interested in replication studies. Journal editors usually insist that articles must offer something ‘new’, and tend to regard replication as a recycling process that lacks sufficient novelty (Madden et al., 1995). This discourages researchers from utilizing perfectly adequate existing theory to explain empirical observations, in a search for novelty that displaces the gradual accumulation of more and more relevant data. But the dearth of replication in organization studies means that rogue findings or poorly designed studies often go unchallenged, and can become conventional wisdom (Tsang and Kwan, 1999). Thus, Kacmar and Whitfield (2000) found that only 9% of the theoretical models in articles published in the Academy of Management Review are ever tested. They concluded that ‘ … the vast majority of published ideas in management are submitted to no tests at all, a handful are submitted to one test, and only a minuscule few are tested over and over again or in multiple ways’ (p. 1350). The evidence base for many management ideas is thus weak. Even where evidence exists, it is often not as firm as its advocates maintain—a point acknowledged by one key advocate of its use in the field of healthcare (Stewart, 2002). Within management studies, for example, researchers may place too much emphasis on statistical significance (Starbuck, 2004) rather than effect sizes (Ziliak and McCloskey, 2008), thereby overstating the generalizability of their data. All this poses a problem for literature reviews that seek to produce actionable knowledge based on robust evidence—there may not be enough of it to serve this purpose, nor much prospect of it emerging. While this remains the case, the potential to produce viable, compelling and agreed integrative literature reviews is limited.
Assumptions of rational decision making
A core assumption within the EBM movement appears to be that decision making is, should be or can be a wholly rational process, in which for example self-interest is subordinated to a calculation of a wider group or societal interest. Rynes et al. (2002) noted with some alarm that less than 1% of HR managers read the academic literature regularly. The proffered solution is that they should do so, thereby overcoming the problem that they are insufficiently educated in how ‘to know or use scientific evidence’ (Rousseau, 2006: 262).
‘Irrational’ management action—i.e. that which damages a wider organizational interest—is depicted here as flowing from a simple absence of knowledge, rather than self-interest, malign intent or wilful ignorance (Learmonth and Harding, 2006). Yet it is clear that action which damages a hypothetical collective organizational need may well be driven by self-interest, vested interest or disinterest in the full awareness that these wider considerations are being endangered. Hope’s (2004) study of the evidence gathered on an initiative to reduce burglary rates found that the policy implications were ignored when they conflicted with the preferred options of those making actual decisions, with even the analysis of the data presented in official reports differing markedly from that of those who conducted the research. Flyvbjerg’s (1998) detailed study of town planning in Denmark demonstrated that the greater the power held by individuals or groups the less they felt the need to assemble evidence or consider means-end justifications. Studies of Enron’s collapse suggest that the organization’s key leaders may have been well aware that their business model was unsustainable, but set this inconvenient truth aside in the interests of their own short term personal interests (Tourish and Vatcha, 2005). Beyond these factors, as Hodgkinson (in press) points out, busy decision makers also frequently rely on intuition when making decisions, rather than more formal sources of evidence.
Moreover, it is not clear that even when evidence is accumulated, and considered, it will point unambiguously in a particular direction. For example, a study of health prioritization in New Zealand explored how a number of government agencies all gathered evidence on the issue, driven by the conviction that policy should be informed by better evidence. They then reached different policy conclusions, despite sharing a common evidence base (Tenbensel, 2004). Thus, when it is not obvious that data should be interpreted in only one way, extra information may generate doubts about action rather than clarity on the way forward (Learmonth, 2006; Weick, 2001). The result can be stasis—the opposite of the EBM movement’s intentions.
In short, studies from multiple perspectives demonstrate that action is driven by many imperatives that sometimes include rational reflection but which also often displace it. However, advocates of EBM tend to assume that if managers simply realized that their actions were damaging, irrational, selfish or insufficiently rooted in an evidence base they would change their model of decision making. In violation of their own injunctions to utilize evidence in support of arguments, they offer no data to support this conclusion.
Privileging managerial definitions of reality and evidence
A further critical issue in the field is that the intended audience for management research envisaged by proponents of EBM is entirely composed of managers. Typically, the literature on EBM does not acknowledge such issues as power, conflict and inequality (Learmonth and Harding, 2006). Rather, employee/ managerial interests are conflated in a manner that is even less credible after the recent economic recession. Thus, EBM advocates routinely equate managerial insight and action with the interests of employees (e.g. Bartunek, 2007). It is assumed, without any evidence, that the managerial voice articulates a universal interest, and that what preoccupies managers must also preoccupy other organizational actors, to the same degree and from the same perspective.
It follows that EBM should focus overwhelmingly on issues of concern to managers, and that the research agenda of management academics should be shaped accordingly. It therefore shares common ground with those who advocate that management research should be more ‘relevant’ to the needs of practitioners (e.g. Gulati, 2007; McGahan, 2007; Tushman and O’Reilly, 2007). Rousseau (2006: 256) makes the intent explicit: ‘Through evidence-based management, practicing managers develop into experts who make organizational decisions informed by social science and organizational research—part of the zeitgeist moving professional decisions away from personal preference and unsystematic experience toward those based on the best available scientific evidence’. While Briner et al. (2009: 19) suggests that decision making informed by EBM should also consider ‘the perspectives of those people who might be affected by the decision’, they also affirm that ‘the start of the process is the practitioner’s or manager’s problem, question or issue’ (p. 23).
The general assumption is that managers are the ‘experts’ on the multiple issues that confront most workplaces, and thus are the ultimate arbiters of what matters. This forced correlation between the assumed expertise of managers on the one hand, and ‘science’ and ‘evidence’ on the other, suggests that managerial decisions can be placed beyond dispute (Learmonth, 2011). It is equally plausible to suggest that expertise may be more widely distributed than such hierarchical models acknowledge. Failing to recognize this, some EBM advocates blur the line between research and consulting. For example, Cascio (2007: 1012) urges academics to design their empirical studies with the possibility of implementation to the fore, suggesting that such research ‘will find a ready market of available buyers in the marketplace for ideas’. This, clearly, has implications for the kind of research likely to be done, and which will find eventual inclusion in integrative literature reviews.
The argument has therefore been made that the future of business schools would be more secure if academics were to adopt a greater focus on the practical problems faced by managers (e.g. Bartunek, 2007). If managers and employees shared identical interests this might not matter. But if this symmetry is absent the interests of particular stakeholders will become privileged above those of others, with implications for what kind of questions get asked, what research gets done, what kind of knowledge is accumulated and hence summarized in practitioner friendly summaries, and what its eventual impact on practice is likely to be. This trend is already well advanced. As Collinson (2002: 42) has noted, it is now difficult ‘to undertake qualitative, critical empirical research … Where research is funded, it is increasingly managerialist and prescriptive and tends to be sub-contracted to research assistants’ (emphasis in original). While Hopwood (2009: 517) has complained that critical scholarship puts ‘less and less effort … into really probing fieldwork’, the political, institutional and social context in which such research occurs suggests that the problem cannot be remedied merely by an act of will. Rather, the growth of managerial controls, assessment exercises, audits and associated demands for accountability implies that the tendency towards favouring certain easily quantifiable kinds of research above others will intensify (Bell, 2011; Learmonth et al., in press). The privileging of a managerial voice above others, routine in the EBM oriented literature, further compounds the problem. Integrative literature reviews will therefore draw on data that is inherently biased towards the priorities of powerful interest groups, and on findings which reflect sectional rather than unitarist interests.
Critical realist perspectives, in common with other critically oriented paradigms, attach more significance to what have been described as ‘the institutionalized frameworks of power relations that constitute the ‘structures of domninancy’ within which specific governance mechanisms, programmes and practices emerge and are, eventually, legitimated as stabilized regulative orders’ (Newton et al., 2011: 14). It is such processes of legitimization, reification and naturalization that the EBM movement seems to ignore, and which I argue limits the theoretical utility of the notion of EBM as a means of understanding the uses and abuses of ‘evidence’ within the broad field of organization studies. The issue is whether a reappraisal of its core tenets from a critical realist perspective can offer a more fruitful means of engaging with the need to address the social problems that afflict the world, without attaching excessive priority to sectional interests in the manner which advocates of EBM seem prone to do.
A critical realist perspective
Critical realism offers several inter-related presumptions about the nature of the social world that have the potential to transform the present debate about EBM. In particular, it offers ‘A commitment to a realist ontology, i.e. to the existence of causal mechanisms whose interactions generate the events and occurrences of the world. These mechanisms may be unobservable directly (e.g. social structures or ideas and categories) yet we can accept their existence because of their causal efficacy. This undermines the empiricist illusion of the primacy of empirical data (the actualist fallacy)’ (Syed et al., 2009). But critical realists also acknowledge what Lawson (2003: 162) has described as ‘epistemological relativism’: that is, they recognize that any depiction of social reality is to some degree affected by the social conditions that give rise to its production. This position is, of course, shared by social constructionists (e.g. Pfohl, 2008). However, critical realists emphasize, to a greater extent than most constructionist accounts, that social reality ‘consist of much more than the linguistic, symbolic, and discursive resources and practices through which we come to describe, understand and reflect on it’ (Reed, 2009a: 434). While acknowledging that our knowledge about the world is relative and socially constrained, critical realism affirms a distinction between the transitive or subject dependent aspects of knowledge and the intransitive or actually existing and real objects of our knowledge (Bhaskar, 1998b). Of course, this distinction is not so hard and fast in the social sciences, since the social world is itself socially constructed and includes knowledge within it (Sayer, 2000). As Al-Amoudi and Willmott (2011: 31) summarize it, ‘…the transitive is nested in the intransitive and agents’ concepts are constitutive, though not exhaustive, of the social’.
The objective of inquiry therefore remains to increase what is known and reduce what is not known. This also compels a recognition that the positivistic split between facts and values is fallacious, since all social theories are evaluative as well as descriptive (Mingers, 2009). Management straddles the domain between the objective and the subjective, and while aspiring to achieve greater knowledge about the world must recognize that elements of both ignorance and subjectivity will always remain. To suggest that improvements in research techniques will somehow yield ‘better’ end products and thus overcome this problem is to miss the point (Fox, 2003). Rather, there is space for both knowledge production and a contest about values, ethics, meaning, interpretation, significance and importance.
In particular, critical realists seek to explore the underlying causal factors or mechanisms that produce observable events (Reed, 2005a). This involves crafting hypotheses through a process of conceptualization described as ‘retroduction’. Possible causal mechanisms are proposed from observable events, and further research seeks to test their efficacy. Theoretical abstraction and model building is central to this process (Reed, 2009b). Knowledge is therefore always provisional. As Al-Amoudi and Willmott (2011: 30) express it, ‘Retroductive judgements provide possible but historically contingent explanations of certain states of affairs’. Thus, it is argued that ‘some representations constitute better knowledge of the world than others’ (Fairclough, 2005: 922). The social world exists independently of our perceptions of it, requiring us to distinguish between ontology—what is—and epistemology—how we know it. While linguistic systems permit a theoretically infinite variety of accounts of any given social situation, in actual practice the range of accounts offered is socially limited by the manner in which language systems and other aspects of social structure interact with each other (Fairclough, 2005). This version of realism is critical in that it recognizes the limits human perception and social relations place on our appreciation of reality, and that our perceptions of the social world are inherently value laden (Syed et al., 2009). But it also acknowledges the limits on relativism, stresses the regularities of human behaviour as well as its discontinuities, and explores what is known as well as what is unknown. Each such account of reality is therefore constrained, pinched and tenuous—but seeks to move us ever closer to a more reliable depiction of the world as it is.
Hesketh and Fleetwood (2006) illustrate the management implications in terms of the debate about the relationship between particular bundles of Human Resource Management practices and organizational performance. As they note, statistical associations do not constitute an explanation. If an association between such practices and performance is shown to exist, we still need to know why such an impact has arisen. For critical realists statistical significance is a point of departure for the deeper exploration of complex causal connections and the production of more robust explanations for observable events. This further implies that what is sought is the progressive erosion of ignorance in favour of greater knowledge, and the creation of spaces where ethical and other issues can find a greater purchase in the research process than traditional positivist approaches normally allow.
While some advocates of critical realism in management studies may err on the side of suggesting that foundational truth claims are more accessible, possible and certain than is consistent with a due acknowledgement of epistemological relativism (see Al-Amoudi and Willmott, 2011, for full discussion), this article utilizes a critical realist perspective to assert that we can seek to reduce what is unknown, without imagining that the task will ever be finally completed. Thus, in contradistinction to a purely positivistic perspective, the socially constructed nature of our knowledge about the world is acknowledged. There is no ‘absolute and universal guarantor for anything’ (Reed, 2005b:1665). As Czarniawska (1997: 55) puts it, ‘the construction of meaning is always social’. Facts aren’t just out there, awaiting impersonal discovery. But critical realists emphasize that not all knowledge claims are equal. Some interpretations of the world—including the social world—are closer to an accurate depiction of causality than others, and are subject to empirical tests which may be imperfect but which aspire towards the gradual reduction in what is unknown.
EBM and critical realism: the paradox of inconsistent application
Rousseau et al (2008), in particular, has made some attempt to locate EBM within a perspective informed by critical realism, urging the use of multiple methods and triangulation to probe various causal accounts of organizational phenomena. However, this application of critical realism seems limited to a somewhat formal acknowledgement of methodological pluralism that is, in practice (and as discussed above) subordinated to the greater priority placed on positivistic approaches. Moreover, Rousseau (2006: 267) also asserts that evidence ‘affords higher-quality managerial decisions that are better implemented, and …yields outcomes more in line with organizational goals’. Organizations are thus reified and hierarchy naturalized, with the foregrounded assumption that goals determined by senior managers capture a unitarist organizational interest. This privileging of a managerial voice above others is inconsistent with the critical realist understanding that social reality is not uniformly perceived. While ‘observation, interpretation and explanation are mediated by theory’ (Al-Amoudi and Willmott, 2011: 34), they are also mediated by stratified social relationships and discordant interests. Organizational actors form their own perceptions of phenomena that are more or less accurate and which enter into conflict with accounts produced by others. Accounts of social reality can of course be subject to testing. But the critical perspective which critical realism brings to bear suggests that complete coherence and agreement is unlikely to be achieved. This means that the stress which EBM puts on the value of integrative literature reviews, to establish a consensus which management practitioners will find useful, is misplaced. The invocation of critical realism in the context of EBM is, therefore, a limited and contradictory use of the theoretical possibilities that critical realism affords on the issue.
The question thus is: can a more consistent critical realist orientation inform a more fruitful agenda on the issue of EBM, particularly in terms of the notion of Evidence Oriented Organizing (henceforth, EOO)?
Towards the notion of Evidence Oriented Organizing
The notion of EBM presupposes that organizations have a unitarist interest, and that managers are the prime arbiters of what matters. But, from the critical realist perspective articulated in this article, organizations can be viewed as simultaneously differentiated and integrated, on a number of dimensions. I therefore suggest that the notion of ‘evidence based organizing’ is more likely to acknowledge the variegated interests that populate organizations, lead to research agendas which are more inclusive in nature, and suggest a more pluralistic approach to what counts as evidence. A different view of organization is intrinsic to this approach.
As Cooren et al (2006: 2–3) express it: organization emerges in the interplay of two interrelated spaces: the textual-conceptual world of ideas and interpretations and the practical world of an object-oriented conversation directed to action … The resulting image of organizational interaction is of an essentially fluid and open-ended process of organizing, in which inherited positions of strength are exploited creatively by the participants.
The recognition that sensemaking, agency and the processes whereby co-orientation between organizational actors is mediated through language means to acknowledge that organizing is ‘an act of juggling between co-evolutionary loops of discursive phenomena’ (Guney, 2006: 34). Organizations therefore struggle to create shared meanings between organizational actors. Thus, the privileging of a management voice in determining the importance of issues to be researched or the utility of research outcomes, as is routine in the EBM oriented literature, reifies power relations in a way that is explicitly challenged by critical perspectives (e.g. Clegg, 2009). As Hodgkinson (in press) expresses it: ‘the idea that social science can be applied in the workplace in an apolitical and interest-free fashion is a fundamentally untenable position’.
The notion of EOO is thus consistent with challenges to the view of organizations as having a taken for granted existence as material entitles, separate and apart from their discursive constructions (Gergen, 2000; Westwood and Linstead, 2001). Rather, the organizational world can be depicted as a plenum of agencies. Their intersection is what constitutes reality. Thus: According to this approach, we never leave the terra firma of interaction, but we then need to acknowledge the interplay and contribution of human and nonhuman doings. In other words, the socioconstructionists are right to start from interaction, but they need to widen the extension (and intension) of this concept to recognize that nonhumans also do things. As for the materialists, they are right to recognize that humans are acted on as much as they act, but they need to recognize that using analytical shortcuts like ‘structures, ‘power’, or ‘ideology’ to account for what influences human action and discourse does not do justice to the complexity of the phenomena they are studying. (Cooren, 2006: 82)
But, simultaneously, it is also recognized that ‘real world material conditions and social relations … constrain and shape the discursive construction of organizational reality in any particular socio-historical situation’ (Reed, 2005a: 629). Agents and structures are mutually constitutive. Thus, advocates of EBM afford a privileged epistemological status to managerial accounts of organizational life, with their priorities, perspectives and prescriptions viewed as articulating a reality that simply ‘is’, and which therefore lies beyond interrogation. This kind of discourse can be employed, as Willmott (1993) argues, to impose a uniform definition of reality on organizational actors which, although couched in timeless and universalist terms, may actually be more consistent with particular sectional interests than it acknowledges.
However, it is possible to view organization as dealing with how ‘socially constructed institutions are reproduced and transformed by the accounting activities of people in interdependent (joint) action as they make sense of what they do together’ (Varey, 2006: 191). Organization, ultimately, emerges as a phenomenon that is co-produced and co-reproduced (within certain limits) by the discursive interactions between organizational actors. This does not exclude causal analysis. Rather, and consistent with process perspectives on organization theory, causality is seen as rooted within a ceaseless chain of preceding and succeeding events (Langley, 2009). Organizational phenomena are regarded ‘as (re)created through interacting agents embedded in sociomaterial practices, whose actions are mediated by institutional, linguistic, and objectual artefacts’ (Langley and Tsoukas, 2010: 9). Flowing from this, and informed by critical realism, an EOO perspective suggests the need to acknowledge that all organizational actors have a stake in determining what are important research topics, rather than just managers. It also means that research on these topics has to encompass a plurality of voices and accord them due weight in accounts of organizational phenomena, as well as in identifying courses of action that may seem to be suggested by research investigations.
The issue of appraisal demonstrates the theoretical and practical implications that arise. Rotundo (2009), in a text devoted to showcasing the implications of EBM in the field of organizational behaviour, produces a guide for managers that purports to indicate how performance appraisal can be used ‘to improve individual and firm performance’. Managers’ position of power over others is taken for granted throughout, alongside other familiar domain assumptions such as the unproblematic nature of defining individual and firm performance. Employee perspectives are marginalized. The practice of management is thus reified and naturalized—that is, socially created entities produced by power relations are treated as purely natural and hence enduring artefacts. This approach also avoids any consideration of the relationship between means and ends, and so constitutes an example of a performative agenda that purports to serve unitary ends while actually advancing sectional interests. Consistent with the work of Townley (2003), an EOO perspective would challenge the possibility of managers acquiring objective knowledge about employee performance, and foreground the question of who controls the labour process at work—and how this control is exercised. It would problematize the issue of the information that is available to appraisers, locating its availability in the context of surveillance mechanisms and power interests (Sewell and Barker, 2006). In short, it would suggest that such issues as performance, goals and accountability are co-created constructs and that evidence on any aspect of them needs to address the perspectives of the many organizational actors that are involved rather than restrict its imagined audience to a managerial elite.
This issue has a parallel in ‘evidence based medicine’, which as discussed above has directly inspired the EBM movement. Oakley (1984) notes a tendency within the collection of databases and evidence summaries in medicine provided by the Cochrane Collaboration and various other organizations to prioritize the views and perspectives of professionals. Her work suggests that radically different research questions could be raised in the field of maternal care by drawing on the perspectives of mothers rather than doctors. Since then, particularly in social care, greater care has been taken to involve service users, whose perspective adds a richer layer of insight to discussions of policy-research links (Young et al., 2002). The notion of EOO applies a similar approach to the field of organization.
It may not always point to easily actionable conclusions, as imagined by advocates of EBM—e.g. ‘The manager should recognize employees for goal attainment and explain how the goal helps the group, the firm and the individual’s career development’ (Rotundo, 2009: 94). Such depictions of employees as the relatively passive recipients of wisdom handed down by impartial and objective managers, in pursuit of a unitary interest, are incompatible with the co-creation of organizational action and the view of organizations as a plenum of agencies. Rather, an EOO perspective would make use of a wider range of perspectives in shaping a research agenda, deriving data, evaluating its significance and determining its applied implications. It therefore redefines the notion of ‘relevance’ and implications for practice in terms of a much wider organizational constituency.
‘Evidence Based Management’ or ‘Evidence Oriented Organizing’?—Six key propositions
The previous discussion suggests that the notion of EOO can have a transformative impact on the discussion of EBM in six key respects:
Advocates of EBM have suggested that integrative literature reviews should provide managers with definitive insights into ‘what works’. It follows that managers will be able to take particular courses of action—on such issues as performance appraisal interviews—with a high degree of confidence that they will have predictable effects (e.g. Rotundo, 2009). On the other hand, critics have argued that the evidence base behind management ‘science’ is so affected by the interpretive tropes used by humans to process information that high degrees of ambiguity and contestation will remain. A critical realist epistemology would suggest that literature reviews should aspire to the more modest goal of the progressive reduction of ignorance. Evidence is emergent, fluid, multifaceted, contested and open, rather than definitive, solid, unproblematic and closed. As Bhaskar (1998b: 14) expressed it: ‘reality is a potentially infinite totality, of which we know something but not how much’. But while total knowledge about anything is impossible to achieve, some causal explanations approximate more closely to reality than others. Literature reviews and other forms of knowledge production should therefore aim to acknowledge both what is known and what is unknown about important organizational phenomena. For example, such reviews should clearly discuss what light the evidence base sheds on those causal theories that have been debated within the research community, and where we can be more or less sure that they are incorrect.
Integrative literature reviews should make critical use of a wide range of evidence types—including qualitative and quantitative research, case studies, journalistic accounts and anecdotes. Such forms of evidence are useful for the process of retroduction and can inform the development of theories that will lead to continuously improved forms of empirical research in due course. Research should therefore avoid privileging particular methods of data collection and analysis above others (Ackroyd, 2004). Choices of method ought to be appropriate to the object of study (Lawson, 1997), difficult as this sometimes is to determine. Thus, in opposition to purely positivistic and functionalist theoretical paradigms, the goal of social science is viewed as being to generate plausible rather than definitive hypotheses about causal relationships and from which fresh streams of research can be developed. Both quantitative and qualitative methods have strengths and limitations, including the difficulty of generalizing from inherently limited data sets. But if they are properly utilized they may still serve the aim of reducing ignorance, increasing knowledge and generating insight. Thus, causal accounts are embedded in deeper process studies of preceding and succeeding events, mediated through linguistic and non-linguistic artefacts.
A critical realist perspective recognizes that our perceptions of the social world are constructed by disparate interest groups, and that the interpretations of the world held by these interest groups often reflect competing rather than harmonious material interests. It follows that the discussion of the role of evidence in management practice should not privilege the interests or the insights of one organizational grouping above others in its attempt to produce actionable knowledge. Too often, in discussions of either EBM or the ‘relevance’ of management research, it is assumed that what is socially useful and academically rigorous equates to an agenda that will be deemed useful above all by managers (e.g. Hughes et al., 2011). The notion of ‘user involvement’ in the research process is restricted to managers. But, as Grey (2001: S30) has pointedly asked, ‘by what fiat are the users of management research held to be the corporations and the managers?’ Consistent with Ferlie, McGivern and Moraes’s (2010) recommendation that business schools should embrace a ‘public interest’ school of management, which rejects the primacy of shareholder value and the view that only a managerial interest should be served by business schools, a critical realist perspective would assert a role for management research that prioritizes the wider public interest. Recent events demonstrate afresh that the fate of business concerns not only managers, but society as a whole. It follows that integrative literature reviews, or other forms of knowledge production, would seek to reach beyond a management audience. Rather, evidence should be synthesized and addressed to the wide variety of audiences who have a vested interest in business practices—including employees, customers, trade unions and Government. Accordingly, and in a project of greater inclusivity, it may be more fruitful to recast the field as one concerned with ‘Evidence Oriented Organizing’ in place of ‘Evidence Based Management’.
Issues of ethics and values should have a central place in management research. These cannot be reduced to ‘what works’ or to what managers decree as having immediate, practical significance. It means that ethical principles should guide the selection of research topics as well as the practice of empirical research, and be informed by a wider consideration than immediate business success, to a far greater extent than is normally acknowledged in the EBM literature. The bald invocation of ‘what works’ often masks deeper issues of entrenched power relationships, and is rarely designed to liberate the voices of actors perceived as being marginal in various social forums. However, as Sue Clegg (2005: 415) has argued, ‘One of the underlying themes in critical realism is that of critique and emancipation, and evidence properly understood aligned to clear theoretical argumentation is part of this project’. In terms of EBM, the question ‘what works’ can therefore be reframed as ‘what works for who? Why? For how long? In whose interests?’ Such a realist synthesis, as Pawson (2002) argued, also helps us to delineate under what circumstances this will apply. This requires the open acknowledgement of the limitations on any knowledge that is generated.
A weakness of some positions associated with Critical Management Studies is that they appear to be concerned more with critique than positive engagement (Spicer et al, 2009; Thompson, 2004; Tourish and Barge, 2010). Positivist perspectives, on the other hand, are relatively insensitive to ambiguity and under-estimate the challenges of constructing accounts of reality that reflect the perceptions of differentiated organizational actors. But, from a critical realist perspective, these tensions do not preclude the space for engagement with real world issues and business challenges. Critical realism recognizes ambiguity and indeterminacy, but also embraces the challenge of identifying what actions, policies and solutions are more or less likely to have a positive impact on organizational practices and wider social well-being. It therefore moves towards a positive engagement with the multiple stresses facing the world we inhabit. This means recognizing and seeking to explain the power saturated organizational contexts in which management theory seeks to find traction. But it also acknowledges those aspects of organizational reality other than power and domination which must be addressed.
Critical realism recognizes that human action is impelled by many factors, in addition to and sometimes in place of rational reflection. It follows that merely making evidence available may not be sufficient to change the behaviour of managers or other organizational actors. While rational action should remain a key goal, it should also be recognized that social action flows from socially constructed perceptions of reality that are in part created by the working out of conflicts between vested interests located within organizational structures. As with the notion of progressively reducing what is not known, an approach to these issues informed by critical realism would suggest that we embrace the goal of progressively reducing irrationality and increasing the space for rational thought and action, without imagining that this process can ever be completed.
Conclusion
Consistent with the notion of increasing knowledge that is advocated here there is no suggestion that critical realism offers a perfect boilerplate for management research or can completely overcome many of the difficulties described in this article. For example, I am not suggesting that it can be ‘the’ solution to the problem of how power impacts on our approach to either generating or utilizing evidence. Power interests are at work within the academy just as they are within ‘the real world’ of organizational practice, and often distort the academic effort, including when that effort is undertaken by either constructionist or critically realist minded scholars.
I also recognize that, while not all truth claims are equally valid, we have no infallible or generally agreed means of adjudicating between them. While we can with reasonable confidence divest ourselves of some truth claims (e.g. ‘the world is flat’), we must be much more tentative when we look at claims for the social world, particularly when the claims are of a causal nature. In addition, ‘critiques of social phenomena are enormously contentious because it is difficult to establish agreement about what constitutes problems, solutions or improvements, and whether the latter are feasible’ (Sayer, 2000: 158). It follows that ethical issues are fraught with challenges of defining power, norms and values, and that few definitions are likely to offer a universal appeal. But I have also argued that, if we acknowledge the importance of improving the world in which we live, it is vital that we grapple with these issues theoretically and empirically. Rather than discourse around the notion of ‘Evidence Based Management’, I have therefore advocated that we recast the discussion around the notion of ‘Evidence Oriented Organizing’. The debate between advocates of EBM and its critics has thus far tended towards a dichotomous clash between ‘believers’ and ‘heretics’ (Morrell, in press). I am suggesting that utilizing a critical realist perspective more thoroughly than has been done so far has the potential to ease some of these tensions, even if it can never fully overcome them.
Thus, the idea of EOO envisages a process that gradually reduces ignorance, and embraces the admittedly complex challenge of establishing some evidentiary criteria for adjudicating between rival truth claims. It is important for organization theorists to engage with the complexities of the world, and progress occurs when we explore possible solutions to our ailments (Starbuck, 2003). I argue that the progressive reduction of what is not known is a worthwhile goal for organization research, despite the imperfections that attend to any such effort. A state of absolute knowledge will never be reached: the goal remains more or less beyond our grasp, however much progress towards it is attained. But the multiplicity of problems that beset society suggests that the effort to make such progress is one that scholars should be eager to embrace.