Abstract
The model of evidence-based practice (EBP) directs clinicians to integrate the best available research evidence, clinical expertise, client preferences and values, and social and cultural factors in the assessment and treatment of psychological problems. Despite its many strengths, the five-step inquiry component of the EBP model suffers from several conceptual and practical problems that make it difficult to implement in practice. In this article, we first outline the transdisciplinary EBP model. Second, several criticisms of the overall EBP model are outlined and briefly discussed. Third, five pressing problems in the inquiry component of the EBP model are identified: (a) information overload, (b) a focus on questions rather than tasks, (c) neglect of theory, (d) difficulty dealing with conflicting evidence, and (e) an oversimplified view of the role of values in research and practice. Fourth, we suggest ways of modifying the inquiry part of the model to address these problems.
Keywords
Clinical psychology has a long tradition of empirically oriented practice, in which clinicians are encouraged to base their interventions on the best available research evidence and to think critically during the assessment, case formulation, and treatment phases of practice. Clinical psychology training programs have sought to produce scientist-practitioners, individuals who combine the skills of a researcher and therapist within a single professional role (Haynes et al., 1999; Shapiro, 1979). According to this scientist-practitioner model, the aim of clinical inquiry is to produce scientifically grounded explanations of clients’ problems, and to then select empirically supported interventions specifically designed to address them. The model of evidence-based practice (EBP) is based on this earlier conceptualization of the clinical psychologist role and similarly directs clinicians to integrate the best available research evidence to guide therapy assessment and treatment (Lilienfeld et al., 2013; Spring & Neville, 2011). However, the EBP model goes beyond the earlier scientist-practitioner model by explicitly incorporating clinical expertise, client preferences and values, and social and cultural factors into the clinical decision-making process.
The EBP movement was initially adapted from medicine (Sackett et al., 2000) and has since been adopted by several health disciplines as a helpful practice framework. These disciplines include social work (Drisko & Grady, 2019), nursing (Bucknall & Rycroft-Malone, 2021), public health (Brownson et al., 2003), forensic psychology (Gannon & Ward, 2014), and clinical psychology (Spring & Neville, 2011). The American Psychological Association has endorsed the EBP approach and defined it in the following way: “Evidence-based practice in psychology (EBPP) is the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences” (APA Presidential Task Force on EBP, 2006, p. 271). In recent years, a transdisciplinary model of EBP has been developed, which builds on the common elements and unique strengths of each of the social work, medicine, public health, and psychology models (Satterfield et al., 2009).
Our major argument is that, despite the merits of EBP, there are several problems with current formulations of aspects of the model that threaten its theoretical integrity and arguably make it harder to implement clinically. An uncritical acceptance of science as primarily concerned with empirical evidence acquisition and evaluation is a major weakness and runs the danger of saddling EBP with problematic epistemological and methodological assumptions. Conceptual analysis, problem formulation, classification, theory construction, development, and evaluation are crucial aspects of science and its application to mental disorders. Moreover, evidence is best understood in relation to specific inquiry tasks and what constitutes the right kind of knowledge depends on the task undertaken and the methods employed to achieve it. Relatedly, science is a human practice and therefore values and goal-directed actions underpin every aspect of it.
In this article, we argue that, although the EBP model should be regarded as a major achievement and is the right direction for clinical psychology to follow, the standard five-step inquiry component of the model suffers from several problems that make it difficult to implement in practice. First, we outline the transdisciplinary EBP model. Second, several criticisms of the overall EBP are outlined and briefly discussed. Third, five pressing problems in the inquiry component of the EBP model are identified: (a) information overload, (b) a focus on questions rather than tasks, (c) neglect of theory, (d) difficulty dealing with conflicting evidence, and (e) an oversimplified view of the role of values in research and practice. Fourth, we suggest ways of modifying the inquiry part of the model to address these problems. Finally, some of the major research and clinical implications of the revised EBP model are briefly discussed. Although the transdisciplinary evidence-based practice model (Spring et al., 2019) is the focus of this paper, for pragmatic reasons we refer to it as the EBP rather than T-EBP model. Our aim in the article is the relatively modest one of addressing some significant difficulties in the inquiry model associated with EBP. We do not present a major reformulation of this influential practice framework.
The transdisciplinary evidence-based practice model
The EBP model consists of three key overlapping areas or “circles” involved in the clinical decision-making process (see Figure 1): (a) best available research evidence; (b) resources, including practitioner’s expertise; and (c) client’s/population’s characteristics, state, needs, values, and preferences. Each of these factors is nested within an environmental and organizational context, which moderates or constrains the kinds of interventions that are appropriate for particular individuals. In essence, this represents a fourth evidence circle. The process of clinical decision-making occurs in the overlap between each of these factors, therefore promoting a shared decision-making process, in which both the client and the practitioner are involved.
Factors relevant to the clinical decision-making process in evidence-based practice.
Most EBP models incorporate the best available scientific evidence, where evidence is defined as “research findings derived from the systematic collection of data through observation and experimentation and the formulation of questions and testing of hypotheses” (Satterfield et al., 2009, p. 383). What constitutes the best available research evidence depends on the tasks being undertaken, for example, establishing a causal link between clinical variables or selecting a treatment for a psychological problem (Spring & Neville, 2011). Practitioner’s expertise is viewed as a vital resource drawn on by all practitioners as a means to implement health services. In the EBP model, practitioner’s expertise involves four key areas: (a) capability at executing the EBP process, (b) assessment, (c) communication/collaboration, and (d) intervention (Satterfield et al., 2009). Finally, the area of client’s/population’s characteristics allows for the shared decision-making process that the model emphasizes, where the importance of patient’s preferences and characteristics is considered in the ultimate decisions regarding clinical care.
This simple model outlines the factors to be explicitly considered in any clinical decision-making process. Of particular relevance is the inquiry component of the EBP model, which requires a practitioner to proceed through a five-step process to arrive at the ultimate decision regarding clinical care. These steps are: (a) ask a question; (b) acquire the evidence; (c) appraise the evidence; (d) apply the evidence; and (e) analyze and adjust practice in light of the outcomes of the previous steps. Each of these steps can be elaborated as follows. First, after assessing the patient, the practitioner poses crucial questions that direct the clinical process toward decisions about the causes, assessment, treatment, or ongoing management of a client’s presenting condition. There can be a lot of uncertainty within the clinical process regarding which presenting problems should be prioritized, or which forms of treatment should be the primary focus (Spring & Neville, 2011). Asking questions relating to assessment, treatment, etiology, and the like, can help to direct the focus of the clinician, and hopefully lead to well-informed treatment options.
After well-formulated questions have been posed, the practitioner must then acquire the relevant research evidence that allows them to answer these questions. In addition to considering the range of available primary research relating to treatment options and efficacy, practitioners are often advised to consult the secondary synthesized literature. Such literature includes systematic reviews, whereby researchers have collated empirical results from several research studies and critically appraised the quality of the evidence (Spring & Neville, 2011).
Once the best available research evidence has been collated, the practitioner must then appraise this evidence according to two separate criteria: quality and applicability. Quality of research evidence relates directly to the treatment outcomes presented in the research. Here, the internal validity of the results must be considered; that is, whether the outcomes reported are in fact due to the intervention of interest and not due to extraneous variables or confounding factors. Secondly, it must be considered whether the research evidence is applicable to the client at hand. Such considerations require the clinician to make their own judgment regarding the sample used in the study and whether their client is appropriately similar to the sample cases.
To make a sound decision regarding the intervention given to a patient, the fourth step in this clinical process applies the relevant research evidence, taking into consideration the patient’s characteristics and other resources. The practitioner must apply information from a multiplicity of sources, acknowledge the environmental context of the patient and the patient’s preferences, and utilize all the necessary resources, including clinician expertise, to arrive at the best clinical decision. Given the complexity of such a task, this step of the clinical process is arguably the most difficult (Spring & Neville, 2011). At this stage, the practitioner must also assess the preferences and values of any relevant stakeholders and engage these stakeholders in the shared decision-making process.
The final step in the five-stage process is to analyze and adjust. Here, the practitioner assesses the outcomes of the given treatment, considering whether the chosen therapeutic approach is correct or if the treatment could be adjusted in any way, maximizing the benefits for the patient. This is an iterative process, whereby the treatment plan is in a continuous state of quality improvement. Such a process ensures that the patient is receiving the best treatment possible, and that these intervention decisions can be used to refine the formulation of questions, future evidence search methods, and future research plans (Spring & Hitchcock, 2010).
In summary, the EBP model is designed to prompt clinicians (and researchers) to take four sets of data into account when formulating and implementing an intervention plan. It is usefully construed as a framework model, somewhat like the biopsychosocial model in health from which specific practice tasks are derived (Savulescu et al., 2020). The model is intended to remind clinicians to explicitly consider important sets of data, and to adopt certain constraints, (e.g., contextual, cultural) when engaging in practice. Thus, the visual model (four circles plus organizational and environmental context) informs clinicians about what information to seek while the five steps tell them how to go about collecting and processing it. Our focus in this paper is on the inquiry model.
Criticism of the evidence-based practice model
Ever since it was first developed in medicine in the 1990s, EBP has faced significant criticism. We do not intend to engage with these criticisms in any detail; our purpose in listing them is simply to provide a theoretical context for our more specific concerns with the inquiry model associated with EBP. To outline and address them in detail would take a paper of its own.
Some of the most well-articulated problems of the EBP model include:
First, criticism of what are regarded as overly rigid evidence hierarchies, where randomized controlled trials (RCT) and systematic reviews of RCTs typically occupy the top spot, nonrandomized studies are placed in the middle, and clinical observations and mechanistic knowledge are ranked near the bottom (Stegenga, 2014; Wampold et al., 2007). Interestingly, the American Psychological Association’s Presidential Task Force report on EBP (2006) initially endorsed a pluralistic conception of evidence, which has gradually evolved into the kind of rigid evidence hierarchy outlined above. Reflecting on this process, Lovasz and Clegg (2019) commented that “the concept of evidence in psychotherapy research and practice has been shaped by social and political influences and cannot be understood apart from these” (p. 231). Importantly, critics have noted that what constitutes good evidence for health interventions depends in part on the specific question driving inquiry (e.g., Bluhm & Borgenson, 2011), a point that is sometimes acknowledged by EBP advocates. In other words, it is argued that a pluralistic, nonhierarchical view of evidence and the methods for producing it is more consistent with good science and effective clinical practice.
Second, concerns about external validity issues when extrapolating research findings from research settings to everyday clinical practice (e.g., Steel, 2008). For example, in medicine, experimental groups are often comprised of individuals who are relatively younger, have less complex clinical presentations, and are healthier than control groups, making it difficult to apply study findings to routine clinical practice (Stegenga, 2014). While in the Spring et al. (2019) transdisciplinary version of EBP, the decision-making process is at the heart of the model, it is significantly constrained by research evidence and institutional factors. Clinical expertise and experience is given a downgraded role. Similarly, Tonelli (2019) argues that in evidence-based medicine there is insufficient appreciation of the critical integrating function of clinicians’ experiential knowledge in deciding what type of evidence in a particular case should be prioritized.
Third, the tendency to overlook problems arising from limited research knowledge, and/or to dismiss therapeutic approaches that have not been well researched (e.g., Solomon, 2015). Although researchers understand that absence of evidence does not necessarily entail a lack of a treatment effect, it may be tempting to assume that it does.
Fourth, relatedly, the stifling of research creativity and tendency to adapt a “cookie cutter” or one size fits all approach to treatment, resulting in the neglect of clients’ specific characteristics and unique health needs (e.g., Lilienfeld et al., 2013). According to Tonelli and Shapiro (2020) evidence-based practice should be centered around case-specific information rather than the other way around.
Fifth, the downgrading of clinician experience, judgment, expertise, and an over-reliance on algorithms in clinical decision-making (e.g., Tonelli, 2019). The worry is that under the direction of EBP, clinical judgment will become rigid and unresponsive to meaningful differences between clients, and because of this neglect, substandard treatment will be delivered. Consistent with this view, Tonelli and Shapiro (2020) argue that “primary experience represents a kind of medical knowledge distinct from the propositional knowledge produced by clinical research, both of which are crucial to determining the best diagnosis and course of action for particular patients” (p. 67). In other words, (clinical) experiential knowledge and research-based evidence are different types of knowledge, each with its own optimal cognitive “niche.” It is a mistake to view them simply as representing degrees of evidence, which can be preferentially ranked in an evidence hierarchy (Tonelli, 2019).
Sixth, the failure to fully incorporate cultural practices and values into assessment and intervention, thereby overlooking significant cultural differences in the etiology and presentation of psychological and medical problems (e.g., Drisko & Grady, 2019). Arguably, this is a particular problem in clinical psychology where mental disorders can manifest differently across cultural contexts, and in fact, some disorders may only exist in particular cultures (Washington, 2016).
Seventh, the minimization of values and their pervasive role in science and practice (e.g., Campbell, 2017). Values of all types have important functions in scientific inquiry, although the nature of their impact varies according to the relevant phase of investigation. Failure to appreciate the pervasiveness of values in the generation of knowledge does not mean that they do not influence research, but simply that it is unacknowledged (Douglas, 2009). A danger of this inadequate recognition of values is that theoretical and ideological allegiances may distort the detection, explanation, and treatment of clinical problems. This could translate into weak programs and unethical treatment.
In our view, many of these complaints are legitimate criticisms of the general EBP model developed in clinical psychology. However, some objections are based on misconceptions of EBP, and arguably do not apply to the transdisciplinary version of the model developed by Spring et al. (2019; see also Howick, 2011; Lilienfeld et al., 2013; Spring & Hitchcock, 2010). Furthermore, it is hard to think of a justifiable alternative to science-informed assessment and treatment. A return to eminence- or eloquence-based practice, with its over privileging of practitioners’ opinions and an insufficient attention to researching findings? This practice would be unproductive and unethical. In any event, our focus in this paper is on the inquiry model associated with EBP and not with the EBP model itself.
Problems with the inquiry steps of evidence-based practice model
There are at least five outstanding problems that are not easily dealt with by the current reformulation of the EBP model: (a) information overload, (b) a focus on questions rather than tasks, (c) neglect of theory, (d) difficulty dealing with conflicting evidence, and (e) a limited role for values in the EBP five-step process. It must be stressed that these are problems with the five-step inquiry process model typically embedded within EBP approaches (e.g., Spring et al., 2019; Straus et al., 2019) and not with the overall model itself. We will now discuss each of these difficulties in turn.
Information overload and decision-making intractability
According to the five steps of the EBP inquiry process, clinicians should start with formulating a relevant question, which could be directed at diagnosis, explanation, treatment, prognosis, or treatment maintenance. The formulation of a question in the five-step model follows an initial assessment phase (although confusingly, according to Spring et al., 2019, assessment continues throughout the inquiry process) where salient issues are highlighted (Spring et al., 2019). Depending on the focus of the inquiry process, the question could be centered on several clinical issues ranging from identifying the causes of a condition, to its assessment, prognosis, or treatment. The problem is that if the search process is supposed to be guided by an explicit consideration of all the available research evidence relating to the question under consideration (e.g., characteristics of disorders, subtypes of clients, different classification systems), then there is an enormous number of possible avenues to explore. Formulating the background and foreground questions implies that the clinician has some idea of what problems are being addressed; that is, their structure, core features, and what kind of methods should be used to detect and assess them. However, it is not made clear how one achieves this with the EBP model. This is a problem of decision-making intractability: faced with so many potential questions, numerous conditions, multiple assessment options, and numerous treatment possibilities, a clinician will quickly be swamped and struggle to make sound decisions. Of course, it is possible to default to clinical experience for some initial guidance but that is precisely the type of strategy EBP was established to block, and for good reasons. The worry is that cognitive and motivational biases will distort a clinician’s initial search and result in a questionable starting point—for example, relying on the DSM-5 diagnostic system (American Psychiatric Association, 2013) in order to diagnose psychological conditions, where something like the Hierarchical Taxonomy of Psychopathology (HiTOP; Kotov et al., 2017) may provide a better understanding of the psychological issues in some instances. There needs to be a scientifically defensible way of constraining the inquiry process from the very beginning. The question is how best to do this without narrowing the relevant clinical options too early. The problem of information overload and intractability is not unique to the five-step inquiry model of clinical decision-making, and is evident in any practice framework based on empirical research. However, in our view, it is especially vulnerable to this problem because of its association with EBP, which stipulates that clinical decisions should always be based on the best science. And determining what the best science is entails an extensive and exhaustive search.
Focus on questions rather than tasks
A related problem is that formulating a clinical task in terms of a question, rather than the task being undertaken, directs the inquiry process to the wrong level. A vague or thin initial specification of a task results in insufficient constraints being built into the clinical question to lead to reasonable possible solutions. This matter can be illuminated by making use of the constraint-composition conceptualization of scientific problems (Nickles, 1981; see also Haig, 2014). Briefly stated, the constraint-composition theory asserts that a problem comprises all the constraints on its solution, along with the demand that the solution be found. On this formulation, the constraints are constitutive of the problem itself; they characterize the problem and give it structure. The explicit demand that the solution be found arises from the goals of the research program, the pursuit of which hopefully leads to filling an outstanding gap in the problem’s structure. Also, by including all the constraints in its articulation, the problem enables the researcher to direct inquiry effectively by pointing the way to its own solution. Similarly, developing a rich description of a practice task at the beginning of inquiry makes it easier to grasp what a satisfactory answer might look like. This requires specifying the nature of the task and its relevant patient, problem, and contextual features (constraints).
A possible objection to this criticism of the five-step inquiry model is to argue that there is no significant difference between structuring inquiry on tasks versus questions; that is, this criticism is a merely verbal or trivial point. We disagree; engaging in a task in a clinical inquiry context is a matter of attempting to solve a particular type of problem, which could be descriptive, classificatory, explanatory, or involve intervention selection. It involves formulating the relevant question, outlining the context, (environmental, social, psychological, etc.) and detailing the specific constraints that are operating (ethical obligations, time pressures, etc.). The same question could be directed to quite different tasks. For example, the question “What symptoms does the client display and how should I assess them?” could guide the task of arriving at a diagnosis for intervention purposes or the building of a compositional explanation of a core symptom. Furthermore, tasks involve at least three components: an investigator (epistemic agent), inquiry context, and inquiry plan. On the other hand, a question only requires an epistemic agent. The task is conceptually richer and more complex than a question and links the clinician to a specific set of environmental and client-relevant factors (see below).
Neglect of conceptual and theoretical work
We argue that theoretical assumptions constrain the development of specific explanatory, classificatory, and treatment models, and provide a lens through which empirical evidence is interpreted. Therefore, different types of theories and paradigms influence the way researchers and clinicians approach case formulation and treatment. However, the interaction between theory and practice is a two-way process, and the relevant clinical facts also exert normative and explanatory pressure on practitioners. Choosing the most appropriate theoretical tools (e.g., a classification system) for a particular task depends in part on the clinical circumstances, and this is something that should be thought through in each clinical encounter and not simply taken for granted. That is, theoretical questions arise consistently during routine clinical practice and cannot be solved once and for all prior to engaging in a EBP inquiry process.
The EBP model was originally developed in medicine and designed to help clinicians make good decisions about the most effective treatment for patients. As such, the focus is primarily on interpreting and evaluating the quality of research evidence concerning candidate treatments, produced by methodologically rigorous designs such as RCTs (Straus et al., 2019). However, this is primarily an empirical matter and little value is placed on knowledge of the mechanisms causing disease and its associated symptoms. In fact, in some rankings of evidence quality in the EBP model, knowledge of pathogenesis (disease/disorder processes) is placed at the bottom of the evidence hierarchy along with clinical experience (Solomon, 2015; Tonelli & Williamson, 2020). A consequence of this emphasis on empirical matters is a neglect of theoretical and conceptual analysis, which is also apparent in the five-step inquiry model and its application in clinical decision-making. What matters most to clinicians is that treatment works, not why or how it does. An emphasis on good quality outcome data makes sense if the clinical task is one of selecting and delivering treatment but is not such a good idea if the task is one of assessing and identifying psychological problems. Questions that reflect theoretical assumptions include the following: What is depression and what are its symptoms (Wilshire et al., 2021)? What kind of classification system is optimal in this context, the DSM-5 or something like the HiTOP (Kotov et al., 2017)? What are the relevant mechanisms of change and how do these interact with treatment? Is schizophrenia a coherent construct? What kind of explanation is best for a particular set of phenomena (Borsboom et al., 2021)? These kinds of questions are theoretical ones that require close attention to ideational matters rather than data. As such, the criteria for evaluating them are conceptual and, in the case of theories, involve reference to epistemic properties such as accuracy, consistency, scope, fruitfulness, and simplicity (Kuhn, 1977). These theoretical virtues are important because they are the features we expect good (explanatory) theories to possess. However, constructing theories with such properties helps scientists confront the pervasive challenge of dealing with the underdetermination of theories by empirical evidence. One attractive way of dealing with this problem is to compare theories by employing the method of inference to the best explanation (e.g., Haig, 2014; Thagard, 1992), which enables the scientist to choose the best of competing theories based on assemblies of the relevant theoretical virtues. In this way, the scientist is able to go beyond the limitations of empirical evidence to embrace considerations of theoretical evidence. The responsible clinician will combine knowledge resulting from appraising theories via inference to the best explanation with pragmatic reasoning about what to do in particular clinical contexts (Thagard & Larocque, 2018).
In EBP, the term evidence refers to empirical observations of relationships between events, especially data that arises from RCT research. Data are produced by reliable and valid measures or tests and are statistically transformed, and at times augmented, using appropriate technology. In essence, evidence is empirically observed (Bluhm & Borgenson, 2011). However, Upshur et al. (2001) argue that this view is overly narrow and state that evidence is an observation, fact or organized body of information, offered to support or justify inferences or beliefs in the demonstration of some proposition or matter at issue . . . . Reasoning clearly plays a role in its meaning and use and so evidence is offered in the process of deliberation, usually to justify claims. (p. 92)
Evidence in this broader sense also encompasses conceptual analysis and theoretical reasoning that is employed to justify a set of hypotheses or claims (Anjum et al., 2015).
Therefore, some aspects of the five-step EBP process need to be modified to reflect the crucial role of conceptual analysis and theoretical work in EBP research and practice. Specifically, virtually all the EBP steps explicitly refer to the need to critically scrutinize empirical research evidence to formulate and justify an intervention plan. However, they make no mention of conceptual or theoretical analysis. We argue, it is simply not possible to translate empirical research evidence and other types of evidence into clinical plans without also engaging in conceptual work. The detection and identification of symptoms, diagnosis, construction of case formulations, and selection of treatment plans for psychological problems all presuppose concepts, classification systems, theories, and data acquisition and analysis methods. One way to avoid the problem of misinterpretations associated with the restrictive use of the term evidence is to think of it in terms, such as “relevant epistemic considerations.” This would make it easier to employ different types of argument structures and evidence in clinical reasoning. It could be objected that expanding the term “evidence” with “relevant epistemic considerations” undermines the whole EBP. We disagree. If one accepts that EBP and its application in the inquiry process involves conceptual analysis and theoretical scrutiny, then the customary use of the term “evidence” is overly narrow and misleading. To repeat, something like “relevant epistemic considerations” is more general and covers theoretically oriented work as well as empirical evidential considerations.
Conflicting evidence
What should the clinician do if the evidence acquired from different sources such as clinical experience, treatment outcome research, or basic psychopathology (mechanistic) research conflicts? What evidence should be prioritized if the data from methodologically strong studies does not cohere with that from a patient’s self-report, or clinician’s judgment? Ideally, of course, they would agree, thereby providing confidence that the depiction of the client’s problems is correct, or that the best recommended treatment option is clear. But what if it is not? This is in part a question of convergent validity. It is tempting to give one source of data consistent priority over the others, but in some situations, this is not a reasonable strategy. Certainly, theorists such as Lilienfeld et al. (2013) confer the right of veto on research evidence when there are conflicting views, while others argue that this is unreasonable and clinical judgment should be the ultimate arbiter (Tonelli, 2019). The relevant question is: which rule should we accept? It is difficult to think of a response that does not appear ad hoc and dogmatic. In part, the problem arises because there is no clear demarcation in the EBP between the different tasks confronting clinicians in their practice. Although a distinction is made between different types of activities, such as treatment, diagnosis, etiology, or prediction, there is no explicit provision within the model for describing the nature of the clinical task at the beginning of the EBP process. Later in the article, we argue that the question of how conflicting empirical evidence (or theoretical evidence and/or justified reasoning if the issue is conceptual) ought to be dealt with depends on the nature of the clinical task. For example, if the practice question concerns the most effective treatment for a simple phobia, then the relevant research evidence would suggest some type of exposure approach. If a client argues that they would rather undertake a form of psychodynamic therapy, then the pressing issue would be how best to (respectfully) persuade the client to choose the behavioral intervention, given its demonstrated superior effectiveness. Thus, objective research evidence derived from systematic reviews of RCTs should be relied on to guide the treatment decision. By contrast, if the task is a diagnostic one, then evidence acquired via the systematic inquiry of a client’s mental state and clinical history should be preferred; this would rely more on evidence coming directly from clients’ self-reports, and is, accordingly, more subjective in nature. These points also apply to research on the detection, assessment, explanation, and treatment of psychological disorders as well as individual clinical decision-making. In a research context, the aim would be to investigate how clinicians as a group manage conflicting evidence and derive a set of guidelines based on optimal ways of responding.
Limited role of values
According to Sadler (2005), two notable features of values are: (a) they guide actions, in the sense that they provide reasons for action, and can be translated into specific goals and plans and (b) norms reflecting values are used to evaluate actions, persons, and outcomes as worthwhile or unworthy. The nature of the “worthiness” depends on the type of value in question, the specific context, and the relevant set of practices. There are different types of values, which all play a role in clinical practice and research (Ward & Heffernan, 2017). These include epistemic, or knowledge-related, values (e.g., internal consistency, external coherence, explanatory depth, simplicity, fertility, empirical adequacy), ethical or moral values (e.g., right, wrong, good, bad), social/cultural values (e.g., consensus, discrimination, sacredness, justice, rituals), and prudential values that refer to the goods affecting individuals’ level of well-being or quality of life (e.g., health, agency, mastery, pleasure). According to Douglas (2009) all these types of values have direct and indirect roles in research and practice domains. For example, epistemic values such as consistency, scope, and predictive accuracy, are directly involved in the central tasks of scientific inquiry because their role is to enhance the credibility of knowledge claims and to rule out influences that may compromise the interpretation of the results. Additionally, social and ethical values sometimes play a direct role in the selection of scientific problems (e.g., curing cancer, reducing suffering), when considering what specific methodologies to use (e.g., RCTs in forensic contexts), and when applying the results to the real-world problems that motivated the research in the first place.
The stated role of values in EBP is a relatively restricted one and is typically cashed out in terms of prudential and social/cultural values (Campbell, 2017). Client preferences and values are one of the four types of evidence researchers and clinicians are asked to consider when formulating intervention plans. These are usually unpacked in terms of the kind of interventions individuals prefer or with respect to norms relating to cultural practices. However, it is apparent that epistemic, social, ethical, and prudential values underpin every phase of the EBP inquiry process. For example, in the Ask phase, practitioners seek to determine what are good or relevant questions (epistemic values) and to also think about options that are likely to be cost effective and acceptable to the persons concerned (social and prudential values). The issue is not whether values underpin EBP—they clearly do—but rather the degree to which they should be explicitly considered and factored into decision-making. We will argue below that one of the first steps in an EBP inquiry process should be identifying the task clinicians are engaged in, and the context within which that task is undertaken. This is in large part a normative or value-based process.
A revised evidence-based practice inquiry model
The prevailing EBP model is a significant innovation in clinical psychology, and in the health services more generally (see Figure 1). It builds on the discipline’s long-time commitment to science-informed practice while also being responsive to clinical expertise, the characteristics of clients, and social/cultural factors. The gap between science and practice is difficult to bridge because of their different underlying epistemic and ethical norms, although recent formulations of EBP have made considerable progress in doing so. Despite this progress, several conceptual and methodological problems remain to be faced. We will now outline a revised conception of the EBP five-step inquiry process model that considers the five major problems identified in the transdisciplinary version of the EBP model (Spring et al., 2019). At the core of the revised model (EBP–R) is an appreciation that clinical inquiry consists of several distinct tasks: problem identification, problem description, explanation, treatment, maintenance, risk management, and prevention. Sometimes all the tasks will be sequentially undertaken, while on other occasions only one or two will be attempted. For each task, clinicians are asked to explicitly consider the four circles of the model and to follow the five-step inquiry model to arrive at the best possible clinical decision under the circumstances. The distinction between the task and the inquiry process is as follows: the task refers to a practice goal while the inquiry process refers to the various steps involved in achieving the goal in question.
Steps of evidence-based practice–revised inquiry process
We have made several changes to the EBP inquiry model to overcome the five problems identified earlier. In brief, the key innovations are: (a) starting an inquiry process with a task formulation; (b) developing a description of the relevant task by systematically adding empirical, methodological, and theoretical constraints; (c) clarifying the different types of “evidence” (relevant considerations) that should be awarded priority depending on the task; and (d) placing values at the heart of clinical inquiry. We will describe each innovation with respect to the problem(s) in greater depth as we discuss each step of the inquiry process (see Figure 2).
Evidence-Based practice inquiry model–revised.
Step one: Specify the key tasks
A significant innovation in our revision of the five-stage EBP inquiry process is to replace the Ask step, with its emphasis on asking questions, with a task-formulation step. Arguably, this is the most critical step and is the major focus of our depiction of the revised five-step model. This change has the advantage of making the problem description significantly richer than one formulated purely in terms of questions.
There are a variety of distinct tasks in clinical practice, each requiring the achievement of different types of knowledge. These tasks include problem detection and description, explanation, treatment, maintenance of treatment gains, and risk management. A major aim of science-informed practice is to utilize the best scientific theories and research evidence (i.e., scientific knowledge) available to the clinician to properly carry out these tasks. What we mean by scientific knowledge can be roughly characterized in the following way: “Scientific knowledge is high-grade knowledge, that is, knowledge that satisfies demanding epistemic standards and that, as a result, is highly reliable, robust, or well-established” (de Ridder, 2020, p. 4). However, as well as scientific knowledge, social, cultural, and commonsense knowledge is incorporated into the task description, thereby accounting for all the sources of data covered by the four circles of the EBP model. In other words, more than one type of knowledge is required when engaging in ethical and effective clinical work.
In accordance with the constraint-composition conceptualization of scientific problems described earlier, we argue that an important first step in developing an understanding of clinical phenomena and their management is to create as rich an initial description of them as possible. Each piece of relevant information becomes part of the task under consideration and serves to streamline and guide subsequent investigations. For example, formulating an explanatory task as one involving a young Indigenous male with a major depression, substance use disorder, and avoidant personality features who is struggling with a sense of cultural alienation and poverty. An individual such as this may be strongly motivated to reconnect with his Indigenous roots, value community-based interventions, and be suspicious of medical professionals. Each piece of relevant knowledge about individual clients becomes a component of the task description, and any diagnosis and case formulation should be constructed with these features in mind. The search for, and appraisal of, treatment options for someone falling under this description will be heavily constrained. Thus, right from the beginning, the task description constrains knowledge and relevant evidence, and serves to contextualize clients more. Unlike the standard EBP model, the revised model does this earlier on in the clinical decision-making process, rather than at the final application step. This helps to address the problems of information overload and intractability discussed earlier.
An obvious objection to requiring a front-end rich formulation of a task is that it may narrow things down to the point that any subsequent literature search will yield little valuable information, and even worse, may fail to identify effective treatments. In the example mentioned above, it could be argued that good quality treatments for depression and addiction might not have been undertaken for individuals who share the key features of being Indigenous, impoverished, culturally alienated, and so on. Thus, any potential advantage of a rich initial task description and the subsequent streamlining of an information search may be lost. Therefore, the relevant question becomes, how best to balance building in task constraints early on with ensuring that all effective treatments or assessment strategies are taken into account?
One strategy for balancing task constraints and treatment considerations is to include only those constraints that have been well supported in the literature, or are so well established by other means that it is reasonable to accept them as legitimate filters. This can be achieved by asking two questions at the outset of a clinical investigation (we focus on the task of treatment for convenience). First, are there any factors that preclude certain kinds of intervention, such as significant disabilities like severe autism, language problems, or significant personality dysfunction? This is a process of ruling out possibilities. If so, they should be regarded as filters or constraints and, therefore, as part of the formulation of the treatment task. The second question has two parts: (a) Are there any internal features of a condition, disorder, or persons that have been reliably established by research (or accepted as trustworthy knowledge) that can act as constraints? This could include distinctions between certain types of symptoms or symptom clusters such as anhedonia, diagnostic subtypes, gender, client preferences, and values. Then, (b) are there any external characteristics that have been reliably established by research (or are accepted as trustworthy knowledge) such as social class, cultural identification, or geography that are likely to markedly affect a person’s responsiveness to certain interventions? Question two rules in certain features as part of the problem, that is, as constraints that any satisfactory conceptualization, classification, explanation, treatment, and so on should consider. Deciding what constraints to use as task descriptors is a challenging problem and, in our view, one that ought to be guided by research and good theory. In any event, something like this filtering process occurs at step four in the standard EBP inquiry process and, therefore, faces similar problems, but at a later stage.
A notable aspect of our revision to the EBP five-step process is that values are a pivotal part of the method and provide an organizing point for clinical inquiry; they provide a normative searchlight that frames key problems and challenges. Clinical tasks such as problem description, or the planning and delivery of a treatment, always occur within a practice niche (context), which comprises the client’s characteristics, therapeutic resources, relevant knowledge, and environmental and organizational contexts. These are essentially the four circles of the EBP and collectively they are manifested at an individual level within a specific context; that is, the lifeworld of the client and practitioner. Each of the four EBP components contains goals and actions, which are incorporated into practices. More specifically, practices involve the application of practical knowledge within goal-directed action sequences, governed by a “structured body of norms” (Wallace, 2009, p. 11). Norms are evaluative in nature and spell out whether an activity is done properly and whether it meets the socially accepted relevant standards. Goals reflect values, the type depending on the task, and the actions designed to realize them. For example, diagnosis presupposes a conception of mental disorders, which are typically defined as conditions involving harmful dysfunction (Kendler et al., 2011), embodying prudential- and knowledge-related values respectively (Nielsen & Ward, 2020). The knowledge-related values are utilized in the process of deciding what mechanisms are associated with the dysfunction component of mental disorders, for example, an impaired threat detection system in anxiety disorders (Barlow, 2004). The prudential-related values are involved in establishing that, as a consequence of this psychological dysfunction, an individual is suffering and having problems coping in their day-to-day life (Wakefield, 1992). Each of the four circles of the EBP comprises both factual and normative elements, but at the very center are clients’ values, reflecting what is at stake for them. Is it social rejection, failure at work, self-recrimination, fear of persecution, or some other kind of disvalued state? To fully understand the nature of the relevant task, clinicians are obliged to grasp the core normative dimensions of encounters with clients. Values are pervasive and have direct and indirect influences on research and practice.
Step two: Acquire relevant information
In the standard EBP five-step inquiry model, the second step is that of acquiring relevant empirical evidence to answer the question posed in step one. Step two has a slightly different function in our revised model and instead seeks to take into account the relevant epistemic considerations pertinent to the task formulated in the first step. On the constraint-composition conceptualization of a problem, a task is a problem comprised of constraints that can be posed as a particular type of rich question. For example, if the task is to detect a disorder, then its formulation will include pertinent information about the patient’s presenting condition, social and cultural context, values, clinician expertise, agency resources, and relevant research findings (drawn from the four circles of the EBP model). A core clinical question will then be formulated and used to structure the systematic collection of clinically relevant data and ideas. In our revised model, a question is only posed once a task has been identified and at least moderately described in terms of the relevant constraints. Otherwise, the danger is that the subsequent inquiry will be unfocused and diffuse.
The nature of the information-acquisition process will vary depending on the specific practice task being undertaken and will likely draw from a mix of scientific theory and research, personal experience and clinical expertise, cultural knowledge, and common-sense folk psychological understandings. The critical difference in the revised inquiry model is that the concept of evidence is expanded to include a host of relevant justificatory considerations. These considerations may be empirical and derived from research studies using RCTs, or conceptual in the form of scientific models and theories, classification systems, methodological (epistemic) norms, ethical norms, and theory appraisal criteria. Thus, there is much more to well-reasoned research and practice than judgments based solely on empirical evidence; these judgments are also guided by logic, meaning, and a variety of epistemic standards. The worry that arises from centering the acquisition step of EBP on the collection of (empirical) evidence is that conceptual aspects of practice will be overlooked.
Introducing conceptual and theoretical resources into the second step of an EBP inquiry process will ensure that simple errors are not made early on in the investigation. For example, in the explanatory case-formulation phase of clinical practice, drawing from invalid diagnostic categories or using psychometric tests that have poor validity and reliability or are not simply fit for purpose (Younstrom et al., 2017) is likely to lead therapists down the wrong investigative path. This will have adverse knock-on effects and could result in poor downstream treatment decisions and suboptimal outcomes for clients.
With respect to the problems of EBP discussed earlier, our revisions to this step go some way to addressing them. First, because of the inclusion of conceptual issues in the acquire step, there is less chance of basing interventions on poor theories and problematic concepts because the process is in part guided by the explicit consideration of epistemic, ethical, and social values. Second, only information that is germane to the task being undertaken is collected, making this aspect of the inquiry process more targeted and efficient. Third, information is prioritized according to the task that drives clinical work. It will, therefore, be easier to address the problem of conflicting data or information. For example, if the task is one of selecting the best treatment option for someone with a major depression, then a meta-analysis, or systematic review, of RCTs will provide the strongest evidence. However, if instead a therapist is seeking to explain a client’s set of symptoms or problems, then a reliance on empirical evidence alone will not be enough. Epistemic considerations such as the explanatory coherence and depth of any theories appealed to when formulating the case will be of critical importance and may well trump empirical data.
Step three: Appraise information
The significant differences between our version of step three and the EBP inquiry model have their origins in the previous two steps: the task engaged in constrains the kind of information or “evidence” sought, which, in turn, has implications for the type of appraisal undertaken. The criteria employed to evaluate responses to task questions may be either empirical or theoretical, or both in some instances. Each set of criteria is underpinned by epistemic values such as validity, coherence, or logical consistency. Therefore, strict evidence hierarchies are not a requirement for the appraisal step of some tasks, and empirical evidence may not be relevant at all. To clarify these points, we will discuss the appraisal process with respect to the different tasks of detecting, explaining, and treating psychological problems.
The identification of problems such as symptoms or mental disorders is a signal-detection task, where clinicians are asked to discern clinically meaningful patterns in a sea of data. The detailed analysis and discovery of data patterns helps practitioners to make judgments about the likely existence of clinical phenomena, empirical regularities that indicate clinically relevant facts such as depressed mood, auditory hallucinations, or low self-esteem. An important methodological requirement in this signal-detection task is that the discovery of data patterns should be replicated, ideally by using multiple methods. Haig (2014) stresses the distinction between data and clinical phenomena (i.e., symptoms, signs, or problems) for which the data serve as evidence; some of the data will be “noise,” and other data markers of genuine phenomena. The ability to assess evidence in successfully detecting a problem depends in part on the convergent validity of different methods, whereas the methods themselves are evaluated through examining their psychometric properties (Younstrom et al., 2017).
According to our revised five-step inquiry model, the evaluation of a clinical explanation for a person’s problems should be based on more than its empirical adequacy. Just as scientific theories in general are underdetermined by the relevant empirical evidence (McMullin, 1995), so are psychological formulations. In any practice situation, there will commonly be several plausible case conceptualizations consistent with the clinical evidence. The clinician should, therefore, consider which of the number of competing case formulations provides the best explanation of a client’s symptoms or problems. As with a scientific theory, the evaluation of a case formulation will often involve the determination of its explanatory coherence in terms of the major criteria of explanatory breadth, simplicity, and analogy (Thagard, 1989, 1992). More generally, theoretical literacy, awareness of the normative nature of theory construction, and evaluation of a case formulation are crucial aspects of this third step.
Our last example of the third step in action is concerned with treatment selection. The choice of a science-informed intervention depends on existing knowledge and, although it makes sense to follow an evidence hierarchy, any decision to do so ought to be constrained by ethical and pragmatic considerations. A client’s values, cultural identity, and the resources available to practitioners will influence the way they go about deciding what the best treatment option is. It is important to consider the best available research evidence (e.g., from systematic reviews of RCTs) in light of client and resource information. Thus, an evaluation is multifaceted and should never be based on research evidence alone. The constraints built into the task identification, problem formulation, and information acquisition steps serve to determine what considerations are relevant and how they should be appraised. As discussed above, there are different evaluation strategies depending on the specific task being undertaken.
Step four: Construct and apply intervention plan
In the standard five-step inquiry model, it is at the application phase that clinicians explicitly consider information relating to the other three circles of the EBP model. In our view, consideration of these factors at this point is too late because of the problem of information overload and the sheer complexity of any well-made decision. Certainly, therapists do make good decisions using the standard model, but there is still room for significant improvement for assessing most disorders. Currently, treatments in psychology are not as effective as they could be (e.g., Craighead et al., 2015; Springer et al., 2018). Arguably, this situation could be due in part to problems of validity with current classification manuals (Berenbaum, 2013), as well as vague, heterogenous therapeutic targets such as major depressive disorders whose diagnostic criteria can be met by dozens of different clinical profiles (Fried & Nesse, 2015).
An additional possible (although speculative) explanation for the relatively weak therapeutic effectiveness of clinical psychology interventions is the persistent flaws in the EBP model, notably, the five problems discussed above. In the revised five-step model, clinicians apply and formulate intervention plans that contain guidelines about how to achieve the task within the relevant context by using appropriate knowledge. The nature of the intervention plan is constrained by the task and its formulation. The innovations made in the earlier steps exert an influence here, primarily by constraining the way the clinical problem is defined, classified, and explained.
Step five: Analyze and adjust plan
The standard five-step model is a dynamic, iterative process in which clinicians gradually adjust their plans. These plans will take different forms as intervention unfolds. Although treatment may begin as “a one size fits all” approach, this is adjusted if necessary to the unique features of the client and their situation (Spring et al., 2019). We argued earlier that this is a mistake and runs the risk of locking psychologists into an incorrect view of a client’s problems and a corresponding inappropriate intervention framework. Our suggestion was to utilize the constraint-composition depiction of practice tasks and their related problems at the beginning of inquiry and, in this way, ensure that consideration of a client’s relevant features are built in at an early stage. By considering a client’s prudential, ethical, and social values in direct relation to their clinical situation, the clinician will have a better idea of what is at stake for them, and what kind of intervention plan is likely to be acceptable. In addition, by paying greater attention to the conceptual aspects of the inquiry process, a greater focus will be placed on mechanisms of change and their relationship to treatment. This is a theoretical task, based on modeling mechanisms thought to constitute and cause psychological problems. To ascertain whether a treatment is working, and why, it is insufficient to rely on evidence from RCTs. RCTs will only tell you whether an intervention is more effective than other options but will not tell you why. And arguably, the “why” is what is needed for rational adjustments to a treatment plan, particularly if the initial treatment was based on incorrect theories, problematic diagnostic systems, or vague treatment targets (Ward & Clack, 2019).
Conclusions
The EBP model is a normative practice model. It is intended to guide and structure the clinical decisions of individual practitioners by incorporating information from multiple domains and levels of analysis. It is an ecological model because it assumes that good decisions need to take contextual features into account; these include a client’s mental state and characteristics, clinician competency, cultural variables, institutional structure and dynamics, and scientific research. The model rejects authority-based decisions, and instead seeks to encourage clinicians to take epistemic responsibility for their decisions and subsequent actions. A core aim of EBP education is to equip practitioners with the cognitive skills necessary to make effective ongoing, “online” decisions and to learn throughout their careers. Importantly, the skills required to do this include those directly concerned with clinical inquiry, and not only content-specific knowledge of disorders and their assessment and treatment.
In this article, we have made several suggestions for revising the inquiry process component of the EBP model in order to address its problems. In hindsight, a better term for EBP would be science-informed practice. This has the advantages of (a) making it clear that theoretical and empirical research are both part of the application of research to practice and (b) underlines the fact that other forms of knowledge are also essential for effective clinical work, including social and cultural knowledge. Although we hope that the proposed revisions will result in more streamlined and effective treatment, evaluating this is a conceptual and empirical issue; only time will tell. However, one thing we are sure of is that clinical inquiry depends crucially on theoretical analysis as well as on empirical research, just as science does. If a model is hampered by the implausibility of its underlying assumptions, then it makes sense to consider alternative formulations. The process of gaining scientific knowledge is a fallible one. Therefore, it is better to explore and identify weaknesses in theories and practice frameworks rather than incorrectly assume that all of the action in science-informed practice occurs at the level of data acquisition. Evidence-based inquiry is more complex than that.
Footnotes
Acknowledgements
We would like to thank Dr. Carolyn Wilshire for her helpful comments on an earlier version of this manuscript.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
