A Criterial Framework for Concept Evaluation in Social Practice and Scholarship

Abstract

This article responds to the argued lack of clearly articulated, consistent, and agreed criteria that might be used by researchers for determining the adequacy of a given concept for a given task. It does so by describing the development of a framework of such criteria, presenting that framework, and illustratively applying it to the evaluation of the concept of warm-up in psychodrama. The framework comprises eight criteria in three categories: the intrinsic qualities of a concept (the criteria of clarity, comprehensiveness, parsimony, and resonance), the contextualization of a concept (differentiation and connectedness), and its application (epistemic utility and practical utility). Using the framework to evaluate the concept of warm-up in the context of its use in psychodrama suggests its potential to make significant differentiations. It is argued that this framework may contribute to evaluating other concepts in other contexts, although the extent of such generalizability remains to be ascertained.

Keywords

concepts criterial framework reconceptualization terms conceptual adequacy

This article presents and grounds a framework for evaluating concepts in professional practice and scholarship. The need for such a framework arose in a research project being undertaken by the lead author, examining the integrity of the concept of warm-up, as it has been used in psychodrama. A review of pertinent literature indicated that the concept, in that context, was both poorly articulated and inconsistently used (Howie and Bagnall 2015). That finding precipitated a search for criteria to (1) determine the ways in which and the extent to which the present concept was inadequately theorized, (2) guide its reconceptualization, and (3) permit a determination of the extent to which that reconceptualization addresses the problems identified.

That search revealed a complex web of meanings, terms, definitions, and discussion regarding the notion of a concept, its role in theory and practice, its development, its evaluation, and its refinement (e.g., Bickhard 2011; Laverty 2016; Machery 2010; Medin and Smith 1984; Slaney and Racine 2011; Weiskopf 2008). There was very little literature on what a concept of high quality might be and no generally recognized criteria, or regularly applied criteria, used with any consistency in making such determinations. The search revealed a general lack of epistemological agreement on or discussion of the criteria that might indicate concept quality, although this lack was itself noted by some authors, such as O’Raghallaigh, Sammon, and Murphy (2011), who specifically recorded their surprise at the lack of writing, independent of the data, on the goodness of a concept, and Daigneault (2012), who suggested that reconceptualization was only seen as a prelude to research, not as important research in itself (Gerring 1999; Machery 2010). Sartori (1984) and Wilson (1969) argued that there was a general lack of epistemological clarification and theorization focused on the qualities of a good concept, despite their being the basic units with which social science worked. Wilson (1969) further suggested that the deficit of knowledge about what makes a good concept arose because concepts rarely had well-articulated meanings, relying instead on shared understanding and other implicit factors for their comprehensibility. He highlighted the lack of procedural rigor and of shared criteria when he wrote that “there are few, if any, fixed rules” (p. 21). From the literature, it appeared that attempts to identify criteria appropriate to concept evaluation or reconceptualization often focused on the processes of concept evaluation and synthesis, only rarely even acknowledging the place of criteria in reconceptualization, and then in a largely post hoc fashion: after the process had been completed (ref., e.g., Avant 2006; Beckwith, Dickinson, and Kendall 2008; Fawcett 1995; Gabriel 2011; Gerring 2012; Hager and Beckett 1995; Hupcey and Penrod 2005; Morse, Hupcey, and Cerdas 1996; Walker and Avant 2011).

For instance, Gerring (1999, 2001, 2012) developed three distinct but overlapping sets of criteria designed as part of a methodology for creating new concepts in the social sciences. While using different terms for his concepts across the three criterial sets—and including a different number of terms in each—the three sets nevertheless evidence significant conceptual overlap. In those three works, he focused on how the creation of new or improved theory (which he termed discovery) could be integrated with the empirical testing of theory (which he termed appraisal). His work was devoted to addressing the problems of appraisal once a specific hypothesis has been identified and to the development of indicated concepts. The criteria he developed for concept formation thus served part of that purpose. Gerring’s work is specifically about the creation of new concepts—concept creation or concept formation. He did not argue that his criterial frameworks were for evaluating the conceptual adequacy of existing concepts. Some authors have, however, used his criteria to evaluate existing concepts, including Tortola (2017) who used Gerring’s 2012 criterial framework to clarify the concept of multilevel governance and Bjørnskov and Sønderskov (2012) who used Gerring’s 1999 framework to examine the adequacy of the concept of social capital. None of those uses, though, provided any substantial argument for using a concept-forming methodology as a vehicle for assessing conceptual adequacy or for the selection of the particular framework. We were faced, then, with both a lack of commonly agreed criteria and a lack of argument for criteria for the particular task with which we were concerned: that of evaluating conceptual adequacy to inform any indicated reconceptualization. The challenge, then, was to develop and apply a framework appropriate to that task. That use of the framework could then be seen as standing as a test and an example, as is done in this article.

The substantive part of this article begins with a background outline of the notion of a concept and its reconceptualization, together with comment on the situations generating an impetus for reconceptualization. It then presents the methodology used in developing the criterial framework, followed by the framework itself, before a concluding discussion of the place and possible utility of the framework in concept evaluation more broadly across other fields of social practice and scholarship.

Concepts and Their Reconceptualization

The notion of a concept in this article is that which is encompassed by a descriptive term, as distinct from the Deweyan psychological notion of a concept propounded by Blumer (1931), Laverty (2016), and others. It thus identifies the scope of the concept signified by that term. That which is encompassed by a descriptive term is also understood as its denotation, extension, or referent and is standardly represented by a description of its scope, which is commonly referred to as a definition, definiens, intension, or the connotation, or meaning of the term (Salmon 1984; Sartori 1984). Such a description will inevitably use further concepts to elucidate the properties, attributes, characteristics, or qualities of the concept including classes of objects, phenomena, or situations covered by it. For example, warm-up in psychodrama is a term that may be defined as “the inner motivation that prepares a person for action,” which we could also say is the meaning or connotation of the term “warm-up.” Such a description may also include stipulative, lexical, or precising properties (Gardner 1972).

The formal (deliberative) activity of formulating a concept—determining its scope and properties and articulating a description of it—may be seen as the activity of conceptualization (Sartori 1984). The reconceptualization of a concept, then, may be understood as the formal activity of modifying the scope and properties of a concept and of articulating a revised description (definition) of it. Previous work suggested that the judgment that a concept required reconceptualization was highly contextualized and was undertaken in a manner idiosyncratic to individual authors and their particular research concerns. The impetus for such an evaluation was seen as arising, variously, from inadequacy of either the research data or the research methodology (de Lange and Mavondo 2000; Saylor 2013; Whiteside and Varley 1998), or existing conceptualizations (hypotheses, frameworks, or analyses; Algesheimer, Bagozzi, and Dholakia 2015; Fairweather et al. 2001; Foord 1986; Kesson and Henderson 2010; Kwak 2007; O’Sullivan 2004; Renshaw 1998; Sellar 2015), or both (Dillon and Howe 2007; Green 1999; Thakker and Ward 2010). According to Blumer (1931, 1954), the development of adequate concepts is central to scientific endeavor, as they create the potential for varied points of view, symbolic means of interacting with the environment, and inferential reasoning, as well as theoretical exploration about what is not available to our senses (Weller 2000). Similarly, Sartori (1984, 2009) and Blumer (1931) argued that the importance of clear concept theorization lay in the role of concepts in mediating our perception of, objects or phenomena, and their interrelationships. Well-constructed, clearly identifiable, and well-defined concepts are thus accepted here as being centrally important to any scholarly endeavor.

Method

The task then became that of developing an appropriate criterial framework. This was done first by focusing on the key features of any concept that would serve to underpin its practicability in intelligent social practice: the intrinsic qualities of the concept, its contextualization, and its application. The next step entailed the in-depth examination, consideration, and defining of worthwhile key criteria for such a framework, selected on the basis of their usefulness for the task at hand. The criteria were then clarified and clearly articulated, before being assessed through a comprehensive review of the pertinent literature. The literature reviewed for that purpose was selected through a comprehensive database search with a wide variety of search terms, seeking to find any references to literature that reviewed, used, developed, or examined the development, modification, or evaluation of concepts, theories, ideas, models, or frameworks. Each work revealed by the search was perused to identify any references to evaluative criteria, including any that were judged to be implied, regardless of whether or not the notion of “criteria” or any of its cognates was used. The texts thus identified were then examined to track the theoretical origins of such conceptualizations, including any primary reference sources, to ascertain their contribution to the criteria presented in each text. Authors who were judged to have made an original and significant contribution to the generation of criteria for the evaluation of concepts (including conceptualizations, theories, ideas, models, or frameworks) and the pivotal texts to which they contributed were then singled out. These works were all significant texts developed to support the generation of new concepts, conceptual models, constructs, or theories, or the modification of existing conceptualizations. Each one also involved a theorization that promoted one or another approach to the generation, definition or theorization of, variously, concepts, conceptual models, constructs, or theories. Primary sources were used here instead of the more modern derivative versions in those cases where the primary source provided clearer or more coherent arguments or where the newer version added little new material. The search process was continued repeatedly with new sources from the database searches and investigating primary sources until it was clear that no new material was being identified. This resulted in 18 works from 14 authors being selected as relevant for the task at hand.

The 14 authors, and their 18 selected works, are briefly enumerated here in published date order: (1) Two papers by Blumer (1931, 1954) “Science Without Concepts,” a paper arguing the value and purpose of concepts for adequate scientific endeavor, and “What Is Wrong With Social Theory?” a critique of the proliferation of inadequate concepts in sociology, with suggestions for improvement. (2) A book by Wilson (1969), Thinking With Concepts, a practical textbook for the analysis and improvement of concepts, conceptual thinking, and conceptual explication, which had been highly influential in concept analysis in the discipline of nursing (Beckwith et al. 2008). (3) An essay by Kuhn (1977), “The essential tension: Selected studies in scientific tradition and change,” a presentation and discussion of how to use criteria for choosing which among two or more theories should be taken as the theory of choice for any particular purpose, grounded in his research in the history of science, his exploration of philosophy, and his initial training as a physicist. (4) A book by Dubin (1978), Theory Building (2nd ed.), grounded in the rational empiricist tradition of science, as part of an attempt to improve researchers’ appreciation of the nexus between theory and data. (5) Two complimentary works by Blalock: a book chapter (Blalock 1979), Dilemmas and Strategies of Theory Construction, and a textbook (Blalock 1982), Conceptualization and Measurement in the Social Sciences, exploring concept development and measurement in the social sciences and problems in and strategies for theory construction. (6) A book by Sartori (1984), Social Science Concepts: A Systematic Analysis, an analysis of social science concepts from a rational perspective, grounded in semantics and political science, and “Concept Misformation in Comparative Politics” (Sartori 2009), a paper discussing the relationship between intension and extension in concept construction and theory building. (7) A textbook of comparative psychology by Patterson (1986), Theories of Counseling and Psychotherapy, which sets out a series of eight criteria for theory evaluation in psychology. (8) A well-cited paper by Bacharach (1989), “Organizational Theories: Some Criteria for Evaluation,” focusing on presenting arguably appropriate ways to evaluate and improve organization theory, grounded in a pragmatic approach to the theorization of concepts and theory, with inspiration from ideas by Popper and Hempel. (9) A textbook by Fawcett (1995), Analysis and Evaluation of Conceptual Models of Nursing, written to clarify an ongoing confusion between conceptual models and theories in nursing. (10) Two works from Gerring, the first (1999), a paper, “What Makes a Concept Good? A Criterial Framework for Understanding Concept Formation in the Social Sciences,” and the second (2012), a book, Social Science Methodology: A Unified Framework, both focusing on criteria for developing sound concepts and theory in political science. (11) A paper by Glaser (2002), “Conceptualization: On Theory and Theorizing Using Grounded Theory,” addressing the properties that make a good concept in a grounded theory approach to theory development. (12) A textbook by Proctor and Capaldi (2006), Why Science Matters: Understanding the Methods of Psychological Research, discussing the criteria that might be used when determining which among a group of theories is the best fit for the circumstances in psychology. (13) A paper by Prochaska, Wright, and Velicer (2008), “Evaluating Theories of Health Behavior Change: A Hierarchy of Criteria Applied to the Transtheoretical Model,” in applied psychology, comparing the usefulness of related and competing concepts of health behavior change and describing a criterial framework for comparing the worth of each one. And (14) A lengthy textbook by Meleis (2012), Theoretical Nursing: Development and Progress, focusing on concept and theory development in nursing, drawing strongly on the perceived need of the nursing profession for stronger theorization.

The development of the criterial framework and its presentation here have been undertaken, it must be acknowledged, with an acute awareness of the limitations of language to such a task. The meanings of concepts, including criterial concepts, are always contextualized, and hence, the definitions of concepts will always be partial and open to different interpretations. We hope to have moderated the limitations thus arising, through the careful and consistent use of terms throughout this work, but that moderation will be, irremediably, limited.

It should also be noted that the interdependencies between concepts in this criterial framework have been accepted here as being beyond the scope of this article. There are indications that they are so highly contextual that generalizations would be very limited, but further work is needed to explore that question. We have, accordingly, largely put the question aside, although we return to briefly in the final discussion.

The Criterial Framework

Concepts may be used for different purposes in different contexts including describing, advocating, requiring, evaluating, or combinations of such uses. The criteria articulated here may be understood as a category of concepts specifically designed for evaluating other concepts. The eight terms labeling these criterial concepts are four that address the intrinsic qualities of a concept: (1) clarity, (2) comprehensiveness, (3) parsimony, and (4) resonance; two that address the contextualization of a concept: (5) differentiation and (6) connectedness; and two that address the application of a concept: (7) epistemic utility and (8) practical utility. Each of the criteria is to be understood as a more or less continuous variable, ranging from high to low, the higher the value being that which is desired in a concept being assessed.

The following articulations of the evaluative criteria constitute the developed criterial framework. Each is presented, firstly, by a concise description of it, along with defining properties, characteristics, and attributes within that description. Each description is followed by an illustrative grounding of the criterion in the literature. Lastly, to demonstrate the use of each criterion, a brief assessment of the concept of warm-up in the context of psychodrama is presented.

Clarity

Conceptual clarity here refers to the degree to which a concept is coherent, intelligible, comprehensible, and lacking ambiguity. Coherence identifies the concept’s intrinsic unity, that is, the extent to which it is (1) logically and rationally structured; (2) explicated in a theoretically straightforward, methodical, and systematic manner; (3) congruent, such that there is a complementarity and mutual support from supplementary concepts, propositions, and arguments, which are, themselves, strongly bound together; and (4) consistent in its use of descriptive terms. Being intelligible identifies the understandability and cohesion of a concept’s internal semantic structure. Comprehensibility identifies the explicability of a concept to the extent that it is readily learned, taught, or used in a manner consistent with the meaning intended by those who are acknowledged as defining it. A lack of ambiguity here identifies the degree to which the meaning of a concept is definitive in its referents.

Conceptual clarity may be seen in the work of Gerring (1999) who, along with Proctor and Capaldi (2006), argued for the importance of a concept being coherent, describing it as the capacity for a concept to hang together, such that the attributes used to define it, straightforwardly fit with the characteristics of the phenomenon in question, and exhibit an internally consistent logic. Sartori (1984) provides three further illustrations: first, his suggestion that coherence is essential in the relationships between a concept’s identifying term, its meaning, and the phenomenon it is explaining; second, his argument that the first rule in concept analysis is for a concept to be unambiguous, in the sense that the meaning unquestionably and unequivocally relates to its labeling term; and third, his recommendation that there needs to be denotational adequacy, such that the referents unmistakably (unambiguously) relate to the meaning and the term. Elements of Fawcett’s (1995) criteria of logical congruence, conceptual clarity, and credibility address comparable properties to those of this criterion. The criterion is also shown in Prochaska et al.’s (2008) focus on the importance of what they called clarity, which they argued required consistency. Bacharach’s (1989) two criteria of logical adequacy and logical relationships are also cognate to properties of the criterion, as both of them point to an adequately logical and cohesive elucidation of a concept being aspects of intelligibility and comprehensibility, and hence of clarity. Dubin’s (1978) criterion of boundaries is congruent with the criterion of clarity to the extent that it requires that a concept be well bounded in itself and have clear relationships between its properties, characteristics, or attributes. Kuhn (1977) argued that a good concept has consistency, identifying the degree to which it is, internally consonant, homogeneous, and congruent. The criterion of clarity is also supported by Meleis’s (2012) own version of the criterion of clarity, which she described as the degree of precision of boundaries, how the concept holds together, the degree of coherence of the descriptive propositions, and the ease with which it may be understood. It is further demonstrated by her idea of consistency, as the degree with which the elements of a theory interrelate in a comparable manner across the whole theory. Patterson’s (1986) criteria echo the criterion developed here, where he refers to preciseness and clarity, as the degree to which a theory is understandable, internally consistent, and free from ambiguities.

Applying the criterion of clarity to the concept of warm-up in psychodrama indicates that there is no single description or explanation. Moreno (2007), for example—who is responsible for coining the term and writing extensively about it—defined it in a number of ways, including “He must have been guided by the warming-up process inherent in his own organism, his master tool, isolated in space, unspecialized yet, but working as a totality, projecting into facial expressions, sounds, movements, the vision of his mind” (p. 41); “…[warm-up] manifests itself in every expression of the living organism as it strives towards an act. It has a somatic expression, a psychological expression, and a social expression. The varieties of its expression depend upon the differentiation of the organism and the environment in which it exists” (Moreno 1956:56); “…perspiration is the folk word for the warming up, preceding and accompanying the creative acts…[and] is not always a straight line to the creative act; the track to it is full of handicaps and hazards, blind alleys and retreats” (Moreno 1955:386); and “The warming up process is the operational expression of spontaneity. Spontaneity and warming up process operate on all levels of human relations, eating, walking, sleeping, sexual intercourse, social communication, creativity, in religious self realization and asceticism” (Moreno 2007:42).

It is clear from those quotations that the concept of warm-up (which includes warming up and warming up process) is indistinct and that a variety of terms are introduced that are, likewise, indistinct—terms such as spontaneity, creativity, creative acts, striving toward an act, process of developing, somatic, psychological, and social expression—thus reducing the coherence of the concept. The concept is ambiguous for the same reason and from the scope of the referents becoming unwieldy and almost impossible to deal with, for instance, when it is claimed that it “operates on all levels of human relations.” This lack of coherence and the ambiguity gives the concept a low level of clarity (Howie and Bagnall 2015).

Comprehensiveness

Comprehensiveness here identifies the extent to which a concept is sufficient for the task at hand. This sufficiency is reflected in the completeness with which the connotation is constructed and with which the denotation is specified. Comprehensiveness necessitates that the relationship between the term and both its connotation and denotation be unmistakably and completely described for the task at hand. If a concept is insufficiently comprehensive, it is likely that it will contain partial meanings or refer to an incomplete or inconclusive set of referents in the task for which it is being used.

This definition is illustrated in Fawcett’s (1995) criterion of comprehensiveness of content, which necessitates, firstly, that there be complete guidance for its use including teaching, making it immediately applicable in practice or research and, secondly, that the elements of the concept descriptively link and relate to one another. Comprehensiveness here is also similar to Gerring’s (1999) criterion of depth to the extent that he suggested a concept is improved if it includes all those properties germane to it. It is also illustrated in Bacharach’s (1989) suggestion that a definition of a concept describing all its properties, characteristics, or attributes was better than one failing to do so to the extent that such a description relates to the task at hand. Patterson’s (1986) criterion of comprehensiveness—as the degree to which the theory covers the area of concern—is also congruent with the scope of the concept here defined.

Applying the criterion of comprehensiveness to the concept of warm-up in psychodrama requires the consideration of its use in that context. The analysis of Howie and Bagnall (2015) has shown the concept to be limited in its comprehensiveness, with neither its connotation nor its denotation being adequately described, to the extent that it exhibits a wide range of different connotations and referents.

Parsimony

Conceptual parsimony identifies the extent to which a concept is focused on the task at hand. The descriptive terms used in its definition should, in other words, all be necessary for its complete description, any surplus connotative elements being seen as reducing its parsimony. The criterion draws attention to the importance of the development of simple and elegant explications of a concept that align to its use in the given context (Hubbard 1995).

This criterion aligns with the notion of parsimony common in the literature. It may be seen as dating back to at least Aristotle who advised that the number of explanatory or defining postulates should be kept to a minimum, consistent with establishing just what has to be established, which, following Carey (2010) is congruent with the definition of the criterion being described here. Sartori (1984:55) advised researchers to “Confine your defining to the necessary properties,” suggesting that, as new characteristics or attributes are added to a definition, its parsimony decreases. Others have, similarly, focused on the degree of brevity and concision with which a concept is explained (Bacharach 1989; Gerring 1999, 2012; Hempel 1966; Prochaska et al. 2008; Proctor and Capaldi 2006; Sartori 1984), the argument being that, if a concept can be defined straightforwardly, using as few descriptive terms as possible, it is, ipso facto, preferable to an otherwise more complex one. Gerring (2012:67) suggested of description that “the fewer assumptions required,…the more believable” and Meleis’s (2012) description of her criterion of simplicity/complexity included an argument for the degree of parsimony in the context for which it is required, with the minimization or elimination of logical circularity, repetition, or recourse to further undefined concepts or theories. Patterson (1986) identified a criterion of parsimony or simplicity as the degree to which complexity is minimized. Baker (2003) argued that parsimony is demonstrably rational, in that any hypothesis will have greater explanatory power than its less parsimonious alternatives. While being included in the majority of the works reviewed that dealt with the development or analysis of concepts or theories, parsimony was rarely defined, grounded in prior scholarship, or argued for in any critical fashion. It was more generally used as a notion that a reader was expected to understand intuitively: an issue noted by other scholars such as Cutcliffe and Harder (2009), Epstein (1984), and Hubbard (1995).

Applying the criterion of parsimony to the concept of warm-up in psychodrama, it is clear that the latter has low parsimony due to their being a very wide diversity of descriptive concepts included in a variety of definitions, none of which is fit for the purpose, none of which has been critically evaluated, and none of which has been researched to any great extent (Howie and Bagnall 2015).

Resonance

Conceptual resonance is the ability of the written or spoken articulation or name of a concept to be catchy, memorable, alliterative, consonant, and appealing in and of itself, rendering a concept attractive, readily remembered and recalled, engendering an interested or pleasing response, and creating or serving as a lyrically or poetically unified description. It is a property of a concept that is carried both in its labeling term and its intension or connotation. High resonance in a concept’s labeling term is valuable on its own and may well be sufficient to increase the concept’s use or recognition, even if the meaning of the concept is theoretically weak.

Part of Gerring’s (1999) criterion of familiarity illustrates this criterion of resonance, where he notes that familiar sounds and phraseology are often incorporated into the explanations of resonant concepts or the labeling term of the concept itself. He noted (2012:118), when discussing a concept’s labeling term, that this type of familiarity “is achieved by finding that word within the existing lexicon that most accurately describes the phenomenon of interest.” Klimoski (1991) suggested that a good idea can get lost for want of a decent articulation and presentation, and O’Raghallaigh et al. (2011) suggested that a concept needs an effective linguistic design to be effective in communicating itself to its audience or stakeholders. Sartori (1984:1) wrote that selecting the label to be used in naming a concept was a significant process and a central decision in clarifying or forming concepts because language was not only expressive but also associative and evocative. Resonance may be further seen as relevant when considering what Glaser (2002) referred to as enduring grab: an evocative image of how much a concept is able to command attention, and which he considered essential for developing theory. Fawcett’s (1995) criterion of credibility may be seen as similar to the criterion of resonance, where she included the elements of social utility (that the concept can be useful in a broad social sense), social congruence (that the concept is not aversive to other socially accepted frameworks), and social significance (that the concept can tap into well recognized and valued conceptual spheres). Any concept that is seen as socially useful, congruent, and significant will rely to a significant degree for its credibility on its inherent resonance. As Gerring (2012:119) noted, “terms and definitions…ought to resonate as much as possible with established usage,” so that they may also resonate with the reader.

Applying the criterion of resonance to warm-up in psychodrama, we see that the concept has strong resonance, as is evidenced in the extent to which it has infiltrated psychodramatic practice while also exhibiting low levels of clarity, comprehensiveness, and parsimony. It may be argued that a concept having such low levels on those latter three criteria, but high general acceptance, must have strong resonance, sufficient to compensate in its acceptance for the conceptual inadequacies (Howie and Bagnall 2015). The resonance of the concept might also be argued to be strong on the ground that the number of its referents continues to grow in the absence of conceptual refinement: the concept now being applied to all manner of situations and contexts in the psychodrama literature (Carter 1997, 2005).

Differentiation

Conceptual differentiation here is the degree to which a concept is distinguishable from other concepts pertinent to the task at hand. Distinguishability is thus with respect to similar or overlapping concepts that are also of significance, or potential significance, to the context of the concept’s use. The criterion is thus important with regard to what the concept, contextually, is conceptually different from and, at the same time, closely related to. A highly differentiated concept will be clearly distinguishable from such related concepts, one with weak or low differentiation will be barely, uncertainly or disputably distinguishable.

The criterion may be seen in Dubin’s (1978) criterion of boundaries to the extent that the latter focuses on external conceptual boundaries. A well-bounded concept is, according to Dubin, one with strong differentiation and is to be preferred over a less bounded concept, which will have weak differentiation. Wilson (1969) considered it important that a concept’s boundaries be mapped through the development of model, contrary, related, borderline, or invented cases, as part of his process of differentiating what is and is not included in any concept’s meaning and referents. Sartori (1984) noted that the discriminating capacity of a concept is central to avoiding its becoming too broad. He argued for the avoidance of synonyms wherever possible in order to further increase such discriminability. Gerring (2012:128) argued that “…it is incumbent upon writers to clarify how their chosen concept(s) differ from neighbouring concepts sharing the same semantic and phenomenal space. This requires establishing clear contrasts with what lies outside the boundaries of a concept”: an activity he termed one of establishing field utility within a semantic field—the language, concepts, and terminology in which a concept is embedded.

Applying the criterion of differentiation to the concept of warm-up in psychodrama, we see that it is very low. The low values of the concept on the criteria of clarity, comprehensiveness, and parsimony ensure that outcome. In the absence of clear, sufficient, and necessary description of the concept, it cannot be clearly differentiated with any reliability from other pertinent concepts. Its boundaries are highly porous or labile and open to ad hoc extension.

Connectedness

Conceptual connectedness is the extent to which a concept is linked substantively with other concepts important in the context of its use. Connectedness is thus based in the importance of a concept’s relationships with other significant concepts being made clear in its articulation. Those relationships may be either explicitly stated in the definition or implicit in the meaning of other descriptive concepts used in the definition. Connectedness has two aspects: one being the number or the proportion of significant other concepts with which the concept has connections and the other being the strength of the connections. Strong connections make epistemically or practically significant linkages, in that they add meaning and value to one or other or both concepts. Weak connections add little or nothing to either concept. These two aspects clearly may vary independently of each other.

Connectedness was identified as being important in the work of Sartori (1984), where he argued that a concept needed to be part of a semantic field in order for it to have meaning due, in part, to its connectivity with the related ideas in that field. Gerring (2012:127-28) argued for clearly specifying the connectivity of a concept when he wrote that “A key element is to specify carefully how a concept fits within a larger semantic field composed of neighbouring concepts,” noting that “These neighbouring terms…give meaning to a concept, precisely because of the interconnectedness of language.” MacKay (1980:330) pointed out that “No concept is an island unto itself: All are interconnected in many different ways…,” which property he termed conceptual connectivity. Bacharach’s (1989) criterion of connectivity is described as the capacity of a concept to bridge gaps between existing concepts, ideas, or theories. Proctor and Capaldi’s (2006) criterion of external consistency is related to that of connectedness: as the extent to which a concept may benefit from being broadly consistent with related concepts. Kuhn’s (1977) criterion of consistency relates to connectedness, in that he suggested the need for a concept or theory to be seen to be “conforming to currently established and verifiable aspects of reality,” suggesting the importance of the connectedness of a concept with related concepts. Meleis’s (2012) criterion of external components touches on connectedness to the extent that she argued for the necessity of being conscious of the degree to which a concept was congruent with the larger social forces within which it existed, arguing also for the importance of congruence with professional values (how the concept connects with other philosophical systems), social values (how it connects with the larger social community in which it fits), and social significance (the degree to which it connects with the lives of others).

Applying the criterion of connectedness to the concept of warm-up in psychodrama, we see that it has high or strong proportional connectedness, in that its multitudinous interpretations give it a connectedness with numerous related concepts (Carter 1997). The concept is central to psychodrama’s group process, its individual process, and its central tenets of spontaneity and creativity, being considered the operationalization of that spontaneity (Clayton and Carter 2004; Moreno 1956, 2007). However, the strength of those connections tends to be low or weak, again because of the lack of conceptual clarity and comprehensiveness renders any one connection of limited explanatory power or epistemic utility. That weakness, though, is offset somewhat by the strength of the concept’s practical utility in psychodrama.

Epistemic utility

Epistemic utility is here the extent to which a concept is descriptively, explanatorily, and predictively useful in the context. Such utility will be through a concept’s contribution to understanding realities, whether experienced, imagined, or foreseen. It will thus commonly be through the concept’s involvement in generating new theory or in supplementing or enriching existing theory. It may also be through its provision of new, different, or modified perspectives on realities including understandings and theories themselves.

Epistemic utility is cognate with Gerring’s (1999) criterion of theoretical utility, Fawcett’s (1995) criterion of generation of theory, and Prochaska et al.’s (2008) criterion of productivity, each of which focuses on the applicability of a concept for the generation of new ideas. Epistemic utility is also illustrated in Gerring’s (1999) criterion of depth, in relation to which he suggested that, if a concept could bundle characteristics that would otherwise be seen as disparate, then such an epistemic outcome would add value to the concept. Proctor and Capaldi’s (2006) criterion of explanatory power, as the inductive potential of a theory or concept to make sense of phenomena, and their concept of predictive power, as the deductive potential of a concept or theory to develop testable hypotheses, are embraced by epistemic utility. Bacharach’s (1989) criteria of explanatory potential and predictive adequacy are similarly components of epistemic utility, and his criterion of construct scope—as the range of concepts or theories that may be better explained, augmented, or challenged, by a concept—is analogous to it. The criterion identified by Walker and Avant (2011) of identifiable consequences would also be included here, since it identifies the potential outcomes of a concept in terms of what it may explain. Epistemic utility may also be seen as including four of the Kuhn’s (1977) criteria: that of accuracy (as the degree of demonstrable agreement between products deducible from a theory and any outcomes of experiments or observations including a theory’s predictive and explanatory power), that of scope (as the breadth of a theory’s epistemic consequences, and how much, and how far, such consequences may also extend beyond the empirical and theoretical situation the theory was initially designed to explain), that of fruitfulness (as the capacity for a concept to produce new research questions, predict new phenomena, or illuminate new relationships between phenomena or concepts), and that of simplicity (as the degree to which the theory brings coherence to phenomena without which the latter would be perceived as a confused and disjointed set of observations or data).

Applying the criterion of epistemic utility to the concept of warm-up in psychodrama, we see that it has low epistemic utility, largely due to its low clarity and differentiation and the weakness of its connectedness (Howie and Bagnall 2015). Those limitations render it effectively incapable of being used epistemically, except in the simplest of explanations, or the most general of predictions.

Practical utility

The criterion of practical utility here identifies the extent to which a concept is useful in informing what persons actually do—their actions—in the context of interest. That utility will commonly but not exclusively be pertinent to social and professional practice. It may be through enriching or modifying understanding of matters pertinent to action or through its direct impact on the actions themselves. Through understanding, it may be of the nature of pertinent actions through influencing factors or through their possible effects. It may also be understanding of oneself as an active agent or of the actions of others.

Practical utility embraces a number of criteria identified in the literature including: Meleis’s (2012) criterion of usefulness (as a concept’s usefulness to professional, research, and administrative practice), Patterson’s (1986) criteria of practicality (as the provision of a conceptual framework to guide professional practice) and importance (as relevance to life or human action, its acceptability by competent professionals, and its persistence in the scholarly literature over time), Walker and Avant’s (2011) criterion of identifiable consequences (as the potential outcomes of a concept in terms of what it could be found to be used for), and Fawcett’s (1995) criterion of credibility, with its focus on social utility and significance. More commonly, practical and epistemic utility have been treated together in the literature, for example, in Bacharach’s (1989) criterion of empirical adequacy, which, he suggested, directly implied the usability of a concept, and in the criterion of generalizability identified by Prochaska et al. (2008) and Blalock (1979), as the reach of applications of a concept: the number and variety of systems and problems to which it may be applied.

Applying the criterion of practical utility to the concept of warm-up in psychodrama, we can see that it has strong practical utility as evidenced by the concept’s utilization in psychodrama practice and training and its extensive use in psychodrama literature (Blatner 2013; Carter 2011; Clayton and Carter 2004; Howie and Bagnall 2015; Karp, Holmes, and Bradshaw-Tauvon 1998; Somov 2008).

Discussion and Application

These criteria were developed to serve as a framework for evaluating the integrity of a particular concept in a particular context: that of warm-up in the context of psychodrama. The illustrative application of the framework to that task suggests that the framework has the potential to differentiate the areas of strength and weakness of a concept in use. Those strengths and weaknesses, then, may be used to guide the ways in which the concept may be strengthened for use in that context through its formal reconceptualization. In the case at hand, there is indicated the need for any reconceptualization of warm-up in psychodrama to focus on strengthening the criteria of clarity, comprehensiveness, and parsimony, as these underpin the weaknesses identified in the criteria of differentiation, connectedness, and epistemic utility. The criteria, then, may effectively be used in monitoring the effectiveness of the reconceptualization in addressing those weaknesses.

The question remains, then, of the extent to which this framework may contribute to evaluating other concepts in other contexts. In response to that question, we argue for the a priori generalizability of the framework to both other concepts and other contexts. The grounds for our doing so are that the framework developed here sought explicitly to identify general qualities of criteria in general categories of conceptual reality. The three aspects of conceptual reality on which the framework was developed to focus—those of a concept’s intrinsic qualities, its contextualization, and its application—apply no more to the concept of warm-up in the context of psychodrama than they may do for any other concept in any other context, except that contexts of a purely theoretical nature would sensibly not bother with the criterion of practical utility. Similarly, the criteria identified within each of those aspects of conceptual reality were formulated as a way of giving expression to the scope of the aspect in each case. In other words, the particularity of the concept and its context of use were not seen as driving the recognition of the criteria. Nevertheless, it must be acknowledged that the framework was developed for the specific purpose here used in illustration of its application, leaving open the possibility that radically different concepts or contexts of use may require somewhat different or additional criteria.

Any application of the framework will, of course, be limited in its reliability by the irremediable limitations of the language used to describe each criterion. All the descriptive terms used in our definitions or explanations of the criteria are open to variability in their interpretation. Definitional clarity, in other words, can never be absolute. The best that we can hope for is that the cultural context of the work here described—that of the epistemological scholarship in professional practice and research—is sufficiently coherent to constrain the linguistic interpretations of the criteria to a reasonable extent. The framework itself may also serve to limit the diversity of interpretations, since its component aspects and contained criteria make up an integrated whole, in which each criterion is meaningfully related to the others and to the aspect of conceptual reality of which it is a part.

It is worth briefly touching on the interdependencies between and among the criteria within the framework, although a complete consideration is beyond the scope of this article. The criteria are, of course, straightforwardly interdependent, as is evident from the illustrative example used above. Some of those interdependencies may be generalizable such as dependence of epistemic utility on clarity and comprehensiveness. Others, though, may be more contextually variable such as the inverse relationship identified here between practical utility and clarity. Formal relationships, such as the level of comprehensiveness setting a limit to parsimony, will certainly, though, apply regardless of the concept and its context of use.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Algesheimer

Bagozzi

R. P.

Dholakia

U. M.

. 2015. “Key Informant Models for Measuring Group-level Variables in Small Groups Application to Plural Subject Theory.” Sociological Methods & Research 47(2):277–313. doi:10.1177/0049124115613782.

Avant

K. C.

2006. “Concept Analysis.” Pp. 102–103 in Encyclopedia of Nursing Research, 2nd ed., edited by Fitzpatrick

J. J.

. New York: Springer.

Bacharach

S. B.

1989. “Organizational Theories: Some Criteria for Evaluation.” The Academy of Management Review 14(4):496–515.

Baker

2003. “Quantitative Parsimony and Explanatory Power.” The British Journal for the Philosophy of Science 54(2):245–59.

Beckwith

Dickinson

Kendall

. 2008. “The ‘con’ of Concept Analysis: A Discussion Paper Which Explores and Critiques the Ontological Focus, Reliability and Antecedents of Concept Analysis Frameworks.” International Journal of Nursing Studies 45(12):1831–841.

Bickhard

M. H.

2011. “On the Concept of Concept.” Journal of Theoretical and Philosophical Psychology 31(2):102–105.

Bjørnskov

Sønderskov

K. M.

. 2012. “Is Social Capital a Good Concept?” Social Indicators Research 114(3):1225–242.

Blalock

H. M.

1979. “Dilemmas and Strategies of Theory Construction.” Pp. 119–35 in Contemporary Issues in Theory and Research: A Metasociological Perspective, edited by Snizek

W. E.

Fuhrman

W. R.

Miller

M. K.

. Westport, CT: Greenwood Press.

Blalock

H. M.

1982. Conceptualization and Measurement in the Social Sciences. Beverly Hills, CA: Sage.

10.

Blatner

2013. “Warming-up, Action Methods, and Related Processes.” The Journal of Psychodrama, Sociometry, and Group Psychotherapy 61(1):43–50.

11.

Blumer

1931. “Science Without Concepts.” American Journal of Sociology 36(4):515–33.

12.

Blumer

1954. “What Is Wrong With Social Theory?” American Sociological Review 19(1):3–10.

13.

Carey

T. V.

2010. “Parsimony, in as Few Words as Possible.” Philosophy Now 81:6–8.

14.

Carter

P. D.

1997. An Information Systems for Psychotherapy Research Using Expert Commentary of Videotaped Expertise. PhD dissertation, Massey University, Palmerston North, New Zealand.

15.

Carter

P. D.

2005. “Spontaneity Made Explicit.” Australian and New Zealand Psychodrama Association Journal 14:61–69.

16.

Carter

P. D.

2011. “The Dramatic Heart of the Psychodramatic Method.” The Arts in Psychotherapy 38(4):290–97.

17.

Clayton

G. M.

Carter

P. D.

. 2004. The Living Spirit of the Psychodramatic Method. Auckland, New Zealand: Resource Books.

18.

Cutcliffe

J. R.

Harder

H. G.

. 2009. “The Perpetual Search for Parsimony: Enhancing the Epistemological and Practical Utility of Qualitative Research Findings.” International Journal of Nursing Studies 46(10):1401–410.

19.

Daigneault

P. M.

2012. “Introduction to the Symposium ‘Conceptual Analysis in Political Science and Beyond’.” Social Science Information 51(2):183–87.

20.

de Lange

Mavondo

. 2000. “Business Student Progress in Open Learning Courses: A Test and Reconceptualisation of Kember’s Model.” Asian Review of Accounting 8(1):106–22.

21.

Dillon

Howe

. 2007. “An Epistemology of Presence and Reconceptualisation in Design Education.” Interchange 38(1):69–88.

22.

Dubin

1978. Theory Building, revised edition . New York: Free Press.

23.

Epstein

1984. “The Principle of Parsimony and Some Applications in Psychology.” The Journal of Mind and Behavior 5(2):119–30.

24.

Fairweather

Rončević

Rydbjerg

Valentová

Zajc

. 2001. “Reconceptualisation of Social Quality.” European Journal of Social Quality 3(1):118–43.

25.

Fawcett

1995. Analysis and Evaluation of Conceptual Models of Nursing. Philadelphia, PA: F. A. Davis.

26.

Foord

J. O.

1986. “Patriarchy: Towards a Reconceptualisation.” Antipode 18(2):186–211.

27.

Gabriel

J. M

. 2011. Definitions Matter: Reflections on Political Science Concept Formation. Retrieved March 2015 (http://ssrn.com/abstract=1958766).

28.

Gardner

P. L.

1972. “Structure-of-knowledge Theory and Science Education.” Educational Philosophy and Theory 4(2):25–46.

29.

Gerring

1999. “What Makes a Concept Good? A Criterial Framework for Understanding Concept Formation in the Social Sciences.” Polity 31(3):357–93.

30.

Gerring

2001. Social Science Methodology: A Criterial Framework. Cambridge, England: Cambridge University Press.

31.

Gerring

2012. Social Science Methodology: A Unified Framework. Cambridge, England: Cambridge University Press.

32.

Glaser

B. G.

2002. “Conceptualization: On Theory and Theorizing Using Grounded Theory.” International Journal of Qualitative methods 1(2):23–38.

33.

Green

R. H.

1999. “Rehabilitation, Sustainable Peace and Development: Towards Reconceptualisation.” Third World Quarterly 20(1):189–206.

34.

Hager

Beckett

. 1995. “Philosophical Underpinnings of the Integrated Conception of Competence.” Educational Philosophy and Theory 27(1):1–24.

35.

Hempel

C. G.

1966. Philosophy of Natural Science. Englewood Cliffs, NJ: Prentice Hall.

36.

Howie

Bagnall

. 2015. “The Transmogrification of Warm-up: From Drama to Psychodrama.” The Arts in Psychotherapy 44:35–44.

37.

Hubbard

1995. Parsimony and the mind. Retrieved March 2016 (http://apollos.ws/dualism/Parsimony%20and%20the%20Mind.pdf).

38.

Hupcey

J. E.

Penrod

. 2005. “Concept Analysis: Examining the State of the Science.” Research and Theory for Nursing Practice 19(2):197–208.

39.

Karp

Holmes

Bradshaw-Tauvon

. 1998. The Handbook of Psychodrama. London, England: Routledge.

40.

Kesson

K. R.

Henderson

J. G.

. 2010. “Reconceptualizing Professional Development for Curriculum Leadership: Inspired by John Dewey and Informed by Alain Badiou.” Educational Philosophy and Theory 42(2):213–29.

41.

Klimoski

1991. “Theory Presentation in Human Resource Management.” Human Resource Management Review 1(4):253–71.

42.

Kuhn

T. S

. 1977. The Essential Tension: Selected Studies in Scientific Tradition and Change. Chicago, IL: University of Chicago Press.

43.

Kwak

D. J.

2007. “Re-conceptualizing Critical Thinking for Moral Education in Culturally Plural Societies.” Educational Philosophy and Theory 39(4):460–70.

44.

Laverty

M. J.

2016. “Thinking My Way Back to You: John Dewey on the Communication and Formation of Concepts.” Educational Philosophy and Theory 48(10):1029–45.

45.

Machery

2010. “Précis of Doing Without Concepts.” Behavioral and Brain Sciences 33(2-3):195–244.

46.

MacKay

D. G.

1980. “Speech Errors: Retrospect and Prospect.” Pp. 319–32 in Errors in Linguistic Performance, edited by Fromkin

V. A.

. New York: Academic Press.

47.

Medin

D. L.

Smith

E. E.

. 1984. “Concepts and Concept Formation.” Annual Review of Psychology 35(1):113–38.

48.

Meleis

A. I.

2012. Theoretical Nursing: Development and Progress. Philadelphia, PA: Wolters Kluwer Health/Lippincott Williams and Wilkins.

49.

Moreno

J. L.

1955. “System of Spontaneity-creativity-conserve, a Reply to P. Sorokin.” Sociometry 18(4):382–92.

50.

Moreno

J. L.

1956. Psychodrama volume 1. New York: Beacon House.

51.

Moreno

J. L.

2007. Who Shall Survive? Foundations of Sociometry, Group Psychotherapy, and Sociodrama. New York: Beacon House.

52.

Morse

J. M.

Hupcey

J. E.

Cerdas

. 1996. “Criteria for Concept Evaluation.” Journal of Advanced Nursing 24(2):385–90.

53.

O’Raghallaigh

Sammon

Murphy

. 2011. “The Design of Effective Theory.” Systems, Signs and Actions 5(1):117–32.

54.

O’Sullivan

2004. “The Reconceptualisation of Learner-centred Approaches: A Namibian Case Study.” International Journal of Educational Development 24(6):585–602.

55.

Patterson

C. H.

1986. Theories of Counseling and Psychotherapy. New York: Harper and Row.

56.

Prochaska

J. O.

Wright

J. A.

Velicer

W. F.

. 2008. “Evaluating Theories of Health Behavior Change: A Hierarchy of Criteria Applied to the Transtheoretical Model.” Applied Psychology An International Review 57(4):561–61.

57.

Proctor

R. W.

Capaldi

E. J.

. 2006. Why Science Matters: Understanding the Methods of Psychological Research. Oxford, England: Blackwell.

58.

Renshaw

1998. “Sociocultural Pedagogy for New Times: Reframing Key Concepts.” The Australian Educational Researcher 25(3):83–100.

59.

Salmon

W. C.

1984. Logic. Englewood Cliffs, NJ: Prentice Hall.

60.

Sartori

1984. “Guidelines for Concept Analysis.” Pp. 15–85 in Social Science Concepts: A Systematic Analysis, edited by Sartori

. Beverly Hills, CA: Sage.

61.

Sartori

2009. “Concept Misformation in Comparative Politics.” Pp. 13–43 in Concepts and Method in Social Science: The Tradition of Giovanni Sartori, edited by Collier

Gerring

. New York: Routledge.

62.

Saylor

2013. “Concepts, Measures, and Measuring Well an Alternative Outlook.” Sociological Methods and Research 42(3):354–91.

63.

Sellar

2015. “Unleashing Aspiration: The Concept of Potential in Education Policy.” The Australian Educational Researcher 42(2):201–215.

64.

Slaney

K. L.

Racine

T. P.

. 2011. “On the Ambiguity of Concept Use in Psychology: Is the Concept ‘Concept’ a Useful Concept?” Journal of Theoretical and Philosophical Psychology 31(2):73–89.

65.

Somov

P. G

. 2008. “A Psychodrama Group for Substance Use Relapse Prevention Training.” The Arts in Psychotherapy 35(2):151–61.

66.

Thakker

Ward

. 2010. “Relapse Prevention: A Critique and Proposed Reconceptualisation.” Behaviour Change 27(3):154–75.

67.

Tortola

P. D.

2017. “Clarifying Multilevel Governance.” European Journal of Political Research 56(2):234–50.

68.

Walker

L. O.

Avant

K. C.

. 2011. Strategies for Theory Construction in Nursing. Upper Saddle River, NJ: Prentice Hall.

69.

Weiskopf

D. A.

2008. “The Origins of Concepts.” Philosophical Studies 140(3):359–84.

70.

Weller

2000. “Tests of Concepts in Herbert Blumer’s Method.” Social Thought and Research 23(1/2):65–86.

71.

Whiteside

S. P.

Varley

R. A.

. 1998. “A Reconceptualisation of Apraxia of Speech: A Synthesis of Evidence.” Cortex: A Journal Devoted to the Study of the Nervous System and Behavior 34(2):221–31.

72.

Wilson

1969. Thinking With Concepts. Cambridge, England: Cambridge University Press.