Why force a square peg into a round hole? The ongoing (pseudo-)problem of psychological measurement

Confusions regarding the meaning of “meaning”

A first point of disagreement concerns ambiguity surrounding the authors’ meaning of “meaning.” They allude throughout their article to the importance of “meaningful psychological measurements.” In fact, a major premise of their arguments against LVM approaches is that the former rely on “meaningless” latent variables as putative proxies for psychological attributes, whereas, they contend, the theory-based framework provides a “sound foundation for meaningful [emphasis added] psychological measurement” (Borgstede & Eggert, 2023, p. 127). But what do they mean by “meaningful”?

For Borgstede and Eggert (2023), meaningful measurements are those that are defined by substantive formal theory. This view is confused on at least two grounds. The first is that meaningfulness is reduced to explicit definitional criteria—that is, necessary and sufficient conditions for the application of a given term or expression. However, the failure of such a classical view of linguistic meaning for capturing the diversity in forms of explanation of linguistic meaning has been long recognized, most notoriously by Wittgenstein (1953). Whereas the classical view held that the meanings of all concepts are given by explicit definitional criteria wherein higher-order concepts are defined in terms of lower-order concepts (e.g., “bachelor” is defined by “unmarried,” “male,” and “adult”), Wittgenstein (1953) illustrated through the notions of “family resemblance” and “language games” that most concepts lack a definitional structure and, instead, their meanings are revealed by examining their uses in context. Simply put, necessary and sufficient linguistic criteria are by no means necessary and sufficient for the meaningfulness of terms and expressions, even within scientific discourse. ¹ Moreover, the authors’ contention that the primary problem with psychological measurement is that psychological concepts are not “well defined” is not well supported. Despite it being a common trope, there is very little evidence that psychological concepts are as “ambiguously defined” or “vague” as the authors claim. ² That their meanings are indeterminate is a different issue (Wittgenstein, 1953; see also Hacker, 1996, pp. 432–433), but this only means that the grounds for the application of those terms are not fixed by necessary and sufficient criteria (Baker & Hacker, 1982; Maraun, 1998), and whether this produces ambiguity in intended meaning will depend on the specific context of use. That is to say, interpretation may be called for when “an utterance can be understood in more than one way” (Hacker, 2013, p. 11), as is typical with psychological concepts.

The second ground on which the authors’ view of meaningful measurement is confused is that the authors conflate the meaningfulness of a (theoretical) concept with the measurability of the attribute (feature, property, process, etc.) designated by the concept, as well as the conceptual matter of stipulating operational definitions with both the theoretical matter of specifying formal (often causal) explanatory hypotheses and the empirical matter of testing those theorized hypotheses in terms of empirical relations between measurements derived from such operations (Wittgenstein, 1953). A theory from which clear hypotheses can be derived and tested presupposes that the meanings of the concepts out of which the theory is built are already meaningful (Baker & Hacker, 1982). The authors also conflate two different senses of “meaning”: linguistic meaning and (theoretical) importance or relevance; this “meaning–significance” conflation has been long understood by psychological measurement scholars (see Bechtoldt, 1959; Brodbeck, 1957).

Vagueness and ambiguity in the proposed theory-based measurement framework

Borgstede and Eggert (2023) argue that “psychological attributes can only be identified if they are defined within the context of substantive formal theory” (p. 18). They claim to build on a structuralist view of scientific theories and to “show that any successful application of such a theory necessarily produces specific values for the theoretical terms that are defined within the theory” and, therefore, that “substantive formal theory is both necessary and sufficient for psychological measurement” (p. 18). In addition to the points raised above on confusions regarding definition and meaning, there are a few notable problems with their assertions.

First, it is obviously not the case that psychological attributes can only be defined within theory. If it were, we would not be able to make sense of the regular, old, psychological attributes about which we (i.e., the general “we”) make ascriptions, including to ourselves, all the time. The question then becomes: How do the putative formalisms the authors endorse relate to ordinary psychological concepts that do denote or designate psychological attributes? I agree that ambiguously defined common language could genuinely be a problem for developing or testing scientific theory; however, technical terms defined by the formalisms of theory will only be a potential solution in cases where the common language is not ambiguously defined. Unless the authors are willing to defend the view that everything we already understand about psychological attributes (from both ordinary conceptions and current psychological theory) is completely independent of what is or could be articulated within what they call “substantive formal theory,” they must at least sketch an account of the relations between those ordinary psychological concepts and the theoretical formalizations that putatively provide better understandings.

Second, there is a circularity lurking in their statement. If psychological attributes can only be “identified” if defined within a substantive formal theory, then a substantive formal theory must exist in order that they be identifiable. However, such theory must be about something, and the something it is about must be expressible with concepts that are already defined—otherwise, it would not be a “substantive” theory about anything in particular and, moreover, one would have no grounds to “identify” a putative psychological attribute as the attribute to which the theory pertains.

Third, somewhat ironically, many features of the proposed framework the authors describe remain vague and not well defined. For instance, the authors are unclear on the specifics of what constitutes a “formalized psychological theory” and how the “standardized conditions” under their framework differ in important ways relevant to the measurability of psychological attributes from those of current approaches used in psychological research. Also, what does “formal theoretical embedding” (Borgstede & Eggert, 2023, p. 127) mean in the psychological context, and on what grounds should we prefer this formalism to that used in either pragmatic (operational functional forms) or realist (LVM as paraphrases of substantive theory) formalisms? And, what are the criteria for “sufficiently developed formal [psychological] theory” (p. 128)? What are the standards of “standardized application of a theory,” and “suitable” enough to provide what kind of “information” (p. 131)? What structures, or kinds of structures, do the authors envisage substantive psychological theories would pertain to? Their description of semantic theory nets (in contrast to nomological networks) and their definition of “theory elements” as “themselves scientific laws related by a specialization relation” (p. 133) does little to clarify how one would produce or recognize either. From the little the authors describe about the theory-based framework of psychological measurement, it is not clear how it differs in practice from the current construct validation framework (under which LVM methods are subsumed) they seek to replace.

Fourth, the conception of measurement the authors invoke is itself quite vague. They suggest that theory-based measurement may be applied to quantitative as well as qualitative attributes, “as long as they are well-defined within a substantive formal theory” (Borgstede & Eggert, 2023, p. 131). So, it would seem, everything is potentially measurable as long as it is “well defined” within a substantive formal theory. However, without more clarity on what this looks like in practice, one does not get very far. Unfortunately, their examples of “value” and “work” provide little help to ground the proposed framework. In the first case, an ordinary sense of “value” is conflated with a pseudotechnical sense, narrowly defined within the context of reinforcement learning theory. ³ In the second case, the authors compare two quite different senses of a polysemous term—the first a mundane ordinary concept, the other a technically defined homonym defined within classical mechanics theory (i.e., W = Fd). In either case, little can be gleaned regarding why such examples of theory-based “measurements” constitute improvements on either ordinary or theoretical concepts of value and work.

Finally, Borgstede and Eggert (2023) seem to be advocating for a framework for psychological measurement on the barest hope for potential within psychology, given what they say about the lack of substantive formal theory in the discipline (pp. 131–132). Although the authors appear to be still willing to hold out hope for “the future of psychology as a natural science,” (p. 133), they provide little analysis to encourage readers that psychology is, indeed, heading in that direction. Nor do they comment on why, after over 100 years as an independent discipline, psychology still lacks formal substantive theory (despite notable efforts by, among others, neo-behaviorists). Rather, they seem to simply assume that psychological theories will eventually be well specified in the kinds of formalisms that the substantive formal theory they advance requires, thereby reducing the crux of their argument to little more than a promissory note.

On the promise of eliminating “folk” psychological concepts

As noted, Borgstede and Eggert (2023) believe that the “problem” of psychological measurement is due to the fact that “psychological concepts are not well-defined” (p. 120), are “ambiguous and too vague” (p. 127), and “tend to invoke naive theories of human behavior . . . which may hinder scientific progress” (p. 128). Such a view can be objected to from a number of angles. I have already mentioned the distinction between ambiguous or vague and indeterminate meanings, and the problem of conflating conceptual (definitional), theoretical, and empirical issues. A further objection concerns whether ordinary (“folk”) psychological concepts necessarily present a problem for scientific psychology and even could be replaced by theoretical formalisms and still be a science of psychology. Here, again, Wittgenstein’s (1953, 1980) insights provide illumination.

The authors admit that psychological concepts are part of ordinary, everyday language and, as such, are not technical concepts. As with other ordinary language terms and expressions, psychological concepts have “notoriously complicated grammars” (Maraun, 1998, p. 436), and the uses of psychological terms and expressions are “complex and ramified” (Hacker, 1996, p. 421). As noted, this does not imply that their meanings are ineffable, but, rather, that the criteria for their application are manifold (Baker & Hacker, 1982). However, there is nothing in the grammars of ordinary psychological concepts, unlike the grammars of the technical concepts of the physical sciences, that indicates what formalisms should replace the ordinary concepts. Nor can one deduce from the grammar of ordinary psychological concepts how to move back and forth between the ordinary and formal–theoretical “language games” (Wittgenstein, 1953). Hacker (1996) cites several reasons why the view that our ordinary psychological vocabulary will be replaced by technical (formal) vocabulary lacks sense, including that the very claim presupposes the ordinary psychological concepts to be replaced. Moreover, ordinary psychological concepts do not “constitute a . . . hierarchy of concepts preparatory to sophisticated theory-construction, as do the concepts of physics” (Hacker, 1996, p. 444; see also Wittgenstein, 1980). Nor are they embedded in a practice of measurement (Maraun, 1998). Thus, the relative lack of success of measurement in psychology is due not to its lagging theoretical sophistication as compared to the physical sciences, but to “their sharply different conceptual foundations” (Maraun, 1998, p. 436). Thus, building a psychological measurement framework on the model of the physical sciences is likely to create more problems than it solves.

Notably, not being embedded in a practice of measurement does not seem to cause serious problems in other disciplines in which substantive formal theory is also rare (e.g., sociology, anthropology, zoology, political science, etc.). Could it be that the “problem of psychological measurement” is simply that psychological attributes are not measurable in the same way that physical characteristics and processes are? ⁴ And, if so, would it not follow that replacing or eliminating ordinary psychological concepts would “eliminate the very explananda of the science of psychology” (Hacker, 2013, p. 23)? As Hacker (1996) reminds us, our “folk” psychological vocabulary does not merely provide prescientific descriptions of our experience of the external (“the world”) or internal (“the mind”). Rather, it is partly constitutive of the phenomena it is also used to describe. Thus, “we can no more abandon our vocabulary of sensation, perception, emotion, and attitude, our terminology of intention, decision, and purpose, and our concepts of thought, belief, recollection, and expectation than we can cease at will to be human beings” (Hacker, 1996, p. 427). Maybe the issue is not squaring the circle but, rather, recognizing that a square peg is a poor fit for a round hole.