Laying the cornerstone of construct validity theory: Herbert Feigl’s influence on early specifications

Abstract

Although the theoretical foundations of construct validity theory have been fairly well described, there remains equivocation over what should properly be taken to be its philosophical underpinnings, with some characterizing it as an essentially positivist enterprise, others identifying a realist philosophy underlying the theory, and others still characterizing its foundations as containing elements of both positivist and realist thinking. This paper summarizes recent work representing each of these three different stances on the philosophical foundations of construct validity theory. Explicit connections are drawn between the work of Herbert Feigl—who pioneered a philosophy of science whose roots lay in logical positivism, but which contained notably realist overtones—and early specifications of construct validity theory. Finally, an appeal is made for a realist interpretation of construct validity theory based both on the connections between early articulations of the theory and key Feiglian ideas and also on Cronbach and Meehl’s later reflections on the origins of their influential work.

Keywords

construct validity theory Feigl empirical realism logical empiricism

There can be little question that construct validity theory is one of the pillars of contemporary psychological science. Born in 1954 with the publication of the “Technical Recommendations for Psychological Tests and Diagnostic Techniques” (APA, 1954) and subsequently further articulated in Cronbach and Meehl’s seminal paper “Construct Validity in Psychology Tests” (1955), construct validity theory has evolved from a theory for the development and evaluation of quantitative psychological measures to a prescription for approaching the scientific inquiry of everything from achievement motivation to the Zeigarnik effect (Maraun, Slaney, & Gabriel, 2009). The “construct” has become psychology’s unit of analysis and construct validation its modus operandi.

Despite the prominent influence of construct validity in psychological and related research—e.g., a search of “construct validity” in the text fields of the PsycINFO and ERIC databases results in thousands of hits—comparably little work has explicitly addressed the philosophy underlying the theory. Of course, a handful of well-known validity theorists have explicated construct validity theory in greater detail throughout its development (e.g., Cronbach, 1971, 1989; Kane, 2006; Messick, 1989); but very few, shall we say, “metatheoretical” treatments have been provided, and among those that have been given, the philosophical underpinnings of construct validity theory have been variously characterized: some have portrayed it as deriving from a positivist and operationalist philosophical stance, according to which “constructs” are viewed simply as descriptive summaries of particular classes of behaviors that are implied by the content of sets of test items (e.g., Borsboom, 2005; Borsboom, Mellenbergh, & van Heerden, 2004; Engel, 1994; Moss, 1992); others have identified a realist philosophy underlying construct validity theory, with constructs being interpreted as representing objectively existing entities whose effects are reflected in test performance (e.g., Lovasz & Slaney, in press; Maraun et al., 2009; Rozeboom, 1984; Slaney, 2001; Smith, 1985). Still others (e.g., McGrath, 2005; Norris, 1983; Orton, 1987) have identified at the foundations of construct validity theory elements of both positivist and realist thinking, wherein constructs are presumed (at least implicitly) to be labels for classes of observable behaviors in one breath and the objectively existing but unobservable entities which are thus designated in the next. In fact, both the 1954 “Technical Recommendations” and Cronbach and Meehl (1955) imply at different times one or the other sense.

The overriding aim of the current work is to explicitly tie several key aspects of Herbert Feigl’s work to components of construct validity theory, as presented in its earliest articulations (i.e., APA, 1954; Cronbach & Meehl, 1955). A secondary aim is to argue that Feigl’s influence on the early specifications of construct validity theory provides a sound basis for assuming that the theory was motivated in large part by a realist philosophy. To this end, the paper is organized as follows. First, in order to ground the various treatments that have been given to construct validity theory, the fundamental tenets of the theory as initially articulated in the “Technical Recommendations” and Cronbach and Meehl (1955) are delineated. Second, three different perspectives on the philosophical foundations of construct validity theory are described, these corresponding roughly to “positivist,” “realist,” and “mixed” interpretations, respectively.¹ Third, three aspects of Feigl’s work are summarized, namely the brand of logical empiricism he favored, the view of theories he advocated, and his assumptions regarding the interpretation of theoretical concepts. Fourth, a number of clear examples of Feigl’s imprint on early construct validity theory are described. Finally, an appeal is made for a realist interpretation of the foundations of construct validity theory based both on the reliance by Cronbach and Meehl on key Feiglian ideas and on their later reflections regarding the origins of the theory.

Construct validity theory: The core elements

As indicated above, construct validity theory was introduced to psychological and educational researchers (the main target audience) through two works, published in 1954 and 1955, respectively: The “Technical Recommendations” paper (APA, 1954) summarized the findings and recommendations of the first joint committee formed by the American Psychological Association (APA), the American Educational Research Association (AERA), and the National Council on Measurements Used in Education (NCME) in order to establish an explicit set of standards for the development and use of tests in psychological and educational research. One year later, two members of the joint committee, Lee Cronbach and Paul Meehl, elaborated substantially the facets of construct validity theory and the logic on which it was based and in so doing gave the theory a much more public face (Cronbach & Meehl, 1955). Hence, the origins of construct validity theory are often traced to the latter rather than former work. In their paper, Cronbach and Meehl (1955) identify construct validity as one of three different types of validity relevant to quantitative tests and assessment procedures, the other two being content validity and criterion-related validity,² both of which are subsumed under construct validity. In contrast to earlier validation procedures, which consisted primarily in reporting observed correlations among test scores, or between test scores and particular criterion measures, construct validity was identified as being relevant when the research question concerns which “psychological quality” is being measured by a given test. For the originators of construct validity theory, this was not merely a question of which class of observables is summarized by test behavior, but, rather, construct validity was said to be applicable “when the tester has no definitive criterion measure of the quality with which he is concerned, and must use indirect measures to validate the theory” (APA, 1954, p. 14). Furthermore, in construct validity it is “the trait or quality underlying the test [emphasis added] that is of central importance, rather than either the test behavior or the scores on the criteria” (APA, 1954, p. 14).

Based in large part on the recapitulation provided at the end of Cronbach and Meehl (1955), the fundamental features of construct validity theory are summarized as follows:

A “construct” is “some postulated attribute of people, assumed to be reflected in test performance” (p. 283); it is the attribute about which claims are made when interpreting test behavior.

Construct validation is relevant to investigating constructs for which clear operational criteria do not exist: that is, construct validity is important when the researchers want to make inferences to entities (mechanisms, processes, etc.) that lie beyond the observed test data.

The theory underlying both the test and the construct may be conceived as an “interlocking system of laws” which is known as a “nomological network.” The nomological network relates “(a) observable properties or quantities to each other; or (b) theoretical constructs to observables; or (c) different theoretical constructs to one another” (p. 290).

Constructs are defined “implicitly” by the nomological network in which they occur. But constructs employed at different stages of a science may vary with regard to the “definiteness” of these definitions. That is, the network “gives the constructs whatever meaning they do have” (p. 294) at a given stage of science. These meanings, however, remain “open” to revision as a science progresses and knowledge is accumulated.

Construct validation requires that at least some of the statements in the network involve explicitly definable terms: that is, observation terms. In the absence of explicit connections between such observation terms and constructs, claims about construct validity are unjustified.

The nomological network can be viewed as an idealization of “a tidy set of postulates which jointly entail the desired theorems” (p. 293). The extent to which the nomological network can be articulated depends on how advanced is the science at hand.

Investigating the validity of a construct involves consideration of multiple sources and types of evidence: “Construct validity cannot generally be expressed in the form of a single coefficient” (p. 300).

When relations that have been deduced from the network fail to be empirically supported, “the fault may lie in the proposed interpretation of the test or in the network” (p. 300); the investigator must then decide whether modifications to the test or the network (or both) are required, but regardless of where modifications are needed, they must be validated with a “fresh body of data” (p. 295).

Construct validation is not essentially different from the general approach taken in science to developing and confirming theories.

Over the past 55 years, construct validity theory has ballooned from a relatively restricted set of prescriptions bearing on the development and evaluation of tests and diagnostic techniques to a full-blooded theoretical and methodological paradigm for conducting science in psychological and related domains of study (cf. Maraun et al., 2009). Of course, Cronbach and Meehl advanced this very idea in claiming that construct validation should not be considered distinct from general scientific procedures for developing and testing theories. Despite the fact that early articulations of construct validity theory were somewhat opaque regarding the ontology of constructs, there can be little doubt the methodological paradigm has grown into an endeavor which has overtones which are at least at times realist in spirit. One easily and often comes across references to constructs being “unobservable attributes,” “latent traits,” or other entities or processes which are hypothesized to “underlie,” “mediate,” “account for,” and “explain” observable behaviors. The issue to be explored here, however, is the extent to which an empirical (scientific) realist perspective and/or agenda informed the earliest articulations of construct validity theory. The following section describes three general positions that have been given.

The logical and philosophical foundations of construct validity theory: Three different stances

As indicated above, there appears to be at least some measure of equivocation with regard to how to characterize the dominant philosophy of science underlying validity theory, generally, and construct validity theory, in particular. Part of this stems no doubt from the fact that in first articulating the philosophical foundations of the latter, Cronbach and Meehl drew from literature representing a relatively large breadth of philosophical perspectives (see Cronbach & Meehl, 1955, p. 290). Furthermore, depending on which particular features of the “Technical Recommendation” (APA, 1954) and/or Cronbach and Meehl (1955) are emphasized, one is likely able to defend a number of different interpretations regarding the philosophical roots of construct validity theory. Here, I will briefly outline three different interpretations—one positivist, one realist, and one a mix of positivist and realist features—that have appeared relatively recently in the theoretical validity literature.

Positivist interpretations

In their characterizations of construct validity theory, Borsboom and his colleagues (Borsboom, 2005, 2006; Borsboom et al., 2004) have claimed that it is underlain with operationalist thinking wherein theoretical attributes are simply equated with observation attributes (typically test scores). They defend their interpretation on the grounds that Cronbach and Meehl (and, later, Messick and others) emphasize that (a) it is test score interpretations, and not tests per se, that may be deemed valid (or invalid); (b) constructs gain their meaning from the nomological network in which they exist; and (c) construct validity admits of degree, thus implying that construct validation is an ongoing process. In particular, Borsboom et al. (2004) contend that whereas a realist theory of measurement would need to emphasize ontology, reference, and causality, construct validity theory focuses (merely) on epistemology, meaning, and correlation, thus betraying its commitment to a positivist philosophy of science.

Along similar lines, Engel (1994) portrays the early roots of construct validity theory as reflecting “a strong version of logical positivism” (p. 226). He justifies this characterization in reference to Cronbach’s (1989) later reflections on the 1955 paper, which Cronbach notes was delivered in a language that was generally consistent with the “positivists’ ideal of ‘reconstruction of scientific knowledge’ as a formal deductive system” (cited in Engel, 1994, p. 226). Engel further claims that despite Cronbach’s admission that it may have been a “tactical error” to characterize the philosophical foundations of construct validity theory in terms of a logical positivist philosophy of science, the cost was the subsequent adoption by mainstream psychology of “an unnecessarily narrow conception of construct validation” that would be “operationalized in terms of particular types of procedures” (p. 227). Moss echoes this general sentiment, claiming that the view of construct validity, and of validation methods, put forth in Cronbach and Meehl (1955) was “tied closely to the positivist philosophy of science dominant at the time” (Moss, 1992, p. 233). However, Moss, like Engel, notes Cronbach’s later critiques of the initial presentation of construct validity theory (e.g., Cronbach, 1975, 1989) and his recognition of how unrealistic for psychology in its current state is the strong program of construct validity with its emphasis on fully specified nomological networks.

Greenwood (1992) argues that MacCorquodale and Meehl’s (1948) distinction between intervening variables and hypothetical constructs—which anticipated many important aspects of the formal introduction of construct validity theory (APA, 1954; Cronbach & Meehl, 1955)—is predicated on a commitment to an instrumentalist justification of theory which was itself influenced (in part) by the meaning empiricism advocated by logical empiricists such as Hempel and Feigl. Meaning empiricism, unlike methodological empiricism, which merely requires that scientific theories be subjected to empirical test, is the requirement that to be deemed meaningful, theoretical terms must be defined in terms of the observable phenomena which they have been introduced to explain (Greenwood, 1992). Greenwood claims that despite the promissory note issued by MacCorquodale and Meehl that hypothetical constructs would remain “open” and thus allow for the generation of novel empirical laws, the determinant content of such constructs may only be specified by operational definitions and thus they could offer no real advantage over intervening variables and, Greenwood implies, are predicated on essentially the same instrumentalist view of theories.

Realist interpretations

Rozeboom (1960, 1984) has long argued that the “empirical realism” championed by Feigl under the moniker “logical empiricism” was founded on the notion that because all knowledge is gained through experience does not necessarily entail a commitment to the much stronger view that knowledge is about experience (Rozeboom, 1960). Rather, Feigl’s primary thesis, according to Rozeboom (1984), is that although theoretical terms get their meanings from the data-language contexts in which they are introduced, their semantics are bound to the causal features of natural reality which are unobservable and thus only knowable through their data consequences: “Theoretical terms are about whatever features of the world have the observationally describable character that their defining theory says they have” (p. 212). Rozeboom (1984) further contends that the philosophical origins of construct validity theory have been mischaracterized as being positivistic at root; rather, he claims that “[i]n explicit opposition to positivist doctrine, the view that psychology’s theoretical constructs designate real underlying causes through their conceptual roles in the ‘nomological network’ was forcefully articulated by Cronbach and Meehl (1955)” (p. 214). From this perspective, operationalizing constructs, rather than betraying an ontological stance with regard to constructs, is merely an epistemic maneuver which enables the empirical testing of causal hypotheses.

Lovasz and I also characterize the roots of construct validity theory as primarily realist (Lovasz & Slaney, in press). Elsewhere (Slaney, 2001), I claim that despite certain clear commonalities shared by realist thinkers, there also are a number of points of divergence among empirical realists. In particular, I contrast two quite different traditions of empirical realism within psychological science, which I refer to as classical empirical realism and psychological realism, respectively. Whereas the former, I contend, constitutes essentially a “one-to-one mapping” of the empirical realist approach to the study of spatio-temporal entities onto that of psychological or sociocultural phenomena (with its attendant commitment to a strict ontological realism), the latter emphasizes the human-contingent characteristics of psychological phenomena and thus does not restrict the ascription of reality claims strictly to objectively existing spatio-temporal entities (processes, mechanisms, etc.); rather, attempts are made to understand psychological phenomena in the context of their real, but human-contingent, nature. Furthermore, I identify the origins of construct validity theory—with its commitment to the notion of observed behaviors being underlain by “constructs”—as falling squarely within the classical empirical realist camp. Lovasz and I reiterate this general position (Lovasz & Slaney, in press), as do Maraun et al. (2009). In both cases we emphasize that although construct validity theory contains positivistic elements, it has an undeniable commitment to certain core tenets of empirical (scientific) realism, in particular with respect to the theory of meaning the latter promotes.

Smith (1985) also advances an argument in favor of a realist interpretation of construct validity theory. He claims that the conventional approach to validation was located within a realist tradition in which it is presumed that “social reality exists and can be described independent of particular interests and values and that truth can be defined by correspondence” (p. 8). Furthermore, Smith contends that valid instruments or inferences are viewed from this perspective as “those that objectively and hence accurately measure reality” (p. 8), as opposed to those that merely give rise to consistent predictability of observable behaviors.

Mixed interpretations

Norris (1983) presents an argument claiming that despite its having the appearance of a field in a normal-science stage, construct validation is guided by two paradigms, one of which has its roots in logical positivism, the other in scientific realism. Specifically, Norris claims that with respect to views on the nature of constructs, the conceptions of explanation and truth invoked, the standards of adequacy employed for justifying validity claims, and the views of scientific theories adopted, elements of both logical positivism and realism are interwoven throughout the construct validation literature. As a result, Norris contends, the fundamental tenets of construct validity theory consist in an “amalgam of inconsistent ideas” (p. 71), these inconsistencies inherited from the two incompatible philosophies of science on which the individual ideas are based. However, ultimately Norris avers that the foundations of construct validity theory “rest heavily” on logical positivism and, given the arguments in philosophy that have convincingly discredited the latter, there may be little to say in favor of building a comprehensive theory of construct validation.

In his examination of the foundations of construct validity theory, Orton (1987) claims that the network view of theories provided a basis for Cronbach and Meehl’s initial articulation of the theory. However, Orton argues that the nomological network interpretation of theory, despite being based on an antiquated philosophy of science, might provide a better foundation for construct validity theory than what he calls the “narrow” position on validity, which is based on a much more rigid demarcation between scientific concepts that admit readily of operationalization and those that do not. Whereas the latter view is, according to Orton, fully steeped in a logical positivist philosophy, the former admits of some commitment to realist interpretations of theoretical constructs. In particular, Orton notes that the principal disadvantage of a construct validity theory based on the network view of theories is that this provides only a partial interpretation of constructs. And, given that psychologists lack “solid ontological intuitions” about what their theoretical terms refer to, he contends that a better foundation for construct validity theory would be that articulated by Reichenbach (1938), MacCorquodale and Meehl (1948), Feigl (1950a), and Loevinger (1957), according to which a construct has “surplus meaning” because it refers to something in the external world.

McGrath (2005) also presents a somewhat mixed interpretation of construct validity theory, claiming that some of the confusions surrounding it stem from the fact that two relatively distinct aims are served by psychological measures: prediction and representation. The former concerns the ability of item/test scores to predict the behavior of another, non-test, variable, and, hence, need not involve any ontological stance with respect to the construct in question. Representation, conversely, comes into play when psychological measures are used to represent a construct; here, the primary goal of measurement is not prediction, but reflection of individuals’ locations on the construct which is presumed to “underlie” the measure (McGrath, 2005). According to McGrath (2005), when prediction is the aim, the test score is interesting for what it says about an external referent (i.e., the criterion), whereas a score generated in the context of representation is interesting for what it says about “an intrinsic referent” (p. 113). McGrath contends that, although Cronbach and Meehl (1955) introduced the term “construct validity” primarily in interests of the latter, the demonstration of criterion-related (i.e., predictive) validity is also deemed to be an important contribution to the evaluation of construct validity. Thus, as a program, construct validation may be characterized in terms of both operationalist and realist underpinnings.

Hence, clearly several different interpretations of the philosophical foundations of construct validity theory have been given and there are doubtless others which have not been described here. The point of illustrating a few different treatments is to underscore the point that, depending on both which particular features of the theory one focuses on and also what one is prepared to presume regarding the ontological stance adopted by its developers, construct validity theory lends itself to a variety of characterizations ranging along the positivist–realist continuum. In the remainder of the paper, I shall make an appeal for a realist interpretation of the origins of construct validity theory. Specifically, I argue that (a) one can discern throughout Herbert Feigl’s brand of logical empiricism a clearly realist stance, especially with respect to the interpretation of hypothetical constructs; (b) many aspects of the initial articulations of construct validity theory may be traced to the ideas of Feigl; and (c) the originators of construct validity theory would themselves in their later writings demonstrate a clear commitment to a realist platform for the theory.

Feigl’s realist turn: From logical positivism to logical empiricism and beyond

Little can be said about the influence of Feigl’s logical empiricism in the absence of some discussion, however brief, of the classical logical positivism from which it took its starting point. The earliest articulations of logical positivism (e.g., Carnap, Hahn, & Neurath, 1929/1973) were concerned with promoting a philosophy of science concerned with furnishing the natural and social sciences with a rationality that would systematize and unify science and at the same time eliminate any references within scientific discourse to intangible, esoteric (i.e., transcendent) metaphysical domains. In particular, many early logical positivists adopted a rigid empiricist meaning criterion, according to which all theoretical concepts which could not be defined explicitly (i.e., strictly in terms of observables) were to be considered scientifically meaningless. As a consequence, logical positivism fostered a primarily descriptive science whose progress would be measured by increasingly accurate predictions made about observables, but which avoided explanatory hypotheses concerning unobservable causal entities.

It was this position, with its accompanying antirealist tone, which led members or associates of the early Vienna Circle to address the limitations inherent in the original doctrines and bring logical positivist philosophy in line with the requirements and aims of the practicing scientist. Feigl was one of the more outspoken advocates of such changes. In particular, he would urge for the adoption of a more liberal meaning criterion which would draw on the debates that were occurring at the time regarding the distinction between theoretical concepts which serve as mere shorthand notational devices for summarizing classes of observations and those that refer to the unobservable, but real, structures and mechanisms which have observable effects (e.g., Feigl 1943, 1950a, 1950b). He would also be instrumental in promoting the change in name from “logical positivism” to “logical empiricism,” the latter meant to reflect an attempt to maintain the empiricist principle but at the same time contend with the fatal flaws of the overly restrictive verifiability principle. Below, I highlight (albeit in brief) three features of Feigl’s work, namely (a) the explicitly realist ontological stance he adopted regarding scientific practice, (b) the conception of scientific theorizing he promoted, and (c) the view of meaning he assumed. I contend that these elements of Feigl’s work constitute a clear signal of his appeal for a realist philosophy of science which would provide much of the foundation for construct validity theory.

Feigl’s realism

From his early associations with the Vienna Circle, Feigl was an advocate for a realist view of science, maintaining that as long as the existential assumptions made about theoretical entities are subjected to proper empirical test, then there is little fear of introducing into scientific practice any “illegitimate transcendence.” In his 1943 chapter “Logical Empiricism,” Feigl claimed that

[t]he term “real” is employed in a clear sense and usually with good reason in daily life and science to designate that which is located in space-time and is a link in the chains of causal relations. It is thus contrasted with the illusory, the fictitious, and the purely conceptual. The reality, in this sense, of rocks and trees, of stars and atoms, of radiations and forces, of human minds and social groups, of historical events and economic processes, is capable of empirical test. (pp. 390–391)

He further contended that science need not (and, indeed, should not) be limited to the “mere description” of how things are, but should pursue why thing are as they are. The particular brand of realist philosophy he pioneered at that time he called “empirical realism,” which he likened to the scientific empiricism of Hans Reichenbach and his colleagues (p. 408).

In his 1950 paper “Existential Hypotheses: Realistic Versus Phenomenalistic Interpretations,” Feigl (1950a) would once again make a plea for a realist philosophy of science in attempting to reconcile a critical phenomenalism (operationism) with what he therein referred to as a “critical (or empirical) scientific realism” (p. 41). In particular, he argued that existential hypotheses containing “hypothetical constructs” were entirely in keeping with the ordinary realism assumed by practicing scientists. However, because they could not be strictly reduced to the observation language, they are considered only indirectly, as opposed to directly, testable. Nonetheless, they may still be viewed as scientifically meaningful. Specifically, Feigl would argue that scientific propositions need not (and, in many cases—e.g., “absolute temperature,” “gravitational field,” “[psi]-function”—could not) be strictly reduced to observation statements in order to be meaningful. Rather, such terms were taken to have “factual reference”—in that they denote real entities, processes, mechanisms, etc.—and thus the existential hypotheses in which they are contained have “surplus meaning”³ over and above their evidential basis (Feigl, 1950a). This view Feigl called “semantic realism,” a view he took to be harmonious with the critical realism of (but abandoned by) Schlick (1925, as cited in Feigl, 1956) and which Feigl would promote in several other places (e.g., Feigl, 1950a, 1956).

Feigl’s endorsement of the network view of theories

In the “Existential Hypotheses” paper, Feigl (1950a) would claim that the system of statements and concepts that comprise our knowledge base is “best understood as a network that connects the directly confirmable with the indirectly confirmable” (p. 57). He would characterize this network in terms of three primary relationships: among (a) directly observable properties (magnitudes), (b) directly observable and indirectly testable properties (magnitudes), and (c) indirectly testable properties (magnitudes). By linking the theoretical terms which designate indirectly testable properties (magnitudes) to the observation terms that represent directly testable properties (magnitudes), propositions containing theoretical concepts are made confirmable, thus “safeguarding” the requirement of empiricism (Feigl, 1950a). A short time later, Feigl (1956) would express a similar view when he averred that theories are best conceived as a “network of concepts and propositions tied only in a few places to the data of immediate experience, and for the rest a matter of ‘free construction’” (p. 16). This “free construction,” he contended, is not as articulated in Carnap’s Aufbau (1928/1967), which was influenced by the classical empiricist criterion of meaning, but, rather, amounted to the “tentative positing of a language, some of whose particulars and universals correspond to data and features of direct experience, the vast remainder of which, however, designate unobservables” (p. 16).

Feigl (1950a, 1956) identified two important features of nomological networks: (a) they express idealized relations which are progressively confirmed or disconfirmed, elaborated and amended, as a science progresses and (b) they go beyond mere description and, as such, have “explanatory power.” As regards the first, Feigl (1949, 1950a) was clear that the nomological network is an idealization of the thus far confirmed state of affairs, and, hence, is always open to revision. It is, he contended, an essential feature of science that claims about the nature of the world always be considered provisional and subject to change as a consequence of some set of disconfirming evidence:

The progress of science may be viewed as the successive trials to achieve maximum predictability of the observable facts by means of adaptations, revision, emendations or simplifications of a network that includes the unobservable (but indirectly confirmable) facts as well. (Feigl, 1950a, p. 57)

Thus, a science progresses through successive approximations toward an ideal conception of how the world is, but can, at the same time, open up such idealizations to modification if the observational base deems it necessary to do so. For Feigl (and others), it is this successively-confirmed-but-always-open-to-modification feature of theories that is “of the greatest importance for the progress of science” (Feigl, 1956, p. 12).

A second important characteristic of nomological networks identified by Feigl is their explanatory power. For Feigl and other logical empiricists, although providing descriptions of observable phenomena is a requirement for testability of a scientific proposition, it gives merely an account of, but is insufficient for accounting for, the observed phenomena (Feigl, 1943). The nomological network, because it links indirectly testable properties (magnitudes) to directly testable properties (magnitudes), is well suited to (Feigl might say essential for) the testing of existential hypotheses. A key feature of existential hypotheses, which can only be given “indirect” empirical meaning, is that they assume that nomological (causal) relations exist among unobservable theoretical entities and observable phenomena (Feigl, 1950a). From Feigl’s perspective, the observables which are connected to “hypothetical constructs” in the nomological network are “indicators” of the theoretical entity that is denoted by the construct. He claimed that “the ‘dualistic’ (i.e., realistic) assertion of the independent existence of the referents of hypothetical constructs is an essential and indispensable feature of any satisfactory explanatory system” (Feigl, 1950a, p. 47).

“Factual reference” and “surplus meaning”: Feigl on the meaning of theoretical concepts

The general view of meaning endorsed by Feigl is captured in his above-described “semantic realism.” At the crux of this conception of meaning are (a) the distinction between epistemic reduction (i.e., the evidential basis for testing scientific propositions) and the semantical relation of designation (i.e., the denotational relation between theoretical concepts and their referents) and (b) the notion that the (full) meanings of theoretical concepts reside in the theoretical entities they are presumed to denote. Regarding the first, Feigl (1950a) contended that whereas the radical empiricist’s meaning of “meaning” amounts only to the former, semantic realism assumes the more commonsense meaning of “meaning” which is adopted by the practicing scientist—that is, that many of the theoretical concepts employed in scientific discourse refer to real, but unobservable, entities. An epistemological requirement for subjecting existential hypotheses to empirical test is that the theoretical concepts contained therein need to be defined in terms of directly observable (measurable) phenomena to which they have been theoretically linked. This does not, however, carry with it the implication that such concepts are semantically reduced to the observables in terms of which they have been operationalized: that is, their meanings cannot be reduced strictly to the epistemic grounds on the basis of which existential hypotheses are tested. Rather—and this regards the second major presupposition of Feigl’s semantic realism—theoretical concepts are viewed as having “surplus meaning” which consists in their “factual reference,” the latter of which “may never coincide with their epistemic reduction” (Feigl, 1950a, p. 56). That is, the meanings given to theoretical terms is merely partial, the “surplus” residing in the unobservable theoretical entities that have been hypothesized as underlying the observables in which they have been (partially) defined (Orton, 1987).

Hence, in the logical empiricism vigorously advocated by Feigl we witness a transition from an emphasis on the strictly observable and thus directly testable to the real, but possibly unobservable, and thus only indirectly testable. Theoretical concepts are now thought to be imbued with “surplus meaning” in virtue of having “factual reference,” that is, in virtue of referring to real but currently unobservable entities whose effects, however, may be directly assessed. This realist position adopted by Feigl and other influential contributors to the later logical empiricist philosophy constituted an important shift in the stance taken with respect to interpreting propositions containing theoretical concepts: in particular, philosophers of science were now explicitly recognizing that science is in large part concerned with describing the (often causal) properties of unobservable, but real, entities, and that the terms that designate such entities refer in a meaningful way.

Laying the cornerstone: Feigl’s imprint on early construct validity theory

That aspects of Feigl’s work were influential to the development of construct validity theory is certainly not a novel insight. Indeed, Cronbach and Meehl (1955) give explicit credit to him and to other individuals for articulating the philosophical bedrock on which the theory was built. Despite this, many of the fundamental ideas of construct validity theory, whose origins lay in the works of Feigl and other well-known philosophers of science, have been and continue to be solely credited to Cronbach and Meehl. The aim here is to make explicit those features of construct validity theory which can be linked to a number of ideas put forth earlier by Feigl.⁴ Specifically, in the next section, core aspects of the foundations of construct validity theory are directly connected to Feigl’s logical empiricism in three primary ways: (a) the central role played by (hypothetical) constructs, (b) the network view of theories, and (c) the adoption of a realist criterion of meaning.

As regards the first, clearly, both Feigl’s logical empiricist philosophy and construct validity theory rely heavily on the “construct” concept. Although Feigl (1950a) accepts that constructs can be either “empirical” or “theoretical,” much of his work was concerned with how to define and properly construe the role of so-called “hypothetical constructs” (or “hypothetical existential constructs,” as he referred to them in Feigl, 1943), which he took to be a special case of theoretical concepts and distinct from mere mathematical idealizations or formulizations of empirical terms. For Feigl, what made these constructs “hypothetical” was that they were reserved for referring to those unobservable theoretical entities with whose existence existential hypotheses were concerned.

Obviously, construct validity theory also hinges on the notion of “construct.” In “Technical Recommendations” (APA, 1954), no explicit definition of “construct” is given, but the concept is implicitly defined both as a “trait or quality” possessed by individuals and reflected in test performance (e.g., pp. 13–14) and as a theoretical heuristic (e.g., p. 15) used to summarize classes of observable variables. In Cronbach and Meehl (1955), a construct is defined as a “postulated attribute of people, assumed to be reflected in test performance” (p. 283). Although none of these characterizations makes explicit reference to unobservability, Cronbach and Meehl claim that constructs may be “remote from observation” (p. 290) and there are a number of references in both the “Technical Recommedations” paper and in Cronbach and Meehl to constructs (or propositions containing constructs) that are in some way only “indirectly” measurable or testable as compared to directly testable (i.e., operationally defined) qualities. Furthermore, it is implied in “Technical Recommendations” (APA, 1954) that constructs are “hypothetical internal” processes (factors, structures, states, etc.; p. 15), or “hypothetical attributes” (p. 28), and are “explanatory” (p. 14) in that they account for, at least to some extent, test behavior. One might also surmise from the fact that construct validity was explicitly endorsed for situations in which an investigator lacks a “definitive criterion measure” of the quality he or she wishes to measure, and thus must resort to “indirect measures” (APA, 1954), that the “constructs” of construct validity theory are meant to refer, at least some of the time, to unobservable attributes (processes, structures, states, etc.).

Interestingly, in their 1948 article, “On a Distinction between Hypothetical Constructs and Intervening Variables,” MacCorquodale and Meehl propose that the term “hypothetical construct” be reserved for designating theoretical concepts that fail to meet the requirement of “intervening variables,” the latter of which they define as abstractions of strictly empirical relations. Yet, in this work, both intervening variables and hypothetical constructs are characterized as being conceptual categories—that is, classes of concepts which are used to denote particular kinds of phenomena; what differentiates them is whether they are mere abstractions of observable variables (in the former case) or concepts which denote qualities (entities, processes, etc.) which are not among the observed (in the latter case). In Cronbach and Meehl (1955), reference is made to MacCorquodale and Meehl’s earlier-made distinction between intervening variables and hypothetical constructs, but the authors are noncommittal as to whether their use of “construct” corresponds more closely to one of these senses of the term than to the other. However, the aforementioned references in “Technical Recommendations” (APA, 1954) to “hypothetical internal” processes and “hypothetical attributes” which are postulated to explain observed test behaviors suggest that the constructs of construct validity theory may overlap more with hypothetical constructs as characterized by MacCorquodale and Meehl (and also by Feigl) than with intervening variable constructs.

A second clear connection of construct validity theory to Feigl’s logical empiricism is in the adoption of the view that theories are networks of postulates containing both theoretical and observation statements. In the “Existential Hypotheses” paper, Feigl (1950a) uses the term “nomological network” in reference to introducing new theoretical concepts by means of a network of logical relationships. Although he does not there explicitly employ the label “nomological network” in reference to this system of relations, he uses the term in later papers to describe a similar system of postulates (e.g., Feigl, 1956). Furthermore, given that Cronbach and Meehl cite the “Existential Hypotheses” paper as articulating (in part) the philosophy of science on which construct validity theory is based, it seems unlikely that the nomological network they describe was conceived independently of that described by Feigl. Moreover, Feigl’s insistence that statements about unobservable (i.e., “hypothetical”) entities, to be considered testable, must “link the (directly) unverifiable with the directly verifiable” (Feigl, 1950a, p. 59) is mirrored in Cronbach and Meehl’s (1955) stipulation that “[a]n admissible psychological construct must be ‘behavior-relevant’”: that is, “unless the network makes contact with observations, and exhibits explicit, public steps of inference, construct validation cannot be claimed” (p. 291). However, as is stated in “Technical Recommendations” (APA, 1954), behavior-relevance is not meant to imply behavior-equivalence. This, too, is consistent both with the distinction Feigl drew between epistemic reduction and designation and with his endorsement of the network view of theories: “The realism here suggested allows for hypotheses only if they are at least indirectly confirmable, and for theoretical constructs only if they are part of the network which connects them with terms designating data of direct observation” (Feigl, 1956, p. 17).

Finally, it is argued that construct validity theory may be linked to Feigl’s logical empiricism by its implied endorsement of the revised logical empiricist criterion of meaning. The distinction between “explicit” and “implicit” definition is clearly a crucial component to the revisions made by logical empiricists to the empiricist criterion of meaning, and references to the distinction arise in the later works of Feigl (e.g., 1950a, 1956). As regards the meanings of hypothetical constructs, because they cannot be explicitly defined, it was believed that whatever meaning that may be attached to them comes from the entire set of postulates of which the nomological network consists: “The meaning of theoretical constructs is best explicated in terms of their locus in the nomological network, i.e. by means of postulates” (Feigl, 1956, p. 17). Cronbach and Meehl (1955) also clearly endorse this view of meaning, at least for some constructs, claiming that “the system [of postulates] constitutes an implicit definition of the theoretical primitives and gives them an indirect empirical meaning” (p. 293), and, moreover, that “[s]ince the meaning of theoretical constructs is set forth by stating the laws in which they occur, our incomplete knowledge of the laws of nature produces a vagueness in our constructs” (p. 294).⁵

Cronbach and Meehl also borrowed from the notion that nomological networks are “projections” of an ideal science which will be confirmed (or disconfirmed) as a science advances. In their discussion of the vagueness of present psychological constructs (pp. 293–294), they note that even for the most advanced physical sciences, the “idealized picture … of a tidy set of postulates” (p. 293) only approximates the ideal. Yet, however incomplete the network may be, it is crucial that a sketch of it exists, for if not, “we would not be saying anything intelligible about our constructs” (p. 294). As the nodes of the network become progressively articulated through the advancement of an empirically based science, the meanings of the terms contained within, although still only given implicitly by the network, become progressively more completely defined. Nowhere in Cronbach and Meehl (1955) is this idea better captured than in the claims they make about anxiety as a paradigm case of a psychological construct: for example, that “[w]e will be able to say ‘what anxiety is’ when we know all of the laws involving it; meanwhile, since we are in the process of discovering these laws, we do not yet know precisely what anxiety is” (p. 294).

Avowals from the originators: Cronbach and Meehl’s realist commitments

That Paul Meehl would ultimately champion a realist philosophy and approach to scientific inquiry is incontrovertible given his many explicit claims along these lines: for example,

The connotation of “natural kind” is that it would exist as a taxon in the perfect conceptual system of Omniscient Jones, that it is in some sense really out there, whether human scientists identify it or not. That way of saying it hinges upon taking a realist view of scientific concepts and theories, rather than a fictionist view as is common among psychologists; but I am a scientific realist, and I presuppose that position throughout this article. (Meehl, 1992, p. 122)

Doubtless I disagree with many of you on a philosophy of science issue, in that I am a scientific realist rather than a fictionalist or instrumentalist. (Meehl, 1979, p. 568)

[A]side from our own epistemology which is realist in its aims, we would challenge the fictionalists or instrumentalists who claim to reject the goal of getting at the “true underlying state of nature” responsible for a cluster or syndrome. … Suffice it to say that we are interested, as scientific realists, in the dichotomous causal entity alleged to underlie a family of fallible indicators. (Golden & Meehl, 1980, p. 489)

It is reasonable to ask, though, whether his realist commitments stem as far back as the publication of Cronbach and Meehl (1955) or developed over the course of his career in conjunction with the transition in science more generally toward scientific realism as a dominant philosophical paradigm. However, both Meehl himself (e.g., 1986, 1992) and others (e.g., Norris, 1983; Rozeboom, 1984) have intimated that Meehl’s realist commitments pervaded throughout the full range of his work. Furthermore, Meehl (1986) hints that in much of his work he intentionally kept vague his philosophical presuppositions, perhaps because of the reactions they might have provoked, especially early in his career when scholars were generally more positivist-minded:

Having worried about [the] reality-status of factors for some 45 years (meanwhile using it sparingly in my research with a fairly clear conscience) I will not be so foolish as to engage that terrible problem here. (p. 326)

In the same passage, he describes both himself and MacCorquodale as holding a realist view of scientific constructs.

Lee Cronbach’s philosophical stance, as it turns out, is much more difficult to pin down. Cronbach would vacillate between adopting either a realist or instrumentalist (and also pragmatic) tone in the general definitions of validity he provided, describing it, on one hand, as a property of tests (as measures of a given attribute, process, state, etc.) and, on the other hand, as a property of test scores or test score interpretations (Hood, 2009). Norris (1983) also notes that there is equivocation in Cronbach’s philosophical stance, claiming that Cronbach committed himself in different writings to either correspondence (realist), instrumentalist, or coherence (positivist) theories of truth and causation. This may be, at least to some extent, a consequence of Cronbach’s wanting to portray construct validity theory as being relatively agnostic to the particular metaphysical perspective assumed by the researcher who uses it. In fact, he admitted that his own perspective on constructs was perhaps more instrumentalist than Meehl’s realist talk of “entities,” but claims that his instrumentalism stopped short of antirealism (see Cronbach, 1989). Furthermore, in his reflections on the early articulations of construct validity theory, Cronbach admitted that the largely positivistic language in which the theory was articulated in the 1955 Cronbach and Meehl paper could be seen as a “pretentious” attempt to “dress up” their immature (but ultimately realist) science in language which would make it appear to be in line with the (logical positivist) philosophy of science that dominated at the time. Hence, rather than promoting an explicitly instrumentalist stance, Cronbach may well have simply been more willing (than Meehl) to grant the construct concept—and also the practices deemed to fall under the moniker construct validation—greater breadth. This would seem to be supported by his claim that the “Technical Recommendations” (and so, too, the subsequent versions of Standards for Educational and Psychological Testing) were “deliberately eclectic, so as to facilitate the work of psychologists of all persuasions” (Cronbach, 1989, p. 161).

Summary and conclusion

In this paper it has been argued that at its origins, construct validity theory was not a byproduct of a strictly positivist philosophy of science, or a haphazard cobbling together of elements of positivism, on one hand, and scientific realism, on the other. Rather, an attempt has been made to demonstrate that the fundamental elements of construct validity theory, including its (implicit) commitment to a realist framework, the network view of theories, and a revised empiricist criterion of meaning, may be explicitly traced to a number of ideas articulated and developed by Herbert Feigl, who was himself promoting a largely realist view of science.

Although Cronbach and Meehl do credit Feigl (and other individuals associated with logical empiricism) for providing the philosophy of science from which construct validity theory derived many of its fundamental tenets, they were still somewhat cautious in the language they employed. For example, they were themselves vague on the ontology of constructs, claiming that construct concepts need not be restricted to designating real entities (processes, states, etc.), but could in some cases be thought of as mere “inductive summaries,” especially in the early stages of the development of a construct or when the purpose of testing is purely pragmatic. Furthermore, Cronbach and Meehl avoided the use of explicit causal or explanatory talk in their explication of construct validity theory. In fact, they were decidedly evasive with regard to the ontological status of the referents of constructs and clearly wanted to—or felt they needed to—leave the door wide open as to what, if any, interpretations can and should be made by the construct validator.

However, here it is maintained that it would be a misconstrual of Cronbach and Meehl (1955)—and also of the earlier published “Technical Recommendations”—to assert that a predominantly positivistic philosophy of science was being assumed. Although there were no explicit avowals of a commitment to a realist framework, a realist ontological stance is betrayed to large extent by the very fact that the authors pioneered the development and promotion of construct validity theory in the first place. Had its creators adopted a strictly positivist conception of constructs, operational definitions of constructs (i.e., in terms of the testing procedures) and classical conceptions of validity would have sufficed and there would have been no need to conceptualize this new and special sense of validity. It is argued, rather, that the apparent ambiguities surrounding the ontological stance assumed in Cronbach and Meehl (1955) are entirely in keeping with Feigl’s logical empiricism, which it is claimed here was an essentially realist philosophy, but one which remained loyal to some of the ideas (e.g., the bifurcation of the theoretical and observation languages) and much of the terminology of the earlier logical positivist philosophy from which it grew.

In parting, it might be pertinent to ask of what relevance are the philosophical underpinnings of construct validity—whatever they may be—to psychological researchers charged with investigating constructs? In other words, how, if at all, might researchers’ construct validation practices be affected by a clarification of the philosophical commitments of the originators (or, subsequent developers) of construct validity theory? In his reflections on the philosophy of science that informed construct validity theory, Cronbach (1989) would note that the choice among various forms of instrumentalism or realism has negligible significance for investigators: “Validation follows the same path in both perspectives. …The Standards were and are deliberately eclectic, so as to facilitate the work of psychologists of all persuasions” (p. 161).

However, although clearly Cronbach implies this to be a strength of construct validity theory, it could as easily be taken to be a weakness: realist and positivist stances, respectively, regarding a given construct are sure to lend themselves to quite different interpretations of data generated from the application of measures of the construct, and such equivocations will then leave the validity of the construct in limbo, at least to some extent. To make matters worse, most researchers are at best completely opaque about the particular philosophy of constructs they presume, and, thus, attempts to reconcile the empirical findings of a body of validation research on a given construct can involve more guesswork than delineation of a set of well-established facts. Although the present work does not address this important dimension of the philosophy of construct validity theory, there certainly is much room and need for future work investigating what the construct validator believes about the ontology and epistemology of the constructs that are focal to his or her research. Furthermore, more research is needed on the extent of congruence between the philosophical stances adopted by theoretical and applied researchers, respectively, with respect to validating constructs. My colleagues and I are currently engaged in research along these lines.

Footnotes

Acknowledgements

A version of this paper was presented at the American Psychological Association Annual Convention in Toronto in August 2009.

This study was supported in part by a Social Sciences and Humanities Research Council of Canada (SSHRC) Standard Research Grant 410-2010-0194 awarded to the author.

Notes

Kathleen L. Slaney is an Assistant Professor in the Theory and Methods stream of the Department of Psychology at Simon Fraser University (SFU). Her interests span a number of areas, including history and philosophy of science, the application of conceptual analysis to philosophical problems in psychological science, and theoretical and applied psychometrics. She is currently involved in a large-scale project investigating the historical and philosophical foundations of construct validity theory and the congruity between theoretical and applied conceptualizations of validity and related measurement concepts. Address: Department of Psychology, Simon Fraser University, 8888 University Drive, Burnaby, BC, V5A 1S6, Canada. Email: klslaney@sfu.ca

References

American Psychological Association. (1954). Technical recommendations for psychological tests and diagnostic techniques. Psychological Bulletin, 51(2, Pt. 2), 1–38.

Borsboom

(2005). Measuring the mind. Cambridge, UK: Cambridge University Press.

Borsboom

(2006). The attack of the psychometricians. Psychometrika, 71, 425–440.

Borsboom

Mellenbergh

G.J.

van Heerden

(2004). The concept of validity. Psychological Review, 111, 1061–1071.

Carnap

(1967). The logical structure of the world ( George

, Trans.). Berkeley: University of California Press. (Original work published 1928)

Carnap

Hahn

Neurath

(1973). The scientific conception of the world: The Vienna Circle. In Neurath

Cohen

R.S.

(Eds.), Empiricism and sociology (pp. 299–318). Dordrecht, The Netherlands: Reidel. (Original work published 1929)

Cronbach

L.J.

(1971). Test validation. In Thorndike

R.L.

(Ed.), Educational measurement (2nd ed., pp. 443–507). Washington, DC: American Council on Education.

Cronbach

L.J.

(1975). Beyond the two disciplines of scientific psychology. American Psychologist, 30, 116–126.

Cronbach

L.J.

(1989). Construct validation after thirty years. In Linn

R.L.

(Ed.), Intelligence: Measurement theory and public policy: Proceedings of a symposium in honor of Lloyd G. Humphreys (pp. 147–171). Urbana: University of Illinois Press.

10.

Cronbach

L.J.

Meehl

P.E.

(1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.

11.

Engel

J.D.

(1994). Narrative of construct validation. Evaluation & the Health Professions, 17, 222–235.

12.

Feigl

(1943). Logical empiricism. In Runes

D.D.

(Ed.), Twentieth-century philosophy: Living schools of thought (pp. 373–416). New York, NY: Greenwood.

13.

Feigl

(1949). Naturalism and humanism. American Quarterly, 1, 135–148.

14.

Feigl

(1950a). Existential hypotheses: Realistic versus phenomenalistic interpretations. Philosophy of Science, 17, 36–62.

15.

Feigl

(1950b). Logical reconstruction: Realism and pure semantic. Philosophy of Science, 17(2), 186–195.

16.

Feigl

(1956). Some major issues and developments in the philosophy of science of logical empiricism. In Feigl

Scriven

(Eds.), Minnesota studies in the philosophy of science (Vol. 1, pp. 3–37). Minneapolis: University of Minnesota Press.

17.

Feigl

(1968). The Wiener Kreis in America. In Fleming

Bailyn

(Eds.), The intellectual migration: Europe and America, 1930–1960 (pp. 630–673). Cambridge, MA: Harvard University Press.

18.

Golden

R.R.

Meehl

P.E.

(1980). Detection of biological sex: An empirical test of cluster methods. Multivariate Behavioral Research, 15, 475–496.

19.

Greenwood

J.D.

(1992). Realism, empiricism and social constructionism. Theory & Psychology, 2, 131–151.

20.

Hempel

C.G.

(1952). Fundamentals of concept formation in empirical science. Chicago, IL: University of Chicago Press.

21.

Hood

S.B.

(2009). Validity in psychological testing and scientific realism. Theory & Psychology, 19, 451–473.

22.

Kane

M.T.

(2006). Validation. In Brennan

R.L.

(Ed.). Educational measurement (4th ed., pp. 17–64). Washington, DC: American Council on Education/Praeger.

23.

Loevinger

(1957). Objective tests as instruments in psychological theory. Psychological Reports, 3, 635–694.

24.

Lovasz

Slaney

K.L.

(in press). What makes a hypothetical construct “hypothetical”? Tracing the origins and uses of the “hypothetical construct” concept in psychological science. New Ideas in Psychology. doi:10.1016/j.newideapsych.2011.02.00510.1016/j.newideapsych.2011.02.005

25.

MacCorquodale

Meehl

P.E.

(1948). On a distinction between hypothetical constructs and intervening variables. Psychological Review, 55, 95–107.

26.

Maraun

M.D.

Slaney

K.L.

Gabriel

S.M.

(2009). The Augustinian methodological family of psychology. New Ideas in Psychology, Special Edition: Mind, Meaning, and Language: Wittgenstein’s Relevance for Psychology, 27, 148–162.

27.

McGrath

R.E.

(2005). Conceptual complexity and construct validity. Journal of Personality Assessment, 85, 112–124.

28.

Meehl

P.E.

(1979). A funny thing happened to us on the way to the latent entities. Journal of Personality Psychology, 43, 564–577.

29.

Meehl

P.E.

(1986). Trait language and behaviorese. In Thompson

Zeiler

M.D.

(Eds.), Analysis and integration of behavioral units (pp. 315–334). Hillside, NJ: Erlbaum.

30.

Meehl

P.E.

(1992). Factors and taxa, traits and types, differences of degree and differences in kind. Journal of Personality, 61, 117–174.

31.

Messick

(1989). Validity. In Linn

R.L.

(Ed.), Educational measurement (3rd ed., pp. 13–103). New York, NY: Macmillan.

32.

Moss

P.A.

(1992). Shifting conceptions of validity in educational measurement: Implications for performance assessment. Review of Educational Research, 62, 229–258.

33.

Norris

S.P.

(1983). The inconsistencies at the foundation of construct validation theory. In House

(Ed.), Philosophy of evaluation (pp. 54–74). San Francisco, CA: Jossey-Bass.

34.

Orton

(1987). The foundations of construct validity: Towards an update. Journal of Research and Development in Education, 21, 23–35.

35.

Reichenbach

(1938). Experience and prediction. Chicago, IL: University of Chicago Press.

36.

Rozeboom

W.W.

(1960). Studies in the empiricist theory of scientific meaning. Philosophy of Science, 27, 359–373.

37.

Rozeboom

W.W.

(1962). The factual content of theoretical concepts. In Feigl

Maxwell

(Eds.), Minnesota studies in the philosophy of science (Vol. 3, pp. 257–273). Minneapolis: University of Minnesota Press.

38.

Rozeboom

W.W.

(1984). Dispositions do explain: Picking up the pieces after hurricane Walter. In Royce

J.R.

Mos

L.P.

(Eds.), Annals of theoretical psychology (Vol. 1, pp. 205–224). New York, NY: Plenum.

39.

Schlick

(1925). Allgemeine Erkenntnislehre [General theory of knowledge] (2nd ed.). Berlin, Germany: Springer.

40.

Sellars

(1972). The language of theories. In MacKinnon

E.A.

(Ed.), The problem of scientific realism (pp. 182–207). New York, NY: Appleton-Century-Crofts.

41.

Slaney

K.L.

(2001). On empirical realism and the defining of theoretical terms. Journal of Theoretical and Philosophical Psychology, 22, 132–152.

42.

Smith

J.K.

(1985). Social reality as mind-dependent versus mind-independent and the interpretation of test validity. Journal of Research and Development in Education, 19, 1–9.

43.

Tuomela

(1973). Theoretical concepts. New York, NY: Springer-Verlag.