The role of auxiliary assumptions for the validity of manipulations and measures

Abstract

Psychologists are rightly concerned with validity of their manipulations and measures but it is far from clear what it means for these to be valid. Psychologists and philosophers who are interested in the issue have proposed several competing definitions. My goal is not to evaluate these definitions or even to review them. Rather, I start from the literature on the falsification of theories and use that literature to illustrate the importance of auxiliary assumptions in theory testing. Subsequent to this illustration, I argue that auxiliary assumptions should play a more central role in how psychologists and philosophers think about validity.

Keywords

auxiliary assumptions construct validity falsification theory validity

It is almost impossible to imagine a psychology experiment with no experimental manipulations or measures. Given this, it is not surprising that much effort has been devoted to improving them and to assessing how good they are. Psychologists also have been concerned with the general principles by which manipulations and measures can be said to be better or worse. In short, one of the important issues in the field of psychology pertains to the goodness of manipulations and measures—the issue of validity.

Researchers have proposed a variety of definitions of validity. Possibly the most well-known view, under the label of construct validity, was proposed by Cronbach and Meehl (1955; see Kane, 2001 for a recent review). According to this view, validity refers to the similarity of relations between theoretical concepts and empirical relations resulting from manipulations or measures of those theoretical concepts. To the extent that the two types of relations match, there is more validity. This definition assumes, of course, that one actually has a theory containing two or more constructs and that there are measures of at least some of them. Alternatively, Messick (1989, 1998) defined validity as the degree to which using the measures of concern produce correct conclusions or positive social outcomes. Although theoretical issues do not have a direct role in Messick’s definition, he carefully considered them in his seminal work (Messick, 1989).

The views put forth by Cronbach and Meehl (1955) or by Messick, (1989, 1998) are not the only views; there are many others (e.g., see Borsboom, Mellenbergh, & van Heerden, 2004 for a recent review). If my goal were to critically review the literature on validity, it would be necessary to explain all of these and then weigh their relative strengths and weaknesses, thereby resulting in conceptions or combinations of conceptions to support or denigrate. However, this is not my goal. I use the foregoing conceptions merely to (a) point out that validity has been, and continues to be, an important issue; (b) make clear that different competing definitions have been proposed; and (c) provide the reader with a brief reminder of the “flavor” of previous conceptions. To see where we are heading, consider that the most obvious option is to criticize previous views and then show how the view to be proposed does not suffer from the alleged limitations. In contrast, I will propose a view that is not in competition with previous views thereby rendering criticisms of those views as unnecessary. Rather, my proposal can be viewed as a clarification that potentially can aid researchers from a variety of different perspectives. Put simply, all researchers must make auxiliary assumptions—assumptions that are secondary to the theory under investigation—and I propose to place them front and center rather than letting them languish in the background. The quickest way to accomplish this is to use the literature on falsification as a starting point.

How theories and auxiliary assumptions influence falsification and validity

Few would disagree that theories, even the extremely paradigm-bound ones in psychology, posit relations between two or more unobservable variables. Some examples in psychology are that attitudes cause behavioral intentions (Fishbein, 1980; Fishbein & Ajzen, 1975), affect causes attributions (Trafimow, Bromgard, Finlay, & Ketelaar, 2005), or that the combination of a diathesis and a stressor cause depression (Walker & Diforio, 1997). If scientists had magic powers to directly observe attitudes, intentions, and so on, then the issue of validity would not come up; they would observe directly how the variables of interest covary, with or without manipulations on the part of the researcher, and the conclusions would be much more straightforward (although arguments about what is meant by causation might still be contemplated).

But alas, scientists do not have magic powers and so theory testing takes on a much less direct form. For example, one might measure attitudes by observing where check marks have been made on an attitude scale, under the assumption that the placement of the check marks has something to do with attitudes in people’s minds that cannot be observed directly. A similar strategy can be used with intentions. What one observes in a particular study, then, is not the covariance between attitudes and intentions but rather the covariance between check marks on one scale and check marks on another scale. Or, in the case where an experiment is performed to manipulate attitudes (e.g., via an essay) and determine the effect on intentions, what actually is observed is the effect of essays on check marks. In either case, whether the essays or check marks have anything to do with the entity of theoretical interest is a matter for further cogitation.

Before proceeding further, it is necessary to acknowledge that there is no reason to believe that there are only two levels at which variables can be observed—observable and unobservable. Rather, variables can be at various levels on this dimension and so, when they are referred to as observable or unobservable, as I will do in this article, this should be taken to mean in the relative sense.¹

Substantive theory links unobservable variables to each other. What links unobservable variables to observable ones? The answer was stated particularly clearly by Lakatos (1970, 1978; see also Duhem, 1914/1954; Hempel, 1965; Quine, 1952), who pointed out that researchers make assumptions that link unobservable variables to their manipulations or measures. Meehl (1990, 1997) reiterated this point in the context of psychology. These linking assumptions are termed “auxiliary assumptions,” and they have important consequences for validity, though they are usually considered in the context of falsification, which will be addressed presently.

Falsification

Although the notion of falsification has a long history in science and in the philosophy of science, the issue was popularized in the 20th century by Popper (1959, 1972, 1983). Popper was concerned that scientists often make predictions from their theories and interpret successes as proof of the theories. Popper avowed that this was an example of the classic fallacy of affirming the consequent, as can be seen by the fallacious syllogism below.

{Premise 1} If theory T is true, then observation O should occur (T → O).

{Premise 2} Observation O occurs (O).

{Conclusion} Therefore, theory T is true (T).

Observation O could have occurred for some reason other than the truth of theory T, and so the conclusion does not follow logically from the premises. But if Premise 2 were a negation rather than an affirmation, then it would be possible to draw a logical conclusion that theory T is not true, as is illustrated by the syllogism below.

{Premise 1} If theory T is true, then observation O should occur (T → O).

{Premise 2} Observation O does not occur (-O).

{Conclusion} Therefore, theory T is not true (-T).

Popper preferred being logical over being illogical and so he argued that scientists should try to disconfirm theories rather than confirm them; confirmation forces the fallacy of affirming the consequent whereas disconfirmation does not. In turn, this led Popper to consider that to use the logic of disconfirmation it is a necessary precondition that the theory being tested actually is capable of falsification. Based on Popper’s work, the falsification capability of theories has been considered to be an important issue in the sciences, including psychology (e.g., Ponterotto, 2005).

Although Popper recognized that the falsification issue was not this simple (also see Duhem, 1954; Hempel, 1965; Quine, 1952), his colleague, Lakatos, placed particular emphasis on the complications (1970, 1978). Lakatos pointed out that one never predicts an observation solely on the basis of a theory, but rather on the combination of a theory and auxiliary assumptions. For example, rather than using his theory to predict where the planets would be at particular times, Newton used his theory in combination with auxiliary assumptions about the present positions and speeds of the planets to do this. Thus, predictive failures could be attributed either to the theory or to auxiliary assumptions. When some of Newton’s predictions were wrong, he did not attribute the failures to his theory but rather to the presence of unknown and unseen astronomical bodies that were messing up the predictions. To summarize, Lakatos (1978) argued that predictions come from theories and auxiliary assumptions, and so predictive failures can be attributed in either direction. This reasoning can be illustrated by the syllogism below.²

{Premise 1} If theory T is true and auxiliary assumptions A₁, A₂, …, A_n are true, then observation O should occur (T & (A₁ & A₂ … & A_n) → O).

{Premise 2} Observation O does not occur (-O).

{Conclusion} Therefore, theory T is not true (-T) or one or more auxiliary assumptions are false (-T v (-A₁ v –A₂ …. v –A_n)).

Clearly, if a predictive failure can be attributed to either a problem in the theory or in an auxiliary assumption, then absolute falsification is out of the question. It is only if the criteria for falsification are made less stringent that falsification can be used given the presence of auxiliary assumptions. A potential cost of doing this is the possibility of rendering every theory falsifiable. To see why, note that once we grant that predictions always come from the combination of theory and auxiliary assumptions, it suggests the consideration of auxiliary assumptions that have not yet been entertained. So if a Popperian deems a particular theory to be incapable of falsification, an aficionado of the theory can respond that the theory is capable of falsification if combined with the optimal auxiliary assumptions. Until one has exhausted all possible sets of auxiliary assumptions, and noted that none of them can be combined with the theory to make a testable prediction, there is insufficient reason to sacrifice the theory on the altar of falsification. Trafimow (2009) has documented how theories that have been deemed to be incapable of falsification actually were falsified, at least under the criterion of reasonable falsification (remember that no theory meets the criterion of absolute falsification).

Validity

But what does all of this have to do with validity? To see the connection, consider that for the concept of validity to be important, it must pertain to an important goal that psychologists have. The foregoing subsection suggests that although the process of falsification is not nearly as neat as it might appear from Popper’s writings, the goal of evaluating theories is an important factor in why psychologists collect and report data. Whether one favors absolute falsification (Popper, 1959), a lesser level of falsification (Lakatos, 1978), the strong inference version of falsification (Platt, 1964), verification (Hempel, 1965), Bayesian evaluation (Edwards, Lindman, & Savage, 1963; Howson & Urbach, 1989; see Trafimow, 2003, 2005, 2006 for recent discussions), or others, it seems clear that the goal of basic researchers is to be able to make some kind of pronouncement about a theory or at least about the relative worth of alternative theories. Although considerations such as internal logic, parsimony, scope, and other characteristics of theories are important in evaluating them, it is widely agreed that the ability of theories to predict observations also is important.

I emphasize here that internal logic, parsimony, and scope are issues that do not necessitate the involvement of manipulations or measures, whereas the confirmation or disconfirmation of predictions does. To traverse the distance between theories and predictions, it is necessary to have auxiliary assumptions. Although this is not the only function of auxiliary assumptions (for a list, see Meehl, 1990, 1997), it is the function of present concern. In turn, the goodness of these auxiliary assumptions determines the extent to which one can have confidence in one’s evaluation of the theories under consideration. Bad auxiliary assumptions suggest that the data have very little to do with the theory whereas good auxiliary assumptions suggest that the data provide important input into its evaluation (see Trafimow, 2003, for a quantitative demonstration of this point). Or, to include manipulations and measures in the foregoing statement, it seems obvious that data obtained with manipulations and measures that stemmed from good auxiliary assumptions provide better theory evaluation material than data obtained with manipulations and measures that stemmed from bad auxiliary assumptions. Consequently, why not define a type of validity in terms of auxiliary assumptions? Hereafter, I define auxiliary validity as it pertains to the goodness or badness of the auxiliary assumptions used in creating experimental manipulations and tests of variables, though I shall become more specific.

The meaning of non-observational terms in auxiliary assumptions

Thus far, I have proposed that (a) auxiliary validity is to be defined in terms of auxiliary assumptions and (b) the crucial function of auxiliary assumptions is to bridge the gap between the non-observational terms in theories with the observational terms in experiments. But auxiliary assumptions logically cannot act as a bridge between non-observational and observational terms unless both are included; each auxiliary assumption must have at least one non-observational term and one observational term.³ I presume the observational terms to not be too much of a problem as they are likely to invoke wide agreement (though I hasten to add that there are cases in the history of science where this was not so or where the alleged observational terms were shown to be less observational than was originally thought). But the non-observational terms are another matter. What do these terms mean?⁴ If we do not know what they mean, do the auxiliary assumptions that contain them mean anything?

The meaning of non-observational terms in theories has been a point of argument for centuries and I have no expectation of solving it here. Instead, this subsection was designed for two purposes. First, I wish to impart to the reader the general flavor of how I think about the issue of meaning. More important, I wish to clarify that because auxiliary assumptions, by their very nature, must contain at least one non-observational term, there is no way to dodge that meaning is as much of a problem for auxiliary assumptions as it is for theories.

One possible solution to the problem is to assert that the non-observational terms mean nothing other than the observational terms connected to them but this road leads to an operationist dungeon to which I wish to avoid being sentenced. At the other extreme, one can assert that good scientists should be able to define their terms and thereby avoid ambiguity but there is a problem with this assertion (Hempel, 1965; Trafimow & Rice, 2009). Suppose that one theorizes about an unobservable entity U in a theory but desires a precise definition. To do this, one will need another term to include in the definition (call it U₁); to define one term in terms of another term means that one must have two terms. But what does U₁ mean? Unless one wishes to define U₁ in terms of U, which would involve circularity, it is necessary to invoke yet another term (call it U₂). To avoid circularity, it is necessary to keep invoking new terms indefinitely (U, U₁, U₂, U₃, …, U_N). Thus, researchers have the unpalatable choice of having circular definitions, definitions that go on forever, or they have to admit that some terms have no definitions (Hempel, 1965; Trafimow & Rice, 2009). This problem is well recognized by physicists. For example, Leon Lederman (1993), a Nobel Prize-winning physicist, pointed out that Newton’s equation of Force = Mass x Acceleration is the most important equation in the history of physics despite the fact that Newton never defined mass. Consistent with these considerations, I reject that notion that unobservable variables in theories must have explicit definitions.

Having rejected both extremes, I am forced into the uncomfortable position of saying that non-observational terms in theories and auxiliary assumptions have meaning above and beyond the way they are manipulated and measured but that there is no way to state explicitly what that surplus meaning entails. A time-honored solution is to say that the surplus meaning is implicit in the relations between non-observational terms that are specified in the theory. I agree with this but believe there is a further source of meaning. To see where it lies, consider that the word chosen for a non-observational term is unlikely to have been chosen at random and that it has a meaning based on the way it has been used in ordinary English as well as in the technical way it might be used in the field of concern. As an example, consider again Newton’s use of the word “mass.” It gains some meaning from its relations with “force” and “acceleration” but it also gains some meaning from the connotation of the word itself, as implying the presence of substance. Finally, non-observational terms also gain some meaning from the experiments researchers perform; by connecting non-observational terms to observational ones in auxiliary assumptions, meaning cannot help but be transmitted “backwards” by dint of this link. This last point is extremely important. Although few would argue that experiments can help to clarify the meanings of non-observational terms, without the explicit consideration of auxiliary assumptions it is not clear why this is so. Explicitly acknowledging that auxiliary assumptions link non-observational and observational terms immediately suggests the possibility that meaning can be transmitted along this link. Finally, although my immediate focus is on backward meaning transmission from observational terms to non-observational ones, it also can happen that the link between the two kinds of terms allows the “forward” transmission of meaning from non-observational terms to observational ones; that even seemingly observational terms can be theory laden is another consequence of the linking function of auxiliary assumptions.

The goodness of auxiliary assumptions

To summarize thus far, theories have non-observational terms with definitions that cannot be explicitly stated that are connected to observational terms used in data collection by auxiliary assumptions. Each auxiliary assumption contains at least one non-observational term, one observational term, and the relation between them. Finally, auxiliary validity depends on the goodness of the auxiliary assumptions. But what does “goodness” mean, particularly in light of the implicitness of the meaning of non-observational terms?

The most obvious meaning of “good” is “true” and this leads to a definition of auxiliary validity; manipulations and measures have auxiliary validity if the auxiliary assumptions on which they are based are true. Although the definition seems straightforward, there are some complications, especially when considered in the context of other notions of validity.

Consider again the traditional notion of construct validity as a matching of empirical with theoretical relations; better matching implies more construct validity. There can be little doubt that construct validity is an important concept that has played an important role in the history of psychology and likely will continue to do so. Yet, it is not sufficient. As an example, consider phlogiston theory and the auxiliary assumptions that (a) burning an object releases the phlogiston it contains, and (b) phlogiston has mass and so releasing it decreases the mass (and weight) of the object. Thus, when 17th-century researchers burned wooden objects and found that these objects weighed less after being burnt than before, there was a good match between theoretical relations and empirical ones. Theoretically, releasing phlogiston should decrease mass and empirically, burning wooden objects actually did decrease their weight. Thus, construct validity was strong and phlogiston theory did very well until Lavoisier (see Trafimow & Rice, 2009, for a description). Of course, we know today that phlogiston does not exist and so the auxiliary assumption that burning objects releases phlogiston is incorrect. Thus, although the manipulations and measures the phlogiston researchers used have good construct validity, they are poor in auxiliary validity, thereby making it easy to see the contrast between construct validity and auxiliary validity. The general point is that theoretical and empirical relations can match for all kinds of reasons and so researchers concerned with construct validity can gain by also considering auxiliary validity.

Construct validity is not the only kind of validity. Recently, Borsboom et al. (2004) suggested that measures are valid if the findings obtained on those measures are caused by the hypothesized latent trait. A problem with this definition is that causality is an extremely tricky issue that has yet to be adequately conceptualized by philosophers; as philosophers often have noted, existing conceptions of causality run into serious problems (Beauchamp, 1974; Sosa, 1975; von Wright, 1974, but see Salmon, 1998, for a potential exception). However, along with Borsboom and colleagues (2004), I will ignore this. Instead, I wish to focus on a hypothetical example. Suppose a political psychologist is interested in the prejudice that members of the Republican Party have against members of the Democratic Party and assumes that this prejudice is a latent trait that has causal effects (as opposed to a label to summarize a variety of behaviors). The political psychologist measures this prejudice by asking questions about the negative characteristics of members of the Democratic Party; greater endorsement of these negative characteristics is taken to indicate greater prejudice. It is easy to imagine a variety of auxiliary assumptions linking prejudice to responses on the measure but let us consider only two of these. One potential auxiliary assumption is that Republicans with more prejudice against Democrats actually will believe that Democrats have more of the negative characteristics on the scale and that this belief will cause Republicans with more prejudice to endorse more negative items than Republicans with less prejudice. A second potential auxiliary assumption is that more prejudice causes more anticipated enjoyment of endorsing negative characteristics and so differences in anticipated enjoyment cause differences in the endorsement of negative characteristics. Even if the researcher assumes that one of these auxiliary assumptions is true when in fact the other one is true, the latent trait causes responses on the measure and so the measure is valid according to the Borsboom et al. (2004) criterion. In contrast, according to the proposed notion of auxiliary validity, the measure is valid if the researcher makes the correct auxiliary assumption but not if the researcher makes the incorrect one. Because it is possible for the measure to be valid in the Borsboom et al. (2004) sense but to be invalid in the sense of auxiliary validity, the contrast between these two types of validity is easy to see.

The existence of the unobservable

Applying the material conditional to auxiliary validity can cause a problem. If one considers an auxiliary assumption to be a material conditional, then by definition the statement as a whole is true so long as the antecedent is false. And if the antecedent uses a non-observational term that refers to an unobservable that does not exist, thereby making the antecedent false, then the auxiliary assumption as a whole is necessarily true. This means that all auxiliary assumptions are true so long as they are based on a non-observational term that refers to a nonexistent unobservable and this might seem inconvenient.

There are at least two ways to deal with this inconvenience. For those researchers who take seriously the voluminous literature criticizing the material conditional (see Sanford, 1989, for an exhaustive review), the inconvenience can be considered to be an artifact. Interpreting auxiliary assumptions as subjunctive conditionals rather than as material conditionals eliminates the artifact (Pearl, 2000; Sanford, 1989).

For those who favor the material conditional over the subjunctive conditional despite the criticisms, there is a different way to deal with the problem, that is, to differentiate between “trivially” versus “nontrivially” true auxiliary assumptions. A trivially true auxiliary assumption would be one where its truth is dependent on the falsity of the antecedent whereas a nontrivially true auxiliary assumption would be one that would be true even with a true antecedent. In this case, then, manipulations and measures would have auxiliary validity if they are based on nontrivially true auxiliary assumptions (as opposed to simply true auxiliary assumptions). For present purposes, it is a matter of indifference which solution is used, or even whether researchers consider the issue at all.

Discussion

My proposal can be summarized quickly as follows. There is no way to test theories without manipulations or measures of variables and it is crucial that these are valid, which means that validity is an extremely important issue. Taking seriously that an important goal of research is to test theories, there is no way to sidestep the necessity of linking the non-observational terms in theories with the observational ones in data collection; the auxiliary assumptions through which this linkage is accomplished are absolutely crucial. Consequently, because auxiliary assumptions are crucial, it makes sense to focus on them, which leads to the proposal of auxiliary validity. Put simply, if the auxiliary assumptions upon which manipulations and measures are based are true, then these manipulations and measures have auxiliary validity.

Does auxiliary validity offer anything that other concepts of validity do not offer? We already have seen that it is possible for measures to have construct validity or causal validity without having auxiliary validity. Therefore, it is clear that auxiliary validity is different from other kinds of validity. But more can be said about this.

Consider that it is important not only to recognize the importance of auxiliary assumptions, but also to recognize that they are very different from both theories and data collection. Theories tend to establish links between non-observational terms (e.g., attitudes cause intentions) and data collection tends to establish links between observational terms (e.g., particular essays influence check marks on intention scales). How is it that links between observational terms obtained in an experiment can be considered to support theorized links between non-observational terms? There is no way to answer this question without establishing links between non-observational and observational terms; auxiliary assumptions fulfill this role. The difference between auxiliary validity and other kinds of validity is the explicit recognition of this, which allows us to mentally distinguish auxiliary assumptions from both theory and data collection. The notion of auxiliary validity explicitly keeps the distinction in the foreground whereas other notions of validity fail to distinguish auxiliary assumptions from theory and data collection in a sufficient way to do this. For example, construct validity focuses on a matching of theoretical and empirical relations and so the role of auxiliary assumptions is reduced to an implicit status. This reduction is largely responsible for the fact that manipulations and measures can have construct validity while nevertheless failing to have auxiliary validity. In addition, Borsboom and colleagues’ (2004) conception of causal validity only specifies that the latent trait causes responses on the measure and not that the latent trait causes responses on the measure because of a specific reason or reasons. Therefore, if the proposed reason is not correct, and the latent trait causes responses on the measure because of some other reason not considered by the researcher, causal validity is nevertheless maintained though auxiliary validity clearly is not.

We have seen that manipulations and measures can have other kinds of validity while nevertheless failing to have auxiliary validity. Is the reverse also possible? Can manipulations and measures have auxiliary validity without having other kinds of validity? I think this is so.

Let us remember that manipulations and measures have auxiliary validity so long as the auxiliary assumptions are true. However, auxiliary assumptions can be true without necessitating that the theory from which they were derived is true. For example, consider again the theory that attitudes cause intentions. Let us now pretend that we have certain knowledge of the following: First, attitudes do not cause intentions; second, the attitude and intention measures both have auxiliary validity; third, because attitudes do not cause intentions, the correlation between our attitude and intention measures equals zero. In this example, because of the wrongness of the theory, the empirical relations (no correlation) and theoretical correlations (correlation) do not match. Therefore, although the attitude and intention measures have auxiliary validity, they clearly do not have construct validity.

As another example, suppose that one specifies a chain of auxiliary assumptions linking attitudes to an attitude measure, not through a causal mechanism but rather through correlated variables. In other words, although attitudes do not cause responses on the attitude measure there is a correlation between the two through correlations with a third variable. In this case, the attitude measure would not be valid in the Borsboom et al. (2004) sense, because people’s attitudes would not cause their responses on the measure, but the measure would have auxiliary validity so long as the chain of auxiliary assumptions is true.

Conclusion

Philosophers and philosophically minded scientists have known about the importance of auxiliary assumptions for theory testing for centuries. Although previous conceptions of validity have taken auxiliary assumptions into account, auxiliary assumptions have been bit players in those conceptions rather than central characters. The present proposal of auxiliary validity changes that; auxiliary assumptions now are central. But it is important not to misconstrue this change in emphasis in the validity arena as also supporting a change in emphasis in science in general. For those scientists who perform basic research to test theories, theories likely will retain, and should retain, the lion’s share of attention. Nor should my present emphasis on auxiliary assumptions change that; the goal of basic research has been, and will continue to be, to propose and test theories. Given this, how might the change in emphasis engendered by the notion of auxiliary validity influence basic research?

To answer this question, consider again that auxiliary validity is concerned with auxiliary assumptions and not directly with assumptions linking non-observational terms in theories. Therefore, to the extent that a researcher is thinking purely at the theoretical level, it is unlikely that thinking about auxiliary validity will be helpful. However, when a researcher is thinking about how to test a theory, or about how to test competing theories against each other, it is necessary to bridge the gap between the non-observational terms in theories and the observational ones in manipulations or measures. By placing the requisite auxiliary assumptions that accomplish this bridging in the foreground, as opposed to the background, the researcher is likely to give more attention to them and thereby improve the manipulations and measures relative to what they otherwise would have been. In turn, the improved manipulations and measures should make for better theory tests and for stronger theory corroborations and disconfirmations than would otherwise be achieved.

But this is not the only potential gain. We already have seen that the meaning of non-observational terms in theories is a tricky issue, and that one of the sources from which these non-observational terms derive their meaning is from the observational terms in experiments that derive from them. Well, then, if meaning is to flow from observational terms in experiments back to the non-observational terms in theories, it seems inescapable that better connections between observational and non-observational terms implies better backwards flow of meaning. Stated in more general language, better auxiliary assumptions imply more meaningful theories, both because of better theory tests and because of increased flow of meaning backwards from experiments to theories. Thus, an explicit concern with auxiliary validity, not ignoring other kinds of validity that also matter, should ultimately result in better theories which, after all, is the goal of those who choose to perform basic research.

Footnotes

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Notes

David Trafimow is Professor of Psychology at New Mexico State University. He has published articles on a variety of topics, including philosophical and statistical issues in psychology, attitudes, attribution, and self-concepts. Address: Department of Psychology, MSC 3452, New Mexico State University, PO Box 30001, Las Cruces, NM 88003–8001, USA. Email: dtrafimo@nmsu.edu

References

Beauchamp

T. L.

(1974). Philosophical problems of causation. Encino, CA: Dickenson Publishing Company, Inc.

Borsboom

Mellenbergh

van Heerden

(2004). The concept of validity. Psychological Review, 111(4), 1061–1071.

Cronbach

L. J.

Meehl

P. E.

(1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.

Duhem

(1954). The aim and structure of physical theory ( Wiener

, Trans.). New York, NY: Atheneum. (Original work published 1914)

Edwards

Lindman

Savage

L. J.

(1963). Bayesian statistical inference for psychological research. Psychological Review, 70, 193–242.

Fishbein

(1980). Theory of reasoned action: Some applications and implications. In Howe

Page

(Eds.), Nebraska symposium on motivation, 1979 (pp. 65–116). Lincoln: University of Nebraska Press.

Fishbein

Ajzen

(1975). Belief, attitude, intention and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.

Hempel

C. G.

(1965). Aspects of scientific explanation and other essays in the philosophy of science. New York, NY: The Free Press.

Howson

Urbach

(1989). Scientific reasoning: The Bayesian approach. La Salle, IL: Open Court.

10.

Kane

M. T.

(2001). Current concerns in validity theory. Journal of Educational Measurement, 38, 319–342.

11.

Lakatos

(1970). Falsification and the methodology of scientific research programs. In Lakatos

Musgrave

(Eds.), Criticism and the growth of knowledge (pp. 91–195). Cambridge, UK: Cambridge University Press.

12.

Lakatos

(1978). The methodology of scientific research programmes: Vol. 1. Philosophical papers. Cambridge, UK: Cambridge University Press.

13.

Lederman

(1993). The god particle: If the universe is the answer, what is the question? New York, NY: Houghton Mifflin Company.

14.

Meehl

P. E.

(1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant using it. Psychological Inquiry, 1, 108–141.

15.

Meehl

P. E.

(1997). The problem is epistemology, not statistics: Replace significance tests by confidence intervals and quantify accuracy of risky numerical predictions. In Harlow

Mulaik

S. A.

Steiger

J. H.

(Eds.), What if there were no significance tests? (pp. 393–425). Mahwah, NJ: Lawrence Erlbaum Associates.

16.

Messick

(1989). Validity. In Linn

R. L.

(Ed.), Educational measurement (3rd ed., pp. 13–103). Washington, DC: American Council on Education and national Council on Measurement in Education.

17.

Messick

(1998). Test validity: A matter of consequence. Social Indicators Research, 45, 35–44.

18.

Pearl

(2000). Causality: Models, reasoning, and inference. Cambridge, UK: Cambridge University Press.

19.

Platt

J. R.

(1964). Strong inference. Science, 146, 347–353.

20.

Ponterotto

J. G.

(2005). Qualitative research in counseling psychology: A primer on research paradigms and philosophy of science. Journal of Counseling Psychology, 52(2), 126–136.

21.

Popper

K. R.

(1959). The logic of scientific discovery. New York, NY: Basic Books.

22.

Popper

K. R.

(1972). Objective knowledge. Oxford, UK: Oxford University Press.

23.

Popper

K. R.

(1983). Realism and the aim of science. London, UK: Routledge.

24.

Quine

W. V. O.

(1952). The dogmas of empiricism. Reprinted from “A logical point of view,” Cambridge, MA: Harvard University Press.

25.

Salmon

W. C.

(1998). Causality and explanation. Oxford, UK: Oxford University Press.

26.

Sanford

D. H.

(1989). If p, then q: Conditionals and the foundations of reasoning. London, UK: Routledge.

27.

Sosa

(1975). Causation and conditionals. Oxford, UK: Oxford University Press.

28.

Swets

J. A.

Tanner

W. P.

Jr. Birdsall

T. G.

(1961). Decision processes in perception. Psychological Review, 68(5), 301–340.

29.

Trafimow

(2003). Hypothesis testing and theory evaluation at the boundaries: Surprising insights from Bayes’s theorem. Psychological Review, 110, 526–535.

30.

Trafimow

(2005). The ubiquitous Laplacian assumption: Reply to Lee and Wagenmakers. Psychological Review, 112, 669–674.

31.

Trafimow

(2006). Using epistemic ratios to evaluate hypotheses: An imprecision penalty for imprecise hypotheses. Genetic, Social, and General Psychology Monographs, 132, 431–462.

32.

Trafimow

(2009). The theory of reasoned action: A case study of falsification in psychology. Theory & Psychology, 19, 501–518.

33.

Trafimow

Bromgard

I. K.

Finlay

K. A.

Ketelaar

(2005). The role of affect in determining the attributional weight of immoral behaviors. Personality and Social Psychology Bulletin, 31, 935–948.

34.

Trafimow

Rice

(2008). Potential performance theory (PPT): A general theory of task performance applied to morality. Psychological Review, 115, 447–462.

35.

Trafimow

Rice

(2009). What if social scientists had reviewed great scientific works of the past? Perspectives on Psychological Science, 4, 65–78.

36.

von Wright

G. H.

(1974). Causality and determinism. New York, NY: Columbia University Press.

37.

Walker

Diforio

(1997). Schizophrenia: A neural diathesis-stress model. Psychological Review, 104(4), 667–685.