The fashionable scientific fraud: Collingwood’s critique of psychometrics

Abstract

In his review of Charles Spearman’s The Nature of ‘Intelligence’ (1923), R. G. Collingwood launched an attack upon psychometrics that was expanded in his Essay on Metaphysics (1940). Although underrated by friend and foe alike, Collingwood’s critique identified a number of defects in the thinking of psychometricians that subsequently became entrenched. However, his main complaint was that psychology generally (and, by implication, psychometrics) was a ‘fashionable scientific fraud’. This charge was inspired by his more general views on logic and metaphysics, which, however, as I argue, are logically unsustainable. Ironically, other elements of his philosophy – his ‘fallacy of calculation’ and concept of ‘scale of forms’ – are relevant to psychometrics and tip the scales in favour of his otherwise unwarranted charge.

Keywords

R. G. Collingwood fallacy of calculation psychometrics scales of forms Charles Spearman

R. G. Collingwood, an habitual critic of psychology and a philosopher with a ‘lifelong love of sweeping generalisations’ (Passmore, 1996: 156), in his Principles of History (1999: 43) proclaimed,

No science can be criticized except from inside.…The idea of philosophy as a ‘consulting science’, criticizing the work of the other sciences and judging it as sound or unsound, is merely a disguise for a kind of inquisition whose unavowed purpose is to save inquisitors the trouble of growing up.¹

Despite this anti-intellectual pronouncement,² he criticised psychometrics from outside with all the obduracy of the inquisitors denounced, and never adjudged his criticising a disguise for anything.³ On the contrary, he saw himself as defending science against barbarians.

Collingwood’s philosophy gave metaphysical primacy to history, and reacted against ahistorical ‘realism’ and logical positivism’s conceptual cleansing.⁴ At the time, the new psychology, as Dewey (1884) called the research programme of Fechner, Wundt, and their followers, was flaunting allegedly quantitative and experimental credentials to gain entry to universities as an independent science. To Collingwood (1939: 95), it epitomised the barbarism of the age: It was what phrenology, astrology, and alchemy had been to earlier times, he said, a ‘fashionable scientific fraud’.

Accompanying his single-handed running battles against the idols of his age, a series of strokes punctuated his final years. Knox (1946) says these marred his thinking, but Passmore (1957: 306) reckoned, ‘When one contemplates the speculative freedom of these later works, one can only wish that his contemporaries could have been similarly afflicted’. Collingwood’s (1923) criticisms of Spearman predated this final flourish, but he repackaged them in an original way (1940), making their philosophical wellspring explicit. These criticisms may have slowed the entry of psychology into British universities (Leudar, 2009) and fanned psychologists’ hostility to philosophy, but Collingwood’s critique of Spearman, along with his criticisms of other psychologists (not considered here; see Connelly and Costall, 2000) reveal a mind guiding its course by stars suddenly eclipsed by the rise of the faux ‘rigorism’ of the new ‘scientific’ philosophy.

Friend and foe alike dismiss his critique of psychometrics: Connelly and Costall, otherwise sympathetic, described it as ‘rather intemperate and overstated’ (2000: 151), while Hearnshaw (1942: 164) damned it as ‘sophistry, misconception, and prejudice’. However, while Collingwood did not always hit his target squarely, he was in every case reacting to real flaws flowing from Spearman’s scientistic aspirations. A mind out of step with its age views things via different presuppositions, glimpsing problems and solutions insiders cannot see.

I first consider Collingwood’s criticisms of Spearman, and whether these sum to scientific fraud. Collingwood was not driven to his verdict by these alone, but by a deeper animus. Considering this exposes the heart of his metaphysics, which, it turns out, cannot support his verdict. So what is to be made of it? Just when it seems lost, rescue arrives unexpectedly from his ‘fallacy of calculation’ (Collingwood, 1933: 80). This argument, never aimed by him at psychometrics, when fired at this target, justifies his verdict entirely.

Collingwood’s critique of Spearman

A pivotal pioneer, Spearman invented psychometrics’ methodological mainstay, factor analysis; an enduring theory of intellectual ability, g theory (Spearman, 1904); and the foundations of classical test theory (see, for example, Ng, 1974), which still infuses psychometric discourse. Francis Galton constructed psychometrics’ quantitative paradigm, and Alfred Binet laid the template for future mental tests, but it was Spearman who bequeathed the prototype for psychometric theories and methods. Spearman set in concrete psychometrics’ commitment to quantitative attributes, and in his book, The Nature of ‘Intelligence’ and the Principles of Cognition (1923), he attempted to underwrite this commitment via a new cognitive theory. Collingwood could not have selected a target more representative of psychometrics’ aspirations to scientific status.

Mere tendencies are not quantitative principles

First, Collingwood criticised Spearman for expressing his ‘quantitative principles’ as mere tendencies. For example, one of these ‘principles’, Spearman’s ‘law of retentivity’, said that ‘the occurrence of any cognitive event produces a tendency for it to occur afterward’ (Spearman, 1923: 132). Spearman’s exalted description of this banal generalisation was window-dressing: ‘quantitative principles’ expressible only as tendencies are neither quantitative (involving no quantitative terms) nor ‘principles’ (‘A tends to produce B’ states neither a necessary nor a sufficient condition for B’s occurrence, but only a condition perhaps sufficient in unspecified circumstances).

Curiously (given his interest in presuppositions), Collingwood neglected the aetiology of Spearman’s excesses in the unquestioned presupposition that mental attributes were quantitative and, therefore, measurable. Since Galton (1865), psychometricians had appropriated the rhetoric of measurement, adopting the linguistic trappings of quantitative science and via their quantitative theories committing themselves to the meaning of measurement long established in quantitative physics. Measurement has a precise meaning in physics (Wildhack, 2002), but in psychology its application expressed only unrealised hopes (Michell, 1997, 1999). While not deliberately misleading, the pre-emptive adoption of quantitative terminology foreclosed investigation into foundational questions begged in their quantitative presupposition.

Spearman’s ‘principles’ invite a contradictory interpretation

Spearman (1923: 134) likewise proposed a ‘principle of fatigue’, viz. that ‘the occurrence of any cognitive event produces a tendency opposed to its occurring afterwards’, which Collingwood (1940) alleged contradicted that of retentivity. Curiously (given his ‘logic’ of question and answer), Collingwood did not consider the question Spearman was answering with these allegedly contradictory ‘principles’.⁵ Had Spearman been asking, ‘How does the occurrence of A (any cognitive event) affect the probability of B (its occurring afterwards)?’ then his ‘principles’ mean that A simultaneously increases and decreases the probability of B (a contradiction); alternatively, had Spearman been asking, ‘How does the occurrence of A affect internal processes relating to B?’ the ‘principles’ mean that A simultaneously produces competing internal states, one working to facilitate and the other to impede B (not a contradiction). However, understood this way, these ‘principles’ are no advance on our common conviction that practice promotes recall and produces fatigue, and psychologists are not needed to tell us that these effects must be mediated by (unknown) internal states.

Plagiarism

Collingwood (1940: 128) also accused Spearman of plagiarism. This arose out of the issue of whether the existence of mental entities, such as ‘impressions’ (subjective visual percepts, for example) had been demonstrated using scientific methods. Spearman (1923: 229) claimed that thanks to psychological research, their existence was proven by experiment, whereas assertions by philosophers like Plato and Hume that they existed were speculations. While Collingwood’s assessment of Spearman’s claim as ‘plagiarism’ appears mistaken, the sympathetic reader might remember that ‘plagiarism’ derives from the Latin for kidnapping: Perhaps Collingwood was accusing Spearman of kidnapping kudos.⁶ He thought Spearman presumed, on behalf of modern psychology, scientific credit for proving Plato’s and Hume’s speculations. Whatever his rationale, Collingwood’s accusation obscures a significant defect in Spearman’s thinking.

Psychologists overestimated the power of experimental methods to probe what were hitherto adjudged philosophical issues. For example, Fechner (1860) thought his psychophysical experiments solved the mind–body problem! Psychologists failed to distinguish questions answerable by experiment from those requiring conceptual analysis. The question at issue here is of the latter kind. For example, it might be argued that the concept of a subjective visual percept is logically incoherent because percepts are both properties (internal to the mind) and relations to something outside the mind (percepts always being of objects). As argued elsewhere (Michell, 1988), because nothing can be both a property and a relation, no experiment could ever prove their existence. The defect in Spearman’s thinking was not plagiarism or kidnapping kudos; it was ignoring the role of conceptual analysis in science.

Intelligence not scientifically defined

Collingwood’s final criticism was that ‘intelligence is not scientifically definable’, since ‘the word denotes not a scientific concept but a vaguely-defined and fluctuating mass of attributes’, and trying to pin it down scientifically was like ‘trying to plot the edge of a fog with a theodolite’ (Collingwood, 1923: 118). Spearman agreed, noting that the concept was ‘scientifically unusable’ (1923: 21), and attempted to redefine intelligence as g (general ability), which he defined via factor analysis. This failed because factor analysis uses test scores (which g was thus both defined by and hypothesised to be a partial cause of), and since effects are logically independent of their causes, they cannot define them. Psychometricians still cannot agree on what g is, only what they think it does. For example, g is said to be that attribute of a person causing differences in performance on reasoning tasks (see, for instance, Flynn, 2007), which does not tell us what the attribute is, only what it causes, and so repeats the fallacy. Collingwood was reacting to a significant flaw, viz. reliance upon what are now called (thanks to his successor, Gilbert Ryle (1949)⁷) dispositional concepts in ‘defining’ the ‘constructs’ tests are said to measure (Michell, 2009a, 2013). In this way, the fiction that tests measure the hidden causes of behaviour is maintained. However, if we do not know these hidden causes (and we do not), their character remains opaque and it is not known whether they are quantitative, in which case there is no evidence that tests measure them.

Because mental tests rely upon a question and answer format, Collingwood missed an opportunity to show via his ‘logic’ of question and answer just how intelligence testing is like plotting ‘the edge of a fog with a theodolite’ (Collingwood, 1923: 118). Intelligence is the attribute hypothesised to cause individual differences in cognitive achievements. Collingwood (1940) suggested that behind such achievements were processes of questioning and answering – in which case, if intelligence is manifest anywhere, it is in the questions we ask and the answers worked out in situations where the interplay of life’s circumstances and our motives impel us to investigate things. But unless two people are asking the same question in similar circumstances and for similar reasons, their answers are incommensurable. Standardised intelligence testing, which involves setting everyone the same series of questions (the test items), may seem to avoid incommensurability, but it does not because test items, being irrelevant to people’s life circumstances, may be engaged by different motives within different people. Standardised instructions are used to harness extraneous motives to the testing task (such as exhortations to answer accurately and quickly), but the effect of these will be uneven. When extraneous motives are recruited, the questions any person is answering are thereby not merely the test’s questions, but broader ones located within the person’s matrix of motives, such as how answering the test items aids in qualifying for entry to some course or demonstrating intellectual superiority over others, and so on. Passmore, also criticising Spearman, noted the same issue:

If we inquire what 2 + 2 equals, and the answer is given as ‘Sydney’, then indeed our problem has not been solved, but the utterance of the word Sydney must have solved some problem in the mind of the speaker. It is a response to some situation, even though it is not the response we require. (Passmore, 1935: 281)

Any two people asked the same irrelevant question may locate it within the frame of different personal questions and to pretend that this might not influence performance is unrealistic. Who is to say that the intelligent person is not the one who, surmounting co-operative and conformist impulses, thumbs a nose at testing and gives the likes of ‘Sydney’ in answer to the psychometrician’s irrelevant questions? The fog obscuring intelligence from our view is generated by the inevitably inscrutable questions people are really answering whenever they engage any intellectual task.

While Collingwood’s criticisms reacted to real flaws in Spearman’s treatment, they do not sum to scientific fraud, or, at least, not as that term features in contemporary discussions (that is, implying deliberate misrepresentation). In their disregard of logical issues, psychologists were inept, but not deliberately looking to deceive. However, perhaps Collingwood had in mind another meaning, viz. that a fraud is someone who falsely presents themselves, deliberately or not. The evidence for this is that his examples of phrenology, astrology, and alchemy did not always involve deliberate deception.

At the relevant historical times, phrenology, astrology, and alchemy were continuous with science (for instance, Galileo practised astrology and Newton ardently pursued alchemy). These pseudosciences involved fraud only in the sense that their unquestioned presuppositions were too unrealistic to sustain the scientific knowledge promised. However, there is nothing in Collingwood’s critique showing that Spearman’s programme was similarly out of touch. To sustain his verdict, Collingwood needed to show that Spearman’s unquestioned quantitative presupposition promised knowledge it was incapable of delivering.

Nevertheless, Collingwood thought the new psychology (including Spearman’s psychometrics) was a scientific fraud, and his verdict was based on considerations transcending his criticisms of Spearman. These criticisms merely identified symptoms of deeper defects, which could only be displayed, Collingwood thought, via his special doctrines about the mind, logic, and metaphysics.

The philosophical roots of Collingwood’s critique

Collingwood thought that psychology could never deliver knowledge because psychologists presupposed their discipline was the science of mind, while, in reality, it was ‘the science of feeling’ (Collingwood, 1940: 106; emphasis added).⁸ Metaphysics studied the mind, he claimed. Psychology aped natural science, boasting of experimental and quantitative methods, while metaphysics employed ‘criteriological’ methods (ibid.: 109–11), and only the latter penetrated the mind.⁹ Psychology, because it used natural science methods, and not logic, had to hold that

the distinction between truth and falsehood is part of the antiquated lumber which has at last, thanks to its own success in superseding a logical science of thought by a psychological, been thrown on the dust heap. (ibid.: 120)

Hence, psychology was ‘no mere addition to the long list of pseudo-sciences; it is an attempt to discredit the very idea of science. It is the propaganda of irrationalism’ (ibid.: 142).

Collingwood’s point was that psychology (as the putative science of thought) was a self-refuting enterprise: for psychology, the distinction between truth and falsity did not exist; if the distinction between truth and falsity did not exist, science (understood as the pursuit of truth) did not exist; if science did not exist, psychology (as a science) did not exist either.

Collingwood’s major premise was that for psychology, the distinction between truth and falsity did not exist. He based this on his historical thesis that psychology grew out of the ‘materialistic epistemology’ of the 18th century (Collingwood, 1940: 114). This was an attempt to study thought by the methods of psychology (hitherto exclusively the science of feeling), which treated thought as ‘aggregations and complexes of feelings and thus special cases of sensation and emotion’ (ibid.: 113). If feelings, sensations, and emotions are not the sorts of things that can be either true or false, it follows that not only will the resulting science of thought have no place for the concepts of truth and falsity in the assessment of thoughts, but also these concepts must be eliminated from science altogether, because science is nothing if there can be no true thoughts.

Leaving aside the claim that before the 18th century psychology was exclusively the science of feeling (and there is reason to doubt it; see Vidal, 2011), it is true that during the 18th century, investigators of the mind, such as Hume, attempted to reduce thoughts to non-propositional elements (such as ideas and impressions), and that this effectively made thought impossible (Passmore, 1952). Hume (1739), for example, attempted to explain thoughts as associations between ideas, but association of ideas is insufficient to explain the complex character of propositions. It is also true that Hume’s associationism influenced the new psychology. Collingwood was correct to reject such approaches, because anything admitting of truth or falsity (such as propositions) cannot be reduced to ideas and their associations. However, there is no necessity for psychology to adopt such non-propositional approaches. If it is recognised that thoughts are irreducible to sub-propositional elements, cannot thinking be investigated naturalistically?

Psychology, thought, and logic

Collingwood reasoned that since a naturalistic approach was merely descriptive and logic was normative, no matter how accurately psychology described thinking it would never be able to conjure logic’s norms out of naturalistic observations. He wrote,

Logic is concerned with thought as its subject-matter. It has a double character. On the one hand it is descriptive, and aims at giving an account of how we actually think; on the other it is normative, and aims at giving an account of the ideal of thought, the way in which we ought to think. (Collingwood, 1933: 128)

Therefore, because is never entails ought, logic’s norms do not follow from descriptions of ‘how we actually think’, and so, he reasoned, normative concepts like truth or validity are beyond the reach of psychology. However, prescriptions about ‘the way in which we ought to think’ cannot spring out of thin air, so how are logic’s norms determined?

He did not address this, but because he thought that ‘philosophical reasoning leads to no conclusions which we did not in some sense know already’ (Collingwood, 1933: 161), ‘in some sense’ we must already know the locus of logic’s normativity. He was right: Starting from his mundane maxim that ‘all thought exists for the sake of action’ (Collingwood, 1924: 15), logic’s normative character unfolds.

For thought to guide action successfully, it must grasp those situations, knowledge of which is necessary for successful action. Situations are always a matter of things possessing properties and things standing in relation to other things. Thus, to capture situations, thought must be propositional, that is, it must involve affirming or denying a predicate of a subject. It might seem that Collingwood denied this in, first, denying that propositions were the ‘unit of thought’ (Collingwood, 1939: 34) and, second, asserting that question–answer complexes (ibid.: 37) constituted thought. However, if a phase of successful thinking begins with questioning (that is, raising the issue of a proposition’s truth) and ends with answering (affirming a proposition’s truth), thinking is both propositional and more than an unstructured collection of propositions.

If thought is propositional in this sense, what we relate to in thinking (viz. the objects of thought) must be the sorts of things propositions express, that is, situations. If, for example, I think, ‘All humans are mortal’, then my thought has the objective situation of all humans being mortal as its object. Furthermore, thoughts can only guide actions to successful outcomes if the situations constituting their objects are real. This is as it must be if thought exists for the sake of action; otherwise thought would confound action.

Hence, the idea of thought existing for the sake of action entails a concept of truth: If real situations are as thought, the thought is true (which is why true thoughts strengthen our grip on reality and promote successful action); and if real situations are not as thought, the thought is false (which is why false thoughts weaken our grip on reality and impede successful action).¹⁰ The distinction between truth and falsity is not normative but natural, residing in the veridicality or otherwise of thought’s relations to situations.

Furthermore, since thought exists for the sake of action, the distinction between logical validity and invalidity is likewise not normative but natural. For example, the thoughts that all humans are mortal and Socrates is human logically entail the conclusion that Socrates is mortal, and this is because the real situations of all humans being mortal and Socrates being human jointly include the situation of Socrates being mortal. That is, logical validity resides in relations of inclusion between situations. Such relations between situations obtain because of situations’ form, not their matter. That is, whenever there are situations of the form all X are Y and all Y are Z (where X, Y, and Z stand for kinds of things), they jointly contain a situation of the form all X are Z.

So Collingwood’s maxim that thought exists for the sake of action entails that laws of logic are first and foremost descriptive of natural relations between situations and are only descriptive of thought (or language) derivatively, because when we consider situations we typically do so in terms of what can be said about them. If infants and animals can reason validly about situations in their environments, and obviously they can, it follows that language is not necessary for valid reasoning, only convenient. Hence, the study of logic is not, in the first instance, the study of language, even of so-called formal languages. Collingwood’s maxim locates validity in the structure of the world.

These conclusions, unfolded from Collingwood’s maxim, in turn imply that logic is a descriptive science, like physics or chemistry, and like them, not intrinsically normative. Furthermore, like them, its laws acquire prescriptive force only conjoined with human motives. For example, for the engineer wanting to construct safe bridges, the laws of physics prescribe conditions necessary and sufficient for safe bridges in the relevant circumstances, and for anyone wanting to draw true conclusions from true premises, the laws of logic have prescriptive force because they describe the forms of valid inference.

Indeed, it is Collingwood’s ‘logic’ of question and answer that transforms the descriptive laws of logic into prescriptions for valid reasoning, because genuinely asking a question is the same as wanting to know the truth regarding some issue, and this motive bestows prescriptive force upon the laws of logic. Collingwood’s ‘logic’ is not logic: It is a schema of investigation within which logic acquires the prescriptive force Collingwood mistakenly thought was intrinsic to it.

From none of the conclusions arrived at above does it follow that psychology, with its preference for natural science methods, cannot employ logic prescriptively. If investigation proceeds according to Collingwood’s ‘logic’, questions guide observations but logic is still needed to guide inferences from observations to the answers arrived at. Furthermore, because logic deals with the forms of situations, not their matter, and because all thought is propositional, logic is applicable to any subject matter.

Crucially, logic is needed to properly address whether the natural science methods of measurement and experiment apply to the phenomena of the human sciences. The experimental method is a means of observing causal relations in situations where the experimenter has control over all of the significant causal factors relevant to the nominated effect, and measurement is a method of observing (or estimating) magnitudes of quantitative attributes. Hence, these methods apply only where such control is achievable (in the case of experiment) or where the attributes investigated possess quantitative structure (in the case of measurement). Psychologists omitted this step prior to committing themselves to these methods and, consequently, their conclusions are compromised accordingly; but it was not the methods of experiment and measurement that somehow made use of logic impossible in this instance; it was the scientistic imperatives endorsed by psychologists that barred this application of logic and caused psychologists to commit to these natural science methods.

Collingwood attempted to derive from, first, the fact that thoughts were true or false and, second, his opinion that logic was normative, the conclusions that, first, logic was necessary to study the mind and, second, that purely descriptive methods precluded the use of logic. Thus, he concluded that psychology was not the science of thought. However, his argument fails because his true maxim, that thought exists for the sake of action, entails that logic is not intrinsically normative.

Psychology and metaphysics

This argument was not the only string to Collingwood’s bow. Another was his view that it was metaphysics, not psychology, that studied the mind. He held that our experience of things was shaped by constellations of absolute presuppositions. These regulated the different forms of human experience, such as morality, religion, art, history, or natural science (Collingwood, 1940; d’Oro, 2002). By an ‘absolute presupposition’ he meant, first, a presupposition that did not answer a question in the mind of the person holding it (that is, one presupposed unquestioningly) and, second, one logically shaping its associated form of experience, as say, presupposing God’s existence shapes certain experiences as religious or presupposing concepts of right and wrong constructs experience of actions as moral or immoral. He saw metaphysics as investigating these fundamental, regulative, mental principles. However, as I will show, if absolute presuppositions have this character, and they are genuinely absolute, they do not shape experience.

If, as Collingwood thought, absolute presuppositions, as regulative principles within our minds, shape our experience of things, then either things themselves, independent of our minds, do not conform to these presuppositions, in which case our presuppositions are erroneous and our experience of things correspondingly non-veridical, or things themselves, independent of our minds, do conform to these presuppositions, in which case these presuppositions characterise things objectively. That is, absolute presuppositions are either false or they are true, and only in the latter case would knowledge of things be possible. But if a constellation of absolute presuppositions is true (viz. characterises reality), then they would be part of what is cognised in veridical mental acts, and Collingwood’s metaphysics, in taking absolute presuppositions to construct experience within our minds, would be mistaken. Therefore, either knowledge of things is possible, in which case Collingwood’s metaphysics is otiose, or his metaphysics tells how experience is confabulated and we know nothing about anything. Accepting any of Collingwood’s nominated forms of experience – art, morality, natural science, religion, or history – as sustaining knowledge (and he certainly wanted this for history¹¹) means that his metaphysics is false.¹²

As if anticipating this argument, Collingwood (1940: 32) insisted, ‘the distinction between truth and falsehood does not apply to absolute presuppositions’. This followed from his premises, first, that truth and falsity applied only to question–answer complexes (not propositions) and, second, that absolute presuppositions were absolute because they were not answers to questions. Hence, he thought, absolute presuppositions were not part of question–answer complexes, and consequently neither true nor false. However, his first premise is false, because to ask a question is to raise the issue of a freestanding proposition’s truth. For example, to ask, ‘Is it because number one plug is not sparking that my car won’t go?’ (Collingwood, 1939: 32) is to raise the issue of whether the freestanding proposition, ‘My car won’t go because number one plug is not sparking’ is true. Likewise, to answer ‘yes’ to that question is to affirm that proposition’s truth, and to answer ‘no’ is to deny it. Hence, question–answer complexes presuppose the truth (or falsity) of freestanding propositions. They cannot replace freestanding propositions. If freestanding propositions can be either true or false, and asking and answering questions presumes they can, Collingwood’s argument fails.

It also fails because ‘the priority affirmed in the word presupposition is logical priority’ (Collingwood, 1940: 21). When the logical work Collingwood expected of presuppositions is considered, it is clear that it can be done only if they are true. For example, if someone takes God’s existence as an absolute presupposition, they take God’s existence to ground further questions, such as, ‘does God answer prayer?’ If this absolute presupposition is not true (that is, if God does not exist), such questions have no basis in reality, and thus have no true answer. For any absolute presupposition to do the work Collingwood requires, it must be true. This is not to say that we must know whether our presuppositions are true, but objectively, independently of us, any presupposition either gets things right (things are as presupposed) or it does not, and if the latter, questions logically depending upon it are without any realistic basis. If a presupposition is neither true nor false, it claims nothing of reality and if nothing is claimed, nothing is presupposed (Llewelyn, 1961).

Defenders of Collingwood might deny that if reality does not conform to our absolute presuppositions, experience is correspondingly non-veridical, objecting that it ‘rests on the erroneous assumption that there is such a thing as epistemically unconditioned knowledge of objects’ (Connelly and d’Oro, 2005: xcii). However, it does not; it rests on the premise that if experience is conditioned by absolute presuppositions, it can be veridical only if the corresponding absolute presuppositions are true. My argument does not require that we must know whether absolute presuppositions are true but, of course, if it is possible for us to know absolute presuppositions (as Collingwood insisted), it is possible for us to know whether they are true. Indeed, Collingwood gave us a method for investigating this issue: reasoning via the hierarchy of presuppositions underlying propositions we already take to be true.

Collingwood’s concept of absolute presupposition actually includes two different sorts: first, those that are not genuinely absolute because they are relative to a restricted class of experiences, as, for example, ‘God exists’ sustains only religious experiences; and second, those that are genuinely absolute because they subserve all experience, as, for example, our conviction that anything we experience is always a thing of some kind subserves every experience we have. Only genuinely absolute presuppositions belong to the subject matter of metaphysics. When presuppositions subserving all experience are true, they affirm something about the general structure of reality (for example, the above-mentioned presupposition affirms that all things are things of some kind). Collingwood (1940: 14) rejected the view that metaphysics studied ‘pure being’ (that is, what was common to all things). However, this rejection is not incompatible with the idea that metaphysics studies features common to all situations (not to all things), and while it may study this by investigating genuinely absolute presuppositions (by investigating something mental), this is only a means to investigating a non-mental subject matter (what is presupposed).¹³ Thus, if Collingwood was right and metaphysics studies absolute presuppositions, it follows that the fundamental subject matter of metaphysics is not the mind, but the structure of reality. Thus, he has produced no reason why psychologists cannot study mind.

Collingwood’s case is dead: Psychology, as a natural science, neither desecrates logic nor invades metaphysics’ domain, and he did not demonstrate that psychometrics was a ‘fashionable scientific fraud’. Yet, before his verdict is interred, there is a further critique of mental measurement deep in his Essay on Philosophical Method, his ‘fallacy of calculation’ (Collingwood, 1933: 80). Applied to psychometrics, his dead indictment quickens.

The fallacy of calculation and scales of forms

Collingwood (1933) introduced this fallacy discussing goodness and pleasure as scales of forms. Thinking of virtue, knowledge, and pleasure as three forms of goodness, he suggested that not only did they differ in kind but also in degree (virtue being a higher form than knowledge, which was higher than pleasure); likewise, different forms of pleasure differed in both kind and degree and, agreeing with Mill that it was ‘better to be Socrates dissatisfied than a fool satisfied’ (ibid.: 79), he inferred, ‘Pleasures form a scale in which the higher are more pleasant than the lower’ (ibid.: 80), a fact ‘fatal to the project of the hedonistic calculus’ (ibid.), he concluded. His reasoning was that while some differences of degree were measurable (such as between degrees of heat in physics), others were not (such as degrees of goodness or pleasure). Measurable differences were ‘differences of degree pure and simple’ (ibid.), while non-measurable differences were differences not only of degree but also of kind. The latter are scales of forms.

Collingwood (1933) used this concept discussing philosophical issues not of relevance here. Its present importance is his claim that scales of forms were not measurable. A scale of forms is an ordered attribute, levels of which differ in both kind and degree, which differs from measurable attributes because the latter possess ‘pure differences of degree’ (ibid.: 73). By a measurable attribute he meant an attribute in which differences between any two degrees could be expressed quantitatively (for instance, one degree always being r times any other, where r is a positive real number).¹⁴

This feature of measurable attributes depends upon the fact that any two degrees of a measurable magnitude are completely homogeneous, differing only quantitatively, never qualitatively. For example, the only difference between a length of 10 cm and one of 100 cm is quantitative (the latter being 10 times the former), and there is no qualitative difference between them so far as length is concerned. Because they are composed entirely of thoroughly homogeneous parts, such attributes possess quantitative structure and, so, are measurable.

Contrast this with Collingwood’s example of an ordered attribute in which degrees differed qualitatively:

As I move my hand nearer to the fire, I feel it grow hotter; but every increase in the heat I feel is also a change in the kind of feeling I experience; from a faint warmth through a decided warmth it passes to a definite heat, first pleasant, then dully painful; the heat at one degree soothes me, at another excites me, at another torments me. I can detect as many differences in kind as I can detect differences in degree; and these are not two sets of differences but one single set. I can call them differences of degree if I like, but I am using the word in a special sense, a sense in which differences of degree not merely entail, but actually are, differences in kind. (Collingwood, 1933: 72–3)

In this case, experiences of increasing heat were qualitatively different feelings: a pleasant warmth, followed by a barely comfortable warmth, followed by a quite uncomfortable sensation, followed by a very painful burning, and so on. Note that Collingwood was not discussing heat as a physical attribute, but an effect physical heat had upon us, which was of qualitatively different sensations, from the pleasure of warmth to increasing levels of painfulness as the physical heat increased.

A scale of forms is a kind of attribute. Attributes are fundamental to science. An attribute consists of a range of properties or relations. Some attributes are quantitative (such as length or velocity); some are merely ordered (hardness, military rank); and some are purely classificatory (nationality, biological species). Science is concerned with the identification of specific attributes, their internal structure, and relationships between them. For example, much of physics involves identifying quantitative attributes (such as mass, volume, and density) and relationships between them (for example, the fact that density equals the ratio of mass to volume). With his scale of forms concept, Collingwood specified a kind of attribute already known to exist, but one all too often ignored because investigators, especially in aspiring sciences, covet quantitative attributes and their promise of measurement and, so, are easily seduced into thinking that all ordered attributes are actually quantitative, just waiting to be measured, if only a way can be invented.

This is an illusion (Michell, 2009b, 2012). If degrees of an attribute differ qualitatively, the attribute cannot be quantitative because the degrees cannot be quantitative multiples of each other. If, on the other hand, the degrees of an attribute are completely homogeneous, they may be quantitative multiples of each other. Hence, in the former case, the attribute’s structure is logically incompatible with quantitative structure. In characterising the logical incompatibility between a scale of forms and quantitative structure, Collingwood was a step ahead of psychometricians and tantalisingly close to showing that the attributes psychologists aspired to measure were unmeasurable.

There is a concept in psychometrics that dovetails with Collingwood’s scale of forms: the Guttman scale. ¹⁵ Psychometricians typically condemn it as implausible, but actually it is ‘one of the very clearest examples of a good idea in all of psychological measurement’ (Cliff, 1983: 284). It applies to mental tests composed of dichotomous items, and such a scale occurs when and only when for any person X, if X is able to answer any item correctly, X also answers all easier items correctly.¹⁶ Consequently, a Guttman scale entails that the number of items a person answers correctly indicates exactly which items that person answers correctly.

The reason this concept is a good idea is that it entails a transparent ordering of those completing the test. Obviously, one person, X, performs at least as well as another, Y, whenever X correctly answers every item Y correctly answers. With a Guttman scale, given any two people, one always performs at least as well as the other in this obvious sense. This order (the Guttman order) of people is directly observable, and the interpretation of test scores involves no recourse to theories purporting to connect answers to unobservable psychological attributes, like abilities.

The reason Guttman scales are condemned as implausible is that with most tests, the Guttman order relation rarely holds between every pair of people. With most tests, when the number of correct answers one person gives exceeds that of another, the latter person may still correctly answer one or more items that the former answers incorrectly. Hence, the test is not a Guttman scale. Most mental tests order only some pairs of people via the Guttman order relation and leave most pairs unordered.

Psychometricians attempt to get around this by using probabilistic theories of test performance, in the process committing themselves to the assumption that unobservable psychological attributes are quantitative and to assumptions about the distribution of erroneous responses, that is, responses in which people inadvertently (given their ability relative to an item’s difficulty) answer correctly or incorrectly. Via these assumptions, ‘measurements’ of hypothetical ‘abilities’ and so on are arrived at. Since the issues of, first, whether the ‘abilities’ are quantitative and, second, whether inadvertent responses behave in the way assumed, have not been independently investigated over the century of testing’s history, psychometric theories remain speculations, no matter how entrenched the practice of using them to underwrite the technology of testing has become.

However, precisely because it is a rarely attained ideal, the Guttman scale idea reinforces the conclusion that mental tests deliver only an ordering at best (not measurements) of people, and in most cases only a crude approximation to an ordering at that. ‘Measurements’ are obtained only at the cost of untested assumptions. The Guttman ideal is realised when test items are so constructed that the ensemble of cognitive resources (knowledge, skills, and strategies) required to answer any item correctly includes those required for easier items. For example, Abdi (2010: 558) considers a mathematics test for children: ‘1) counting from 1 to 50, 2) solving addition problems, 3) solving subtraction problems, 4) solving multiplication problems, and 5) solving division problems’. Here, for each item, the ensemble of cognitive resources required to answer correctly includes those required for all easier items, and at each step, the cognitive increases are heterogeneous, involving different knowledge, skills, and strategies. That is, in improving from, say, mere counting to being able to solve addition problems, the increase in cognitive resources required is qualitatively different to that required in moving from being able to solve addition problems to being able to solve subtraction problems, and so on. Performances on such a test indicate increases in the degree of mastery displayed, but these increases in degrees are qualitatively different. This means that the hierarchy of ensembles of cognitive resources is a scale of forms. What is more, in such cases, the observable attribute assessed by the test is degree of mastery displayed of the hierarchy of ensembles of relevant cognitive resources, and this attribute is a scale of forms because it involves increasing degrees differing in kind. As a scale of forms, such an observable attribute is not quantitative. Indeed, as already argued, its structure is logically incompatible with quantitative structure.

To explain individual differences between people on such observable attributes, the typical psychometric approach is to postulate underlying cognitive abilities, which are assumed to be quantitative attributes. However, to do the required explanatory job, any such hypothesised attribute requires only a structure no more complex than that of the attribute it is invoked to explain (the observable attribute). Thus, if the structure of the observable attribute is not quantitative, quantitative structure is not required of hypothesised abilities in order for them to explain test performances.

Since the concept of a Guttman scale is an ideal that tests generally only approximate, and the observable attribute assessed by a Guttman scale is a scale of forms, it follows that for any test, the observable attribute assessed is at best only ever a scale of forms. Furthermore, since scales of forms are incompatible with quantitative structure, psychometrics has no need of quantitative theories to do the scientific job required of it. Therefore, psychometrics has no place for the concept of measurement. The introduction of metric concepts into mental testing is scientifically unnecessary and their introduction is for extraneous (that is, non-scientific) reasons.

This conclusion generalises beyond ‘ability’ tests to those purporting to be of personality traits and social attitudes, that is, to the full range of hypothetical attributes psychometricians aspire to measure. Psychometricians uncritically presuppose that attributes underlying test performance are quantitative, but the ‘scale of forms’ concept shows this presupposition to be without foundation in the phenomena observed in testing.

Psychometricians have introduced the rhetoric of measurement into their discourse and theorising about mental tests because of the secondary gains derived: the social status accorded quantitative sciences and the economic rewards accrued by marketing tests as instruments of psychological measurement. But psychometrics does not deliver the measurements (in other words, the knowledge) it promises and, therefore, in Collingwood’s colloquial sense of the term, it is a scientific fraud.

But a fashionable scientific fraud? Psychometricians presuppose the ‘quantitative imperative’ (‘measure what is measurable, and make measurable what is not so’ (Michell, 1990, 2003)), which Collingwood (2005), like psychometricians (Suen, 2008), attributed to Galileo. Galileo did not endorse it (Kleinert, 2009), the relevance of measurement to investigation depending only upon the attributes studied being quantitative. Nonetheless, it is a very fashionable view in psychology, so in that sense psychometrics is a fashionable fraud.

However, it is so also in a wider sense: As a technology, psychometrics instantiates a broader fashion, viz. that of ‘metrics’ for assessing qualitative features of human performance. As Muller describes it,

There is a cultural pattern that has become ubiquitous in recent decades, engulfing an ever-widening range of institutions. Depending upon taste, one could call it a cultural ‘meme’, an ‘episteme’, a ‘discourse’, a ‘paradigm’, a ‘self-reinforcing rhetorical system’ or simply a fashion. It comes with its own vocabulary and master terms. It affects the way in which people talk about the world, and thus how they think about the world and how they act in it. For convenience, let’s call it metric fixation. (Muller, 2018: 17; original emphasis)

This cultural pattern is a fraud because its ‘metrics’ are not measures – they are frequency counts, ratings, and so on – and they obscure, indeed, in some cases destroy, the very qualities they promise to measure (Meyer, 2017); and because any such ‘metric’ will be ‘gamed’, a phenomenon called Campbell’s Law (Campbell, 1979).

Conclusion

Turning Collingwood’s fallacy of calculation argument against psychometrics and using his scale of forms concept reinstates his verdict of its being a fashionable scientific fraud. Collingwood’s own attempts failed for three reasons: first, his criticisms of Spearman, while not without merit, were insufficient; second, his argument from psychology’s alleged incompatibility with logic failed, because if thought exists for the sake of action, logic describes objective relations and derives its prescriptive force from our wanting to find true answers to the questions we investigate, including psychological ones; and third, his view that metaphysics studies the mind because it investigates absolute presuppositions fails, because if such presuppositions are genuinely absolute and sustain knowledge, they describe invariant features of objective situations, not mental structures. Despite these failures, he already had, nestled in the depths of his ‘best book’ (Collingwood, 1939: 118), materials sufficient for the job.

These were his scale of forms concept and his fallacy of calculation argument. Psychological tests in general only ever approximate Guttman scales, and the latter are scales of forms, the structure of which is logically incompatible with quantitative structure. Because the attributes observed in performances on psychological tests at best only approximate scales of forms and, therefore, cannot be quantitative, it follows that explaining test performances does not require hypothesising quantitative attributes. Hence, presenting psychological tests as instruments of measurement is a fraud.

Why did he not fashion his own attack from these materials? Collingwood rejected the very idea of psychology as the science of mind. His criticisms of Spearman identified symptoms of a fatal flaw, which was exposed, he thought, through his logical and metaphysical doctrines. On the other hand, using his fallacy of calculation argument against psychometrics would have been constructive, offering a solution to psychologists, whereas he, like a grand inquisitor, damned the entire discipline for intellectual apostasy.

Footnotes

Declaration of conflicting interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

Notes

Earlier versions of this article were read to the Theory Group in the School of Psychology, University of Sydney, and at the joint meeting of Cheiron and ESHHS, Montreal, Canada, 2012. I acknowledge the useful comments received at both venues. My thanks also to Agnes Petocz, Fiona Hibberd, and Sharon Medlow for their assistance.

References

Abdi

(2010) ‘Guttman Scaling’, in Salkind

N. J.

(ed.) Encyclopedia of Research Design. Thousand Oaks, CA: Sage, pp. 558–62.

Anderson

(1930) ‘Realism and Some of Its Critics’, Australasian Journal of Psychology and Philosophy 8(2): 113–34.

Anderson

(2007) Space, Time and the Categories: Lectures on Metaphysics, 1949-50. Sydney: Sydney University Press.

Beaney

(2013) ‘Collingwood’s Critique of Oxbridge Realism’, in Boucher

Smith

(eds) R. G. Collingwood: An Autobiography and Other Writings. Oxford: Oxford University Press, pp. 247–69.

Campbell

D. T.

(1979) ‘Assessing the Impact of Planned Social Change’, Evaluation and Program Planning 2(1): 67–90.

Cliff

(1983) ‘Evaluating Guttman Scales: Some Old and New Thoughts’, in Wainer

Messick

(eds) Principles of Modern Psychological Measurement: A Festschrift for Frederic M. Lord. Hillsdale, NJ: Erlbaum, pp. 283–301.

Collingwood

R. G.

(1923) ‘Review of Spearman’s The Nature of Intelligence’, Oxford Magazine 42: 117–18.

Collingwood

R. G.

(1924) Speculum Mentis, or the Map of Knowledge. Oxford: Clarendon Press.

Collingwood

R. G.

(1933) An Essay on Philosophical Method. Oxford: Clarendon Press.

10.

Collingwood

R. G.

(1938) The Principles of Art. Oxford: Clarendon Press.

11.

Collingwood

R. G.

(1939) An Autobiography. Oxford: Clarendon Press.

12.

Collingwood

R. G.

(1940) An Essay on Metaphysics. Oxford: Clarendon Press.

13.

Collingwood

R. G.

(1999) The Principles of History and Other Writings in the Philosophy of History, ed. Dray

W. H.

van der Dussen

W. J.

. Oxford: Oxford University Press.

14.

Collingwood

R. G.

(2005) ‘The Metaphysics of F. H. Bradley: An Essay on “Appearance and Reality”’, in Collingwood

R. G.

, An Essay on Philosophical Method (rev. ed.), ed. Connelly

d’Oro

Oxford: Clarendon Press, pp. 227–52.

15.

Collingwood

R. G.

Myres

J. N. L.

(1936) Roman Britain and the English Settlements. Oxford: Clarendon Press.

16.

Connelly

Costall

(2000) ‘R. G. Collingwood and the Idea of a Historical Psychology’, Theory & Psychology 10(2): 147–70.

17.

Connelly

d’Oro

(2005) ‘Editors’ Introduction’, in Collingwood

R. G.

, An Essay on Philosophical Method (rev. ed.), ed. Connelly

d’Oro

. Oxford: Clarendon Press, pp. xiii–cxvi.

18.

Dewey

(1884) ‘The New Psychology’, Andover Review 2: 278–89.

19.

d’Oro

(2002) Collingwood and the Metaphysics of Experience. London: Routledge.

20.

Fechner

G. T.

(1860) Elemente der Psychophysik [Elements of Psychophysics] (Vols 1–2). Leipzig: Breitkopf und Härtel.

21.

Flynn

J. R.

(2007) What Is Intelligence? Beyond the Flynn Effect. Cambridge: Cambridge University Press.

22.

Galton

. (1865) ‘Hereditary Talent and Character’, Macmillan’s Magazine 12: 157–66, 318–27.

23.

Guttman

(1944) ‘A Basis for Scaling Qualitative Data’, American Sociological Review 9(2): 139–50.

24.

Hearnshaw

L. S.

(1942) ‘A Reply to Professor Collingwood’s Attack on Psychology’, Mind 51(202): 160–9.

25.

Hibberd

(2009) ‘John Anderson’s Development of (Situational) Realism and Its Bearing on Psychology Today’, History of the Human Sciences 22(4): 63–92.

26.

Hume

(1739) A Treatise of Human Nature. London: John Noon.

27.

Kleinert

. (2009) ‘Der messende luchs. Zwei verbreitete Fehler in Der Galilei-Literatur’ [The Measuring Lynx: Two Widespread Mistakes in the Galileo Literature], NTM Zeitschrift für Geschichte der Wissenschaften, Technik und Medizin 17(2): 199–206.

28.

Knox

T. M.

(1946) ‘Editor’s Preface’, in Collingwood

R. G.

, The Idea of History. New York: Galaxy, pp. i–xxiv.

29.

Leudar

(2009) ‘What Can R. G. Collingwood Do For Psychology Today?’, Ethnographic Studies 11: 39–60.

30.

Llewelyn

J. E.

(1961) ‘Collingwood’s Doctrine of Absolute Presuppositions’, Philosophical Quarterly 11(42): 49–60.

31.

Mackie

J. L.

(1973) Truth, Probability and Paradox: Studies in Philosophical Logic. Oxford: Clarendon Press.

32.

Meyer

H.-D.

(2017) ‘The Limits of Measurement: Misplaced Precision, Phronesis, and Other Aristotelian Cautions for the Makers of PISA, APPR, etc.’, Comparative Education 53(1): 17–34.

33.

Michell

(1988) ‘Maze’s Direct Realism and the Character of Cognition’, Australian Journal of Psychology 40(3): 227–49.

34.

Michell

(1990) An Introduction to the Logic of Psychological Measurement. Hillsdale, NJ: Erlbaum.

35.

Michell

(1997) ‘Quantitative Science and the Definition of Measurement in Psychology’, British Journal of Psychology 88(3): 355–83.

36.

Michell

(1999) Measurement in Psychology: A Critical History of a Methodological Concept. Cambridge: Cambridge University Press.

37.

Michell

(2003) ‘The Quantitative Imperative: Positivism, Naïve Realism and the Place of Qualitative Methods in Psychology’, Theory & Psychology 13(1): 5–31.

38.

Michell

(2009a) ‘Invalidity in Validity’, in Lissitz

R. W.

(ed.) The Concept of Validity: Revisions, New Directions, and Applications. Charlotte, NC: Information Age, pp. 111–33.

39.

Michell

(2009b) ‘The Psychometricians’ Fallacy: Too Clever by Half?’ British Journal of Mathematical and Statistical Psychology 62(1): 41–55.

40.

Michell

(2012) ‘“The Constantly Recurring Argument”: Inferring Quantity From Order’, Theory & Psychology 22(3): 255–71.

41.

Michell

(2013) ‘Constructs, Inferences, and Mental Measurement’, New Ideas in Psychology 31(1): 13–21.

42.

Muller

J. Z.

(2018) The Tyranny of Metrics. Princeton: Princeton University Press.

43.

K. T.

(1974) ‘Spearman’s Test Score Model: A Restatement’, Educational and Psychological Measurement 34(3): 487–98.

44.

Passmore

J. A.

(1935) ‘The Nature of Intelligence’, Australasian Journal of Psychology and Philosophy 13(4): 279–89.

45.

Passmore

J. A.

(1952) Hume’s Intentions. Cambridge: Cambridge University Press.

46.

Passmore

(1957) A Hundred Years of Philosophy. London: Duckworth.

47.

Passmore

(1996) ‘Interpretation Reconsidered’, Journal of Political Philosophy 4(2): 155–68.

48.

Ryle

(1949) The Concept of Mind. London: Hutchinson’s University Library.

49.

Spearman

(1904) ‘“General Intelligence,” Objectively Determined and Measured’, American Journal of Psychology 15(2): 201–92.

50.

Spearman

(1923) The Nature of ‘Intelligence’ and the Principles of Cognition. London: Macmillan.

51.

Stevens

S. S

. (1946) ‘On the Theory of Scales of Measurement’, Science 103(2684): 677–80.

52.

Suen

H. K.

(2008) ‘Measurement’, in Salkind

N. J.

(ed.) Encyclopedia of Educational Psychology: Vol. 2. Thousand Oaks, CA: Sage, pp. 641–7.

53.

Vidal

(2011) The Sciences of the Soul: The Early Modern Origins of Psychology, trans. Brown

. Chicago: University of Chicago Press.

54.

Wildhack

W. A.

(2002) ‘Physical Measurement’, in Blumel

Rappaport

(eds) McGraw-Hill Concise Encyclopedia of Physics. New York: McGraw-Hill, pp. 483–7.