Mistaking the instance for the rule: A critical analysis of the truth-table evaluation paradigm

Abstract

Many studies probe for interpretations of < if A then C> by having people evaluate truth-table cases (<A and C>, < A and not-C>, < not-A and C>, < not-A and not-C>) as making the rule true or false, or being irrelevant. We argue that a single case can never prove a general rule to be true, as philosophy of science has taught any researcher. Giving participants the impossible “true” option would therefore bias results away from this response. In Experiment 1 people judged instead whether cases make a rule false, do not make the rule false, or are irrelevant to the rule. The experimental group (N = 44) showed a significant increase in not-false responses compared with true responses of the control group (N = 39). In Experiments 2 and 3 the experimental groups judged whether cases make a rule true, corroborate it (i.e., make the rule more plausible, but neither true nor false), make it false, or are irrelevant. There was a significant reduction of irrelevant responses as compared to the default true/false/irrelevant task for the control groups. Even < A and C> cases were often no longer considered to make an < if A then C> rule true and were correctly judged to corroborate (vs. verify) rules. Results corroborate our conceptual analyses of the unsuitable “true” response option and put into question arguments that hinge on the presumed likelihood by which people consider truth contingencies to make a rule “true”.

Keywords

Reasoning Deduction Induction Logic Conditionals If

Get full access to this article

View all access options for this article.

References

Barres

Johnson-Laird

P. N.

(2003). On imagining what is true (and what is false). Thinking & Reasoning 9, 1–42.

Barrouillet

Gauffroy

Lecas

J.-F.

(2008). Mental models and the suppositional account of conditionals. Psychological Review 115, 3 760–772.

Barrouillet

Lecas

J.-F.

(1998). How can mental models theory account for content effects in conditional reasoning? A developmental perspective. Cognition 67, 209–253.

Barrouillet

Lecas

J. F.

(1999). Mental models in conditional reasoning and working memory. Thinking and Reasoning 5, 289–302.

Blackburn

(2006). Truth: A guide for the perplexed London: Penguin Books.

Boghossian

(2006). Fear of knowledge: Against relativism and constructivism Oxford, UK: Clarendon Press.

Byrnes

J. P.

Overton

W. F.

(1988). Reasoning about logical connectives: A developmental analysis. Journal of Experimental Child Psychology 46, 194–218.

Damer

E. T.

(2005). Attacking faulty reasoning: A practical guide to fallacy-free arguments. (5). Belmont, CA: Thomson Wadsworth.

Dekeyser

Schroyens

Schaeken

Spitaels

d'Ydewalle

(2000). Preferred premise order in propositional reasoning: Semantic informativeness and co-reference. Deductive reasoning and strategies. Schaeken

De Vooght

Vandierendonck

d'Ydewalle

Mahwah, NJ: Lawrence Erlbaum Associates, Inc 73–95.

10.

Edgington

(2003). What if? Questions about conditionals. Mind & Language 18, 380–401.

11.

Evans

J. St. B. T.

(1972). Interpretation and “matching” bias in a reasoning task. Quarterly Journal of Experimental Psychology 24, 193–199.

12.

Evans

J. St. B. T.

(1983). Linguistic determinants of bias in conditional reasoning. Quarterly Journal of Experimental Psychology 35A, 635–644.

13.

Evans

J. St. B. T.

(2007). Hypothetical thinking: Dual processes in reasoning and judgement Hove, UK: Psychology Press.

14.

Evans

J. St. B. T.

Clibbens

Rood

(1996a). The role of implicit and explicit negation in conditional reasoning bias. Journal of Memory and Language 35, 392–409.

15.

Evans

J. St. B. T.

Ellis

C. E.

Newstead

S. E.

(1996b). On the mental representation of conditional sentences. Quarterly Journal of Experimental Psychology 49A, 1086–1114.

16.

Evans

J. St. B. T.

Handley

S. J.

Nielens

Over

(2007). Thinking about conditionals: A study of individual differences. Memory & Cognition 35, 7 1772–1784.

17.

Evans

J. St. B. T.

Newstead

S. E.

(1977). Language and reasoning: A study of temporal factors. Cognition 5, 265–283.

18.

Evans

J. St. B. T.

Over

D. E.

(2004). If New York: Oxford University Press.

19.

Evans

J. St. B. T.

Over

D. E.

Handley

(2005). Supposition, extensionality, and conditionals: A critique of the mental model theory of Johnson-Laird and Byrne (2002). Psychological Review 112, 1040–1052.

20.

Johnson-Laird

P. N.

Byrne

R. M. J.

(2002). Conditionals: A theory of meaning, pragmatics, and inference. Psychological Review 109, 646–678.

21.

Johnson-Laird

P. N.

Tagart

(1969). How implication is understood. American Journal of Psychology 2, 367–373.

22.

Marcus

S. L.

Rips

L. J.

(1979). Conditional reasoning. Journal of Verbal Learning and Verbal Behavior 18, 199–223.

23.

Moshman

(1979). Development of formal hypothesis-testing ability. Developmental Psychology 15, 104–112.

24.

Oaksford

Stenning

(1992). Reasoning with conditionals containing negated constituents. Journal of Experimental Psychology: Learning, Memory, and Cognition 18, 835–854.

25.

Oberauer

Geiger

Fisher

Weidenfeld

(2007). Two meanings of “if”? Individual differences in the interpretation of conditionals. Quarterly Journal of Experimental Psychology 60, 790–819.

26.

Ormerod

T. C.

(1997). Rationalities 1 and 2: Dual processes or different task demands?. Cahiers De Psychologie Cognitive 16, 181–189.

27.

Ormerod

T. C.

Manktelow

K. I.

Jones

G. V.

(1993). Reasoning with three types of conditional: Biases and mental models. Quarterly Journal of Experimental Psychology 46A, 653–677.

28.

Ormerod

T. C.

Richardson

(2003). On the generation and evaluation of inferences from single premises. Memory & Cognition 31, 467–478.

29.

Paris

(1973). Comprehension of language connectives and propositional logic relationships. Journal of Experimental Child Psychology 16, 278–291.

30.

Popper

K. R.

(1963). Conjectures and refutations: The growth of scientific knowledge London: Routledge and Kegan Paul.

31.

Schroyens

(2007). Mistaking the instance for the rule: A critical analysis of the truth-table paradigm and implications for theories of conditional reasoning Leuven, Belgium: University of Leuven (Rep. No. 319.) Laboratory of Experimental Psychology. Retrieved January 1, 2009, from http://ppw.kuleuven.be/reason/schroyens/report319.pdf.

32.

Schroyens

A critical review of thinking about what is true, possible and irrelevant in reasoning from or reasoning about conditional propositions

Proceedings of the 30th Annual Cognitive Science Society. Washington, DC Sloutsky

Love

McRae

Cognitive Science Society

2008 2116–2121 Retrieved January 1, 2009, from http://www.cogsci.rpi.edu/CSJarchive/Proceedings/2008/pdfs/p2116.pdf.

33.

Schroyens

Schaeken

(2003). A critique of Oaksford, Chater, and Larkin's (2000) conditional probability model of conditional reasoning. Journal of Experimental Psychology: Learning, Memory, and Cognition 29, 140–149.

34.

Sevenants

Schroyens

Schaeken

Dieussaert

(2008). Truth table tasks: The relevance of irrelevant. Thinking & Reasoning 14, 409–433.

35.

Thompson

V. A.

(2000). The task-specific nature of domain-general reasoning. Cognition 76, 209–268.