This article considers the application of cumulative meta-analysis, defined as the procedure of performing a (new) meta-analysis at every point during the history of a research domain. Two distinct facets of cumulative knowledge are identified: sufficiency (“Are additional studies needed to establish the existence of the phenomenon?”) and stability (“Will additional studies change the aggregate picture of the phenomenon?”). These two facets of cumulative knowledge define the purpose of the present effort: How can we determine whether a cumulative meta-analytic database has achieved sufficiency and stability? The authors delineate indicators of sufficiency and stability that might be derived from cumulative meta-analyses and explore the use of these indicators in a set of previously published meta-analytic databases. Discussion explores both retrospective and prospective implications of this approach to cumulative knowledge and compares the implications of this approach to cumulative knowledge with alternative views of social psychology as history.
Get full access to this article
View all access options for this article.
References
1.
Abelson, R. P. (1997). On the surprising longevity of flogged horses: Why there is a case for the significance test. Psychological Science, 8, 12-15.
2.
Adorno, T. W. , Frenkel-Brunswik, E., Levinson, D. J., & Sanford, R. N. (1950). The authoritarian personality. New York: Harpers.
3.
Anthony, T. , Copper, C., & Mullen, B. (1992). Cross-racial facial identification: A social cognitive integration. Personality and Social Psychology Bulletin, 18, 296-301.
4.
Barnir, A. (1998). Can group- and issue-related factors predict choice shift? A meta-analysis of group decisions on life dilemmas. Small Group Research, 29, 308-338.
5.
Beardon, J. A. , & Thomsen, J. S. (1957). A survey of atomic constants. Nuovo Cimento, Suppl. 5, 267-360.
6.
Beck, C. T. (1996). A meta-analysis of the relationship between postpartum depression and infant temperament. Nursing Research, 45, 225-230.
7.
Berkson, J. (1942). Tests of significance considered as evidence. Journal of the American Statistical Association, 37, 325-335.
8.
Bond, C. F. , & Titus, L. J. (1983). Social facilitation: A meta-analysis of 241 studies. Psychological Bulletin, 94, 265-292.
9.
Bornstein, R. F. (1994). Dependency as a social cue: A meta-analytic review of research on the dependency-helping relationship. Journal of Research in Personality, 28, 182-213.
10.
Braden, J. P. , Gonzalez, G. M., & Miller, M. D. (1990). Use of time series, ARIMA designs to assess program efficacy. School Psychology Review, 19, 224-231.
11.
Carson, K. P. , Schreisheim, C. A., Kinicki, A. J. (1990). The usefulness of the failsafe statistic in meta-analysis. Educational and Psychological Measurement, 50, 233-243.
12.
Chambers, J. M. , Cleveland, W. S., Kleiner, B., Tukey, P. A. (1983). Graphic methods for data analysis. Belmont, CA: Wadsworth.
13.
Cochran, W. G. (1937). Problems arising in the analysis of a series of similar experiments. Journal of the Royal Statistical Society, Suppl. 4(1), 102-118.
14.
Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-1312.
15.
Coladarci, A. (1983). Paradise regained: An apodictic analysis of the relationship between school size and pupil achievement. Research in Rural Education, 2, 79-82.
16.
Cooper, H. M. (1979). Statistically combining independent studies: A meta-analysis of sex differences in conformity research. Journal of Personality and Social Psychology, 37, 131-135.
17.
DeProspero, A. , & Cohen, S. (1979). Inconsistent visual analysis of intrasubject data. Journal of Applied Behavioral Analysis, 12, 573-579.
18.
Diaconis, P. (1985). Theories of data analysis: From magical thinking through classical statistics. In D. C. Hoaglin, F. Mosteller, & J. W. Tukey (Eds.), Exploring data tables, trends, and shapes (pp. 1-36). New York: John Wiley.
19.
Driskell, J. E. , & Mullen, B. (1990). Status, expectations and behavior: A meta-analytic review and test of the theory. Personality and Social Psychology Bulletin, 16, 541-553.
20.
Fisher, R. A. (1932). Statistical methods for research workers. London: Oliver and Boyd.
21.
Gergen, K. J. (1973). Social psychology as history. Journal of Personality and Social Psychology, 26, 309-320.
22.
Gergen, K. J. (1976). Social psychology, science, and history. Personality and Social Psychology Bulletin, 2, 373-383.
23.
Gilbert, D. T. (1998). Ordinary personology. In D. T. Gilbert, S. T. Fiske, & G. Lindzey (Eds.), The handbook of social psychology (Vol. II, pp. 89-150). Boston: McGraw-Hill.
24.
Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Research, 5, 3-8.
25.
Goldberger, J. (1907). Typhoid “bacillus carriers.” In M. J. Rosenau, L. L. Lumsden, & J. H. Kastle (Eds.), Report on the origin and prevalence of typhoid fever in the District of Columbia. Hygenic Laboratory Bulletin, 35.
26.
Hardaway, R. A. (1990). Subliminally activated symbiotic fantasies: Facts and artifacts. Psychological Bulletin, 107, 177-195.
27.
Hedges, L. V. (1987). How hard is hard science, how soft is soft science? The empirical cumulativeness of research. American Psychologist, 42, 443-455.
28.
Hedges, L. V. , & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
29.
Herbert, T. B. , & Cohen, S. (1993). Depression and immunity: A meta-analytic review. Psychological Bulletin, 113, 472-486.
30.
Hovland, C. I. , Lumsdaine, A. A., & Sheffield, F. D. (1949). Experiments on mass communication. Princeton, NJ: Princeton University Press.
31.
Hunter, J. E. , & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
32.
Ito, T. A. , Miller, N., & Pollock, V. E. (1996). Alcohol and aggression: A meta-analysis on the moderating effects of inhibitory cues, triggering events, and self-focused attention. Psychological Bulletin, 120, 60-82.
33.
Jaensch, E. R. (1938). Der Geqentypus. Leipzig, Germany: Barth.
34.
Johnson, B. , Mullen, B., & Salas, E. (1995). A comparison of the three major meta-analytic approaches. Journal of Applied Psychology, 80, 94-106.
35.
Jones, E. E. (1985). Major developments in social psychology during the past five decades. In G. Lindzey & E. Aronson (Eds.), Handbook of social psychology (Vol. I, pp. 47-108). New York: Random House.
36.
Lau, J. , Antman, E. M., Jimenez-Silva, J., Kupelnick, B., Mosteller, F., & Chalmers, T. C. (1992). Cumulative meta-analysis of therapeutic trials for myocardial infarction. New England Journal of Medicine, 327, 248-254.
37.
Levine, N. (1976). On the meta-physics of social psychology: A critical view. Human Relations, 29, 385-400.
38.
Lush, J. L. (1931). Predicting gains in feeder cattle and pigs. Journal of Agricultural Research, 42, 853-881.
39.
McCain, L. J. , & McCleary, R. (1979). The statistical analysis of the simple interrupted time-series quasi-experiment. In T. D. Cook & D. T. Campbell (Eds.), Quasi-experimentation, design, and analysis issues for field settings. Chicago: Rand McNally.
40.
Mosteller, F. , & Tukey, J. W. (1968) Data analysis, including statistics. In G. Lindzey & E. Aronson (Eds.), The handbook of social psychology (Vol. II). Reading, MA: Addison-Wesley.
41.
Mullen, B. (1989). Advanced BASIC meta-analysis. Hillsdale, NJ: Lawrence Erlbaum.
42.
Mullen, B. (1991). Group composition, salience, and cognitive representations: The phenomenology of being in a group. Journal of Experimental Social Psychology, 27, 297-323.
43.
Mullen, B. , Anthony, T., Salas, E., & Driskell, J. E. (1994). Group cohesiveness and quality of decision making: An integration of tests of the groupthink hypothesis. Small Group Research, 25, 189-204.
44.
Mullen, B. , Brown, R., & Smith, C. (1992). Ingroup bias as a function of salience, relevance, and status: An integration. European Journal of Social Psychology, 22, 103-122.
45.
Mullen, B. , Bryant, B., & Driskell, J. E. (1997). The presence of others and arousal: An integration. Group Dynamics, 1, 52-64.
46.
Mullen, B. , & Copper, C. (1994). The relation between group cohesiveness and performance: An integration. Psychological Bulletin, 15, 210-227.
47.
Mullen, B. , Copper, C., & Driskell, J. E. (1990). Jaywalking as a function of model behavior. Personality and Social Psychology Bulletin, 16, 320-330.
48.
Mullen, B. , Driskell, J. E., & Salas, E. (1998). Meta-analysis and the study of group dynamics. Group Dynamics, 2, 1-17.
49.
Mullen, B. , & Hu, L. (1988). Social projection as a function of cognitive mechanisms: Two meta-analytic integrations. British Journal of Social Psychology, 27, 333-356.
50.
Mullen, B. , & Hu, L. (1989). Perceptions of ingroup and outgroup variability: A meta-analytic integration. Basic and Applied Social Psychology, 10, 233-252.
51.
Mullen, B. , Johnson, C., & Salas, E. (1991). Productivity loss in brainstorming groups: A meta-analytic integration. Basic and Applied Social Psychology, 12, 3-23.
52.
Mullen, B. , & Rosenthal, R. (1985). BASIC meta-analysis. Hillsdale, NJ: Lawrence Erlbaum.
53.
Mullen, B. , Salas, E., & Driskell, J. E. (1989). Salience, motivation, and artifact as contributions to the relation between participation rate and leadership. Journal of Experimental Social Psychology, 25, 545-559.
54.
Myers, D. G. (1999). Social psychology. Boston: McGraw-Hill.
55.
Narby, D. J. , Cutler, B. L., & Moran, G. (1993). A meta-analysis of the association between authoritarianism and jurors’ perceptions of defendant culpability. Journal of Applied Psychology, 78, 34-42.
56.
Pearson, K. (1933). On a method of determining whether a sample size n supposed to have been drawn from a parent population having a known probability integral has probably been drawn at random. Biometrika, 25, 379-410.
57.
Pogue, J. M. , & Yusuf, S. (1997). Cumulating evidence from randomized trials: Sequential monitoring boundaries for cumulative meta-analysis. Controlled Clinical Trials, 18, 580-593.
58.
Rosenthal, R. (1979). The “file drawer problem” and tolerance for null results. Psychological Bulletin, 86, 638-641.
59.
Rosenthal, R. (1991). Meta-analytic procedures for social research. Newbury Park, CA: Sage.
60.
Rosenthal, R. , & Rubin, D. B. (1978). Interpersonal expectancy effects: The first 345 studies. Behavioral and Brian Sciences, 3, 410-415.
61.
Rosnow, R. L. , & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276-1284.
62.
Rossi, J. S. (1985). Comparison of physical and behavioral science: The roles of theory, measurement, and effect size. Paper presented at the annual meeting of the Eastern Psychological Association, Boston.
63.
Schlenker, B. R. (1974). Social psychology and science. Journal of Personality and Social Psychology, 29, 1-15.
64.
Schmidt, F. L. (1992). What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology, American Psychologist, 47, 1173-1181.
65.
Secord, P. F. (1976). Transhistorical and transcultural theory. Personality and Social Psychology Bulletin, 2, 418-420.
66.
Sheeran, P. , & Orbell, S. (1998). Do intentions predict condom use? Meta-analysis and examination of six moderator variables. British Journal of Social Psychology, 37, 231-250.
67.
Sohn, D. (1997). Questions for meta-analysis. Psychological Reports, 81, 3-15.
68.
Steblay, N. M. (1987). Helping behavior in rural and urban environments: A meta-analysis. Psychological Bulletin, 102, 346-356.
69.
Steiner, I. (1972). Group process and productivity. San Diego, CA: Academic Press.
70.
Stouffer, S. A. (1949). The American soldier: Vol. 1. Adjustment during army life. Princeton, NJ: Princeton University Press.
71.
Summers, I. , Coffelt, T., & Horton, R. E. (1988). Work-group cohesion. Psychological Reports, 63, 627-636.
72.
Sweeney, P. D. , Anderson, K., & Bailey, S. (1986). Attributional styles and depression: A meta-analytic review. Journal of Personality and Social Psychology, 50, 974-991.
73.
Tedeschi, J. T. , Gaes, G. G., Riordan, C., & Quigley-Fernandez, B. (1981). Social psychology and cumulative knowledge. Personality and Social Psychology Bulletin, 7, 161-172.
74.
Thorndike, R. L. (1933). The effect of the interval between test and retest on the constancy of the IQ. Journal of Educational Psychology, 24, 543-549.
75.
Tryon, W. W. (1982). A simplified times series analysis for evaluating treatment interventions. Journal of Applied Behavioral Analysis, 15, 423-429.
76.
Tufte, E. R. (1990). Envisioning information. Cheshire, CT: Graphics Press.
77.
Tufte, E. R. (1997). Graphical explanations. Cheshire, CT: Graphics Press.
78.
Tukey, J. W. (1969). Analyzing data: Sanctification or detective work?American Psychologist, 24, 83-91.
79.
Tukey, J. W. (1990). Data based graphics: Visual display in the decades to come. Statistical Science, 5, 327-329.
80.
van Ijzendoorn, M. H. , & Bus, A. G. (1994). Meta-analytic confirmation of the non-word reading deficit in dyslexia. Reading Research Quarterly, 30, 3-12.
81.
Vecchio, R. P. (1983). Assessing the validity of Fiedler’s contingency model of leadership effectiveness: A closer look at Strube and Garcia. Psychological Bulletin, 93, 404-408.
82.
Wainer, H. (1997). Visual revelations: Graphical tales of fate and deception from Napoleon Bonaparte to Ross Perot. New York: Copernicus.
83.
Wallach, M. A. , Kogan, N., & Bem, D. J. (1964). Diffusion of responsibility and level of risk taking in groups. Journal of Abnormal and Social Psychology, 68, 263-274.
84.
Webster’s New World Dictionary. (1984). New York: Warner Books.
85.
Wilson, E. J. , & Sherrell, D. L. (1993). Source effects in communication and persuasion research: A meta-analysis. Journal of the Academy of Marketing Science, 21, 101-112.
86.
Wolff, M. (1977). Social psychology as history: Advancing the problem. Personality and Social Psychology Bulletin, 3, 211-212.
