Using illustrations from HIV prevention research, the current article advocates approaching meta-analysis as a theory-testing scientific method rather than as merely a set of rules for quantitative analysis. Like other scientific methods, meta-analysis has central concerns with internal, external, and construct validity. The focus of a meta-analysis should only rarely be merely describing the effects of health promotion, but rather should be on understanding and explaining phenomena and the processes underlying them. The methodological decisions meta-analysts make in conducting reviews should be guided by a consideration of the underlying goals of the review (e.g., simply effect size estimation or, preferably, theory testing). From the advocated perspective that a health behavior meta-analyst should test theory, the authors present a number of issues to be considered during the conduct of meta-analyses.
Get full access to this article
View all access options for this article.
References
1.
Ajzen, I.
, & Fishbein, M. (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice Hall.
2.
Albarracin, D.
, Johnson, B. T., Fishbein, M., & Muellerleile, P.A. (2001). Theories of reasoned action and planned behavior as models of condom use: A meta-analysis. Psychological Bulletin, 127, 142–161.
3.
Bandura, A.
(1997). Self-efficacy: The exercise of control. New York: Freeman.
4.
Baron, R. M.
, & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182.
5.
Bukoski, W. J.
(Ed.). (1997). Meta-analysis of drug abuse prevention programs (NIDA Research Monograph 170). Rockville, MD: NIDA.
6.
Carey, M. P.
, & Lewis, B. P. (1999). Motivational strategies can augment HIV-risk reduction programs. AIDS and Behavior, 3, 269–276.
7.
Carlson, M.
, & Miller, N. (1987). Explanation of the relation between negative mood and helping. Psychological Bulletin, 102, 91–108.
8.
Cohen, J.
(1990). Things I have learned (so far). American Psychologist, 45, 1304–1312.
9.
Cohen, J.
(1994). The earth is round (p > .05). American Psychologist, 49, 997–1003.
10.
Cook, T. D.
, & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues for field settings. Boston: Houghton Mifflin.
11.
Deci, E. L.
, & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum.
12.
Eagly, A. H.
, & Wood, W. (1994). Using research syntheses to plan future research. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 485–500). New York: Russell Sage.
13.
Fishbein, M.
, & Middlestadt, S. E. (1989). Using the Theory of Reasoned Action as a framework for understanding and changing AIDS-related behaviors. In V. M. Mays, G. W. Albee, & S. F. Schneider (Eds.), Primary prevention of AIDS. Psychological approaches. Primary prevention of psychopathology (Vol. 13, pp. 93–110). Newbury Park, CA: Sage.
14.
Fisher, J. D.
, & Fisher, W. A. (1992). Changing AIDS-risk behavior. Psychological Bulletin, 111, 455–474.
15.
Fisher, W. A.
, & Fisher, J. D. (1993). A general social psychological model for changing AIDS risk behavior. In J. B. Pryor & G. D. Reeder (Eds.), The social psychology of HIV infection (pp. 127–153). Hillsdale, NJ: Erlbaum.
16.
Gerrard, M.
, Gibbons, F. X., Warner, T. D., & Smith, G. E. (1993). Perceived vulnerability to HIV infection and AIDS preventive behavior: A critical review of the evidence. In J. B. Pryor & G. D. Reeder (Eds.), The social psychology of HIV infection (pp. 59–84). Hillsdale, NJ: Erlbaum.
17.
Glass, G. V.
(1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3–8.
18.
Gollub, E. L.
, Rey, D., Obadia, Y., & Moatti, J. P. (1998). Gender differences in risk behaviors among HIV+ persons with an IDU history. The link between partner characteristics and women’s higher drug-sex risks. The Manif 2000 Study Group. Sexually Transmitted Diseases, 25,483–488.
19.
Greenwald, A. G.
, Pratkanis, A. R., Leippe, M. R., & Baumgardner, M. H. (1986). Under what conditions does theory obstruct research progress?Psychological Review, 93, 216–229.
20.
Helweg-Larson, M.
, & Collins, B. E. (1997). A social psychological perspective on the role of knowledge about AIDS in AIDS prevention. Current Directions, 6, 23–26.
21.
Holmbeck, G. N.
(1997). Toward terminological, conceptual, and statistical clarity in the study of mediators and moderators: Examples from the child-clinical and pediatric psychology literatures. Journal of Consulting & Clinical Psychology, 65, 599–610.
22.
Johnson, B. T., Carey, M. P., Marsh, K. L., & Levin, K. D. (2001). Meta-analysis of interventions to reduce HIV risk in adolescents. Unpublished manuscript.
23.
Johnson, B. T.
, Carey, M. P., & Muellerleile, P. A. (1997, February 5). Large trials vs. meta-analysis of smaller trials [Letter to the editor]. Journal of the American Medical Association, 277, 377-377.
24.
Johnson, B. T.
, & Eagly, A. H. (2000). Quantitative synthesis of social psychological research. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social psychology (pp. 496–528). Cambridge, UK: Cambridge University Press.
25.
Johnson, B. T.
, Marsh, K. L., Carey, M. P., Levin, K. D., & Lewis, B. P. (2000, June). Prevention of sexually transmitted HIV: A meta-analytic review. Paper presented at the International Conference on Psychology, Haifa, Israel.
26.
Johnson, B. T.
, & Nichols, D. R. (1998). Social psychologists’ expertise in the public interest: Civilian morale research during World War II. Journal of Social Issues, 54, 53–77.
27.
Johnson, B. T.
, & Turco, R. M. (1992). The value of goodness-of-fit indices in meta-analysis: A comment on Hall and Rosenthal. Communication Monographs, 59, 388–396.
28.
Kalichman, S. C.
(1998). Preventing AIDS: A sourcebook for behavioral interventions. Mahwah, NJ: Erlbaum.
29.
Kalichman, S. C.
, Carey, M. P., & Johnson, B. T. (1996). Prevention of sexually transmitted HIV infection: A meta-analytic review of the behavioral outcome literature. Annals of Behavioral Medicine, 18, 6–15.
30.
Kamb, M. L.
, Peterman, T. A., & Wolitski, R. J. (2000). Prevention counseling for HIV-negative persons [Letter to the Editor]. American Journal of Public Health, 90, 1152-1152.
31.
Kelly, J. A.
(1995). Changing HIV risk behavior: Practical strategies. New York: Guilford.
32.
Kelly, J. A.
, St. Lawrence, J. S., & Stevenson, L. Y. (1992). Community AIDS/HIV risk reduction: The effects of endorsements by popular people in three cities. American Journal of Public Health, 82, 1483–1489.
33.
Linville, P. W.
, Fischer, G. W., & Fischhoff, B. (1993). AIDS risk perceptions and decision biases. In J. B. Pryor & G. D. Reeder (Eds.), The social psychology of HIV infection (pp. 5–38). Hillsdale, NJ: Erlbaum.
34.
Lipsey, M. W.
, & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48, 1181–1209.
35.
Matt, G. E.
(1997). Drawing generalized causal inferences based on meta-analysis. In W. J. Bukoski (Ed.), Meta-analysis of drug abuse prevention programs (NIDA Research Monograph 170). Rockville, MD: NIDA.
36.
Matt, G. E.
, & Cook, T. D. (1994). Threats to the validity of research syntheses. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 503–520). New York: Russell Sage.
37.
Miller, N.
, & Pollock, V. E. (1994). Meta-analysis and some science-compromising problems of social psychology. In W. R. Shadish & S. Fuller (Eds.), The social psychology of science (pp. 230–261). New York: Guilford.
38.
Miller, N.
, & Pollock, V. E. (1995). Use of meta-analysis for testing theory. Evaluation & the Health Professions, 18, 370–392.
39.
Mook, D. G.
(1983). In defense of external invalidity. American Psychologist, 38, 379–387.
40.
Paul, G. L.
(1967). Strategy of outcome research in psychotherapy. Journal of Counseling Psychology, 31, 109–118.
41.
Rosenstock, I. M.
, Stretcher, V. J., & Becker, M. H. (1988). Social learning theory and the health belief method. Health Education Quarterly, 13, 73–92.
42.
Sheeran, P.
, Abraham, C., & Orbell, S. (1999). Psychosocial correlates of heterosexual condom use: A meta-analysis. Psychological Bulletin, 125, 90–132.
43.
Skinner, E. A.
(1995). Perceived control, motivation, & coping (Vol. 8). Thousand Oaks, CA: Sage.
44.
Smith, M. L.
, & Glass, G. (1977). Meta-analysis of psychotherapy outcomes. American Psychologist, 32, 752–760.
45.
Weinhardt, L. S.
, & Carey, M. P. (2000). Human Immunodeficiency Virus testing and behavior change [Letter to the Editor]. Archives of Internal Medicine, 160, 1538–1539.
46.
Weinhardt, L. S.
, Carey, M. P., Johnson, B. T., & Bickham, N. L. (1999). Effects of HIV counseling and testing on sexual risk behavior: A meta-analytic review of published research, 1985–1997. American Journal of Public Health, 89, 1397–1405.
47.
Weinhardt, L. S.
, Carey, M. P., Johnson, B. T., & Bickham, N. (2000). Effects of HIV Counseling and testing on sexual risk behavior [Letter to the Editor]. American Journal of Public Health, 90, 1152–1153.
48.
Wilson, T. D.
, DePaulo, B. M., Mook, D. G., & Klaaren, K. J. (1993). Scientists’ evaluations of research: The biasing effects of the importance of the topic. Psychological Science, 4, 322–325.
