In May 2004, 12 experts in evaluating large-scale health communication programs came to the Centers for Disease Control and Prevention to participate in an electronic focus group discussion. They offered advice on topics ranging from the role of logic models to the best strategies for controlling for self-selection bias in surveys regarding outcomes of exposure to mass media health messages. The experts also highlighted health communication evaluation topics that have received too little scientific attention. Finally, they made strategic policy recommendations. Use of the state-of-the-art evaluation methods that they recommended could improve the communication of factual and persuasive health messages and help to guard the public health of the nation. Their advice may also advance evaluation practice in other substantive areas, especially where it is difficult or impossible to implement randomized designs.
Abadie, A., Diamond, A., & Hainmueller, J. (2007). Synthetic control methods for comparative case studies: Estimating the effect of California's tobacco control program (Working Paper No. 12831). Cambridge, MA : National Bureau of Economic Research. Retrieved April 30, 2007, from http://www.nber.org/papers/w12831.pdf
2.
Ajzen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice Hall.
3.
Allison, P.D. (2000). Multiple imputation for missing data: A cautionary tale. Sociological Methods & Research, 28(3), 301-309.
4.
Andreasen, A.R. (1985, May—June). “Backward” marketing research. Harvard Business Review, pp. 176-182.
5.
Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs, NJ: Prentice Hall.
6.
Bollen, K.A., Guilkey, D.K., & Mroz, T.A. (1995). Binary outcomes and endogenous explanatory variables: Tests and solutions with an application to the demand for contraceptive use in Tunisia. Demography, 32(1), 111-131.
7.
Boulay, M., & Valente, T.W. (2005). The selection of family planning discussion partners in Nepal. Journal of Health Communication, 10(6), 519-536.
8.
Brehm, J.W. (1966). A theory of psychological reactance. New York: Holt, Rinehart & Winston.
9.
Bronfrenbrenner, U. (1979). The ecology of human development. Cambridge, MA: Harvard University Press.
10.
Campbell, D.T. (1969). Reforms as experiments. American Psychologist, 24, 409-429.
11.
Centers for Disease Control and Prevention. ( 1999). Framework for program evaluation in public health. Mortality and Morbidity Weekly Review, 48(RR-11), 1-40.
12.
Cole, G.E. (1999). Advancing the development and application of theory-based evaluation in the practice of public health. American Journal of Evaluation, 20(3), 453-470.
13.
DesJardines, S., & McCall, B. (2006). The impact of the Gates Millennium Scholars Program on selected outcomes of low-income minority students: A regression discontinuity analysis. Retrieved April 30, 2007, from http://www.ers.princeton.edu/McCall.pdf
14.
Des Jarlais, D.C., Lyles, C., & Crepaz, N. (2004). Improving the reporting quality of nonrandomized evaluations of behavioral and public health intervention: The TREND statement . American Journal of Public Health, 94(3), 361-366.
15.
DiClemente, R. J., Crosby, R. A., & Kegler, M. C. (Eds.). (2002). Emerging theories in health promotion practice and research. San Francisco: Jossey-Bass .
16.
Figueroa, M.E., Bertrand, J.T., & Kincaid, D.L. (2002). Evaluating the impact of communication programs . Summary of an expert meeting organized by the MEASURE Evaluation Project and the Population Communication Services Project, Elkridge, MD, August 2001. Retrieved May 8, 2007, from www.cpc.unc.edu/measure/publications/pdf/ws-02-09.pdf
17.
Fortmann, S.P., Taylor, C.B., Flora, J.A., & Jatulis, D.E. (1993). Changes in adult cigarette smoking prevalence after 5 years of community health education: The Stanford Five-City Project . American Journal of Epidemiology, 137(1), 82-96.
18.
Fridinger, F., & Kirby, S. (2002). The “Doublemint” factor: Issues and challenges in marketing nutrition and physical activity behaviors in 1 program . Social Marketing Quarterly, 8(4), 1-13.
19.
Glanz, K., Rimer, B., & Lewis, F.M. (2002). Health behavior and health education: Theory, research, and practice (3rd ed.). San Francisco: Jossey-Bass.
20.
Green, L.W. (2006). Public health asks of systems science: To advance our evidence based practice, can you help us get more practice-based evidence?American Journal of Public Health, 96, 406-409.
Hahn, J., Todd, P., & Van der Klaaw, W. (2001). Identification and estimation of treatment effects with a regression-discontinuity design. Econometrica: Notes and Comments, 69(1), 201-209.
23.
Hedeker, D., & Gibbons, R.D. (1997). Application of random-effects pattern-mixture models for missing data in longitudinal studies. Psychological Methods, 2(1), 64-78.
Henry, G.T. (2003). Beyond use: Understanding evaluation's influence on attitudes and actions. American Journal of Evaluation, 24, 293-314.
26.
Huhman, M., Heitzler, C., & Wong, F. (2004). The VERB campaign logic model: A tool for planning and evaluation. PreventingChronic Disease, 1(3), A11. Retrieved May 8, 2007, from http://www.cdc.gov/pcd/issues/2004/jul/04_0033.htm
27.
Institute of Medicine. (2002). Speaking of health: Assessing health communication strategies for diverse populations. Washington, DC: National Academy Press.
28.
Kennedy, M. G., & Abbatangelo, J. (Eds.). (2005). Guidance for evaluating mass communication health initiatives: Summary of an expert panel discussion . Atlanta, GA: Centers for Disease Control and Prevention. Retrieved May 11, 2007, from http://www.healthcommunication.net/Evaluating_Mass_Comm.pdf
29.
Kincaid, D.L. (2000). Mass media, ideation, and behavior: A longitudinal analysis of contraceptive change in the Philippines. Communication Research, 27, 723-763.
30.
Kincaid, D.L. (2004). From innovation to social norm: Bounded normative influence. Journal of Health Communication, 9(Suppl. 1), 37-57.
31.
Little, R.J.A., & Rubin, D. (1987). Statistical analysis with missing data. New York: John Wiley.
32.
Luke, D.A., & Harris, J.K. (2007). Network analysis in public health: History, methods and applications. Annual Review ofPublic Health, 28, 69-93.
33.
McElroy, L., Bibeau, D., Steckler, A., & Glanz, K. (1988). An ecological perspective on health promotion programs. Health Education Quarterly, 15, 351-377.
34.
McGuire, W. (1981). Theoretical foundations of campaigns. In R. Rice & W. Paisley (Eds.), Public communication campaigns (pp. 41-70). Thousand Oaks, CA: Sage.
35.
Mroz, T.A. (1997). Discrete factor approximations in simultaneous equation models: Estimating the impact of a dummy endogenous variable on a continuous outcome. Retrieved May 11, 2007, from http://www.unc.edu/~mroz/pagesdata/papers/discretefactorapprox/dfmnew5.htm
36.
National Cancer Institute. (2003). Theories at a glance. Retrieved April 30, 2007, from http://www.cancer.gov/cancerinformation/theory-at-a-glance
37.
Oud, J.H.L. (2002). Continuous time modeling of the cross-lagged panel design. Kwantitatieve Methoden, 69, 1-26.
38.
Palmgreen, P., Donohew, L., Lorch, E.P., Hoyle, R.H., & Stephenson, M.T. (2002). Television campaigns and sensation seeking targeting of adolescent marijuana use: A controlled time series approach. In R. C. Hornik (Ed.), Public health communication: Evidence for behavior change (pp. 35-56). Mahwah, NJ: Lawrence Erlbaum.
39.
Petty, R.E., & Cacioppo, J.T. (1979). Elaboration likelihood model: Issue involvement can increase or decrease persuasion by enhancing message-relevant cognitive responses. Journal of Personality and Social Psychology, 37, 1915-1926.
40.
Prochaska, J.O., DiClemente, C.C., & Norcross, J.C. (1992). In search of how people change. Applications to addictive behaviors. American Psychologist , 47, 1102-1114.
41.
Rice, R.E., & Atkin, C.K. (2001). Public communication campaigns (3rd ed.). Thousand Oaks, CA: Sage.
42.
Rogers, E.M. (1995). Diffusion of innovations (4th ed.). New York: Free Press.
43.
Rosen, L., Manor, O., Englehard, D., & Zucker, D. (2006). In defense of the randomized controlled trial for health promotion research. American Journal of Public Health, 96, 1181-1196.
44.
Roth, P. (1994). Missing data: A conceptual review for applied psychologist. Personnel Psychology, 47, 537-560.
45.
Siegel, M. (2002). The effectiveness of state-level tobacco control interventions: A review of program implementation and behavioral outcomes . Annual Review of Public Health, 23, 45-71.
46.
Slater, M.D., & Kelly, K.J. (2002). Testing alternative explanations for exposure effects in media campaigns: The case of a community-based, in-school media drug prevention project. Communication Research, 29, 367-389.
47.
Thistlethwaite, D.L., & Campbell, D.T. (1960). Regression-discontinuity analysis: An alternative to the ex-post facto experiment. Educational Psychology, 51(6), 309-317.
48.
Unaids. (2000). National AIDS programmes: A guide to indicators for monitoring and evaluating national HIV/AIDS prevention programmes for young people. Retrieved May 8, 2007 , from http://data.unaids.org/Publications/IRC-pub06/JC949-NAP-YoungPeople_en.pdf
49.
Valente, T.W. (2002). Evaluating health promotion programs. New York: Oxford University Press.
50.
Valente, T.W., Foreman, R.K., Junge, B., & Valhov, D. (1998). Satellite exchange in the Baltimore needle exchange program. Public Health Reports, 113(S1), 91-96.
51.
Valente, T.W., & Saba, W.P. (1998). Mass media and interpersonal influence in the Bolivia National Reproductive Health Campaign. Communication Research, 25, 96-124.
52.
Valente, T.W., & Saba, W.P. (2001). Campaign exposure and personal communication as factors in contraceptive use in Bolivia. Journal of Health Communication, 6(4), 303-322.
53.
Wakefield, M., & Chaloupka, F. (2000). Effectiveness of comprehensive tobacco control programmes in reducing teenage smoking in the USA. Tobacco Control, 9, 177-186.
Weiss, C.H. (2005). An alternate route to policy influence. American Journal of Evaluation, 26, 12-30.
56.
Willis, G.B. (1999, August). Cognitive interviewing: A “how to” guide. Short course presented at the annual meeting of the American Statistical Association, Baltimore. Retrieved May 8, 2007, from http://appliedresearch.cancer.gov/areas/cognitive/interview.pdf
57.
Wothke, W. (1998). Longitudinal and multi-group modeling with missing data. In T. D. Little, K. U. Schnabel, & J. Baumert (Eds.), Modeling longitudinal and multiple group data: Practical issues, applied approaches, and specific examples (pp. 1-24). Mahwah, NJ: Lawrence Erlbaum.
58.
Yanovitzky, I., Zanutto, E., & Hornik, R. (2005). Estimating causal effects of public health education campaigns using propensity score methodology. Evaluation and Program Planning, 28, 209-220.
59.
Yin, R. ( 2003). Case study research: Design and methods. Thousand Oaks, CA: Sage.
60.
Zaza, S., Wright-de Aguero, L., Briss, P.A., Turman, B.I., Hopkins, D.P., Hennessy, M.H., et al. (2000). Data collection instrument and procedure for systematic reviews in The Guide to Community Preventive Services. American Journal of Preventive Medicine, 18(1S), 44-74.
