A Randomized Trial of the Impact of Survey Design Characteristics on Response Rates Among Nursing Home Providers

Abstract

An experiment was conducted to maximize participation of both the Director of Nursing (DoN) and the Administrator (ADMIN) in long-term care facilities. Providers in each of the 224 randomly selected facilities were randomly assigned to 1 of 16 conditions based on the combination of data collection mode (web vs. mail), questionnaire length (short vs. long), and incentive structure. Incentive structures were determined by amount compensated if the individual completed and an additional amount per individual if the pair completed (a) $30 individual/$5 pair/$35 total; (b) $10 individual/$25 pair/$35 total; (c) $30 individual/$20 pair/$50 total; and (d) $10 individual/$40 pair/$50 total. Overall, 47.4% of eligible respondents participated; both respondents participated in 29.3% of facilities. In multivariable analyses, there were no differences in the likelihood of both respondents participating by mode, questionnaire length, or incentive structure. Providing incentives contingent on participation by both providers of a facility was an ineffective strategy for significantly increasing response rates.

Keywords

surveys response rate incentives nursing home

Get full access to this article

View all access options for this article.

References

Ahern

N. R.

(2005). Using the Internet to conduct research. Nurse Research, 13, 55–70.

Asch

D. A.

Jedrziewski

M. K.

Christakis

N. A.

(1997). Response rates to mail surveys published in medical journals. Journal of Clinical Epidemiology, 50, 1129–1136.

Berk

M. L.

Edwards

W. S.

Gay

N. L.

(1993). The use of a prepaid incentive to convert nonresponders on a survey of physicians. Evaluation & the Health Professions, 16, 239–245.

Boyce

B. F.

Bob

Levenson

S. A.

(2003). The preliminary impact of Maryland’s medical director and attending physician regulations. Journal of American Medical Director Association, 4, 157–163.

Braithwaite

Emery

De Lusignan

Sutton

(2003). Using the Internet to conduct surveys of health professionals: a valid alternative?. Family Practice, 20, 545–551.

Camunas

Alward

R. R.

Vecchione

(1990). Survey response rates to a professional association mail questionnaire. Journal of the New York State Nurses Association, 21, 7–9.

Carley-Baxter

L. R.

Hill

C. A.

Roe

D. J.

Twiddy

S. E.

Baxter

R. K.

Ruppenkamp

(2009). Does response rate matter? Journal editors use of survey quality measures in manuscript publication decisions. Survey Practice, November. Available at: http://surveypractice.org/2009/10/17/editors-decisions/

Colon-Emeric

C. S.

Casebeer

Saag

Allison

Levine

Suh

T. T.

… Lyles

K. W.

(2005). Barriers to providing osteoporosis care in skilled nursing facilities: perceptions of medical directors and directors of nursing. Journal of the American Medical Directors Association, 6, S61–S66.

Comondore

V. R.

Devereaux

P. J.

Zhou

Stone

S. B.

Busse

J. W.

Ravindran

N. C.

Guyatt

G. H.

(2009). Quality of care in for-profit and not-for-profit nursing homes: systematic review and meta-analysis. British Medical Journal, 339, b2732

10.

Couper

M. P.

Miller

P. V.

(2008). Web survey methods. Public Opinion Quarterly, 72, 831–835.

11.

Cull

W. L.

O’Connor

K. G.

Sharp

Tang

S. F.

(2005). Response rates and response bias for 50 surveys of pediatricians. Health Service Research, 40, 213–226.

12.

Cummings

S. M.

Savitz

L. A.

Konrad

T. R.

(2001). Reported response rates to mailed physician questionnaires. Health Service Research, 35, 1347–1355.

13.

Dal Bo

Foster

Putterman

(2010). Institutions and behavior: Experimental evidence on the effects of democracy. American Economic Review, 100, 2205–2229

14.

Daly

J. M.

Jogerst

G. J.

(2005). Association of knowledge of adult protective services legislation with rates of reporting of abuse in Iowa nursing homes. Journal of American Medical Directors Association, 6, 113–120.

15.

Fennell

M. L.

Feng

Clark

M. A.

Mor

(2010). Elderly Hispanics more likely to reside in poor-quality nursing homes. Health Affiliation (Millwood), 29, 65–73.

16.

Fisher

Bosley

Goldenberg

Mockovak

Tucker

(2003). A qualitative study of nonresponse factors affecting BLS establishment surveys: Results. Proceedings of the Survey Research Methods Section, American Statistical Association. Retrieved September 27, 2010, from http://www.bls.gov/osmr/pdf/st030230.pdf

17.

Flanigan

T. S.

McFarlane

Cook

(2008). Conducting survey research among physicians and other medical professionals—A review of current literature. Retrieved September 27, 2010, from http://www.amstat.org/sections/srms/Proceedings/y2008/Files/flanigan.pdf

18.

Groves

R. M.

Peytcheva

(2008). The impact of nonresponse rates on nonresponse bias. Public Opinion Quarterly, 72, 167–189.

19.

Guise

Chambers

Valimaki

Makkonen

(2010). A mixed-mode approach to data collection: Combining web and paper questionnaires to examine nurses' attitudes to mental illness. Journal of Advanced Nursing, 66, 1623–1632.

20.

Handler

S. M.

Perera

Olshansky

E. F.

Studenski

S. A.

Nace

D. A.

Fridsma

D. B.

… Hanlon

J. T.

(2007). Identifying modifiable barriers to medication error reporting in the nursing home setting. Journal of the American Medical Directors Association, 8, 568–574.

21.

Harrington

(2001). Regulating nursing homes: Residential nursing facilities in the United States. British Medical Journal, 323, 507–510.

22.

Hill

C. A.

Fahrney

Wheeless

S. C.

Carson

C. P.

(2006). Survey response inducements for registered nurses. Western Journal of Nursing Research, 28, 322–334.

23.

James

K.M.

Ziegenfuss

J.Y.

Tilburt

J.C.

Harris

A.M.

Beebe

T.J.

(2011). Getting physicians to respond: The impact of incentive type and timing on physician survey response rates. HSR, 46, 232–242.

24.

Jepson

Asch

D. A.

Hershey

J. C.

Ubel

P. A.

(2005). In a mailed physician survey, questionnaire length had a threshold effect on response rate. Journal of Clinical Epidemiology, 58, 103–105.

25.

Jogerst

G. J.

Daly

J. M.

Dawson

J. D.

Peek-Asa

Schmuch

(2006). Iowa nursing home characteristics associated with reported abuse. Journal of the American Medical Directors Association, 7, 203–207.

26.

Kagel

J. H.

Roth

A. E.

(1995). In The handbook of experimental economics Princeton, NJ: Princeton University Press.

27.

Kellerman

S. E.

Herold

(2001). Physician response to surveys. A review of the literature. American Journal of Preventive Medicine, 20, 61–67.

28.

Leece

Bhandari

Sprague

Swiontkowski

M. F.

Schemitsch

E. H.

Tornetta

Guyatt

G. H.

(2004). Internet versus mailed questionnaires: A randomized comparison (2). Journal of Medical Internet Research, 6, e30

29.

Odon

Price

J. H.

(1999). Effects of a small monetary incentive and follow-up mailings on return rates of a survey to nurse practitioners. Psychological Reports, 85, 1154–1156.

30.

Poon

E. G.

Jha

A. K.

Christino

Honour

M. M.

Fernandopulle

Middleton

Kaushal

(2006). Assessing the level of healthcare information technology adoption in the United States: A snapshot. BMC Medical Informatics and Decision Making, 6, 1

31.

Resnick

H. E.

Manard

Stone

R. I.

Castle

N. G.

(2009). Tenure, certification, and education of nursing home administrators, medical directors, and directors of nursing in for-profit and not-for-profit nursing homes: United States 2004. Journal of the American Medical Directors Association, 10, 423–430.

32.

Shih

T. H.

Fan

(2008). Comparing response rates from web and mail surveys: A meta-analysis. Field Methods, 20, 249–271.

33.

Shirts

B. H.

Perera

Hanlon

J. T.

Roumani

Y. F.

Studenski

S. A.

Nace

D. A.

Handler

S. M.

(2009). Provider management of and satisfaction with laboratory testing in the nursing home setting: Results of a national internet-based survey. Journal of the American Medical Directors Association, 10, 161–166.

34.

Singer

Bossarte

R. M.

(2006). Incentives for survey participation when are they “coercive”?. American Journal of Preventive Medicine, 31, 411–418.

35.

Smith

D. B.

Feng

Fennell

M. L.

Zinn

J. S.

Mor

(2007). Separate and unequal: Racial segregation and disparities in quality across U.S. nursing homes. Health Affiliation (Millwood), 26, 1448–1458.

36.

The American Association for Public Opinion Research. (2008). In Standard definitions: Final dispositions of case codes and outcome rates for surveys 5th ed Lenexa, KS: Author.

37.

Thran

S. L.

Hixon

J. S.

(2000). Physician surveys: Recent difficulties and proposed solutions. Paper presented at the ASA 2000 Proceedings

38.

Ulrich

C. M.

Danis

Koziol

Garrett-Mayer

Hubbard

Grady

(2005). Does it pay to pay? A randomized trial of prepaid financial incentives and lottery incentives in surveys of nonphysician healthcare professionals. Nursing Research, 54, 178–183.

39.

VanGeest

J. B.

Johnson

T. P.

Welch

V. L.

(2007). Methodologies for improving response rates in surveys of physicians: A systematic review. Evaluation & the Health Professions, 30, 303–321.

40.

VanGeest

J. B.

Wynia

M. K.

Cummins

D. S.

Wilson

I. B.

(2001). Effects of different monetary incentives on the return rate of a national mail survey of physicians. Medical Care, 39, 197–201.

41.

Ward

N. S.

Teno

J. M.

Curtis

J. R.

Rubenfeld

G. D.

Levy

M. M.

(2008). Perceptions of cost constraints, resource limitations, and rationing in United States intensive care units: Results of a national survey. Critical Care Medicine, 36, 471–476.

42.

Wyatt

J. C.

(2000). When to use web-based surveys. Journal of American Medical Information Association, 7, 426–429.

43.

Young

Inamdar

Barhydt

N. R.

Colello

A. D.

Hannan

E. L.

(2010). Preventable hospitalization among nursing home residents: Varying views between medical directors and directors of nursing regarding determinants. Journal of Aging Health, 22, 169–182