The Effect of Varying Incentive Amounts on Physician Survey Response

Abstract

A major challenge with surveying physicians is low response. In this article, we present results of an experiment conducted to determine the optimal monetary incentive amount for gaining response from physicians to a short screener survey. Sampled physicians were randomly assigned to three prepaid cash incentive conditions (US$2, US$5, US$10) compared to a control (US$0). This study found using any incentive increased response versus no incentive. The US$10 incentive produced the highest response and was significantly greater than the US$2 incentive group. However, we did not find a statistical difference between the $5 and US$10 incentives or between the US$2 and US$5 incentives. In addition, any incentive amount increased the likelihood of early response compared to no incentive. This study builds on previously mixed results about the effects of various incentive amounts and effect on early survey response. These findings provide practical advice for researchers surveying physicians.

Keywords

physician survey response rate incentive early response

A major challenge with surveying physicians is low response (Asch, Jedrziewski, & Christakis, 1997; Cook, Dickinson, & Eccles, 2009; Cull, O’Connor, Sharp, & Tang, 2005; Flanigan, McFarlane, & Cook, 2008; Kellerman & Herold, 2001). Similar to general population surveys, physician survey response rates have steadily declined over the last half century (Cho, Johnson, & VanGeest, 2013; McLeod, Klabunde, Willis, & Start, 2013). Barriers to physician survey response include their busy schedules, frequent survey requests, fear of responses being used against them, and office policies that deter participation (Cook et al., 2009; Klabunde, Willis, & Casalino, 2013; Moore, Post & Smith, 1999; VanGeest, Beebe, & Johnson, 2015). As response rates decline, efforts to encourage response, such as providing an incentive and design-based strategies, become increasingly important (Cho et al., 2013; Delnevo, Abatemarco, & Steinberg, 2004; Field et al., 2002; McLeod et al., 2013; Thorpe et al., 2009).

Previous research has shown that design-based factors contribute to increasing physician survey response. For example, mail surveys are more effective than online or telephone (Cho et al., 2013; McLeod et al., 2013; Pit, Vo, & Pyakurel, 2014), registered mail is more effective than standard mail (Pit et al., 2014; Thorpe et al., 2009), and including nonrespondent follow-ups increases response (Cho et al., 2013; McLeod et al., 2013). In addition, it is well-established that monetary incentives significantly increase survey response among physicians compared to no incentive (Cho et al., 2013; Field et al., 2002; Thorpe et al., 2009; VanGeest et al., 2015; VanGeest, Johnson, & Welch, 2007) or a nonmonetary incentive (Abdulaziz et al., 2015; Flanigan et al., 2008; Kellerman & Herold, 2001; VanGeest et al., 2007). Additionally, prepaid incentives elicit a significantly higher response from physicians compared to promised incentives post survey completion (Delnevo et al., 2004; Kellerman & Herold, 2001; Pit et al., 2014). In a systematic review of incentive and design-based strategies to improve physician response, monetary incentives were the most effective strategy (Pit et al., 2014). However, results around the ideal monetary incentive amount are mixed.

Several studies do not show meaningful differences in physician response by incentive amount (Kasprzyk, Montano, Lawrence, & Phillips, 2001; VanGeest et al., 2007; VanGeest, Wynia, Cummins, & Wilson, 2001). While other studies found differences between smaller incentive amounts (US$2–$10; Deehan, Templeton, Taylor, Drummond, & Strang, 1997; Halpern, Ubel, Berlin, & Asch, 2002) and between larger incentives (US$20–$50; Keating, Zaslavsky, Goldstein, West, & Ayanian, 2008; McLeod et al., 2013), Turnbull, O’Connor, Lau, Halpern, and Needham (2015) let physicians choose their incentive amount up to US$50 and found that only one third chose any incentive, but those who did take it generally took the maximum amount.

Another important outcome is whether incentives increase earlier response (e.g., mailing a survey back sooner). If responses are received earlier in the field period, fewer costly follow-up activities are needed (Dillman, Smyth, & Christian, 2014). However, findings regarding the effect of incentives on early response are sparse and mixed. In a randomized control trial, Turnbull et al. (2015) found that physicians offered an Amazon gift card up to US$50 responded to a short web survey sooner than physicians not offered an incentive. Similarly, Keating, Zaslavsky, Goldstein, West, and Ayanian (2008) found a higher percentage of physicians responded to the first mailing in the US$50 incentive condition compared to the US$20 incentive. In contrast, VanGeest, Wynia, Cummins, and Wilson (2001) did not find that higher levels of prepaid cash incentives reduced the number of mail and/or telephone interventions required to reach the target response rate in a mail survey of physicians.

In this study, we assess the following: (1) Is any incentive more effective than no incentive in increasing response to a short, mailed screener survey? (2) Does the incentive amount have a differential effect on screener response? (3) Do incentives have an effect on early response?

Method

Study Design

This study was conducted as part of a larger program evaluation about facilitators and barriers for primary care physicians in implementing and achieving meaningful use of their electronic health record (EHR) systems. We sampled 4,638 primary care physicians using the American Medical Association Physician Masterfile and program administrative data. Physicians were mailed a screener to determine whether they had an EHR system and worked in a practice with 10 or fewer providers or worked in a practice with more than 30% of patients with Medicaid or without insurance. We randomized all sampled physicians to four cash incentive amounts (no incentive, US$2, US$5, or US$10) and included it with the short, mailed screener.

The screener asked seven questions about EHR system use at the physician’s practice and the type of practice/setting in which the physician works. The screener indicated that the person most familiar with EHR selection, implementation, and use in the practice should complete the survey. We believe that the physician was most likely the respondent because the screener was addressed to the physician. We expect that the physician would be knowledgeable about their practice’s EHR use and practice characteristics at a summary level. These instructions were more important for the in-depth follow-up survey, which is not the focus of this article. However, we do not know for sure who completed the screener so we refer to the “physician practice” when describing results. Nonrespondents received a second screener mailing and were called to complete the screener over the phone after 1 month. The screener survey response rate was 50%. The authors’ company institutional human subjects committee approved this study.

Variables

Dependent variables

The response rate outcome was defined as response to the screener, either by mail or phone = 1, versus no response = 0. The early response outcome is a continuous variable defined as the number of days between the start of data collection and screener response either by mail or phone.

Independent variables

Incentive inclusion was coded 1 = incentive included or 0 = no incentive. Incentive amount had four categories (no incentive, US$2, US$5, and US$10).

Control variables

We controlled for physician age (continuous), physician gender (male = 0, female = 1), program evaluation participation (yes = 1, no = 0), practice type (group practice, two physician practice, solo practice, government run practice, and no classification), and provider type (family practice, internal medicine, pediatrics/adolescents, OB-GYN, general practice and public health, geriatrics).

Statistical Analysis

We used ordinary least squares and logistic regression analyses to investigate predictors of screener response. Wald tests were used to test for differences between incentive amounts.

Results

The sample is 58.3% male with a mean age of 50.7 years. Sixty-six percent of the sample works in a group practice and 14.6% work in a solo practice. Nearly 40% of physicians work in family practice, 25.9% in internal medicine, and 18.3% in pediatrics/adolescents. About half (49.3%) of the starting sample participates in the broader program under evaluation. Screener response rates by incentive group were as follows: 55.3% for the US$10 incentive group, 51.3% for the US$5 incentive group, 50.8% for the US$2 incentive group, and 44.0% for the no incentive group. Of the 2,306 screener respondents, 49.1% returned the screener by mail and 50.9% completed the screener over the phone during follow-up.

Research Question 1: Is any incentive more effective than no incentive in increasing response to the short mailed screener?

In adjusted logistic regression models (Table 1), receiving a prepaid cash incentive increased the odds of screener response by 41% (odds ratio [OR] = 1.41; 95% CI [1.24, 1.60]) compared to no incentive.

Research Question 2: Does the incentive amount have a differential effect on screener response?

Table 1.

Logistic and OLS Regression Results Predicting Screener Response by Incentive Amount.

Physician/Practice Characteristics	Research Question 1	Research Question 2	Research Question 3
	Screening Questionnaire Response		Number of Days for Screening Questionnaire Response
	N = 4,649 OR [95% CI]	N = 4,638 OR [95% CI]	N = 2,306 β [95% CI]
Received incentive	1.41 [1.24, 1.60]***	—	—
Incentive amount
No incentive	—	Reference	Reference
US$2 incentive	—	1.32 [1.12, 1.55]***	−15.90 [−21.30, −10.51]***
US$5 incentive	—	1.36 [1.16, 1.60]***	−18.71 [−24.10, −13.33]***
US$10 incentive	—	1.60 [1.36, 1.88]***	−19.97 [−25.25, −14.69]***
Program participation	1.59 [1.42, 1.79]***	1.59 [1.42, 1.79]***	−8.85 [−12.74, −4.95]***
Age	1.00 [0.99, 1.01]	1.00 [0.99, 1.01]	−0.26 [−0.48, −0.04]*
Gender (female)	0.85 [0.75, 0.97]*	0.85 [0.75, 0.97]*	3.94 [−0.28, 8.17]
Practice type
Group	Reference	Reference	Reference
Self-employed solo	1.28 [1.08, 1.53]**	1.28 [1.08, 1.53]**	−9.39 [−14.81, −3.96]***
Two physician	1.35 [1.00, 1.83]*	1.36 [1.01, 1.84]*	−0.03 [−9.13, 9.08]
Government run	0.98 [0.44, 2.18]	1.06 [0.47, 2.42]	−0.20 [−26.08, 25.68]
No classification	0.44 [0.37, 0.53]***	0.44 [0.37, 0.53]***	8.62 [1.66, 15.59]*
Provider type
Family practice	Reference	Reference	Reference
Internal medicine	0.74 [0.63, 0.85]***	0.72 [0.62, 0.84]***	4.20 [−0.82, 9.23]
Pediatrics/adolescents	1.05 [0.88, 1.24]	1.05 [0.88, 1.24]	−2.40 [−7.80, 3.00]
OB-GYN	0.99 [0.82, 1.20]	0.98 [0.81, 1.19]	−0.50 [−6.66, 5.66]
General practice and public health	0.57 [0.39, 0.84]**	0.57 [0.39, 0.83]**	−0.81 [−14.36, 12.73]
Geriatrics	0.67 [0.35, 1.25]	0.67 [0.36, 1.27]	22.42 [−0.29, 45.12]*

Note. β = beta; OLS = ordinary least squares; OR = odds ratio; 95% CI = 95% confidence interval.

*p < .05. **p < .01. ***p < .001.

Each incentive amount (US$2, US$5, and US$10) had a statistically significant monotonic increase in the odds of screener response over no incentive; the largest incentive had the largest effect (Table 1). Compared to no incentive, the US$2 incentive increased the odds of screener response by 32% (OR = 1.32; 95% CI [1.12, 1.55]), the US$5 incentive increased the odds of response by 36% (OR =1.36; 95% CI [1.16, 1.60]), and the US$10 incentive increased the odds of response by 60% (OR = 1.60; 95% CI [1.36, 1.88]).

Wald tests among the incentive amounts determine that the effect of the US$10 incentive was significantly greater than the US$2 incentive. However, there was no statistical difference between the US$5 and US$10 incentives or between the US$2 and US$5 incentives.

Research Question 3: Do incentives have an effect on early response?

In Table 1, Research Question 3 presents the relationship between the number of days until screener response and incentive amount. The data collection lasted 178 days (about 6 months). We found a negative, monotonic relationship such that physician practices who received the US$10 incentive responded about 20 days earlier compared to those who received no incentive. Physician practices who received the US$5 incentive responded about 19 days earlier and those who received the US$2 incentive responded about 16 days earlier compared to those who received no incentive. Wald tests among the incentive amounts did not show statistically significant differences.

Discussion

Previous literature on physician survey response indicates that monetary incentives are effective, particularly if they are prepaid (Cho et al., 2013; Delnevo et al., 2004; Field et al., 2002; Flanigan et al., 2008; James, Ziegenfuss, Tilburt, Harris, & Beebe, 2011; Thorpe et al., 2009; VanGeest et al., 2007). This study also found that prepaid cash incentives significantly increased physician practice response compared to no incentive.

This study contributes to the knowledge gap about the optimal incentive amount to increase physician survey response. While several studies did not show meaningful differences in physician response by incentive amount (Kasprzyk et al., 2001; VanGeest et al., 2001, 2007), other studies did find differences (Deehan et al., 1997; Halpern et al., 2002; Keating et al., 2008; McLeod et al., 2013). This study compared no incentive, US$2, US$5, and US$10 prepaid cash incentives and found mixed results; the US$10 incentive was more effective than $2, but no difference was found between the US$5 and US$10 incentives or between the US$2 and US$5 incentives.

This study also contributes to an area of research needing more attention: the effect of incentives on early response. By encouraging earlier response, the prepaid incentive may ultimately decrease the need for costly reminder mailings or phone calls (Dillman et al., 2014). This study found incentives were effective in obtaining earlier responses: use of a US$10 incentive produced responses about 20 days earlier, US$5 incentive about 19 days earlier, and US$2 incentive about 16 days earlier compared to no incentive. Our finding is similar to Turnbull et al. (2015) who also found physicians completed a short web survey sooner if offered any monetary incentive versus no incentive. Although we did not find significant differences between incentive amounts, other research such as Keating et al. (2008) found a higher percentage of physicians responded to the first mailing in the larger incentive condition compared to the smaller incentive condition. While incentives may encourage earlier response and reduce the sample size of follow-ups, additional contacts may still be needed to get the desired response rate (Cho et al., 2013; McLeod et al., 2013; VanGeest et al., 2001).

Several characteristics of this study are worth noting when comparing to other research. This study included a short screener survey, and shorter questionnaires have been shown to be associated with higher response rates (Edwards et al., 2002; Flanigan et al., 2008; Jepson, Asch, Hershey, & Ubel, 2005; VanGeest et al., 2007). Half of our sample was participating in the program about EHR adoption under evaluation and therefore may have been primed to respond to this screener asking about EHR use. We did control for program participation and found program participants were more likely to respond than nonparticipants. Additionally, we surveyed primary care physician practices, so results may not apply to specialists. While we sent the screener to the physician and believe they could answer the questions, it is possible the physician delegated this task to someone else in their practice. The effect of incentives on response could vary between physicians and other practice staff.

Conclusion

Based on the findings from this study, small prepaid cash incentives can be used to increase response to short screener surveys of primary care physician practices. All incentives resulted in a higher response than no incentive, with the US$10 incentive achieving the highest response rate. When choosing an incentive amount, the US$2 incentive is the most cost effective and still increases response rate compared to no incentive. However, to get the highest response rate possible, the US$10 incentive is best if the budget can support it. Furthermore, all prepaid incentive amounts increased the likelihood of early response, which can have cost benefits by reducing the sample size for nonresponse follow-up efforts. In future research, we suggest studying whether the cost savings from early responses outweigh incentive costs. Other future research investigating the interactive effects of survey length and incentive amounts would contribute to understanding the factors most likely to increase response to surveys of physicians.

Footnotes

Authors’ Note

The content is solely the responsibility of the authors.

Acknowledgments

The author would like to acknowledge Dr. Grace Wang and Dr. Steve Garfinkel for their review and improvements to earlier drafts.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: These data were collected under a project funded by The Office of the National Coordinator for Health Information Technology.

ORCID iD

HarmoniJoie Noel

References

Abdulaziz

Brehaut

Taljaard

Émond

Sirois

M. J.

Lee

J. S.

… Perry

J. J.

(2015). National survey of physicians to determine the effect of unconditional incentives on response rates of physician postal surveys. BMJ Open, 5, e007166. doi:10.1136/bmjopen-2014-007166

Asch

D. A.

Jedrziewski

M. K.

Christakis

N. A.

(1997). Response rates to mail surveys published in medical journals. Journal of Clinical Epidemiology, 50, 1129–1136.

Cho

Y. I.

Johnson

T. P.

VanGeest

J. B.

(2013). Enhancing surveys of health care professionals: A meta-analysis of techniques to improve response. Evaluation & the Health Professions, 36, 382–407.

Cook

J. V.

Dickinson

H. O.

Eccles

M. P.

(2009). Response rates in postal surveys of healthcare professionals between 1996 and 2005: An observational study. Health Services Research, 9, 160.

Cull

W. L.

O’Connor

K. G.

Sharp

Tang

S. S.

(2005). Response rates and response bias for 50 surveys of pediatricians. Health Services Research, 40, 213–226.

Deehan

Templeton

Taylor

Drummond

Strang

(1997). The effect of cash and other financial inducements on the response rate of general practitioners in a national postal study. British Journal of General Practice, 47, 87–90.

Delnevo

C. D.

Abatemarco

D. J.

Steinberg

M. B.

(2004). Physician response rates to a mail survey by specialty and timing of incentive. American Journal of Preventive Medicine, 26, 234–236.

Dillman

D. A.

Smyth

J. D.

Christian

L. M.

(2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method. Hoboken, NJ: John Wiley.

Edwards

Roberts

Clarke

DiGuiseppi

Pratap

Wentz

Kwan

(2002). Increasing response rates to postal questionnaires: Systematic review. British Medical Journal, 324, 1183.

10.

Field

T. S.

Cadoret

C. A.

Brown

M. L.

Ford

Greene

S. M.

Hill

… Zapka

J. M.

(2002). Surveying physicians do components of the “Total Design Approach” to optimizing survey response rates apply to physicians? Medical Care, 40, 596–605.

11.

Flanigan

T. S.

McFarlane

Cook

(2008). Conducting survey research among physicians and other medical professionals: A review of current literature. Paper presented at the Proceedings of the Survey Research Methods Section, American Statistical Association.

12.

Halpern

S. D.

Ubel

P. A.

Berlin

J. A.

Asch

D. A.

(2002). Randomized trial of 5 dollars versus 10 dollars monetary incentives, envelope size, and candy to increase physician response rates to mailed questionnaires. Medical Care, 40, 834–839.

13.

James

K. M.

Ziegenfuss

J. Y.

Tilburt

J. C.

Harris

A. M.

Beebe

T. J.

(2011). Getting physicians to respond: The impact of incentive type and timing on physician survey response rates. Health Services Research, 46, 232–242. doi:10.1111/j.1475-6773.2010.01181.x

14.

Jepson

Asch

D. A.

Hershey

J. C.

Ubel

P. A.

(2005). In a mailed physician survey, questionnaire length had a threshold effect on response rate. Journal of Clinical Epidemiology, 58, 103–105.

15.

Kasprzyk

Montano

D. E.

Lawrence

J. S.

Phillips

W. R.

(2001). The effects of variations in mode of delivery and monetary incentive on physicians’ responses to a mailed survey assessing STD practice patterns. Evaluation & the Health Professions, 24, 3–17.

16.

Keating

N. L.

Zaslavsky

A. M.

Goldstein

West

D. W.

Ayanian

J. Z.

(2008). Randomized trial of $20 versus $50 incentives to increase physician survey response rates. Medical Care, 46, 878–881.

17.

Kellerman

S. E.

Herold

(2001). Physician response to surveys: A review of the literature. American Journal of Preventive Medicine, 20, 61–67.

18.

Klabunde

C. N.

Willis

G. B.

Casalino

L. P.

(2013). Facilitators and barriers to survey participation by physicians: A call to action for researchers. Evaluation & the Health Professions, 36, 279–295.

19.

McLeod

C. C.

Klabunde

C. N.

Willis

G. B.

Start

(2013). Health care provider surveys in the United States, 2000-2010: A review. Evaluation & the Health Professions, 36, 106–126.

20.

Moore

Post

Smith

(1999). ‘Bin bag’ study: A survey of the research requests received by general practitioners and the primary health care team. British Journal of General Practice, 49, 905–906.

21.

Pit

S. W.

Pyakurel

(2014). The effectiveness of recruitment strategies on general practitioner’s survey response rates: A systematic review. BMC Medical Research Methodology, 14, 76.

22.

Thorpe

Ryan

McLean

S. L.

Burt

Stewart

Brown

J. B.

… Harris

(2009). How to obtain excellent response rates when surveying physicians. Family Practice, 26, 65–68.

23.

Turnbull

A. E.

O’Connor

C. L.

Lau

Halpern

S. D.

Needham

D. M.

(2015). Allowing physicians to choose the value of compensation for participation in a web-based survey: Randomized controlled trial. Journal of Medical Internet Research, 17, e189. doi:10.2196/jmir.3898

24.

VanGeest

J. B.

Beebe

T. J.

Johnson

T. P.

(2015). Surveys of physicians. In Johnson

T. P.

(Ed.), Handbook of health survey methods (pp. 515–543). Hoboken, NJ: John Wiley.

25.

VanGeest

J. B.

Johnson

T. P.

Welch

V. L.

(2007). Methodologies for improving response rates in surveys of physicians: A systematic review. Evaluation & the Health Professions, 30, 303–321.

26.

VanGeest

J. B.

Wynia

M. K.

Cummins

D. S.

Wilson

I. B.

(2001). Effects of different monetary incentives on the return rate of a national mail survey of physicians. Medical Care, 39, 197–201.