Estimating Eligibility Rates: Crucial Component of the Calculation for Response Rates

Abstract

Response rates are used by analysts to assess survey quality: higher response rates usually are desired to reduce the incidence of nonresponse bias. The response rate is defined as the ratio of the number of completed interviews divided by the number of eligible sample units. However, because of the inconsistency of the definition of response rates often quoted in travel surveys, it is difficult to state explicitly that declining response rates are the result of fewer people willing to participate in surveys or are attributed to the calculation of response rates. Most likely it is a combination of these two factors. Two well-known formulas used to calculate response rates are described: the Council of American Survey Research Organizations formula and the American Association for Public Opinion Research formula. The difference between these formulas lies in the estimate of eligible sample units among the sample units of unknown eligibility. Through examination of two call history files, the recruitment phase for two household travel surveys, eligibility rate estimates for the sample units of unknown eligibility were calculated and used in the response rate formulas. The rates of eligibility for the sample units of unknown eligibility were higher than the eligibility rates for the units of known eligibility. These results were not expected and further confirm that agencies must treat units of unknown eligibility carefully when calculating response rates.

Get full access to this article

View all access options for this article.

References

Council of Marketing Opinion Research. Survey Response Definitions. 1999. http://www.cmor.org/downloads/resp_tools/cmor.pdf

De Heer

W. F.

, and Moritz

. Transportation Research Circular E-C008: Data Quality Problems in Travel Surveys: An International Overview. TRB, National Research Council, Washington, D.C., 2000. http://www.TRB.org/trb/publications/ec008/workshop_c.pdf. Accessed Jan. 28, 2003.

Richardson

A. J.

Behavioral Mechanisms of Nonresponse in Mail-Back Travel Surveys. In Transportation Research Record: Journal of the Transportation Research Board, No. 1855, TRB, National Research Council, Washington, D.C., 2003, pp. 191–199.

Kam

H. B.

, and Morris

. Response Patterns in Travel Surveys: The VATS Experience. 1999. http://www.trc.rmit.edu.au/Publications/Papers/responsepatterns.pdf. Accessed Feb. 10, 2003.

Council of American Survey Research Organizations. On the Definition of Response Rates: A Special Report of the CASRO Task Force on Completion Rates. 1982. http://www.casro.org. Accessed Nov. 6, 2002.

American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys, 3rd ed. 2004. http://www.aapor.org/pdfs/standarddefs2004.pdf. Accessed March 12, 2004.

Ezzati-Rice

T. M.

, Coronado

V. G.

, Frankel

M. R.

, Hoaglin

D. C.

, Loft

J. D.

, and Wright

R. A.

. Estimating Response Rates in Random-Digit- Dialing Surveys That Screen for Eligible Subpopulations. 1999 http://www.jpsm.umd.edu/icsn/papers/EzzatiRiceCoronado.htm. Accessed Nov. 7, 2002.

Richardson

A. J.

, and Meyburg

A. H.

. Definitions of Unit Non-Response in Travel Surveys. In Transport Survey Quality and Innovation ( Stopher

P. R.

and Jones

P. M.

, eds.), Pergamon Press, New York, 2003.

Singer

, Van Hoewyk

, and Maher

M. P.

. Experiments with Incentives in Telephone Surveys. Public Opinion Quarterly, Vol. 64, No. 2, 2000, pp. 171–188.

10.

Beerten

, Lynn

, Laiho

, and Martin

. Response Rates as a Measure of Survey Quality. 2000. webfarm.jrc.cec.eu.int/ETK-NTTS/Papers/final-papers/61pdf

11.

Lynn

, Beerten

, Roeland

, Jean

, and Jean

. University of Essex, Colchester, United Kingdom. Recommended Standard Final Outcome Categories and Standard Definitions of Response Rates for Social Surveys. 2001. iser.essex.ac.uk/pubs/workpaps/pdf/2001-23.pdf. Accessed March 5, 2004.

12.

Dillman

D. A.

, and Carley-Baxter

L. R.

. 2000. Structural Determinants of Mail Survey Response Rates over a 12-Year Period, 1988-1999 survey.sesrc.wsu.edu/dillman/papers.htm. Accessed Nov. 5, 2002.

13.

Dillman

D. A.

, Phelps

, Tortora

, Swift

, Koherll

, and Berck

. Response Rate and Measurement Differences in Mixed Mode Surveys: Using Mail, Telephone, Interactive Voice Response and the Internet. 2001. survey.sesrc.wsu.edu/dillman/papers.htm. Accessed Nov. 5, 2002.

14.

Kalfs

, and van Evert

. Non-Response and Travel Surveys. In Transport Survey Quality and Innovation ( Stopher

P. R.

and Jones

P. M.

, eds.), Pergamon Press, New York, 2003.

15.

Brick

J. M.

, Montaquila

, and Scheuren

. Estimating Residency Rates for Undetermined Telephone Numbers. Public Opinion Quarterly, Vol. 66, 2002, pp. 18–39.

16.

Keeter

, Miller

, Kohut

, Groves

R. M.

, and Presser

. Consequences of Reducing Non-Response in a National Telephone Survey. Public Opinion Quarterly, Vol. 64, No. 2, 2000, pp. 125–148.

17.

National Center for Education Statistics. NCES Statistical Standards. 2002. nces.ed.gov/statprog/2002/stdl3.asp

18.

Groves

R. M.

, and Couper

M. P.

. Nonresponse in Household Interview Surveys. John Wiley and Sons, New York, 1998.

19.

Ellis

J. M.

Estimating the Number of Eligible Respondents for a Telephone Survey of Low-Incidence Households. Presented at Annual Meeting of the American Association for Public Opinion Research, Portland, Ore., 2000.