Abstract
Recent research has attempted to examine the proportion of interviewer variance that is due to interviewers systematically varying in their success in obtaining cooperation from respondents with varying characteristics (i.e., nonresponse error variance), rather than variance among interviewers in systematic measurement difficulties (i.e., measurement error variance) - that is, whether correlated responses within interviewers arise due to variance among interviewers in the pools of respondents recruited, or variance in interviewer-specific mean response biases. Unfortunately, work to date has only considered data from a CATI survey, and thus suffers from two limitations: Interviewer effects are commonly much smaller in CATI surveys, and, more importantly, sample units are often contacted by several CATI interviewers before a final outcome (response or final refusal) is achieved. The latter introduces difficulties in assigning nonrespondents to interviewers, and thus interviewer variance components are only estimable under strong assumptions. This study aims to replicate this initial work, analyzing data from a national CAPI survey in Germany where CAPI interviewers were responsible for working a fixed subset of cases.
Keywords
References
1.
Bauer, T.K., Fertig, M., and Vorrell, M. (2011a). Neighborhood Effects and Individual Unemployment. SOEP paper 409, DIW Berlin.
2.
Bauer, T.K., Flake, R., and Sinning, M.G. (2011b). Labor Market Effects of Immigration: Evidence from Neighborhood Data. Ruhr Economic Papers #257, Bochum, Dortmund, Duisburg, Essen.
3.
Beland, Y. and St-Pierre, M. (2008). Mode Effects in the Canadian Community Health Survey: A Comparison of CATI and CAPI. Advances in Telephone Survey Methodology, Chapter 14, J.M. Lepkowski, C. Tucker, J.M. Brick, E.D. de Leeuw, L. Japec, P.J. Lavrakas, M.W. Link, R.L. Sangsler (eds). New York: Wiley.
4.
Bender, S. and Heining, J. (2011). The Research-Data-Centre in Research-Data-Centre Approach: A First Step Towards Decentralised International Data Sharing. FDZ Methodenreport, 07/2011, Nürnberg.
5.
Biemer, P.P. (1980). A Survey Error Model which Includes Edit and Imputation Error. Proceedings of the American Statistical Association, Section on Survey Research Methods, 616-621.
6.
Biemer, P.P. (2010). Total Survey Error: Design, Implementation, and Evaluation. Public Opinion Quarterly, 74, 817-848.
7.
Biemer, P.P. and Stokes, S.L. (1991). Approaches to the Modeling of Measurement Error. Measurement Errors in Surveys, P.P. Biemer, R.M. Groves, L. Lyberg, N.A. Mathiowetz, and S. Sudman (eds). New York: Wiley. Biemer, P.P. and Trewin, D. (1997). A Review of Measurement Error Effects on the Analysis of Survey Data. In Survey Measurement and Process Quality, L. Lyberg, P.P.
8.
Biemer, M. Collins, E.D. de Leeuw, C. Dippo, N. Schwarz, and D. Trewin (eds). New York: Wiley-Interscience.
9.
Brunton-Smith, I., Sturgis, P., and Williams, J. (2012). Is Success in Obtaining Contact and Cooperation Correlated with the Magnitude of Interviewer Variance? Public Opinion Quarterly, 76, 265-286.
10.
Büngeler, K., Gensicke, M., Hartmann, J., Jäckle, R., and Tschersich, N. (2010). IABHaushaltspanel im Niedrigeinkommensbereich Welle 3 (2008/2009). Methoden- und Feldbericht. FDZ Methodenreport, 10/2010, Nu¨rnberg.
11.
Campanelli, P. and O’Muircheartaigh, C. (1999). Interviewers, Interviewer Continuity, and Panel Survey Nonresponse. Journal of Official Statistics, 2, 303-314.
12.
Christoph, B., Müller, G., Gebhardt, D., Wenzig, C., Trappmann, M., Achatz, J., Tisch, A., and Gayer, C. (2008). Codebook and Documentation of the Panel Study “Labour Market and Social Security” (PASS). Volume 1: Introduction and Overview, Wave 1 (2006/2007). FDZ Datenreport, 05/2008, Nürnberg.
13.
Collins, M. and Butcher, B. (1982). Interviewer and Clustering Effects in an Attitude Survey. Journal of the Market Research Society, 25, 39-58.
14.
Davis, P. and Scott, A. (1995). The Effect of Interviewer Variance on Domain Comparisons. Survey Methodology, 21, 99-106.
15.
Durrant, G.B., Groves, R.M., Staetsky, L., and Steele, F. (2010). Effects of Interviewer Attitudes and Behaviors on Refusal in Household Surveys. Public Opinion Quarterly, 74, 1-36.
16.
Fellegi, I.P. (1964). Response Variance and its Estimation. Journal of the American Statistical Association, 59, 1016-1041.
17.
Freeman, J. and Butler, E.W. (1976). Some Sources of Interviewer Variance in Surveys. Public Opinion Quarterly, 40, 79-91.
18.
Gebhardt, D., Müller, G., Bethmann, A., Trappmann, M., Christoph, B., Gayer, C., Müller, B., Tisch, A., Siflinger, B., Kiesl, H., Huyer-May, B., Achatz, J., Wenzig, C., Rudolph, H., Graf, T., and Biedermann, A. (2009). Codebook and Documentation of the Panel Study “Labour Market and Social Security” (PASS). Volume 1: Introduction and Overview, Wave 2 (2007/2008). FDZ Datenreport, 06/2009, Nürnberg.
19.
Groves, R.M. (2004). The Interviewer as a Source of Survey Measurement Error. Survey Errors and Survey Costs, (Second Edition). New York: Wiley-Interscience.
20.
Groves, R.M. and Lyberg, L. (2010). Total Survey Error: Past, Present, and Future. Public Opinion Quarterly, 74, 849-879.
21.
Groves, R.M. and Magilavy, L.J. (1984). An Experimental Measurement of Total Survey Error. Proceedings of the Joint Statistical Meetings of the American Statistical Association, Section on Survey Research Methods, 698-703.
22.
Groves, R.M. and Magilavy, L.J. (1986). Measuring and Explaining Interviewer Effects in Centralized Telephone Surveys. Public Opinion Quarterly, 50, 251-266.
23.
Groves, R.M. and Peytcheva, E. (2008). The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis. Public Opinion Quarterly, 72, 167-189.
24.
Hansen, M.H., Hurwitz, W.N., and Bershad, M.A. (1960). Measurement Errors in Censuses and Surveys. Bulletin of the International Statistical Institute, 32nd Session, Vol. 38, Part 2, 359-374.
25.
Holbrook, A.L., Green, M.C., and Krosnick, J.A. (2003). Telephone Versus Face-to-Face Interviewing of National Probability Samples with Long Questionnaires: Comparisons of Respondent Satisficing and Social Desirability Bias. Public Opinion Quarterly, 67, 79-125.
26.
Hox, J.J. (1998). Multilevel Modeling: When and Why. In Classification, Data Analysis, and Data Highways, I. Balderjahn, R. Mathar, and M. Schader (eds). New York: Springer.
27.
Hox, J.J. and de Leeuw, E.D. (2002). The Influence of Interviewers’ Attitude and Behavior on Household Survey Nonresponse: An International Comparison. Survey Nonresponse, R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little (eds). New York: Wiley.
28.
Kish, L. (1962). Studies of Interviewer Variance for Attitudinal Variables. Journal of the American Statistical Association, 57, 92-115.
29.
Mangione, T.W., Fowler, F.J., and Louis, T.A. (1992). Question Characteristics and Interviewer Effects. Journal of Official Statistics, 8, 293-307.
30.
Morton-Williams, J. (1993). Interviewer Approaches. Aldershot: Dartmouth Publishing Company Limited.
31.
O’Muircheartaigh, C. and Campanelli, P. (1998). The Relative Impact of Interviewer Effects and Sample Design Effects on Survey Precision. Journal of the Royal Statistical Society, Series A, 161, 63-77.
32.
O’Muircheartaigh, C. and Campanelli, P. (1999). A Multilevel Exploration of the Role of Interviewers in Survey Non-Response. Journal of the Royal Statistical Society, Series A, 162(Part 3), 437-446.
33.
Patterson, H.D. and Thompson, R. (1971). Recovery of Inter-Block Information when Block Sizes are Unequal. Biometrika, 58, 545-554.
34.
Pickery, J. and Loosveldt, G. (2002). A Multilevel Multinomial Analysis of Interviewer Effects on Various Components of Unit Nonresponse. Quality and Quantity, 36, 427-437.
35.
Rudolph, H. and Trappmann, M. (2007). Design und Stichprobe des Panels “Arbeitsmarkt und Soziale Sicherung” (PASS). In Neue Daten für die Sozialstaatsforschung. Zur Konzeption der IAB-Panelerhebung “Arbeitsmarkt und Soziale Sicherung”, (ed.) M. Promberger, IAB Forschungsbericht, 12/2007, Nürnberg.
36.
Schaeffer, N.C., Dykema, J., and Maynard, D.W. (2010). Interviewers and Interviewing. In Handbook of Survey Research, J.D. Wright and P.V. Marsden (eds). (Second Edition). Bingley, U.K. Emerald Group Publishing Limited.
37.
Schnell, R. (1997). Nonresponse in Bevölkerungsumfragen: Ausmaß, Entwicklung und Ursachen. Opladen: Leske þ Budrich.
38.
Schnell, R. (2012). Survey-Interviews. Standardisierte Befragungen in den Sozialwissenschaften. Wiesbaden: VS-Verlag.
39.
Schnell, R. and Kreuter, F. (2005). Separating Interviewer and Sampling-Point Effects. Journal of Official Statistics, 21, 389-410.
40.
Snijkers, G., Hox, J.J., and de Leeuw, E.D. (1999). Interviewers’ Tactics for Fighting Survey Nonresponse. Journal of Official Statistics, 15, 185-198.
41.
Trappmann, M., Gundert, S., Wenzig, C., and Gebhardt, D. (2010). PASS: a Household Panel Survey for Research on Unemployment and Poverty. Proceedings of the American Statistical Association, Section on Survey Research Methods, 130, 609-622.
42.
West, B.T. and Olson, K. (2010). How Much of Interviewer Variance is Really Nonresponse Error Variance? Public Opinion Quarterly, 74, 1004-1026.
43.
Wiggins, R.D., Longford, N.T., and O’Muircheartaigh, C.A. (1992). A Variance Components Approach to Interviewer Effects. In Survey and Statistical Computing, A. Westlake, R. Banks, C. Payne, and T. Orchard (eds). Amsterdam: North-Holland.
44.
Zhang, D. and Lin, X. (2008). Variance Component Testing in Generalized Linear Mixed Models for Longitudinal/Clustered Data and Other Related Topics. Random Effect and Latent Variable Model Selection, D.B. Dunson (ed.)., Springer Lecture Notes in Statistics, 192.