An Experiment to Test the Feasibility and Quality of a Web-Based Questionnaire of Teachers

Abstract

The use of web-based surveys to gather information from teachers has become increasingly common primarily based on the premise that they can reduce costs. Yet, relatively, little is known about the quality or cost effectiveness of web-based versus mail surveys for teachers. To study the efficacy of web-based teacher surveys, the author randomly assigned a nationally representative sample of 877 elementary school teachers to a paper or web survey mode. The cost savings from the web administration were not enough to offset the loss of sample due to missing/inaccurate email addresses and the lower response rates that resulted from the web survey.

Keywords

web-based surveys response rates teacher questionnaires survey costs

Get full access to this article

View all access options for this article.

References

Archer, T.M. 2003. ‘‘Web-Based Surveys.’’ Journal of Extension 41. Retrieved online article #4TOT6.

Biemer, P.P. , and L.E. Lyberg. 2003. Introduction to Survey Quality. Hoboken, NJ: John Wiley.

Carbonaro, M. , and J. Bainbridge. 2000. ‘‘Design and Development of a Process for Web-Based Survey Research.’’ The Alberta Journal of Educational Research 46:392-7.

Cobanoglu, C. , B. Warde , and P.J. Moreo. 2001. ‘‘A Comparison of Mail, Fax, and Web-Based Survey Methods.’’ International Journal of Market Research 43:441-52.

Cole, S.T. 2005. ‘‘Comparing Mail and Web-Based Survey Distribution Methods: Results of Surveys to Leisure Travel Retailers.’’ Journal of Travel Research 43:422-30.

Converse, P.D. , E.W. Wolfe , X. Huang , and F.L. Oswald. 2008. ‘‘Response Rates for Mixed Mode Surveys Using Mail and E-Mail/Web.’’ American Journal of Evaluation 29:99-107.

Couper, M.P. 2000. ‘‘Web Surveys: A Review of Issues and Approaches .’’ Public Opinion Quarterly 64:464-94.

Couper, M.P. , J. Blair , and T. Triplett. 1999. ‘‘A Comparison of Mail and E-Mail for a Survey of Employees in Federal Statistical Agencies. ’’ Journal of Official Statistics 15:39-56.

Couper, M.P. , M. Traugott , and M.J. Lamias. 2001. ‘‘Web Survey Design and Administration.’’ Public Opinion Quarterly 65:230-53.

10.

Crawford, S.D. , M.C. Couper , and M.J. Lamais. 2001. ‘‘Web Surveys: Perceptions of Burden.’’ Social Science Computer Review 19:146-62.

11.

Denscombe, M. 2006. ‘‘Web-Based Questionnaires and the Mode Effect: An Evaluation Based on Completion Rates and Data Contents of Near-Identical Questionnaires Delivered in Different Modes.’’ Social Science Computer Review 24:246-54.

12.

Dillman, D.A. 2007. Mail and Internet Surveys: The Tailored Design Method . Hoboken, NJ: John Wiley.

13.

Dillman, D.A. , and J.D. Smyth. 2007. ‘‘Design Effects in the Transition to Web-Based Surveys.’’ American Journal of Preventive Medicine 32:90-6.

14.

Dillman, D.A. , R.D. Tortora , and D. Bowker. 1998. ‘‘Principles for Constructing Web Surveys.’’ Accessed October 25, 2007. http://citeseer.ist.psu.edu/398113.html

15.

Duffy, M.E. 2002. ‘‘Methodological Issues in Web-Based Research .’’ Journal of Nursing Scholarship 34:83-8.

16.

Greenlaw, C. , and S. Brown-Welty. 2009. ‘‘A Comparison of Web-Based and Paper-Based Survey Methods.’’ Evaluation Review 33:464-80.

17.

Groves, R.M. 1989. Survey Errors and Survey Costs. New York, NY: John Wiley.

18.

Groves, R. , F. Fowler Jr. , M. Couper , J. Lepkowski , E. Singer , and R. Tourangeau. 2004. Survey Methodology. Hoboken, NJ: John Wiley.

19.

Heberlein, T.A. , and R. Baumgartner. 1978. ‘‘Factors Affecting Response Rates to Mailed Questionnaires: A Quantitative Analysis of the Published Literature .’’ American Sociological Review 43:447-62.

20.

Hoffer, T.B. , L. Venkataraman , E.C. Hedberg , and E.S. Shagle. 2007. Final Report on the National Survey of Algebra Teachers for the National Math Panel. Chicago : NORC.

21.

Kaplowitz, M.D. , T.D. Hadlock , and R. Levine. 2004. ‘‘A Comparison of Web and Mail Survey Response Rates.’’ Public Opinion Quarterly 68:94-101.

22.

Kiernan, N.E. , M. Kiernan , M.A. Oyler , and C. Gilles. 2005. ‘‘Is a Web Survey as Effective as a Mail Survey? A Field Experiment Among Computer Users.’’ American Journal of Evaluation 26:245-52.

23.

Kiesler, S. , and L.S. Sproull. 1986. ‘‘Response Effects in the Electronic Survey.’’ Public Opinion Quarterly 50:402-13.

24.

Klassen, R.D. , and J. Jacobs. 2001. ‘‘Experimental Comparison of Web, Electronic and Mail Survey Technologies in Operations Management. ’’ Journal of Operations Management 19:713-28.

25.

Kwak, N. , and B. Radler. 2002. ‘‘A Comparison Between Mail and Web Surveys: Response Pattern, Respondent Profile, and Data Quality. ’’ Journal of Official Statistics 18:257-73.

26.

Lazar, J. , and J. Preece. 1999. ‘‘Designing and Implementing Web-Based Surveys’’. The Journal of Computer Information Systems 39:63-7.

27.

Matz, M. 1999. ‘‘Administration of a Web Versus Paper Surveys: Mode Effects and Response Rates.’’ Master’s Thesis. University of North Carolina.

28.

McCabe, S.E. 2004. ‘‘Comparison of Web and Mail Surveys in Collecting Illicit Drug Use Data: A Randomized Experiment.’’ Journal of Drug Education 34:61-72.

29.

Mertler, C.A. 2003. ‘‘Patterns of Response and Nonresponse From Teachers to Traditional and Web Surveys.’’ Practical Assessment, Research and Evaluation 8. Accessed October 15, 2007. http://PAREonline.net/getvn.asp?v=8andn=22

30.

Shannon, D.M. , and C. Bradshaw. 2002. ‘‘A Comparison of Response Rate, Response Time and Costs of Mail and Electronic Surveys. ’’ The Journal of Experimental Education 70:179-92.

31.

Shannon, D. , T. Johnson , S. Searcy , and A. Lott. 2002. ‘‘Using Electronic Surveys: Advice From Survey Professionals.’’ Practical Assessment, Research and Evaluation 8. Accessed January 13, 2011. http://PAREonline.net/getvn.asp?v=8&n=1I

32.

Sheehan, K.B. , and M.G. Hoy. 1999. ‘‘Using E-Mail to Survey Internet Users in the United States: Methodology and Assessment.’’ Journal of Computer-Mediated Communication 4. Accessed http://www.ascusc.org/jcmc/vol4/issue3/sheehan.html

33.

Shih, T. , and X. Fan. 2008. ‘‘Comparing Response Rates From Web and Mail Surveys: A Meta-Analysis.’’ Field Methods 20:249-71.

34.

Solomon, D. 2001. ‘‘Conducting Web-Based Surveys.’’ Practical Assessment, Research and Evaluation 7. Accessed January 13, 2011. http://PAREonline. net/getvn.asp?v=7&n=19

35.

Tomaskovic-Devey, D. , J. Leiter , and S. Thompson. 1994. ‘‘Organizational Survey Nonresponse.’’ Administrative Science Quarterly 39:439-57.

36.

Tourangeau, R. , and K. Rasinski. 1988. ‘‘Cognitive Processes Underlying Context Effects in Attitude Measurement.’’ Psychological Bulletin 103:299-314.

37.

Tourkin, S. , T. Warner , R. Parmer , C. Cole , B. Jackson , A. Zukerberg , and A. Soderberg. 2007. Documentation for the 2003-04 Schools and Staffing Survey. Washington, DC: Institute for Education Sciences.

38.

U.S. Department of Education , National Center for Education Statistics. 1997. Improving the Mail Return Rates of SASS Surveys: A Review of the Literature . Working Paper No. 97-18, by Cornette Cole, Randall Palmer, and Dennis Schwanz. Project Officer, Steven Kaufman. Washington, DC.

39.

Watt, J. 1997. ‘‘Using the Internet for Quantitative Survey Research.’’ Quirk’s Marketing Research Review . Accessed October 15, 2007. http://www.swiftinteractive.com/white1.asp

40.

Wells, J. , L. Lewis , and B. Greene. 2006. Internet Access in U.S. Public Schools and Classrooms: 1994-2005. Washington, DC: National Center for Education Statistics. (NCES Rep. No. 2007-020). Accessed October 20, 2007. http://nces.ed.gov/pubs2007/2007020.pdf