Computers/Ordinateurs/Internet

Concurrent Web Options and Mail Survey Response

Rebecca L. Medway and Jenna Fulton, “When More Gets You Less: A Meta-Analysis of the Effect of Concurrent Web Options on Mail Survey Response Rates”, Public Opinion Quarterly, 2012, 76(4): 733-746. Researchers offer concurrent Web options in mail surveys in hopes of improving response rates. However, existing literature reports mixed effects of doing so. This paper utilizes meta-analytic methods to evaluate the effect that offering a concurrent Web option in a mail survey has on the response rate. Across nineteen experimental comparisons, we find that mail surveys that incorporate a concurrent Web option have significantly lower response rates than those that do not. We propose several explanations for this result: adding a mode option may increase the complexity of responding, selecting the Web option could cause a break in the response process, or attempting to respond online may result in implementation failure. We also find that study characteristics, such as offering an incentive, have little influence on the magnitude of this effect.

Measuring Communication Ties in Web Forums

Andraž Petrovčič, Vasja Vehovar and Aleš Žiberna, “Approaches to Measure Communication Ties in Web Forums”, Quality and Quantity, 2012, 46(3): 829-85. In Web forums, very often almost no direct relational information exists that would indicate the presence of communication ties among contributors. In contrast with the reply-to structure of Usenet newsgroup or mailing list conversations that contain explicit relational information created by the contributors, some Web forums only enable participants to add new posts to threads or to quote preceding posts in threads. When discussions emerge, it is difficult to identify who is replying to whom. Drawing on the social network studies dealing with the conversational patterns in Usenet and Web forums, this paper presents an alternative approach to identifying the ties between authors of posts. Several assumptions are discussed, and different measures are developed and empirically evaluated.

Meta-Analysis for Sociology on the Web

In January 2013, issue 117 of the BMS included an ongoing research note, “Meta-Analysis for Sociology - A Measure-Driven Approach”, by David J. Roelfs, (University of Louisville), Eran Shor* (McGill University), Louise Falzon, (Mt. Sinai Medical Center), Karina W. Davidson (Columbia University) and Joseph E. Schwartz (Stony Brook University). The text’s abstract stated that meta-analytic methods were becoming increasingly important in sociological research, and the article presented an approach for meta-analysis which was especially helpful for sociologists. Conventional approaches to meta-analysis often prioritize “concept-driven” literature searches. However, in disciplines with high theoretical diversity, such as sociology, this search approach might constrain the researcher’s ability to fully exploit the entire body of relevant work. The authors explicated a “measure-driven” approach, in which iterative searches and new computerized search techniques were used to increase the range of publications found (and thus the range of possible analyses) and to traverse time and disciplinary boundaries. This measure-driven search approach was demonstrated with two meta-analytic projects, examining the effects of various social variables on all-cause mortality.

Supplementary information for this article was distributed to BMS subscribers and RC33 members by our BMS-RC33 listserv in early January and stored at https://listes.services.cnrs.fr/wws/admin/bms-rc33 for further consultation. Below is the opening part of the supplementary document.

Additional Details on the Concept Driven and Measure Driven Search Approaches, and the Literature on Meta-Analysis Search Techniques

In recent years sociologists are increasingly recognizing the merits of meta-analysis. Consequently, the use of this method (and of systematic reviews more generally) has grown rapidly over the past few years (see Figure S1). However, Figure S1 also shows that, historically, meta-analysis is an underused technique in the discipline. As of 15 December 2011, out of over 80,000 publications including the term “meta-analysis” or “systematic review” in their title or abstract, only 111 (less than 1%) were identified as sociology articles (as classified by ISI Web of Knowledge) and published in a sociology journal (as classified by Journal Citation Reports). The majority of meta-analyses were identified as psychological, medical, epidemiological, or public health articles.

OPEN IN VIEWER

Figure S1.

Number of meta-analyses and systematic reviews published in sociology journals by publication date.

There are two main explanations for the historical neglect of meta-analysis methods by sociologists. First, there is a lack of meta-analysis training within sociology, which is compounded by the inhibiting effect of traditional disciplinary and sub-disciplinary boundaries on methodological diffusion. Meta-analysis is frequently used by psychologists, medical researchers (epidemiologists, public health scholars, and health care providers), and environmental scientists (Lundahl and Yaffe, 2007). In sociology, it is mostly demographers, sociologists of health, or criminologists who use this technique. As advanced methods courses in sociology tend to focus on regression techniques, training in meta-analysis occurs primarily in research rather than classroom settings. Thus, researchers not working in the sociological sub-fields where meta-analysis is already in use are unlikely to be exposed to the technique. The considerable investment needed to thoroughly learn and employ meta-analysis techniques may serve as an additional barrier.

Second, and perhaps more importantly, meta-analysis requires the repeated use of similar measures, whereas research in sociology is (seemingly) seldom replicated (Cole, 1994; Stinchcomb, 1994; Stinchcomb 1994). However, we argue that sociologists often do employ comparable measures, but this fact is obscured by the use of multiple terms when actually referring to the same concept and by the use of differing proxy measures for the same concept. First, different researchers use widely diverse and often-surprising titles and keywords to describe their research. Since meta-analysis searches rely heavily on these features, researchers will locate primarily those studies that best match pre-conceived ideas about the field. Second, there are often fewer operational measures than theoretical constructs, the result being that multiple concepts are estimated with the same operational measure. Simply put, it is very hard (even for experts in a given sub-field) to specify the entire set of search terms used in multiple sub-fields and disciplines in advance.

[full text at https://listes.services.cnrs.fr/wws/admin/bms-rc33]