Sociological Methods & Research

At least since Pearson’s infamous warning in 1897, ratio and difference measures have provoked dispute and misunderstanding, even as they continue to appear in social science research publications. Using differential calculus and hierarchical model decomposition, this article (a) presents a new, streamlined exposition that directs attention to the “effects” of variables rather than correlations between them, (b) explicates a previously unnoticed measurement problem that affects certain ratio measures, (c) shows that hierarchical model decomposition satisfies many or all of the measurement purposes for which ratio and difference measures are used, (d) presents a brief proof that some widely used difference and log-of-ratio measures are mathematically equivalent to hierarchical model measures that do not suffer the problems noted by Pearson, and (e) illustrates these arguments with an analysis of U.S. Census Bureau’s Current Population Survey data on mean earnings, by state, of white, black, American Indian, and Asian men in 2016.

Exponential random graph models (ERGM) have been widely applied in the social sciences in the past 10 years. However, diagnostics for ERGM have lagged behind their use. Collinearity-type problems can emerge without detection when fitting ERGM, yielding inconsistent model estimates and problematizing inference from parameters. This article provides a method to detect multicollinearity in ERGM. It outlines the problem and provides a method to calculate the variance inflation factor (VIF) from ERGM parameters. It then evaluates the method with a Monte Carlo simulation, fitting 216,000 ERGMs and calculating the VIFs for each model. The distribution of VIFs is analyzed using multilevel regression to determine what network characteristics lend themselves to collinearity-type problems. The relationship between VIFs and unstable standard errors (a standard sign of collinearity) is also examined. The method is shown to effectively detect multicollinearity, and guidelines for interpretation are discussed.

Nonlinear mixed-effects models are models in which one or more coefficients of the growth model enter in a nonlinear manner, such as appearing in the exponent of the growth function. In their applications, the within-individual residuals are often assumed to be independent with constant variance across time, an assumption that implies that the assumed growth function fully accounts for the dependencies and patterns of variation in the data. Studies have shown that a poorly specified within-individual residual covariance structure of a linear mixed-effects model can impact the estimated covariance matrix of the random effects at the second-level, model fit and statistical inference. The consequences for nonlinear mixed-effects models are not, however, clearly understood. This is due in part to the differences in the estimation needs of the two types of models. Using empirical data examples, this work illustrates the impact of fitting alternative residual covariance structures in nonlinear mixed-effects models that do not entirely parallel the results from studies of strictly linear mixed-effects models and call for the need of researchers to consider alternative structures when fitting nonlinear mixed-effects models.

Multiple outcome recurrent event data are typical in social sciences, where several outcomes on an individual are collected. In situations where aggregated counts of events over a long observation period are recorded, rounding is common, leading to counts being heaped at rounded values. We consider situations where multiple outcome recurrent event data are recorded as binary responses indicating presence/absence of events between periodic assessments. By analyzing these jointly through linkage via random effects, we show that a joint outcome analysis of the presence/absence data, that are less prone to recall errors, provides high relative efficiency, compared to the analysis of true counts. Motivated by a study of criminal behavior, we demonstrate the utility of such joint analyses, including that the analysis of longitudinal presence/absence data eliminates the bias arising from the analysis of heaped count data, and hence incorrect conclusions concerning possible risk factors.

Many political and socioeconomic binary outcomes are the result of multiactor interaction: states joining a given international institution (e.g., military alliance, economic block) and not its rivals, people entering the workforce as an employee of a specific firm and not of its competitors, and so on. Yet most empirical studies analyze multilateral phenomena as the (joint) binary choice of either a single or, at most, two decision makers. This is due in part to a lack of empirical estimators that can efficiently deal with multiactor interaction. Analyzing multilateral processes as a set of either monadic or dyadic events, though, introduces bias and has important consequences for the estimates and ultimately the inferences that one would draw. In this article, I develop a new empirical estimator that is specifically designed to analyze multiparty interactions. Specifically, the model can accommodate the input of multiple actors into a unified, overarching decision-making process. Results from a Monte Carlo analysis and an application to real data on alliance formation demonstrate the superior performance of the new estimator relative to the standard approach.

Population sizing is essential in sociology and in various other real-life applications. Gigapixel cameras can provide high-resolution images of an entire population in many cases. However, exhaustive manual counting is tedious, slow, and difficult to verify, whereas current computer vision methods are biased and known to fail for large populations. A design unbiased method based on geometric sampling has recently been proposed. It typically requires only between 50 and 100 manual counts to achieve relative standard errors of 5–10 percent irrespective of population size. However, the large perspective effect introduced by gigapixel images may boost the relative standard error to 30–40 percent. Here, we show that projecting the sampling grid from a map onto the gigapixel image using the camera projection neutralizes the variance due to perspective effects and restores the relative standard errors back into the 5–10 percent range. The method is tested on six simulated images. A detailed step-by-step illustration is provided with a real image of a 30,000 people crowd.

In recent decades, factorial survey experiments (FSEs) have become increasingly widespread and successful for analyzing attitudes and behavioral intentions. FSEs measure the ratings of multidimensional treatments embedded in textual scenarios, which are called vignettes. Analyses of FSEs often assume that these ratings are interval scaled. Past research indicates that this assumption is problematic. Therefore, the following article develops a design for interval scaling tests in FSEs and a method for analyzing FSEs which is sensitive to the scaling level of ratings. An exemplary application of scaling sensitive factorial survey analysis in comparison to standard methods yields effect sizes, which are about a sixth larger with regard to the treatments used in the FSE and about a third larger with respect to influences of between respondent differences. The new method also enables the evaluation of interval in contrast to noninterval scaled rating behavior.

Standard Qualitative Comparative Analysis is especially suited to explain diversity but is often diagnosed with weak findings. Its protocol either can dismiss necessary conditions as irrelevant and make solutions that are untrue to observations, or add irrelevant conditions as causal and make incorrect solutions. Additionally, the algorithm may not recognize the functional dependencies among the conditions. These claims call for different gauges to assess the single conditions that are retrieved by Standard minimizations. This article develops “import” and “essentiality” to establish whether a condition has explanatory merit alone and within the wider model. When applied in exemplary studies, these gauges indicate that Standard solutions are more sound than often conceded.

Qualitative coding procedures emanating from grounded theory were limited by technologies of the 1960s: colored pens, scissors, and index cards. Today, electronic documents can be flexibly stored, retrieved, and cross-referenced using qualitative data analysis (QDA) software. We argue the oft-cited grounded theory framework poorly fits many features of contemporary sociological interview studies, including large samples, coding by teams, and mixed-method analysis. The grounded theory approach also hampers transparency and does not facilitate reanalysis or secondary analysis of interview data. We begin by summarizing grounded theory’s assumptions about coding and analysis. We then analyze published articles from American Sociological Association flagship journals, demonstrating that current conventions for semistructured interview studies depart from the grounded theory framework. Based on experience analyzing interview data, we suggest steps in data organization and analysis to better utilize QDA technology. Our goal is to support rigorous, transparent, and flexible analysis of in-depth interview data. We end by discussing strengths and limitations of our twenty-first-century approach.

Identifying illicit behavior in survey research is inherently problematic, since self-reports are untrustworthy. We argue that fraudulent interviewers can, however, be identified through statistical deviance of the distributional parameters of their interviews. We document that a high proportion of the variation in the data is due to the interviewer. In addition, we show that the incidence of interviewer-induced anomalies is strongly associated with the perceived level of corruption across the countries participating in the European Social Survey 2010. The major implication of the findings is that the data from some countries cannot be used fruitfully for cross-national comparative research.

A key obstacle to measurement is the aggregation problem. Where indicators tap into common latent traits in theoretically meaningful ways, the problem may be solved by applying a data-informed (“inductive”) measurement model, for example, factor analysis, structural equation models, or item response theory. Where they do not, researchers solve the aggregation problem by appeal to concept-driven (“deductive”) criteria, that is, aggregation schemes that do not presume patterns of covariance across observable indicators. This article introduces a novel approach to scale construction that builds on the properties of concepts to solve the aggregation problem. This is accomplished by regarding conceptual attributes as necessary-and-sufficient conditions arrayed in an ordinal scale. While different sorts of scales are useful for different purposes, we argue that “lexical” scales are in many cases superior for research questions where it is relevant to combine the differentiation of an ordinal scale with the distinct, meaningful categories of a typology.

Address-based sampling (ABS) refers to the use a list of addresses derived from the U.S. Postal Service’s Computerized Delivery Sequence File as a sampling frame. While most residential addresses are included on an ABS frame, it still suffers from undercoverage. Undercoverage is problematic only if the uncovered units have different attributes than covered units. In this article, we assess the risk of observing coverage bias on a variety of univariate point estimates in an in-person survey. We accomplish this task using a combination of survey data and Monte Carlo simulation models to vary the coverage rate and the target geography. For some variables, we found low risk of observing coverage bias. But, for others, the risk of observing bias was high for nearly all levels of coverage. Finally, we did not observe any trends by variable type that would give us the ability to predict a priori which variables may suffer bias.

A methodologically sound systematic review is characterized by transparency, replicability, and a clear inclusion criterion. However, little attention has been paid to reporting the details of interrater reliability (IRR) when multiple coders are used to make decisions at various points in the screening and data extraction stages of a study. Prior research has mentioned the paucity of information on IRR including number of coders involved, at what stages and how IRR tests were conducted, and how disagreements were resolved. This article examines and reflects on the human factors that affect decision-making in systematic reviews via reporting on three IRR tests, conducted at three different points in the screening process, for two distinct reviews. Results of the two studies are discussed in the context of IRR and intrarater reliability in terms of the accuracy, precision, and reliability of coding behavior of multiple coders. Findings indicated that coding behavior changes both between and within individuals over time, emphasizing the importance of conducting regular and systematic IRR and intrarater reliability tests, especially when multiple coders are involved, to ensure consistency and clarity at the screening and coding stages. Implications for good practice while screening/coding for systematic reviews are discussed.

Questions on voluntary association memberships have been used extensively in social scientific research for decades. Researchers generally assume that these respondent self-reports are accurate, but their measurement has never been assessed. Respondent characteristics are known to influence the accuracy of other self-report variables such as self-reported health, voting, or test scores. In this article, we investigate whether measurement error occurs in self-reports of voluntary association memberships. We use the 2004 General Social Survey (GSS) questions on voluntary associations, which include a novel resource: the actual organization names listed by respondents. We find that this widely used voluntary association classification scheme contains significant amounts of measurement error overall, especially within certain categories. Using a multilevel logistic regression, we predict accuracy of response nested within respondents and interviewers. We find that certain respondent characteristics, including some used in research on voluntary associations, influence respondent accuracy. Inaccurate and/or incorrect measurement will affect the statistics and conclusions drawn from the data on voluntary associations.

The Karlson–Holm–Breen (KHB) method has rapidly become popular as a way of separating the impact of confounding from rescaling when comparing conditional and unconditional parameter estimates in nonlinear probability models such as the logit and probit. In this note, we show that the same estimates can be obtained in a somewhat different way to that advanced by Karlson, Holm, and Breen in their original article and implemented in the user-written Stata command khb. While the KHB method and this revised KHB method both work by holding constant the residual variance of the model, the revised method makes comparisons across multiple nested models easier than the original method.

Necessary condition analysis (NCA) has recently been proposed to researchers in business, management, organization, psychology, sociology, and even medicine as a new data analysis tool for identifying necessary but insufficient causes of an outcome. In this comment, I demonstrate that NCA is inadequate for performing such inferences. The reason is a mismatch between the method’s purported search target and its actual output. Moreover, I show that, even if its output corresponded to its search target, the method of qualitative comparative analysis would always outperform NCA in all relevant respects.

There are two problems with the comment of Thiem about necessary condition analysis (NCA): First, it is based on wrong assumptions about what NCA aims to do. Second, it applies NCA incorrectly. These are critical errors such that the comment’s conclusions about NCA are flawed. Contrary to what the comment states, NCA is a valid method for identifying necessary conditions.