Sociological Methodology

False positive findings are a growing problem in many research literatures. We argue that excessive false positives often stem from model uncertainty. There are many plausible ways of specifying a regression model, but researchers typically report only a few preferred estimates. This raises the concern that such research reveals only a small fraction of the possible results and may easily lead to nonrobust, false positive conclusions. It is often unclear how much the results are driven by model specification and how much the results would change if a different plausible model were used. Computational model robustness analysis addresses this challenge by estimating all possible models from a theoretically informed model space. We use large-scale random noise simulations to show (1) the problem of excess false positive errors under model uncertainty and (2) that computational robustness analysis can identify and eliminate false positives caused by model uncertainty. We also draw on a series of empirical applications to further illustrate issues of model uncertainty and estimate instability. Computational robustness analysis offers a method for relaxing modeling assumptions and improving the transparency of applied research.

Conventional model selection evaluates models on their ability to represent data accurately, ignoring their dependence on theoretical and methodological assumptions. Drawing on the concept of underdetermination from the philosophy of science, the author argues that uncritical use of methodological assumptions can pose a problem for effective inference. By ignoring the plausibility of assumptions, existing techniques select models that are poor representations of theory and are thus suboptimal for inference. To address this problem, the author proposes a new paradigm for inference-oriented model selection that evaluates models on the basis of a trade-off between model fit and model plausibility. By comparing the fits of sequentially nested models, it is possible to derive an empirical lower bound for the subjective plausibility of assumptions. To demonstrate the effectiveness of this approach, the method is applied to models of the relationship between cultural tastes and network composition.

The relationship between processes and time-varying covariates is of central theoretical interest in addressing many social science research questions. On the one hand, event history analysis (EHA) has been the chosen method to study these kinds of relationships when the outcomes can be meaningfully specified as simple instantaneous events or transitions. On the other hand, sequence analysis (SA) has made increasing inroads into the social sciences to analyze trajectories as holistic “process outcomes.” We propose an original combination of these two approaches called the sequence analysis multistate model (SAMM) procedure. The SAMM procedure allows the study of the relationship between time-varying covariates and trajectories of categorical states specified as process outcomes that unfold over time. The SAMM is a stepwise procedure: (1) SA-related methods are used to identify ideal-typical patterns of changes within trajectories obtained by considering the sequence of states over a predefined time span; (2) multistate event history models are estimated to study the probability of transitioning from a specific state to such ideal-typical patterns. The added value of the SAMM procedure is illustrated through an example from life-course sociology on how (1) time-varying family status is associated with women’s employment trajectories in East and West Germany and (2) how German reunification affected these trajectories in the two subsocieties.

Randomized experiments on a network often involve interference between connected units, namely, a situation in which an individual’s treatment can affect the response of another individual. Current approaches to deal with interference, in theory and in practice, often make restrictive assumptions on its structure—for instance, assuming that interference is local—even when using otherwise nonparametric inference strategies. This reliance on explicit restrictions on the interference mechanism suggests a shared intuition that inference is impossible without any assumptions on the interference structure. In this paper, we begin by formalizing this intuition in the context of a classical nonparametric approach to inference, referred to as design-based inference of causal effects. Next, we show how, always in the context of design-based inference, even parametric structural assumptions that allow the existence of unbiased estimators cannot guarantee a decreasing variance even in the large sample limit. This lack of concentration in large samples is often observed empirically, in randomized experiments in which interference of some form is expected to be present. This result has direct consequences for the design and analysis of large experiments—for instance, in online social platforms—where the belief is that large sample sizes automatically guarantee small variance. More broadly, our results suggest that although strategies for causal inference in the presence of interference borrow their formalism and main concepts from the traditional causal inference literature, much of the intuition from the no-interference case do not easily transfer to the interference setting.

Treatment interference (i.e., one unit’s potential outcomes depend on other units’ treatment) is prevalent in social settings. Ignoring treatment interference can lead to biased estimates of treatment effects and incorrect statistical inferences. Some recent studies have started to incorporate treatment interference into causal inference. But treatment interference is often assumed to follow a simple structure (e.g., treatment interference exists only within groups) or measured in a simplistic way (e.g., only based on the number of treated friends). In this paper, I highlight the importance of collecting data on actual treatment diffusion in order to more accurately measure treatment interference. Furthermore, I show that with accurate measures of treatment interference, we can identify and estimate a series of causal effects that are previously unavailable, including the direct treatment effect, treatment interference effect, and treatment effect on interference. I illustrate the methods through a case study of a social network–based smoking prevention intervention.

In recent years, there has been increasing attention focused on the spatial dimensions of residential segregation—from the spatial arrangement of segregated neighborhoods to the geographic scale or relative size of segregated areas. However, the methods used to measure segregation do not incorporate features of the built environment, such as the road connectivity between locations or the physical barriers that divide groups. This paper introduces the spatial proximity and connectivity (SPC) method for measuring and analyzing segregation. The method addresses the limitations of current approaches by taking into account how the physical structure of the built environment affects the proximity and connectivity of locations. I describe the method and its application for studying segregation and spatial inequality more broadly, and I demonstrate one such application—analyzing the impact of physical barriers on residential segregation—with a stylized example and an empirical analysis of racial segregation in Pittsburgh, Pennsylvania. The SPC method contributes to scholarship on residential segregation by capturing the effect of an important yet understudied mechanism of segregation—the connectivity, or physical barriers, between locations—on the level and spatial pattern of segregation, and it enables further consideration of the role of the built environment in segregation processes.

Survey researchers often ask a series of attitudinal questions with a common question stem and response options, known as battery questions. Interviewers have substantial latitude in deciding how to administer these items, including whether to reread the common question stem on items after the first one or to probe respondents’ answers. Despite the ubiquity of use of these items, there is virtually no research on whether respondent and interviewer behaviors on battery questions differ over items in a battery or whether interview behaviors are associated with answers to these questions. This article uses a nationally representative telephone survey with audio-recorded interviews and randomized placement of items within four different batteries to examine interviewer and respondent behaviors and respondent answers in battery questions. Using cross-classified random-effects models, the authors find strong evidence that there is more interviewer–respondent interaction on items asked earlier in the battery. In addition, interviewer and respondent behaviors are associated with both substantive and nonsubstantive answers provided to battery items, especially if the interviewer decided to reread or probe with the response options. These results suggest that survey designers should follow recommendations to randomize battery items and consider the importance of standardization of question administration when designing battery questions.

Autoregressive latent trajectory (ALT) models combine features of latent growth curve models and autoregressive models into a single modeling framework. The development of ALT models has focused primarily on models with linear growth components, but some social processes follow nonlinear trajectories. Although it is straightforward to extend ALT models to allow for some forms of nonlinear trajectories, the identification status of such models, approaches to comparing them with alternative models, and the interpretation of parameters have not been systematically assessed. In this paper we focus on two forms of nonlinear autoregressive latent trajectory (NLALT) models. The first form allows for a quadratic growth trajectory, a popular form of nonlinear latent growth curve models. The second form derives from latent basis models, or freed loading models, that allow for arbitrary growth processes. We discuss details concerning parameterization, model identification, estimation, and testing for the two forms of NLALT models. We include a simulation study that illustrates potential biases that may arise from fitting alternative models to data derived from an autoregressive process and individual-specific nonlinear trajectories. In addition, we include an extended empirical example modeling growth trajectories of weight from birth through age 2.

In recent studies, latent class tree (LCT) modeling has been proposed as a convenient alternative to standard latent class (LC) analysis. Instead of using an estimation method in which all classes are formed simultaneously given the specified number of classes, in LCT analysis a hierarchical structure of mutually linked classes is obtained by sequentially splitting classes into two subclasses. The resulting tree structure gives a clear insight into how the classes are formed and how solutions with different numbers of classes are substantively linked to one another. A limitation of the current LCT modeling approach is that it allows only for binary splits, which in certain situations may be too restrictive. Especially at the root node of the tree, where an initial set of classes is created based on the most dominant associations present in the data, it may make sense to use a model with more than two classes. In this article, we propose a modification of the LCT approach that allows for a nonbinary split at the root node, and we provide methods to determine the appropriate number of classes in this first split, based either on theoretical grounds or on a relative improvement of fit measure. This novel approach also can be seen as a hybrid of a standard LC model and a binary LCT model, in which an initial, oversimplified but interpretable model is refined using an LCT approach. Furthermore, we show how to apply an LCT model when a nonstandard LC model is required. These new approaches are illustrated using two empirical applications: one on social capital and the other on (post)materialism.

Researchers studying income inequality, economic segregation, and other subjects must often rely on grouped data—that is, data in which thousands or millions of observations have been reduced to counts of units by specified income brackets. The distribution of households within the brackets is unknown, and highest incomes are often included in an open-ended top bracket, such as “$200,000 and above.” Common approaches to this estimation problem include calculating midpoint estimators with an assumed Pareto distribution in the top bracket and fitting a flexible multiple-parameter distribution to the data. The authors describe a new method, mean-constrained integration over brackets (MCIB), that is far more accurate than those methods using only the bracket counts and the overall mean of the data. On the basis of an analysis of 297 metropolitan areas, MCIB produces estimates of the standard deviation, Gini coefficient, and Theil index that are correlated at 0.997, 0.998, and 0.991, respectively, with the parameters calculated from the underlying individual record data. Similar levels of accuracy are obtained for percentiles of the distribution and the shares of income by quintiles of the distribution. The technique can easily be extended to other distributional parameters and inequality statistics.