Sociological Methodology

The opportunities for understanding how treatment effects vary across different segments of the population have led to a rise in the use of quantile regressions for identifying unconditional quantile treatment effects (QTEs). However, existing quantile regression models fall into two categories: those that are unsuitable for identifying unconditional QTEs and those that often struggle with the complex data structures common in sociology and other social sciences. In particular, existing methods face difficulties with large data sets and high-dimensional fixed effects. The authors introduce a two-step approach to estimating unconditional QTEs, which is easy to use and aligns with the needs of sociologists. First, the treatment variable is decomposed into a systematic and random part, and then, the random variation in the treatment status is used as the sole independent variable in a quantile regression model. Through a series of simulations and three empirical applications, the authors provide strong evidence that the residualized quantile regression (RQR) approach provides approximately unbiased estimates of unconditional QTEs comparable with existing methods. Moreover, the RQR approach offers greater flexibility and enhances computational speed compared with existing models, and it can easily handle high-dimensional fixed effects. In sum, the RQR approach fills a pressing void in quantitative research methodology, offering a much-needed tool for studying treatment effect heterogeneity.

Inducing semantic relations in word vector spaces and analyzing how other words or entire documents discursively engage these relations is a popular form of cultural analysis. The authors propose a reliability metric that is easily interpretable and agnostic to the type of relation. The metric, which the authors call the anchor reliability coefficient (relco), is found by creating an artificial document-term matrix of simulated documents that sequentially shift more of their tokens from relation-relevant anchor terms to nonanchor terms and then regressing the documents’ similarity to an induced relation on the anchor inclusion score of the documents. The authors validate the metric at the word level with both expert- and crowdsourced dictionaries and at the document level with expert-annotated social media posts. The authors also provide some heuristic baselines for assessing reliability effect sizes and null hypothesis testing.

Causal inference approaches often emphasize binary treatments. But in many applications, the underlying constructs are continuous. In the potential outcomes framework, a continuous treatment can take on numerous values, each corresponding to a potential outcome that may be realized. In this setting, common estimands may be intractable because of a common issue in social research, particularly research on social inequality: the exposure is highly stratified by confounders. The authors show how to avoid drawing inferences about counterfactuals where data are unlikely to exist by carefully selecting the causal estimand. The authors adopt an additive shift estimand that adds a small, fixed amount to each unit’s income. This approach is preferable to population-average dose-response curves in settings in which some treatment values rarely occur in some subgroups. The authors also show how to estimate and summarize patterns of nonlinearity and effect heterogeneity with continuous treatments. As a motivating example, the authors consider the causal effect of parental income on college attendance, a setting in which the exposure is highly stratified by confounders (e.g., parental education). This approach applies to a wide range of possible treatment conditions in sociology.

In the thriving field of network studies, there has been an emerging practice of comparing optimal modularity across social networks to evaluate the variation of network module–related substantive concepts, such as the level of consensus, polarization, or community boundary rigidity. This practice offers valuable insights and is often thoughtfully motivated, but it faces several conceptual and empirical challenges that merit careful consideration. Conceptually, the selected modularity metric may misalign with the substantive concepts researchers aim to measure. Empirically, estimated optimal modularity is highly sensitive to algorithm choice and to network characteristics unrelated to those substantive concepts, which can bias comparison results. The authors illustrate these issues with toy examples and systematic simulations and offer suggestions for more rigorous comparison practices. To show the practical significance of these lessons, the authors replicate an empirical study that examines the temporal trend of modularity scores for job mobility networks to evaluate the evolution of mobility boundary rigidity in the U.S. labor market.

Computational methods have become widespread in the social sciences, but probabilistic language models remain relatively underused. We introduce language models to a general social science readership. First, we offer an accessible explanation of language models, detailing how they estimate the probability of a piece of language, such as a word or sentence, on the basis of the linguistic context. Second, we apply language models in an illustrative analysis to demonstrate the mechanics of using these models in social science research. The example application uses language models to classify names in a large administrative database; the classifications are then used to measure a sociologically important phenomenon: the spatial variation of religiosity. This application highlights several advantages of language models, including their effectiveness in classifying text that contains variation around the base structures, as is often the case with localized naming conventions and dialects. We conclude by discussing language models’ potential to contribute to sociological research beyond classification through their ability to generate language.