Do Family Dinners Reduce the Risk for Early Adolescent Substance Use? A Propensity Score Analysis

Propensity Score Matching

Most research on family meals and adolescent substance use has used multivariate linear or logistic regression to estimate their empirical associations. An alternative has been to use fixed-effects regression models with longitudinal data to attenuate the bias that results from stationary unobserved variables; this presumably identifies potential causal effects (e.g., Musick and Meier 2012; Sen 2010). However, the causal modeling literature also advocates the use of matching techniques with observational data (Rosenbaum 2010; Stuart 2010). Studies indicate that fixed-effect estimators yield substantial bias and are relatively inefficient for capturing causal effects. Thus, propensity score matching is the recommended alternative because it leads to less bias in point estimates and is more efficient (Dehejia and Wahba 1999; Michalopoulos, Bloom, and Hill 2004; Rubin and Thomas 2000).

The most common matching approach is to establish comparable groups first through the use propensity scores, followed by a model that estimates differences in outcomes, such as substance use, across the groups (e.g., Slade et al. 2008). In general, the idea is to mimic experimental studies by constructing treatment and control groups, thus reducing the selection bias that is inherent in observational studies.

Specifically, it is a virtual certainty that the frequency of family meals is not randomly assigned across families (Davidson and Gauthier 2010). Thus, when examining its potential effects on substance use studies attempt to control for confounding variables (e.g., Eisenberg et al. 2008). But because we do not directly observe substance use among those who do share frequent family meals relative to their substance use if they did not share frequent family meals, we can never examine both potential outcomes for the same person. In other words, the counterfactual is never observed directly.

Researchers typically consider aggregated estimates as a substitute for individual (unobserved) estimates of the family meal effect. To move closer to a causally relevant model, however, we would have to assume that the average level of use for those who experience more family meals is equal to the average use for those who experience fewer family meals if they had experienced more family meals. Yet because selection bias affects the frequency of family meals, this assumption can rarely, if ever, be tested.

Given that we cannot directly observe the counterfactual condition, one solution is to study the nonrandom assignment mechanism—in this case the characteristics that affect family meal frequency. If we can establish a suitable model that predicts this assignment mechanism, we may use this information to (1) partial out the effects of observed covariates by creating treatment and control groups, and (2) estimate expected differences in substance use for these groups, thus estimating the average “treatment” effects of family meals on substance use. An essential step of this approach is the creation of these groups. It is important that the groups closely resemble one another, just as randomization in experimental studies results in similar treatment and control groups. A commonly used approach is to develop a prediction model of the assignment process and then estimate the propensity score, or the conditional probability of treatment given a set of observed covariates: [p(x) = Pr(T = 1 | x)]. A key feature of this is that individuals with the same propensity score, whether they fall in the observed treatment or control group, have the same distribution of the observed covariates. This is known as the balancing property (Rosenbaum and Rubin 1983). Thus, assuming that a good estimate of the true propensity score is found, comparable treatment and control groups that are balanced on a set of covariates may be identified (Rosenbaum 2010).

Next, we may substitute those observed on the same set of covariates for those unobserved to get the expected effect for the counterfactual case. This is done by averaging over units to get the aggregated estimate of the treatment effect. Although matching leads to a modest loss in efficiency, research suggests that matched samples balanced by propensity scores provide estimates with less bias and greater efficiency than linear regression estimates or fixed-effects estimators on the basis of unmatched samples (Dehejia and Wahba 1999, 2002; Rubin and Thomas 2000).

In this study, we use a nationally representative sample of young adolescents to explore the potential causal impact of family dinners on the frequency and onset of substance use. In particular, we develop a selection model on the basis of an extensive set of predictors to account for the nonrandom assignment of these adolescents to different levels of family dinner frequency. The estimated propensity scores from this model are then used to create matched treatment and control samples. Next, we compare the predicted odds of substance use initiation and the estimated means of alcohol, tobacco, and marijuana use across the groups to determine whether there are differences. We also consider family dinner frequency as a binary, categorical, and continuous indicator, thus establishing whether the potential causal effects on substance use depend on the way it is measured.

Measures

Previous research on adolescent drug use has used different scales, including binary indicators (e.g., no use vs. use), discrete categories, or continuous variables (e.g., past 30-day frequency of use). Several scholars argue that continuous measures are most appropriate, because more frequent use is associated with subsequent problem use and other health consequences (e.g., Bahr et al. 2005). Binary measures tend to conflate those who use only occasionally with those who use more often (Macleod et al. 2004). Moreover, studies have addressed initiation of use because its predictors may differ from those that predict the frequency of use, and preventing initiation during adolescence is a key goal of most intervention programs (Kosterman et al. 2000). We therefore rely on continuous measures of three of the most common types of drug use: alcohol use frequency in the past 30 days, cigarette use frequency in the past 30 days, and marijuana use frequency in the past 30 days. Moreover, drug use initiation, or whether respondents initiated substance use between 1997 and 1998, is also examined.

The variables concerning past 30-day use are based on the following questions: “Have you ever used [substance]?” and, for those who answered “yes,” “On how many days have you used [substance] in the last 30 days?” We combined responses to these two sets of questions to create past 30-day frequency variables. Because the frequency items are highly skewed, the natural logarithm of each (+1) was computed and used as outcome variables. The NLSY97 database includes constructed variables for waves 1 and 2 that measure any use of the aforementioned substances. By manipulating these variables, we constructed a categorical measure of initiation versus three other outcomes. Those who reported no lifetime use in 1997 yet reported use of any of these substances in 1998 are identified as initiators, whereas those who did not use in either year are identified as abstainers. The other two categories are those who stopped using and those who continued to use, but we limit our attention to initiation versus abstention.

The key explanatory variable is family dinners. Unfortunately, there is a lack of consistency in how family dinners have been measured. Some have considered categorical indicators of family dinner frequency (e.g., Eisenberg et al. 2008), whereas others have used continuous measures (e.g., Sen 2010). Both have strengths and weaknesses, however. Categorical measures may yield different results depending on the cut points used. Continuous measures may fail to show threshold effects or may mask nonlinearities in the associations. Because there is no clear guidance and we wish to offer a comprehensive view of the potential effects of family dinners, we use two categorical measures and a continuous measure. All of them are based on the following question: “In a typical week, how many days from 0 to 7 do you eat dinner with your family?”

Following Eisenberg et al. (2008), we first create a binary measure, with zero to four days as a low-frequency group and five to seven as a high frequency group. This scheme is used in the propensity score models that address binary treatment. An alternative categorical approach has been to identify three classes of frequency: zero to two days, three to five days, and six to seven days (Fulkerson et al. 2006). We adopt this coding strategy for the categorical treatment models. Finally, recent studies have treated family dinners as a continuous variable, zero to seven days a week (e.g., Musick and Meier 2012; Sen 2010). This coding strategy is used in the generalized propensity score models described later.

As mentioned in the section on sensitivity analyses, several other measurement approaches were also considered, such as a binary variable coded as zero to six days versus seven days and a categorical measure coded as zero to three days, four to six days, and seven days. In addition, because family dinner frequency manifests high negative skew, we estimated models with transformations in an attempt to induce normality. However, none of these approaches yields substantially different empirical results, so the three measures described earlier are used.

Because a key concern is to predict “treatment assignment” (frequency of family dinners) and subsequently examine whether it predicts substance use, we use two overlapping sets of covariates: those that research suggests influence family dinner frequency or affect adolescent substance use. Studies indicate that shared family dinners are predicted by gender, race-ethnicity, family income, parental education, mother’s employment, age, family structure, size of household, time use by adolescents, religiosity, and parent-child relations (e.g., Davidson and Gauthier 2010; Tepper 2001).

Several of these variables are also common predictors of adolescent substance use (e.g., Bailey and Hubbard 1990; Brook et al. 2006). In addition, substance use is consistently associated with parental monitoring, the surrounding physical environment, and peer behaviors (Allen et al. 2003; Hoffmann 2002; Kosterman et al. 2000).

We use several variables to construct measures of parent-child relations and parental monitoring. Parent-child relations are based on a scale developed by Child Trends (Hair et al. 2008). The scale is based on 16 questions that ask respondents about their mothers and fathers. The questions include inquiries such as “I really enjoy spending time with him/her,” “How often does s/he praise you for doing well?,” and “How often does s/he criticize you or your ideas?” Depending on the particular question, response options include “never” (coded 0) to “always” (coded 4) and “strongly disagree” (coded 0) to “strongly agree” (coded 4). Higher values on this scale indicate a closer relationship with one’s parent. Cronbach’s α values for the mother and father sets of responses are .78 and .82. Because some adolescents, especially those living with single parents, did not respond to these questions for both parents, we use the following scheme to retain as much information as available. If adolescents provided responses to these questions for both mothers and fathers, we take the average of the two response sets. If responses were available for only one parent, such as in single-parent families, we use the mean of that response set.

Parental monitoring was the composite of several items that asked how much the adolescent’s parents know about (1) his or her friends, (2) his or her close friends’ parents, (3) who he or she is with when not at home, and (4) his or her teachers and school activities. Responses range from “knows nothing” (coded 0) to “knows everything” (coded 4) (Hair et al. 2008). The monitoring scales have Cronbach’s α values of .76 (mother) and .82 (father). As with parent-child relations, we retain as much information as available by taking the mean of the two response sets about mothers and fathers if both are available or the mean of the response set if available for only one parent.

Time use is based on a series of questions that asked how many weekdays the youth spent doing homework, watching television, and reading for pleasure. Another question asked how many days in the previous week one’s family did something “fun such as play a game, go to a sporting event, and so forth.” A similar question inquired about doing “something religious as a family such as go to church, pray or read the scriptures together?” The first three variables are coded zero to five days, whereas the next two are coded as zero to seven days. Finally, days absent from school asked about the previous school term or semester. Because it manifests high positive skew, we log-transformed it after adding 1 to preserve the zeros (skewness of transformed variable = .21).

Physical environment is a composite measure based on a set of questions that asked about electricity in the home, the interior of the home, the condition of buildings in the neighborhood, whether the residential area is safe, and whether respondents had heard gunshots in a typical week. The items are summed to produce a composite score, with higher values indicating a more risky physical environment (Center for Human Resource Research and Child Trends 1999).

Peer substance use is based on a set of questions that asked adolescent respondents to estimate the percentage of youth in their grades at school who smoked cigarettes, got drunk at least once a month, and used marijuana, inhalants, or other drugs. The response categories range from “almost none (less than 10 percent)” (coded 1) to “almost all (more than 90 percent)” (coded 5). A limitation of these variables is that they do not directly measure friends’ behaviors; they only represent one’s perceived peer environment (Allen et al. 2003). Nevertheless, recent research suggests that beliefs about peer substance use are at least as important as actual peer behaviors among early adolescents (Nargiso, Friend, and Florin forthcoming).

The demographic variables are taken from the youth and parent or head of household NLSY97 surveys. Gender is coded as 1 = female and 0 = male. Some studies have shown that family meals are more consequential for girls (e.g., Eisenberg et al. 2008), but supplementary analyses indicate that this is not applicable to these data. Thus, we simply control for gender in each model (cf. Musick and Meier 2012). Age is coded in years, 12 to 14. Race-ethnicity involves a set of four dummy variables available in the NLSY97: white or other racial or ethnic status, African American, Hispanic, and mixed race-ethnicity. Residence is also a set of three dummy variables: urban or suburban, rural, and unknown. Family structure is measured with a set of four dummy variables: two–biological parent family, stepparent family, single-parent family, and other family type.

Size of household identifies the number of family members who lived in the home. It ranges from 1 to 12, with a mean of 2.44. Parental education is coded in years and indexes the highest level of education among either parent (M = 13.66 years). Mother’s employment is based on the roster of household members and is coded as 0 = mother not employed outside the home and 1 = mother employed outside the home. Family income is measured in dollars and gauges past-year income from all sources. Given its positive skew, we use the square root to normalize its distribution (skewness of transformed variable = .58). Table 1 provides summary information for the outcome and explanatory variables. Note that a majority of respondents reported six or seven family dinners per week.

OPEN IN VIEWER

Table 1.

Descriptive Statistics, 1997 National Longitudinal Study of Youth.

Variable	M or Proportion	SE	Minimum	Maximum
Frequency of family dinners	5.19	.03	0	7
Binary frequency of family dinners (0–4 vs. 5–7 days)	.72	.01	0	1
Categorical frequency of family dinners
0–2	.14	.02
3–5	.30	.01
6–7	.56	.01
Substance use, 1997
Past 30-day alcohol use, logged	.15	.01	0	3.4
Past 30-day cigarette use, logged	.28	.01	0	3.4
Past 30-days marijuana use, logged	.08	.01	0	3.4
Substance use, 1997–1998
Abstained from use	.39	.01
Initiated use	.17	.01
Stopped using	.10	.02
Continued to use	.33	.01
Female	.49	.01	0	1
Age (years)	13.00	.01	12	14
Race-ethnicity
White/other	.71	.01	0	1
African American	.15	.01	0	1
Hispanic	.13	.02	0	1
Mixed race-ethnicity	.01	.02	0	1
Residence type
Urban/suburban residence	.69	.01	0	1
Rural residence	.26	.01	0	1
Residence unknown	.05	.02	0	1
Family structure
Two–biological parent family	.54	.02	0	1
Stepparent family	.14	.01	0	1
Single-parent family	.27	.01	0	1
Other family type	.05	.02	0	1
Size of household	2.44	.02	1	12
Family income, square root	207.68	1.35	0	496
Parental education	13.66	.04	1	21
Mother employed	.57	.01	0	1
Weekdays did homework	3.46	.02	0	5
Weekdays read for pleasure	1.91	.03	0	5
Weekdays watched television	4.30	.02	0	5
Days family did fun activities	2.68	.03	0	7
Days family did religious activities	1.54	.03	0	7
Days absent from school, logged	1.25	.01	0	5
Parental monitoring	9.57	.05	0	20
Parent-child relations	24.92	.07	0	34
Physical environment	11.87	2.00	0	70
Peers who smoke	2.26	.02	1	5
Peers who get drunk	1.69	.02	1	5
Peers who use illegal drugs	1.90	.02	1	5

Note: Means and proportions are averages estimated across 25 multiply imputed samples and are based on weighted data; n = 5,419.

Analytic Methods

Our goal is to estimate the effect of family dinners on adolescent substance use using propensity score matching and weighting. However, as discussed earlier, previous studies have not identified a standard way to measure family dinners. Thus, we use several propensity score matching models in this study. The first type is appropriate when the treatment variable, family dinners, is binary (zero to four vs. five to seven days) or continuous (zero to seven days). In this approach, the first step is to estimate a statistical model that predicts family dinner frequency. For the binary indicator, we use a probit regression model (Stuart 2010) and for the continuous variable, we use a generalized propensity score model (also called a dose-response model), which is based on the standard normal density function (Hirano and Imbens 2004). The key to both of these is to estimate a selection model that accounts well for the assignment to the treatment categories. To meet the conditional independence assumption of these models, all potential variables that are related to processes of assignment into family dinner categories should be included in the model. Experts also recommend that one consider quadratic and interaction terms to better predict the treatment assignment. This is to satisfy the assumption of ignorable treatment assignment (Stuart 2010). To strengthen the predictive ability of models, we conducted a comprehensive assessment of the set of explanatory variables described earlier, along with quadratic terms for continuous covariates and interactions among sets of variables (cf. Slade et al. 2008). The decision rule was to include any covariate with a p value < .20 in the selection model. Two quadratic terms emerged as predictive of family dinners: parental monitoring squared and religious activities squared. Thus, each was included in the models. Moreover, only one of the peer variables was predictive—peers who get drunk—the other two peer variables were included only in the subsequent postmatching models.

Once the estimated propensity scores are determined from the probit or generalized models, they are used to match on the covariates that predict family dinner frequency. Similar to randomized experiments, the goal of matching is to achieve covariate balance or restrict the sample to a set of respondents who are comparable on these covariates. If comparisons indicate only modest mean or proportion differences among the covariates for the matched groups, then we may assume that these characteristics are not responsible for differences in outcomes across the groups. Although a number of matching methods are available, we use a nearest-neighbor model with replacement and limit the number of potential matches with a caliper for the binary treatment models (Dehejia and Wahba 2002). Various caliper levels were examined; a relatively small caliper (.005) is used on the basis of its ability to balance the covariates efficiently and maintain a substantial number of treatment and control cases. Up to five matches of controls to each treatment are specified to ensure adequate reliability across groups. The generalized propensity score model uses a blocking routine with cut points to match groups (Hirano and Imbens 2004). Following matching on covariates, we tested the balancing property for both types of models. This was followed by means/odds comparison tests based on linear or multinomial regression models to determine if the propensity of family dinners affects substance use. The most common approach is to focus on the average treatment effect on the treated, or the expected effect of family dinners on substance use for those experiencing more frequent family dinners.

The second type of model is appropriate for the three categories of family dinner (zero to two, three to five, and six or seven days). This also involves a prediction equation, in this case a multinomial logistic model, which is then used to calculate predicted propensity scores or the probability of falling into each category. The inverse of the scores are used as sampling weights in linear or multinomial regression models designed to predict the outcomes (Guo and Fraser 2010). A presumed benefit of this approach is that it uses all the data, rather than discarding cases that cannot be matched to a treatment unit. A disadvantage is that it includes observations outside the common support area.

Sensitivity Analyses

We conducted several auxiliary analyses to determine whether the results are sensitive to (1) the coding used for family dinners, (2) particular aspects of the propensity score models, (3) possible endogeneity of family dinners, and (4) unobserved factors. First, as mentioned earlier, we varied the measure of family dinners by using different cut points to create alternative binary (e.g., zero to six vs. seven days) and categorical scales. We also evaluated different transformations of the zero-to-seven scale. None of these steps results in different findings in general.

Second, we used different propensity score models by changing the matching algorithm (e.g., different number of matches, matching without replacement, different calipers). We also explored the use of a logistic model for proportions and a boosted regression model to increase the predictive power of the propensity score models (Lee, Lessler, and Stuart 2009). Boosted regression is a data-mining technique that iteratively reweights the regression parameters so that the errors in prediction diminish at each stage until a satisfactory prediction model is obtained (Hastie, Tibshirani, and Friedman 2009, chap. 10). Although it results in an improved prediction model (correct classification increases from 23 percent to 76 percent), it does not achieve sufficient covariate balance.

Third, it is generally recognized that most matching estimators are not consistent when the number of matches is small. Thus, bootstrapped standard errors are often used to attempt to correct this deficiency. According to recent research, however, this approach does not correct for bias (Abadie and Imbens 2008). Thus, we used a bias-corrected matching estimator that is implemented in the Stata package nnmatch to reestimate the binary family dinner models (Abadie and Imbens 2011). A limitation of this package is that it does not provide details about the matching or how it handles the covariates, so we opted to present the models shown earlier. The results of these models reveal that the average treatment effects on the treated are not significantly different from zero, thus supporting the results shown in the first panel of Table 3.

Fourth, to obviate potential endogeneity issues in cross-sectional propensity score models, we reestimated the models using substance use measures at wave 2 as outcome variables. Even the unadjusted difference models using the entire sample do not show a treatment effect; nor do any of the propensity score models. Thus, we do not find support for the effect of family dinners on marijuana use (cf. Table 3, column 4) over a one-year period (1997 to 1998).

Fifth, because the earlier analyses use family dinners measured at the same time as the covariates (1997), there may be reverse causality that is not considered (e.g., family dinners may affect parent-child relations). To guard against this, we estimated a set of models using family dinner frequency during wave 2 (1998) as the treatment indicator. Family dinner frequency and the covariates measured during wave 1 (1997) were used in the prediction equation. The resulting propensity score models showed no statistical relationship with adolescent substance use during wave 2. Once again, even the effects on marijuana use were not confirmed in this model.

Finally, following recent studies (e.g., Miller et al. 2010), we estimated fixed-effects regression models to predict substance use from 1997 to 1998. The fixed-effects model is represented as

Y i t = β 0 + β 1 ( family dinners ) i t + β q X i t + u i + ε i t ,

where X represents the time-varying explanatory variables and u_i is a respondent-specific term, or the fixed effect. Family dinners is denoted it, which indicates that it is allowed to change across the two waves. The fixed-effect represents observed and unobserved characteristics of respondents. The ϵ _it term indicates random noise that exists within and across units but may also include the effects of omitted variables. Because within-respondent variation is used to estimate the effects in this model, there tends to be reduced bias in coefficients designed to predict adolescent drug use (Frees 2004). Unfortunately, in addition to substance use and family dinners, the number of time-varying explanatory factors is limited to parent-child relations, parental monitoring, mother’s employment, family income, and two time use variables (family does something fun or does something religious together). Thus, the models are underfit.

The results of the fixed-effects models suggest that the continuous measure of family dinners has a modest longitudinal impact on alcohol, cigarette, and marijuana use (e.g., β[alcohol use] = –.004, p ≈ .03; cf. Musick and Meier 2012) but no discernible effect on substance use initiation. Nevertheless, a key assumption is that the treatment variable is not correlated with unobserved characteristics housed in ϵ _it . If this assumption is not met, then the fixed-effects models will provide biased results. Moreover, as mentioned earlier, studies indicate that fixed-effects estimators yield substantial bias and are relatively inefficient for capturing causal effects (Dehejia and Wahba 1999; Michalopoulos et al. 2004). Thus, propensity score matching is the recommended alternative.

Limitations

Although we have undertaken a relatively comprehensive analysis, there are several limitations that point to the need for additional research. First, the models may violate the unconfoundedness assumption (Rosenbaum 2010), which notes that the treatment indicator should be independent of the potential outcomes conditional on pretreatment variables. This means that all variables that affect both the outcome and the likelihood of receiving treatment are observed. Yet there are bound to be unobserved covariates that we could not examine. For example, genetic factors may affect the association between family dinners and substance use (Daw et al. 2013; Guo, Roettger, and Cai 2008).

Second, although we consider this issue in the sensitivity analyses, we cannot fully determine whether the covariates precede the treatment. Moreover, it is possible that family dinners and other variables, including the outcomes, are related reciprocally (White and Halliwell 2011). This has implications for attempts to estimate the probability or the distribution of family dinners and how they relate to adolescent behaviors. For example, youth who already use, say, marijuana may subsequently withdraw from family activities, including family dinners. And it may be this withdrawal process that leads to greater involvement in marijuana use rather than any effect of shared family meals or other aspects of family life. Future research should explore these associations in detail.

Third, there may be reporting bias when it comes to substance use and some of the explanatory variables, even though self-administered response cards were used in the NLSY97. Systematic underreporting of substance use is a problem in most large-scale surveys (Shillington and Clapp 2000). The degree of bias is unknown, however, so alternative models are needed that correct for unobserved biases.