Using machine learning to predict help-seeking among 2016–2018 Pregnancy Risk Assessment Monitoring System participants with postpartum depression symptoms

Abstract

Background:

Despite the importance of early identification and treatment, postpartum depression often remains largely undiagnosed with unreported symptoms. While research has identified several factors as prompting help-seeking for postpartum depression symptoms, no research has examined help-seeking for postpartum depression using data from a multi-state/jurisdictional survey analyzed with machine learning techniques.

Objectives:

This study examines help-seeking among people with postpartum depression symptoms using and demonstrating the utility of machine learning techniques.

Methods:

Data from the 2016–2018 Pregnancy Risk Assessment Monitoring System, a cross-sectional survey matched with birth certificate data, were used. Six US states/jurisdictions included the outcome help-seeking for postpartum depression symptoms and were used in the analysis. An ensemble method, “Super Learner,” was used to identify the best combination of algorithms and most important variables that predict help-seeking among 1920 recently pregnant people who screen positive for postpartum depression symptoms.

Results:

The Super Learner predicted well and had an area under the receiver operating curve of 87.95%. It outperformed the highest weighted algorithms which were conditional random forest and stochastic gradient boosting. The following variables were consistently among the top 10 most important variables across the algorithms for predicting increased help-seeking: participants who reported having been diagnosed with postpartum depression, having depression during pregnancy, living in particular US states, being a White compared to Black or Asian American individual, and having a higher maternal body mass index at the time of the survey.

Conclusion:

These results show the utility of using ensemble machine learning techniques to examine complex topics like help-seeking. Healthcare providers should consider the factors identified in this study when screening and conducting outreach and follow-up for postpartum depression symptoms.

Keywords

help-seeking machine learning postpartum depression

Introduction

Approximately 13% of individuals in the United States are estimated to experience postpartum depression (PPD),¹ which presents as a major depressive episode beginning during the postpartum period.² Symptoms include problems with sleep unrelated to care for the infant, anxiety, irritability, excessive worry about health and care of the infant, and in some instances suicidal ideation and thoughts of harming the baby.³ PPD results in a wide range of negative outcomes for mothers, children, and the mother–child relationship.^4–6

Longer and more severe PPD episodes are linked to poorer outcomes for infants and children highlighting the importance of early diagnosis and treatment.⁶ Untreated PPD has been shown to result in poorer outcomes for mothers when compared with treated mothers.^7–9 Despite the importance of early identification and treatment, PPD often remains largely undiagnosed¹⁰ with symptoms unreported by patients.^11–13

In addition to research that has identified factors related to diagnosis of PPD,^5,14–16 several factors have been identified as prompting help-seeking.^17–28 Demographic factors such as income, education, and race or ethnicity can function as facilitators or barriers to care. Pregnant people with lower incomes may not be able to afford care because of lack of insurance coverage, less paid leave from work, childcare coverage, or transportation costs.^17,23–26 Race and ethnicity are associated with social barriers such as lack of trust in the healthcare system, lack of culturally sensitive care, or cultural beliefs that stigmatize help-seeking.^{19,20,23–28} Education and mental health literacy are also germane. If a person or a family member has sought help in the past, they usually have a better understanding of symptoms and the benefits of mental health services and how to access them.^17–27

A 2009 study used Pregnancy Risk Assessment Monitoring System (PRAMS) data from Utah to examine factors associated with help-seeking among 337 people who had PPD symptoms.¹⁹ They report that help-seeking during pregnancy, living in a rural setting, and having an infant in neonatal intensive care were associated with postpartum help-seeking. Variables that decreased likelihood of help-seeking included non-White race, Hispanic ethnicity, and reporting experiencing emotional abuse.

More recently, a 2020 analysis was conducted using PRAMS data from New York City to explore help-seeking among 618 people who had PPD symptoms.²⁰ They found help-seeking was associated with a mental health history, being an Asian Pacific Islander, and having visited a health care provider for chronic illness.

No research has examined help-seeking for PPD using data from a multi-state/jurisdictional survey analyzed with machine learning techniques to address the many potential factors that may influence help-seeking. However, prior work, using electronic health records,^29–31 survey data,^32–36 and social media,³⁷ has demonstrated the utility of machine learning techniques to create algorithms to assist with PPD screening and prediction. For example, Natarajan et al.³² used survey data and discovered that a large number of weak predictors interact to strongly predict presence of PPD symptoms—demonstrating that PPD symptoms can be predicted from sociodemographic survey data. A recent study by Shin et al.³³ demonstrated that PPD symptoms can be predicted from a large, national survey using machine learning techniques. Using data from the 2012–2013 PRAMS data set, they developed nine machine learning models and identified a set of common variables across all the models. Machine learning algorithms may be useful to predict outcomes beyond PPD screening, such as help-seeking.³⁸ Calls have been made to address psychiatric topics in ways that can integrate the workings of the brain with psychosocial influences, and data-driven machine learning can provide one way to address this complexity.³⁹ Traditional analysis methods may be unable to adequately model and address the complexity of factors that influence behaviors around topics such as help-seeking for PPD. Identifying the key variables related to help-seeking can provide important information related to clinical practices as well as prevention efforts.

In this study, we use data from a multi-state/jurisdictional survey matched with birth certificate data to assess a wide range of risk factors to identify the most important variables to predict help-seeking behavior for participants with PPD symptoms.

Method

Measures

This study used cross-sectional data from a sample of individuals from six US states/jurisdictions with a recent birth (2–4 months postpartum) who participated in the 2016–2018/Phase 8 data collection period of the Centers for Disease Control and Prevention (CDC)-sponsored PRAMS.⁴⁰ The PRAMS data set shared with researchers includes a set of core (required for all participating states/jurisdictions) self-report questions,⁴¹ standard (optional),⁴² and state-specific questions and linked birth certificate data.⁴³ Multiple items and outcomes contained in the PRAMS questionnaires have demonstrated acceptable validity evidence, including PPD measures;⁴⁴ participation in Special Supplemental Nutrition Program for Women, Infants, and Children (WIC); use of Medicaid to pay for delivery; and breastfeeding initiation.⁴⁵

Some of the questions from the questionnaire and/or birth certificate data used in this study include race; ethnicity; smoking or drinking before, during, or after pregnancy; maternal weight gain; prenatal and postnatal care; complications during pregnancy; delivery experiences; pregnancy outcomes (e.g. birth weight, gestational age at delivery, length of hospital stay); insurance status before, during, and after pregnancy; and maternal mental health before, during, and after pregnancy. The outcome, help-seeking, was based on the following question: “Since your new baby was born, have you asked for help for depression from a doctor, nurse, or other healthcare worker?” and coded “0” for no and “1” for yes.⁴² PPD symptoms were assessed by the PRAMS questionnaire using two items: (1) “Since your new baby was born, how often have you felt down, depressed, or hopeless? Always Often Sometimes Rarely Never” and (2) “Since your new baby was born, how often have you had little interest or little pleasure in doing things you usually enjoyed? Always Often Sometimes Rarely Never.”⁴¹ Individuals who respond “always” and/or “often” for either question are considered to screen positive for PPD.^46–48 A full list of variables and corresponding coding used in the current analysis is contained in the Supplemental Materials, Table 1.

Data are publicly available for researchers and were obtained upon approved application via the CDC’s formalized PRAMS proposal process.⁴⁰ This research was approved by the first author’s university institutional review board under protocol number 20-005 as exempt, secondary research for which consent was not required.

Sample

Using birth certificates, each state/jurisdiction participating in PRAMS provides a random, stratified sample of between 100 and 250 individuals who have given birth in the past month.⁴⁹ Certain populations are oversampled based on the needs of the state/jurisdictions; most oversample for low birth weight deliveries and some create stratified samples based on race, ethnicity, geographic location, or other factors.⁴⁹ The total annual sampling frame for each state/jurisdiction ranges between 1000 and 3000 participants.⁵⁰ Currently, only Connecticut, Nebraska, New Hampshire, New York, Texas, and New York City include the optional, standard item which assesses the help-seeking outcome variable. The full sample was 14,570 individuals who completed the assessment in those locations. Given we are examining help-seeking among individuals with PPD symptoms, our sample was further limited to only include those who screen positive for PPD symptoms in these locations (n = 2054) (see Figure 1). The sample was further reduced to 1920 to only include individuals who (1) had a live birth and whose infant was alive at the time of data collection and (2) those 18 or over at the time of data collection. This final sample represented 13.2% of the total number of individuals who completed the assessment in the six states/jurisdictions.

Figure 1.

Sample selection and PRAMS data preparation steps.

Data analysis

Means, frequencies, and bivariate tests were conducted using the PRAMS complex survey weighting with Stata 15.1.⁵¹ Several steps were followed to clean, preprocess, train, and test the data (Figure 1).

Data preprocessing

Stata version 15.1⁵¹ was used to clean the data and the R recipes package⁵² was used for preprocessing. We started with 551 variables in our data set (see Figure 1). We cleaned our data by removing string variables and variables that had missing data due to skip patterns or were not asked by all six states/jurisdictions included in the analysis. Next, we examined variables with missing data. Most variables had less than 10% missing data, which has been suggested as a cut off for concern with regard to bias.⁵³ Four variables related to paternal factors (race, education, Hispanic ethnicity, and acknowledgment of paternity) and one related to poverty line status had missing data greater than 10%. Further examination revealed these patterns of missingness were not random and were removed from the analysis as it would be inappropriate to impute the missing data.⁵⁴ We then imputed missing data for the remaining variables using k-nearest neighbors, which used all the remaining variables to impute missing values with the k-nearest neighbors algorithm. Following this step, we removed sparse variables with little to no variance. Sparse variables were defined as (1) having a frequency ratio of greater than 95:5 for most prevalent to second most prevalent value, and (2) percent of unique values divided by the sample size below 10. Finally, we tested for multicollinearity, using correlation or Cramer’s V as appropriate, and removed redundant variables that correlated with other variables greater than 0.6. However, some variables with correlations greater than 0.6 were retained based on theory. Our final data set was reduced to 63 variables. Data were separated randomly into training and test data sets, with 70% of the data used to train the model and the remaining 30% used to test model performances.

Training

An ensemble method, Super Learner⁵⁵ in R, was used. Super Learner uses a v-fold cross-validation with a library of algorithms and arrives at a final, best fitting weighted combination of those algorithms.^56,57 We included the following algorithms in the Super Learner ensemble library: stepwise logistic regression, random forest, bagged classification and regression trees (bagged CART), stochastic gradient boosting (GB), conditional random forest, neural networks, model averaged neural network, Bayesian additive regression trees (BART). After training, we tested the Super Learner ensemble on the remaining test data. We examined the area under the receiver operating curve (AUC) for each individual algorithm plus the Super Learner model, which is the combination of the algorithms. AUC is a measure of discrimination between the classes of the outcome and provides a measure of the method’s ability to distinguish between observations with and without the outcome of interest. Relative to other measures such as accuracy, F1 score, and precision, the AUC has been demonstrated to be better in terms of consistency and discrimination even with imbalanced classes⁵⁸ and invariant to class proportion changes.^59,60 We also examined the variable importance for the highest weighted algorithms.

Results

Of the 1920 participants included in the analysis, most were White (59.3%), followed by Black (19.7%) and then Chinese, Japanese, Filipino, or other Asian (11.6%) (see Table 1 for the weighted percentages). Almost a third of participants (30.0%) were Hispanic/Latino (30.0%). The majority were married (55.6%), between the ages of 25 and 34 (52.9%), and had 12 or more years of education (83.6%). A little over half of the participants had their delivery paid for by Medicaid (54.0%). Most (83.5%) attended a maternal postpartum visit. Only 25.3% reported seeking help from a healthcare provider for depression symptoms after the delivery of their baby despite experiencing depression symptoms.

Table 1.

Full sample participant characteristics (n = 1920).

Participant characteristics	Frequency (weighted percentage)
Race
White	914 (59.3)
Black	430 (19.7)
Chinese, Japanese, Filipino, other Asian	354 (11.6)
Native American, Alaska Native, Hawaiian	12 (0.4)
Other non-White	127 (7.5)
Mixed race	79 (1.6)
Missing	4
Hispanic/Latino
Yes	442 (30.0)
Missing	10
Maternal age (years)
18–19	68 (4.7)
20–24	381 (25.3)
25–29	527 (27.0)
30–34	566 (25.9)
35–39	307 (13.9
40+	71 (3.2)
Missing	–
Primary language
English	1750 (88.8)
Spanish	156 (10.6)
Chinese	14 (0.7)
Missing	–
Poverty line status
Under poverty line	602 (37.1)
Missing	221
Marital status
Married	1110 (55.6)
Other	810 (44.4)
Missing	–
Years of education
0–8	85 (4.5)
9–11	192 (11.9)
12	469 (26.8)
13–15	549 (28.3)
16+	621 (28.5)
Missing	4
Payment method for birth
Private insurance	789 (37.8)
Medicaid	1017 (54.0)
Self-pay	49 (3.9)
Other insurance	57 (4.2)
Missing	8
Rural location
Rural	171 (9.7)
Missing	–
Attended maternal postpartum visit
Yes	1656 (83.5)
Missing	13
Sought help from health care provider for PPD symptoms
Yes	565 (25.3)
Missing	–

PPD: postpartum depression.

Table 2 provides the results of the Super Learner model. The highest weighted algorithms, or those that contributed the most to the final model, were conditional random forest and stochastic gradient boosting. The AUC for the overall model was 87.95% with AUC for the algorithms ranging from 84.18% (bagged CART) to 87.52% (BART).

Table 2.

Super Learner weights and AUC’s for full sample (n = 1920).

	Super Learner weight	Area under the curve (AUC) (95% CI)
Final Super Learner model	1.0	87.95 % (84.42%–91.47%
Stepwise logistic regression	0.10	85.23% (81.32%–89.13%)
Random forest	0.16	87.16% (83.57%–90.75%)
Bagged CART	0.00	84.18% (80.08%–88.27%)
Stochastic GB	0.22	87.22% (83.52%–90.92%)
Conditional random forest	0.23	87.35% (83.72%–90.98%)
Neural networks	0.00	85.14% (81.26%–89.02%)
Model averaged neural network	0.10	85.98% (82.23%–89.72%)
BART	0.18	87.52% (83.84%–91.19%)

CI: confidence interval; CART: classification and regression trees; GB: gradient boosting; BART: Bayesian additive regression trees.

The variables with the 10 highest importance or coefficient estimates were examined for the highest weighted models (conditional random forest and stochastic gradient boosting) as reported in Table 3. The variables are listed in order of overall importance in the models and weighted frequencies and percentages (or means and standard deviations for interval-level data) are reported for each variable by help-seeking status. Although there was some variability in the variable rankings, receiving a PPD diagnosis was the most important variable in both algorithms in predicting help-seeking. Only 17.4% of participants who did not report receiving a PPD diagnosis sought help for PPD compared to 61.2% of those who had received a PPD diagnosis. Similar results were found for depression during pregnancy—those who reported having been told they were depressed during pregnancy were much more likely to seek help for PPD than those not experiencing depression during pregnancy (54.6% versus 13.8%). Interestingly, maternal body mass index (BMI) when the questionnaire was completed also appeared as a top predictor, with those with higher BMI more likely to seek help. Other variables ranking in the top 10 importance/size across the three algorithms included (1) the state in which the mother lives (those in Connecticut, Nebraska, and New Hampshire were most likely to seek help), (2) maternal race (white individuals, Native American people, and those reporting mixed race were most likely to seek help), and (3) depression 3 months prior to pregnancy.

Table 3.

Full sample (n = 1920) selected algorithms’ top 10 important variables: means/frequencies, bivariate tests, and algorithm rankings.

Variables	Weighted frequencies (percentages) or means (standard deviations)		Weighted test statistic and p value	Conditional random forest ranking	Stochastic GB ranking
Variables	Help-seeking	Help-seeking	Weighted test statistic and p value	Conditional random forest ranking	Stochastic GB ranking
Told had depression since baby was born	No	Yes	F(1, 1826) = 120.5, p < 0.001	1	1
No	1154 (82.6%)	138 (17.4%)
Yes	86 (38.9%)	374 (61.2%)
Depression during pregnancy			F(1, 1865) = 120.4; p < 0.001	2	4
No	121 (86.2%)	243 (13.8%)
Yes	192 (45.4%)	309 (54.6%)
Maternal body mass index when questionnaire was completed, means (linearized standard error)	26.2 (0.30)	28.0 (0.6)	OR = 1.3 (95% CI: 1.01–1.06)	7	2
State			F(3, 5604) = 7.8; p < 0.001	3	3
Connecticut	363 (61.2%)	170 (38.8%)
Nebraska	113 (60.6%)	49 (39.4%)
New Hampshire	85 (53.6%)	85 (46.5%)
New York	147 (70.9%)	85 (29.1%)
Texas	182 (74.6%)	81 (25.4%)
New York City	465 (83.3%)	95 (16.7%)
Maternal race			F(4, 8255) = 4.9; p < 0.001	4	5
White	571 (70.4%)	343 (29.7%)
Black	328 (79.1%)	102 (20.9%)
Chinese, Japanese, Filipino, Other Asian	298 (88.0%)	56 (12.1%)
Native American, Alaska Native, Hawaiian	7 (24.1%)	5 (75.9%)
Other non-White	97 (82.9%)	30 (17.1%)
Mixed race	52 (59.3%)	27 (40.8%)
Attended postpartum checkup for self			F(1, 1868) = 11.9; p < 0.001	5	–
No	196 (87.6%)	55 (12.4%)
Yes	1148 (72.3%)	508 (27.7%)
Attended pre-pregnancy health care visit			F(1, 1846) = 9.02; p = 0.003	6	–
No	540 (81.5%)	139 (18.5%)
Yes	788 (70.1%)	418 (29.2%)
Infant age when questionnaire was completed, means (linearized standard error)	122.6 (1.5)	120.1 (2.3)	OR = 1.0 (95% CI: 0.99–1.0)	–	6
Maternal weight gain during pregnancy, means (linearized standard error)	28.7 (0.6)	30.2 (1.3)	OR = 1.01 (95% CI: 0.99–1.02)		7
Method of payment for birth			F(3, 5413) = 0.5; p = 0.7	8
Private insurance	544 (73.7%)	245 (27.3%)
Medicaid	734 (75.3%)	283 (24.7%)
Self-pay	35 (83.8%)	14 (16.2%)
Other insurance	39 (76.8%)	18 (23.3%)
Depression 3 months before pregnancy			F(1, 1847) = 90.4; p < 0.001	10	8
No	1153 (83.8%)	290 (16.2%)
Yes	176 (45.9%)	267 (54.1%)
Current insurance—none			F(1, 1878) = 3.1; p = 0.08	9	–
No	1190 (73.0%)	516 (27.0%)
Yes	162 (81.9%)	49 (18.1%)
Pregnancy intention			^a	–	9
Wanted to be pregnant later	300 (70.3%)	132 (27.7%)
Wanted to be pregnant sooner	270 (81.1%)	83 (18.9%)
Wanted to be pregnant then	457 (81.9%)	174 (18.1%)
Didn’t want to be pregnant then or at any time in the future	87 (64.2%)	57 (35.8%)
Wasn’t sure what wanted	215 (63.7%)	110 (36.4%)
Clinical estimate of gestational age at birth (weeks)			F(2.12, 3972.40) = 1.2508; p = 0.29	–	10
Less than 27	24 (62.0%)	15 (38.0%)
28–33	80 (65.8%)	47 (34.2%)
34–36	152 (72.3%)	81 (27.8%)
37–42	1097 (75.1%)	564 (24.9%)

GB: gradient boosting; OR: odds ratio; CI: confidence interval.

Bold indicates top 10 variables across all three algorithms, italics indicates top 10 variables across at least two algorithms.

Test unable to be calculated because stratum with single sampling unit.

–

Variable did not rank as one of the top 10 important variables in this algorithm.

Sensitivity analysis

Given the high number of participants who attended a postnatal visit for themselves (83.5%), we repeated the analysis with only the individuals who attended the visit. This permitted the inclusion of a previously removed variable (removed due to skip patterns) which asked, “During your postpartum checkup, did a doctor, nurse, or other health care worker do any of the following things . . . Ask me if I was feeling down or depressed.” We included this question as a proxy for PPD screening to examine whether it was related to help-seeking. This new model included 1656 cases with 503 (27.7%) individuals reporting help-seeking. The highest weighted algorithms for this model were neural networks and random forest. The AUCs for the overall model and the individual algorithms (Table 4) were similar to those reported in the full sample (see Table 2). The AUC for the overall model was 87.08% with the AUC for the algorithms ranging from 84.66% (random forest) to 88.96% (stepwise logistic regression).

Table 4.

Super Learner weights and AUCs for postpartum visit attendees only sample (n = 1656).

	Super Learner weight	Area under the curve (AUC) (95% CI)
Final Super Learner model	1.0	87.08% (83.27%–90.88%)
Stepwise logistic regression	0.12	88.96% (85.57%–92.36%)
Random forest	0.17	84.66% (80.44%–88.89%)
Bagged CART	0.00	84.98% (80.88%–89.07%)
Stochastic GB	0.14	87.31% (83.63%–91%)
Conditional random forest	0.14	86.52% (82.51%–90.53%)
Neural networks	0.19	86.01% (82.22%–89.8%)
Model averaged neural network	0.11	85.62% (81.65%–89.58%)
BART	0.13	88.38% (84.89%–91.88%)

CI: confidence interval; CART: classification and regression trees; GB: gradient boosting; BART: Bayesian additive regression trees.

The variables with the highest importance or coefficient estimates were examined for the highest weighted models (neural network and random forest) (see Table 5 for ranking of top 10 variables and survey weighted bivariate analyses).

Table 5.

Postpartum visit attendees only sample (n = 1656) selected algorithms’ top 10 important variables: means/frequencies, bivariate tests, and algorithm rankings.

Variables	Weighted frequencies (percentages) or means (standard deviations)		Bivariate test statistic and p value	Neural network ranking	Random forest ranking
Variables	Help-seeking	Help-seeking	Bivariate test statistic and p value	Neural network ranking	Random forest ranking
Told had depression since baby was born	No	Yes	F(1, 1488) = 250.8; p < 0.001	1	1
No	984 (90.9 %)	121 (9.1%)
Yes	72 (20.7%)	345 (79.3%)
Depression during pregnancy			F(1, 1573) = 123.4; p < 0.001	2	7
No	981 (85.2%)	225 (14.8%)
Yes	133 (38.5%)	273 (61.5%)
Maternal body mass index when questionnaire was completed, means (linearized standard error)	26.3 (0.3)	28.4 (0.7)	OR = 1.04 (95% CI: 1.01–1.07)	–	2
Month before pregnancy, insurance paid by parent			F(1, 1615) = 0.41; p = 0.70	3	–
No	1097 (72.4%)	472 (27.7%)
Yes	49 (69.5%)	36 (30.5%)
Maternal weight gain during pregnancy, means (linearized standard error)	28.7 (0.67)	30.3 (1.37)	OR = 1.00 (95% CI: 0.99–1.02)	–	3
Language			F(1.2, 1943.1) = 6.65; p < 0.01	4	–
English	1039 (70.7%)	477 (29.4%)
Spanish	96 (85.5%)	30 (14.5%)
Chinese	13 (98.1%)	1 (2.0%)
Infant age when questionnaire was completed, means (linearized standard error)	122.7 (1.6)	119.0 (34.0)	OR = 1.00 (95% CI: 0.99 –1.00)	–	4
First trimester of prenatal visit			F(2.5, 3983.8) = 1.63; p = 0.19	5	–
No prenatal care visit	14 (79.5%)	4 (20.5%)
First trimester	961 (71.0%)	445 (29.2%)
Second trimester	128 (82.3%)	43 (17.7%)
Third trimester	8 (66.5%)	5 (33.5%)
State			F(3.0, 4799) = 6.9; p < 0.001	8	5
Connecticut	324 (59.1%)	324 (40.9%)
Nebraska	91 (60.4%)	38 (39.6%)
New Hampshire	72 (52.8%)	77 (47.2%)
New York	129 (70.0%)	75 (30.1%)
Texas	129 (70.0%)	72 (30.0%)
New York City	403 (82.3%)	89 (17.7%)
Smoke during pregnancy			F(1, 1490) = 36.80; p < 0.001	6	–
No	1014 (74.8%)	408 (25.2%)
Yes	46 (30.8%)	61 (69.2%)
Maternal race			F(4.5, 7279) = 6.9; p < 0.001	7	6
White	375 (66.0%)	316 (34.0%)
Black	227 (79.2%)	89 (20.8%)
Chinese, Japanese, Filipino, Other Asian	267 (88.2%)	48 (11.8%)
Native American, Alaska Native, Hawaiian	6 (23.1%)	4 (76.9%)
Other non-White	78 (83.7%)	24 (16.3%)
Mixed race	44 (53.5%)	26 (46.5%)
Pregnancy intention			^a	–	8
Wanted to be pregnant later	252 (70.3%)	122 (30.0%)
Wanted to be pregnant sooner	243 (81.1%)	77 (18.9%)
Wanted to be pregnant then	394 (79.6%)	162 (20.4%)
Didn’t want to be pregnant then or at any time in the future	66 (56.0%)	43 (44.0%)
Wasn’t sure what wanted	169 (58.3%)	96 (41.7%)
Method of payment for birth			F(2.9, 4626.5) = 0.06; p = 0.98	9	–
Private insurance	500 (71.8%)	233 (28.2%)
Medicaid	584 (72.3%)	246 (27.7%)
Self-pay	29 (74.2%)	11 (25.8%)
Other insurance	33 (75.7%)	13 (24.3%)
Maternal age			F(4.6, 7434.2) = 0.83; p = 0.52	–	9
18–19	37 (68.0%)	20 (32.0%)
20–24	197 (68.1%)	117 (31.9%)
25–29	303 (72.3%)	137 (27.7%)
30–34	374 (76.3%)	135 (23.7%)
35–39	192 (70.3%)	84 (29.7%)
40 +	45 (82.3%)	15 (17.6%)
Used electronic cigarettes in past 2 years			F(1, 1592) = 3.83; p = 0.05	10
No	1081 (73.3%)	445 (26.7%)
Yes	49 (58.0%)	56 (42.0%)
Number of weeks breastfeeding			F(3.6, 5632.2) = 1.94; p = 0.11	–	10
Never breastfed, or breastfed less than 1 week	145 (68.8%)	88 (31.2%)
Breastfeeding 1–8 weeks	173 (66.8%)	106 (33.2%)
Breastfeeding 9–16 weeks	150 (69.2%)	66 (30.8%)
Breastfeeding 17 or more weeks	40 (86.6%)	8 (13.4%)
Breastfeeding currently	605 (77.1%)	223 (22.9%)
Asked by health care provider if they are “depressed or down”^b			F(1, 1584) = 22.3; p < 0.001	–	–
No	236 (88.4%)	35 (11.6%)
Yes	884 (68.0%)	468 (32.0%)

OR: odds ratio; CI: confidence interval.

Bold indicates top 10 variables in both algorithms.

Test unable to be calculated because stratum with single sampling unit.

This analysis permitted inclusion of additional variable which assesses if participant was asked by health care provider if they are “depressed or down.” This variable was included in this list to serve as a proxy for “PPD screening” to examine relationship with help-seeking.

–

Variable did not rank as one of the top 10 important variables in this algorithm.

Similar to the full data analysis, receiving a PPD diagnosis was the most important variable in both algorithms in predicting help-seeking. Three other variables were among the most important in both algorithms in predicting help-seeking: (1) depression during pregnancy, (2) the state in which the mother lives, and (3) maternal race. Bivariate trends for these variables were similar as to those for the full data set. Finally, while having been asked whether they were down or depressed was significantly related to help-seeking, this was not among the most variables in predicting help-seeking.

Discussion

The current research answers important questions about help-seeking among patients with PPD symptoms. First, while all 1920 participants screened positive for PPD symptoms, only 25.3% of participants sought help from a health care provider for PPD symptoms. Next, this research demonstrated the utility of an ensemble machine learning algorithm, Super Learner, to develop a best fitting model weighted by algorithm to identify the key variables associated with help-seeking. The AUCs for the overall model and the individual algorithms were high (87.95 % for the overall model). The most important variables were receiving a PPD diagnosis, depression during pregnancy, maternal BMI at the time the survey was completed, the state in which the mother lives, maternal race, and depression 3 months prior to pregnancy. When we reran the analysis with only those who attended a postpartum health care visit for themselves, the AUC remained high (87.08% for overall model). The most important variables in these algorithms were similar to the previous analysis, with variables related to depression history among the most important. Although most participants reported having been asked whether they were down or depressed (80.4%) and this variable had a significant bivariate relationship with help-seeking, this was not a top 10 variable in predicting help-seeking.

Similar to previous research with help-seeking among those with PPD, in this study, variables related to personal depression history were also among the most important predictors for help-seeking for PPD symptoms.^{19,20,24–27} Factors such as low mental health literacy, negative attitudes toward help-seeking, and low emotional clarity have all been demonstrated to be related to lack of personal and family experience with depression.^{18,21,24–27} Increased levels of mental health literacy have been found to be related to more positive attitudes toward professional mental health help-seeking among recently pregnant individuals.²⁷ Recent research suggests that recently pregnant individuals and the public have misperceptions regarding perinatal mental health disorders in terms of symptoms, severity, risk, difference from “the baby blues,” and treatment, and these misperceptions can influence help-seeking.²⁴ For example, believing perinatal mental health disorders are the same as “baby blues” can minimize the experience and discourage help-seeking. This research also suggests that most individuals experiencing perinatal mental health disorders prefer to seek informal help through friends and family believing it is effective treatment, rather than seek formal treatment, which is often necessary.²⁴ The current research reinforces the need to promote PPD education among individuals, families, and the public.

Also similar to previous research, we found that maternal race was an important predictor for help-seeking,^{19,20,23,25,26} with lowest help-seeking for Asian American people followed by Black individuals. Cultural differences have a large impact on the manifestation of mental health concerns, help-seeking, provider interpretation of symptoms, and the relationship formed between the provider and patient.⁶¹ Indeed, research has consistently demonstrated that mental health help-seeking is low among Asian American individuals.^23,62,63 Some reasons for this disparity include stigma and discrimination,⁶⁴ negative attitudes toward mental health services,⁶⁴ preference for informal (e.g. friends and family) rather than formal assistance (e.g. health care provider),⁶⁵ and internalized stereotypes about the experience of mental health concerns among those who are Asian American.⁶⁶ In addition, barriers to help-seeking and mental health utilization include a lack of culturally and ethnically relevant mental health services and providers.^23,65 Help-seeking may be lower among Black individuals experiencing PPD symptoms due to stigma related to depression,^61,67 a lack of trust for providers as a result of historical oppression and trauma,^61,68 as well as systematic discrimination and racism.⁶¹ However, the impact of racism on help-seeking is unclear, as Bossick and others found that emotional upset due to racism was associated with greater PPD symptoms but not with decreased help-seeking.²⁸

As noted above, when analyses were replicated with only those who attended a postpartum maternal health provider visit, depression before, during, and after pregnancy remained among the most important variables for predicting help-seeking. This revised analysis also permitted the inclusion of the variable “Asked by health care provider if they are depressed or down” that can serve as a proxy for PPD screening. Although reporting having been asked whether depressed or down demonstrated a significant positive relationship with help-seeking, interestingly, it was not among the most important predictors. While not a top predictor for help-seeking behavior in the current research, research demonstrates screening for PPD is critical, as it has shown an absolute risk reduction in PPD prevalence ranging between 2% and 9%.⁶⁹ Recent efforts in the United States to increase screening for PPD include Medicaid coverage of maternal screening during child well visits. This represents an important step in increasing maternal screening since Medicaid currently covers nearly half of all pregnancies in the United States.⁷⁰ Under Medicaid fee-for-service policies, currently, 43 states and Washington D.C. recommend, require, or allow coverage for screening;⁷¹ with only 8 of those states requiring PPD screening. In addition, some governing bodies specify screening should occur a minimum of one point in time,⁷² while others such as the American Academy of Pediatrics suggest a more robust PPD screening schedule at 1, 2, 4, and 6 months during well-child visits.⁷³ More robust screening schedules provide additional opportunities to catch PPD symptoms when they are occurring.

Ensuring appropriate screening is only the first step to increase help-seeking for PPD symptoms. The provider response to screening is critical, as is the relationship between the provider and patient, and provider attitude and response to the individual’s disclosure.^74–76 For example, feeling like one’s concerns are being dismissed or minimized will prevent individuals from further disclosure, while feeling encouraged or taken seriously will increase comfort and willingness to share. Having established and supportive relationships with providers will also facilitate help-seeking.⁷⁵ Furthermore, provider outreach and follow-up is critical to increase patient help-seeking and treatment initiation.⁷⁵

Benefits and limitations of machine learning ensemble methods

This analysis also demonstrated the utility of machine learning ensemble methods with data sets with many variables. These data-driven methods can successfully address the nonlinear and complex relationships that influence mental health-related behaviors like help-seeking.³⁹ Machine learning models, such as random and conditional forest, are effective at handling multidimensional and complex data.^77,78 Ensemble models typically include techniques that are nonlinear and nonparametric and therefore are flexible and robust to many different data types, yet can still find good fitting models.⁷⁸ Instead of using a single model or method, we can improve model accuracy by gathering models into ensembles; the Super Learner improves model accuracy by combining several algorithms into a weighted average resulting in typically better performance than any single model.^57,78,79 A strength and recommendation for implementing the Super Learner is to include a mix of various algorithms in the Super Learner and to evaluate not only the final model but each algorithm.^57,79 Using a mix of algorithms in the Super Learner allows for confidence in overall performance since even though an individual algorithm may do well on some observations’ predictions, the individual algorithm’s overall performance could be worse than the Super Learner’s.⁵⁷

Furthermore, machine learning models, including those that use ensemble methods, are good at creating models that can generalize to new data. Machine learning techniques include the random division of the data set into a training data set to develop the model and then a test data set to test the model with new data. If the pool of data is small, data splitting is a critical decision as a small test set’s ability to judge performance is limited.⁶⁰ However, for large data sets, the size of training and test sets is less critical. A typical split in the data might be 80% for training and 20% for testing.⁶⁰ Vabalas et al.⁸⁰ showed that the train/test split approach produces robust and unbiased performance estimates regardless of sample size. However, the higher the ratio of predictors to the number of observations, the more likely the machine learning algorithm will overfit the data and lead to poor performance on test data.⁸⁰ In our analysis, since we have a relatively low number of predictors, our use of the 70% training and 30% testing split is justified. In addition, our test data set did a good job with prediction (e.g. AUCs in range of 0.8), suggesting our model would also predict well to a new data set that contained similar variables.⁵⁹ Finally, the Super Learner is also adaptive and robust for small samples while controlling for overfitting.⁵⁷

There are, however, limitations associated with ensemble methods. Ensemble models are more complex and can often lack of interpretability at the expense of having a very accurate predictive model.⁷⁸ Compared to single-method models such as linear regression or random forest, more accurate prediction comes at the expense of model complexity; it is more challenging to determine the exact nature of relationships or to see which variables are important.^78,79 This necessitates examination of bivariate relationships and comparison of variable importance across the different methods to help interpret relationships and findings. In addition, ensemble methods including the Super Learner are computationally intense requiring greater computing power and time compared to more traditional or single algorithm methods.^78,79

Limitations and future research

This research is not without limitations. The item assessing the outcome, help-seeking, was only included in six states/jurisdictions, hindering our ability to generalize to the rest of the United States or other countries. After limiting our sample to those with PPD symptoms and other inclusion criteria (e.g. live birth/live infant during survey and participant is 18 or over), our sample comprised 13.2% of the total sample, which matches the current population prevalence estimates of PPD.¹ Given the random sampling methods used by PRAMS, our findings should be representative for the six locations where this topic is assessed, hence providing the most comprehensive possible examination of this topic in the United States. This limitation of our research also highlights the need for greater systematic data collection and reporting on this topic.

An additional limitation includes the loss of variables during data preprocessing including variables with low variability, items that were missing as a result of skip logic, or because they were not asked by all locations. These items may have important implications for help-seeking among participants with PPD but could not be assessed in this study. Future research may examine these potentially related topics using smaller samples that have asked the relevant questions or are not missing data due to skip logic.

In addition, while much of the PRAMS data are based on birth certificate information, in this study, the outcome was a single, self-reported item from the questionnaire. As such, the outcome is subject to the limitations associated with self-report data including recall bias, or the inaccurate or incomplete memory of an experience, which may have influenced the quality of the data reported in this study.⁸¹ However, maternal recall of pregnancy-related behaviors, pregnancy outcomes, and postpartum behaviors such as breastfeeding have been found to be highly accurate when compared to patient charts, even as much as 30 years postpartum.^82–84

In this study, among those who attended a postpartum visit for themselves, we do not know when their postpartum visit occurred, how many times the participant may have been asked about their own depression, nor when their present symptoms may have started. While PPD symptom onset typically occurs in early postpartum, or within 4 weeks of birth, late symptom onset (weeks 5–12 postpartum) can occur.^85,86 Consequently, it is possible that participants’ symptoms may have started after their provider visit. Future research should replicate and confirm the results of this study using medical records rather than self-reported data.

Finally, this research was conducted using data gathered just prior to the start of the COVID-19 pandemic. Compared to before the pandemic, the prevalence of PPD symptoms has been generally found to have increased, with estimates ranging from 7% to 80.8% among recently pregnant people.⁸⁷ Risk factors such as lockdown, isolation, job loss, reduced access to healthcare, loss of childcare services, and fear of the baby contracting COVID-19 generally contributed to the increase in symptoms.⁸⁷ While there has been an increase in need for mental health services during the pandemic,⁸⁷ research has just started to examine the prevalence of help-seeking among individuals with PPD symptoms during and post-pandemic.⁸⁸ Qualitative work suggests that reduced access and concerns about in-person appointments may have prevented in-person help-seeking for maternal mental health concerns and virtual alternatives were unable to adequately fulfill these needs.⁸⁹ It will be important to replicate this research using data collected after the start of the pandemic.

Conclusion

This study demonstrates that machine learning techniques can assist providers in identifying individuals who might be hesitant to ask for help for PPD symptoms. Specifically, findings suggest that having a history of depression before or during pregnancy increases the likelihood of asking for help from health care providers for depression and supports previous research which proposes that people of color may be less likely to seek help for PPD. Healthcare providers should be cognizant of these factors when screening for PPD and conducting outreach and follow-up for PPD symptoms.

Supplemental Material

sj-docx-1-whe-10.1177_17455057221139664 – Supplemental material for Using machine learning to predict help-seeking among 2016–2018 Pregnancy Risk Assessment Monitoring System participants with postpartum depression symptoms

Supplemental material, sj-docx-1-whe-10.1177_17455057221139664 for Using machine learning to predict help-seeking among 2016–2018 Pregnancy Risk Assessment Monitoring System participants with postpartum depression symptoms by Rebecca Fischbein, Heather L Cook, Kristin Baughman and Sebastián R Díaz in Women’s Health

Footnotes

Acknowledgements

This research would not be possible without the PRAMS Working Group, Centers for Disease Control and Prevention (CDC), as well as the PRAMS participants.

Declarations

ORCID iDs

Rebecca Fischbein

Heather L Cook

Supplemental material

Supplemental material for this article is available online.

References

Hahn-Holbrook

Cornwell-Hinrichs

Anaya

Economic and health predictors of national postpartum depression prevalence: a systematic review, meta-analysis, and meta-regression of 291 studies from 56 countries. Front Psychiatry 2017; 8: 248.

American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th ed. Washington, DC: American Psychiatric Association.

Stewart

Vigod

Postpartum depression. N Engl J Med 2016; 375: 2177–2186.

Slomian

Honvo

Emonts

, et al. Consequences of maternal postpartum depression: a systematic review of maternal and infant outcomes. Womens Health 2019; 15: 1745506519844044.

O’Hara

McCabe

JE.

Postpartum depression: current status and future directions. Annu Rev Clin Psychol 2013; 9: 379–407.

Josefsson

Sydsjö

A follow-up study of postpartum depressed women: recurrent maternal depressive symptoms and child behavior after four years. Arch Womens Ment Health 2007; 10(4): 141–145.

Vliegen

Casalin

Luyten

The course of postpartum depression: a review of longitudinal studies. Harv Rev Psychiatry 2014; 22(1): 1–22.

Xiao

Kroll-Desrosiers

Goldberg

, et al. The impact of sleep, stress, and depression on postpartum weight retention: a systematic review. J Psychosom Res 2014; 77(5): 351–358.

Dias

Figueiredo

Breastfeeding and depression: a systematic review of the literature. J Affect Disord 2015; 171: 142–154.

10.

Beck

CT.

Postpartum depression: it isn’t just the blues. Am J Nurs 2006; 106(5): 40–50. Quiz 50–51.

11.

Dennis

Chung-Lee

Postpartum depression help-seeking barriers and maternal treatment preferences: a qualitative systematic review. Birth 2006; 33(4): 323–331.

12.

Buist

Speelman

Hayes

, et al. Impact of education on women with perinatal depression. J Psychosom Obstet Gynaecol 2007; 28(1): 49–54.

13.

Byatt

Biebel

Friedman

, et al. Women’s perspectives on postpartum depression screening in pediatric settings: a preliminary study. Arch Womens Ment Health 2013; 16(5): 429–432.

14.

Zhao

Zhang

ZH.

Risk factors for postpartum depression: an evidence-based systematic review of systematic reviews and meta-analyses. Asian J Psychiatr 2020; 53: 102353.

15.

Ghaedrahmati

Kazemi

Kheirabadi

, et al. Postpartum depression risk factors: a narrative review. J Educ Health Promot 2017; 6: 60.

16.

Hutchens

Kearney

Risk factors for postpartum depression: an umbrella review. J Midwifery Womens Health 2020; 65: 96–108.

17.

Bina

Glasser

Factors associated with attitudes toward seeking mental health treatment postpartum. Women Health 2019; 59(1): 1–12.

18.

Fonseca

Gorayeb

Canavarro

MC.

Women’s help-seeking behaviours for depressive symptoms during the perinatal period: Socio-demographic and clinical correlates and perceived barriers to seeking professional help. Midwifery 2015; 31(12): 1177–1185.

19.

McGarry

Kim

Sheng

, et al. Postpartum depression and help-seeking behavior. J Midwifery Womens Health 2009; 54: 50–56.

20.

Manso -Córdoba

Pickering

Ortega

, et al. Factors related to seeking help for postpartum depression: a secondary analysis of New York city PRAMS data. Int J Environ Res Public Health 2020; 17: E9328.

21.

Fonseca

Silva

Canavarro

MC.

Depression literacy and awareness of psychopathological symptoms during the perinatal period. J Obstet Gynecol Neonatal Nurs 2017; 46(2): 197–208.

22.

Lee-Carbon

Nath

Trevillion

, et al. Mental health service use among pregnant and early postpartum women. Soc Psychiatry Psychiatr Epidemiol 2022; 57: 2229–2240.

23.

Kim

Lee

YJ.

Factors associated with mental health help-seeking among Asian Americans: a systematic review. J Racial Ethn Health Disparities 2022; 9: 1276–1297.

24.

Daehn

Rudolf

Pawils

, et al. Perinatal mental health literacy: knowledge, attitudes, and help-seeking among perinatal women and the public—a systematic review. BMC Pregnancy Childbirth 2022; 22: 574.

25.

Eghaneyan

Murphy

ER.

Mental health help-seeking experiences of Hispanic women in the United States: results from a qualitative interpretive meta-synthesis. Soc Work Public Health 2019; 34(6): 505–518.

26.

Jones

Help seeking in the perinatal period: a review of barriers and facilitators. Soc Work Public Health 2019; 34(7): 596–605.

27.

Jones

Postpartum help-seeking: the role of stigma and mental health literacy. Matern Child Health J 2022; 26(5): 1030–1037.

28.

Bossick

Callegari

, et al. Experiences of racism and postpartum depression symptoms, care-seeking, and diagnosis. Arch Womens Ment Health 2022; 25(4): 717–727.

29.

Wang

Pathak

Zhang

Using electronic health records and machine learning to predict postpartum depression. Stud Health Technol Inform 2019; 264: 888–892.

30.

Zhang

Wang

Hermann

, et al. Development and validation of a machine learning algorithm for predicting the risk of postpartum depression among pregnant women. J Affect Disord 2020; 279: 1–8.

31.

Moreira

MWL

Rodrigues

JJPC

Kumar

, et al. Postpartum depression prediction through pregnancy data analysis for emotion-aware smart systems. Inform Fusion 2019; 47: 23–31.

32.

Natarajan

Prabhakar

Ramanan

, et al. Boosting for postpartum depression prediction. In: 2017 IEEE/ACM international conference on connected health: applications, systems and engineering technologies, Philadelphia, PA, 17–19 July 2017, pp. 232–240. New York: IEEE.

33.

Shin

Lee

Adeluwa

, et al. Machine learning-based predictive modeling of postpartum depression. J Clin Med 2020; 9: 2899.

34.

Zhang

Liu

Silenzio

VMB

, et al. Machine learning models for the prediction of postpartum depression: application and comparison based on a cohort study. JMIR Med Inform 2020; 8: e15516.

35.

Jiménez-Serrano

Tortajada

García-Gómez

JM.

A mobile health application to predict postpartum depression based on machine learning. Telemed J E Health 2015; 21(7): 567–574.

36.

Tortajada

García-Gomez

Vicente

, et al. Prediction of postpartum depression using multilayer perceptrons and pruning. Methods Inf Med 2009; 48(3): 291–298.

37.

De Choudhury

Counts

Horvitz

. Predicting postpartum changes in emotion and behavior via social media. In: CHI conference on human factors in computing systems, Paris, 27 April–2 May 2013, pp. 3267–3276. New York: Association for Computing Machinery.

38.

Lee

Sankar

Hammer

, et al. Using machine learning to classify individuals with alcohol use disorder based on treatment seeking status. Eclinicalmedicine 2019; 12: 70–78.

39.

Rutledge

Chekroud

Huys

QJ.

Machine learning and big data in psychiatry: toward clinical applications. Curr Opin Neurobiol 2019; 55: 152–159.

40.

Centers for Disease Control and Prevention (CDC). Are PRAMS data available to outside researchers? https://www.cdc.gov/prams/prams-data/researchers.htm#data (2022, accessed 24 August 2022).

41.

Centers for Disease Control and Prevention. Pregnancy risk assessment monitoring system (PRAMS) phase 8 core questionnaire, 2016, https://www.cdc.gov/prams/pdf/questionnaire/Phase-8-Core-Questions-508.pdf

42.

Centers for Disease Control and Prevention. Pregnancy risk assessment monitoring system (PRAMS) phase 8 standard questions, 2017, https://www.cdc.gov/prams/pdf/questionnaire/Phase-8-Standard-Core-Questions-508.pdf

43.

Centers for Disease Control and Prevention. PRAMS research dataset codebook, https://www.cdc.gov/prams/prams-data/docs/PRAMS-Research-Dataset-Codebook_tagged508.pdf (2022, accessed 24 August 2022).

44.

Davis

Pearlstein

Stuart

, et al. Analysis of brief screening tools for the detection of postpartum depression: comparisons of the PRAMS 6-item instrument, PHQ-9, and structured interviews. Arch Womens Ment Health 2013; 16(4): 271–277.

45.

Ahluwalia

Helms

Morrow

Assessing the validity and reliability of three indicators self-reported on the pregnancy risk assessment monitoring system survey. Public Health Rep 2013; 128(6): 527–536.

46.

Stone

Diop

Declercq

, et al. Stressful events during pregnancy and postpartum depressive symptoms. J Womens Health 2015; 24: 384–393.

47.

Liu

Giallo

Doan

, et al. Racial and ethnic differences in prenatal life stress and postpartum depression symptoms. Arch Psychiatr Nurs 2016; 30(1): 7–12.

48.

Salm Ward

Kanu

Robb

. Prevalence of stressful life events during pregnancy and its association with postpartum depressive symptoms. Arch Womens Ment Health 2017; 20(1): 161–171.

49.

Centers for Disease Control and Prevention (CDC). PRAMS model protocol 2015 version, 2015, https://www.cdc.gov/prams/methodology.htm

50.

Shulman

D’Angelo

Harrison

, et al. The pregnancy risk assessment monitoring system (PRAMS): overview of design and methodology. Am J Public Health 2018; 108(10): 1305–1313.

51.

StataCorp. Stata statistical software: release 15. College Station, TX: Stata Corp LLC, 2017.

52.

Kuhn

Wickham

Tidymodels: a collection of packages for modeling and machine learning using tidyverse principles, 2020, https://www.tidymodels.org

53.

Bennett

DA.

How can I deal with missing data in my study. Aust N Z J Public Health 2001; 25(5): 464–469.

54.

Dong

Peng

CY.

Principled missing data methods for researchers. Springerplus 2013; 2(1): 222.

55.

Polley

LeDell

Kennedy

, et al. Super Learner prediction, 2019, https://github.com/ecpolley/SuperLearner

56.

van der Laan

Polley

Hubbard

AE.

Super Learner. Stat Appl Genet Mol Biol 2007; 6: 25.

57.

Polley Eric

van der Laan Mark

. SuperLearner in prediction. UC Berkeley Division of Biostatistics working paper series, Working paper no. 266, 3 May 2010, https://biostats.bepress.com/ucbbiostat/paper266

58.

Huang

Ling

CX.

Using AUC and accuracy in evaluating learning algorithms. IEEE Trans Knowl Data Eng 2005; 17: 299–310.

59.

Fawcett

An introduction to ROC analysis. Pattern Recogn Lett 2006; 27: 861–874.

60.

Kuhn

Johnson

Applied predictive modeling. New York: Springer, 2013.

61.

Gopalkrishnan

Cultural diversity and mental health: considerations for policy and practice. Front Public Health 2018; 6: 179.

62.

Yamashiro

Matsuoka

JK.

Help-seeking among Asian and Pacific Americans: a multiperspective analysis. Soc Work 1997; 42(2): 176–186.

63.

Zhang

Snowden

Stanley

Differences between Asian and White Americans’ help seeking and utilization patterns in the Los Angeles area. J Community Psychol 1998; 26: 317–326.

64.

Spencer

Chen

Effect of discrimination on mental health service utilization among Chinese Americans. Am J Public Health 2004; 94(5): 809–814.

65.

Lee

Juon

Martinez

, et al. Model minority at risk: expressed needs of mental health by Asian American young adults. J Community Health 2009; 34(2): 144–152.

66.

Kim

Lee

Internalized model minority myth, Asian values, and help-seeking attitudes among Asian American students. Cultur Divers Ethnic Minor Psychol 2014; 20(1): 98–106.

67.

Amankwaa

LC.

Postpartum depression among African-American women. Issues Ment Health Nurs 2003; 24: 297–316.

68.

Office of the Surgeon General (US), Center for Mental Health Services (US), National Institute of Mental Health (US). Mental health: culture, race, and ethnicity: a supplement to mental health: a report of the surgeon general. Rockville, MD: Substance Abuse and Mental Health Services Administration (US), http://www.ncbi.nlm.nih.gov/books/NBK44243/ (2001, accessed 6 September 2022).

69.

O’Connor

Rossom

Henninger

, et al. Primary care screening for and treatment of depression in pregnant and postpartum women: evidence report and systematic review for the US Preventive Services Task Force. JAMA 2016; 315: 388–406.

70.

Kaiser Family Foundation. Births financed by Medicaid. State Health Facts, https://www.kff.org/medicaid/state-indicator/births-financed-by-medicaid/?currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D

71.

Policy NA for SH. Medicaid policies for maternal depression screening during well-child visits, by state (as of January 2021), https://healthychild.nashp.org/wp-content/uploads/2021/04/Maternal-Depression-Screen-updates-4-1-2021.pdf (January 2021, accessed 5 June 2021).

72.

ACOG committee opinion no. 757: screening for perinatal depression. Obstet Gynecol 2018; 132: e208–e212.

73.

American Academy of Pediatrics. Recommendations for preventive pediatric health care, https://downloads.aap.org/AAP/PDF/periodicity_schedule.pdf (March 2021, accessed 5 June 2021).

74.

Newman

Hirst

Darwin

What enables or prevents women with depressive symptoms seeking help in the postnatal period?

Br J Midwifery 2019; 27: 219–227.

75.

Sword

Busser

Ganann

, et al. Women’s care-seeking experiences after referral for postpartum depression. Qual Health Res 2008; 18(9): 1161–1173.

76.

Bell

Feeley

Hayton

, et al. Barriers and facilitators to the use of mental health services by women with elevated symptoms of depression and their partners. Issues Ment Health Nurs 2016; 37(9): 651–659.

77.

Boulesteix

A-L

Janitza

Kruppa

, et al. Overview of random forest methodology and practical guidance with emphasis on computational biology and bioinformatics: random forests in bioinformatics. Wires Data Mining Knowl Discov 2012; 2: 493–507.

78.

Seni

Elder

JF.

Ensemble methods in data mining: improving accuracy through combining predictions. Cham: Springer, 2010.

79.

Sinisi

Polley

Petersen

, et al. Super learning: an application to the prediction of HIV-1 drug resistance. Stat Appl Genet Mol Biol 2007; 6: 7.

80.

Vabalas

Gowen

Poliakoff

, et al. Machine learning algorithm validation with a limited sample size. PLoS ONE 2019; 14(11): e0224365.

81.

Hassan

Recall bias can be a threat to retrospective and prospective research designs. Int J Epidemiol 2005; 3: 1–11.

82.

Elliott

Desch

Istwan

, et al. The reliability of patient-reported pregnancy outcome data. Popul Health Manag 2010; 13(1): 27–32.

83.

Tomeo

Rich-Edwards

Michels

, et al. Reproducibility and validity of maternal recall of pregnancy-related events. Epidemiology 1999; 10: 774–776.

84.

Troude

L’Hélias

Raison-Boulley

, et al. Perinatal factors reported by mothers: do they agree with medical records. Eur J Epidemiol 2008; 23(8): 557–564.

85.

Mori

Tsuchiya

Matsumoto

, et al. Psychosocial risk factors for postpartum depression and their relation to timing of onset: the Hamamatsu Birth Cohort (HBC) study. J Affect Disord 2011; 135(1–3): 341–346.

86.

Postpartum Depression: Action Towards Causes and Treatment (PACT) Consortium. Heterogeneity of postpartum depression: a latent class analysis. Lancet Psychiatry 2015; 2(1): 59–67.

87.

Usmani

Greca

Javed

, et al. Risk factors for postpartum depression during COVID-19 pandemic: a systematic literature review. J Prim Care Community Health 2021; 12: 21501327211059348.

88.

Wang

Song

, et al. Intentions to seek mental health services during the COVID-19 pandemic among Chinese pregnant women with probable depression or anxiety: cross-sectional, web-based survey study. JMIR Ment Health 2021; 8: e24162.

89.

Jackson

De Pascalis

Harrold

, et al. Postpartum women’s experiences of social and healthcare professional support during the COVID-19 pandemic: a recurrent cross-sectional thematic analysis. Women Birth 2022; 35(5): 511–520.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB