A Community-Based Study of Cognitive Impairment Caregiving Outcomes Pre- and During the COVID-19 Pandemic

Abstract

The impact of the COVID-19 pandemic on informal caregiving was examined in a Mexican American (MA) and Non-Hispanic White (NHW) population-based cohort. 395 participants age > 65 years were recruited via door-to-door and phone recruitment as part of the Brain Attack Surveillance in Corpus Christi-Cognitive (BASIC-C) project. Both recipients and caregivers answered questions regarding the recipient’s health and the COVID-19 pandemic. 15% of caregivers saw their caregiving recipient less than before the pandemic and 18% saw their recipient more than before. 55% of caregivers reported a slight to severe impact of the pandemic on their caregiving, and 45% reported no impact. For most caregivers, their caregiving role did not change markedly during the pandemic. MA and NHW caregivers had similar survey responses.

Keywords

Alzheimer’s/dementia caregiving Alzheimer’s related dementia research COVID-19

What this paper adds

• New perspective on the impact of COVID-19 on older adults

• Potential positive impact on caregiving ability by quarantining

• Examination of the role of ethnicity in informal caregiving in the context of the COVID-19 pandemic

Applications of study findings

• Indications for future research on ethnic disparities in caregiving resources

• Policy development regarding aid to caregivers in pandemic situations

• In the event of future pandemics, resource allocation needs to be targeted toward caregivers as well as communities at high risk

Introduction

From the onset of the COVID-19 pandemic, significant concern existed regarding social isolation of older populations, especially those with cognitive impairment. While isolation for older adults was especially necessary for the prevention of spreading disease, many studies have corroborated that social isolation is positively associated with cognitive decline (Cardona & Andrés, 2023). Alternatively, social support, especially interaction with family and relatives, is associated with improved memory and language function (Ying et al., 2020). The literature suggests that social isolation and lack of social support during the pandemic may be a quick route to worsening of cognitive function (Soysal et al., 2022). In some circumstances, though, older adults and family members who provide informal care began living together during the pandemic, and increased interaction and social support was possible since family members were mostly confined to being in the home (Goyer, 2021). The impact of the pandemic on informal caregivers is also important to note. In addition to the usual pandemic challenges of changes in work and family responsibilities, additional needed care for older loved ones may have significant impact (Paplikar et al., 2022). While this may be beneficial for older adults, reports point out that the pandemic lockdown negatively impacted the mental health of informal caregivers including higher levels of depression and increased caregiver burden (Altieri & Santangelo, 2021; Pongan et al., 2021).

The issue of caregiving roles is relevant especially amongst the Mexican American (MA) community, where nursing home facilities for aging parents are less likely to be utilized, and “familism” is exhibited at a higher rate than their Non-Hispanic White (NHW) counterparts (Crist et al., 2009) (Falzarano et al., 2022). In Latinx culture, familism, the high importance of a bond among members of a family, cements life-long relationships, provides considerable strength in adverse environments and has been shown to be a protective mechanism against stressors (Umaña-Taylor et al., 2011). In various reviews, familism either shows no association with caregiver burden, or lower degrees of burden (Maximiano-Barreto et al., 2022; Tian et al., 2022). This may indicate that in households with a culture of familism, more time spent between caregiver and recipient can be beneficial for the recipient, and less harmful for the caregiver. Consequently, there may be ethnic differences in caregiver outcomes throughout the duration of the pandemic.

The Brain Attack Surveillance in Corpus Christi- Cognitive (BASIC-C) is a longitudinal prospective cohort study investigating informal caregiving at home and cognitive impairment/dementia in aging populations in a MA and NHW community in Corpus Christi, Texas. The purpose of this paper is to report the project’s findings on MA and NHW caregivers’ roles in caring for their loved ones pre- and during the pandemic, including frequency of interaction, self-reported impact on caregiving ability, and financial changes and burden.

We hypothesized that overall, informal caregivers will report significant changes in their caregiving roles in comparison to before the pandemic. Additionally, it is expected that MA caregivers report less significant changes in their caregiving roles, potentially due to cultural differences like familism.

Methods

The methods of the community-based BASIC-C project were previously reported (Briceño et al., 2020). Briefly, households were randomly identified in Nueces County, Texas, USA, a predominantly urban county with residents living in the city of Corpus Christi. The population estimate was 340,233 in 2020, 14.9% of which are age 65 and older. Of the population aged 65 and older, 50.7% are of Hispanic/Latinx ethnicity, and 43.4% are of non-Hispanic White ethnicity (US Census Bureau, 2022).

Participants and Procedure

Recipients of informal care ≥ age 65 were recruited using a random sample door-to-door addresses from 5/1/2018–3/15/2020. Households were identified by random ascertainment using US Census Data. Recruitment continued after the COVID pandemic onset from 4/20/2020–2/11/2022 using a random sample of phone numbers. Individuals with possible cognitive impairment were identified using the Montreal Cognitive Assessment (MoCA) (Nasreddine et al., 2005) during door-to-door recruitment and Telephone Montreal Cognitive Assessment (T-MoCA) (Katz et al., 2021) during phone recruitment. Participants who scored ≤25 and ≤18 on the MoCA or T-MoCA respectively were eligible for participation. These eligible participants are termed “recipients” for the purposes of our studies. Recipients named a non-paid caregiver who provided them with any level of care, with a very broad definition of “caregiving.” Both the recipient and caregiver answered questions about the recipient’s health and demographics. Recipients took a long form Harmonized Cognitive Assessment Protocol cognitive assessment test (Langa et al., 2020), and caregivers answered additional questions about the recipient.

Beginning in May of 2020, we added on a survey to our caregiver interview regarding the COVID-19 pandemic. The survey included three domains, with nine questions in the first domain for a total of 11 total items. These three domains included: “In what ways throughout the duration of the coronavirus pandemic was your work (paid employment, outside of informal caregiving) affected?”; having them rank the extent to which “COVID-19 has impacted my role as a caregiver,” and “Compared to before the COVID-19 pandemic, I see my [recipient] more than before, less than before, or the same.” These questions were generated from surveys such as the Environmental Influences on Child Health Outcomes survey (Blackwell et al., 2018), and the presentation was similar to the Panel Study of Income Dynamics (McGonagle et al., 2012).

Statistical Analysis

Analyses were performed using data from 395 baseline Informants were comprised of MA or NHW caregivers who also completed the 11-item COVID survey between 5/28/20 and 3/31/22 (153 at baseline interview; 242 at follow-up interview). Missing income and education for one caregiver was imputed using SAS Proc MI. All analyses were performed using SAS version 9.4 (SAS Institute Inc).

Inverse propensity weighting (IPW) (Chen, et al., 2020; Elliott & Valliant, 2017) was used to address selection bias due to sample nonresponse. Population weights were created to calibrate MoCA screen sample demographics to 2015–2019 U.S. Bureau of Census population estimates for Nueces County by age, sex, race, ethnicity, and education level (U.S. Bureau of Census, 2022). Additional IPW adjustments were made to account for attrition of eligible dyads and informants who did not complete the COVID survey.

Unweighted descriptive estimates and population weighted estimates and 95% confidence intervals for caregiver demographics were generated along with weighted percentages of their responses to each of the COVID survey items (9 items covered impact of COVID on work and 2 items asked about COVID impact on caregiver roles). To account for sampling and weight variability, Rao-Scott design-adjusted chi-square was used to test for group differences in caregiver responses to COVID survey items. Ethnic group comparisons (MA vs. NHW) to COVID survey items included comparisons stratified by caregiver household income (< $40,000.00 vs. ≥ $40,000.00) and recipient cognitive status (T-MoCA ≤15 vs. T-MoCA >15). Logistic regression was used to estimate the ethnic effect after adjusting for caregiver’s age, gender, income, relationship with the respondent, and respondent’s cognitive status.

Statement of Ethics

Caregiving recipients recruited to be part of the study who have significant cognitive dysfunction gave their written or verbal assent and written or verbal consent was obtained from their caregiver. This study protocol was reviewed and approved by the University of Michigan Institutional Review Board, approval number HUM00133485.

Reasonable requests for data sharing will be considered by the corresponding author based on IRB approvals and existing data use agreements.

Results

This study included 395 recipient and caregiver dyads. The demographics of the caregivers are detailed in Table 1. For NHW caregiver and recipient dyad relationships, 35% of caregivers were a spouse/partner, 47% were the child of the recipient, and the remaining 18% were another family member, friend, or neighbor. For MA relationships, 25% of caregivers were a spouse/partner, 54% were the recipient’s child, and the remaining 21% were another family member, friend, or neighbor. A higher percentage (44%) of NHW caregivers than MA caregivers (28%) were 65 or older.

Table 1.

Caregiver Demographics (n = 395).

	Unweighted		Weighted
	n	% or Mean	% or Mean	95% CL
Age (years)	395	58.9	58.5	57.1–59.8
Gender
Men	87	22.0%	21.5%	17.3–25.8
Women	308	78.0%	78.5%	74.2–82.7
Ethnicity
MA	260	65.8%	64.5%	59.5–69.6
NWH	135	34.2%	35.5%	30.4–40.5
Education
Less than high school	54	13.7%	14.5%	10.7–18.2
High school or more^a	341	86.3%	85.6%	81.8–89.3
Relationship of informant
Spouse/partner	103	26.1%	29.4%	24.5–34.3
Child	213	53.9%	50.7%	45.5–55.9
Other family member	56	14.2%	14.1%	10.4–17.7
Friend/neighbor/other	23	5.8%	5.8%	3.4–8.3
Yearly household income
Under $20,000	66	16.7%	17.2%	13.2–21.2
$20–39,000	87	22.0%	22.4%	18.0–26.8
$40,000 or above	242	61.3%	60.4%	55.3–65.6

Note. MA = Mexican American; NHW = Non-Hispanic White.

^a“high school or more” includes high school graduation, skill or trade school, some college, college, and graduate education.

The first series of COVID-related questions asked caregivers how the pandemic affected their work outside of their unpaid caregiving (Table 2). About 20% of caregivers changed to work remotely upon the onset of the pandemic, and 17% reduced their work hours. About 8% lost their job temporarily and 5% lost their job permanently. Over one quarter of caregivers had jobs that put them at increased risk of getting COVID-19.

Table 2.

Responses to “How Did the COVID-19 Pandemic Affect Your Work” Questionnaire Items.

	n ^a	Percent^b	95% CL
I moved to work remotely or from home
No	268	79.7	[75.2–84.1]
Yes	70	20.3	[15.9–24.8]
I lost my job permanently
No	314	95.0	[92.5–97.5]
Yes	16	5.0	[2.5–7.5]
I lost my job temporarily, or was not told for how long
No	303	91.4	[88.2–94.6]
Yes	27	8.6	[5.4–11.8]
I got a new job
No	307	92.4	[89.2–95.5]
Yes	23	7.6	[4.5–10.8]
I reduced my work hours
No	287	82.6	[78.1–87.0]
Yes	53	17.4	[13.0–21.9]
I increased my work hours
No	278	82.3	[78.0–86.6]
Yes	59	17.7	[13.4–22.0]
My job put me at increased risk of getting COVID-19
No	251	73.7	[68.7–78.7]
Yes	88	26.3	[21.3–31.3]
I laid off employees
No	326	98.7	[97.3–100.0]
Yes	4	1.3	[.0–2.7]
I did not have a paying job before the COVID-19 outbreak
No	207	57.3	[51.9–62.8]
Yes	151	42.7	[37.2–48.1]

^a395 participants included, participants could choose not to answer.

^bWeighted percentages represent population of adults 65 years and older in Nueces County, Texas.

Table 3 details the comparison of time the caregivers spent with their recipient before and after the pandemic. About 15% of caregivers saw their recipient less than before the pandemic. Over two-thirds of caregivers saw their recipient the same amount as they did before, and about 17% saw their recipient more than before.

Table 3.

Responses to Question: “Compared to Before the COVID-19 Pandemic, Do You See the Recipient More Than Before, the Same as Before, or Less Than Before?”

	n	Percent^a	95% CL
More than before	70	17.5	[13.5–21.4]
The same as before	264	67.9	[63.1–72.7]
Less than before	61	14.6	[11.0–18.2]

^aWeighted percentages represent population of adults 65 years and older in Nueces County, Texas.

Table 4 shows estimates based on responses to the question of how much their caregiving role has been impacted by the pandemic on a scale from “Not at all” to “Extremely.” Over 40% of caregivers reported that their caregiving role had not been impacted at all. One fifth reported their role had been “very” or “extremely” impacted, with the remaining approximately 40% reporting a “slight” or “moderate” impact. We also noted a higher percent of child compared to spousal (26% vs. 15%, respectively) or other (26% vs. 14%, respectively) caregivers were more likely to report their caregiving role as “very”/“extremely” impacted (X² (1, N = 395) = 6.7, p < .03).

Table 4.

Responses to Question: “To What Extent has Your Caregiving Role Been Impacted by the COVID-19 Pandemic?”

	n	Percent^a	95% CL
Not at all	168	45.0	[39.8–50.2]
Slightly	85	20.4	[16.3–24.5]
Moderately	58	14.4	[10.8–18.1]
Very	41	10.1	[7.0–13.3]
Extremely	43	10.1	[7.1–13.1]

^aWeighted percentages represent population of adults 65 years and older in Nueces County, Texas.

The impact of COVID on caregiver role by ethnicity and income was also examined. For caregivers with an income of less than $40,000 per year, a higher percentage of NHWs (39.4%) compared to MAs (15.3%) reported very/extreme impact on role as a caregiver (X² (1, N = 395) = 7.57, p < .01). For the impact on work items, for caregivers with a household income of $40,000 or more, a higher percentage of NHWs compared to MAs (26.6% vs. 14.8%, respectively) increased their work hours (X² (1, N = 395) = 4.1, p < .05). These ethnic differences remained statistically significant after accounting for caregiver’s age, gender, income, relationship with the respondent, and respondent’s cognitive status. Additionally, for caregivers with an income of $40,000 per year or more, a higher percentage of NHWs (47.9%) compared to MAs (30.4%) did not have a paying job before COVID (X² (1, N = 395) = 6.2, p < .02). However, this effect was no longer significant after adjusting for caregiver’s age and relationship with the respondent in a multivariate model. No other ethnic differences were detected, with or without adjusting for caregiver’s demographics and respondent’s cognitive status.

Discussion

This study found that 20% of caregivers noted “very” or “extreme” changes in their caregiving role during the pandemic, but the remaining 80% reported little or no change. As we navigate life in a post-COVID world, it is important to consider how pandemics may also influence the lives of those caring for the most vulnerable populations. Examining caregiver burden and impact of COVID on the close relation of older adults should influence how we distribute resources, education, and allocation of vaccines and treatments. For the majority of caregivers in our study sample, however, little changed during the pandemic. In this study community only about 15% of caregivers saw their care recipient less than before the pandemic. The same or more time was spent with caregivers, and a majority of caregivers reported their ability to care informally for their recipient was unchanged. This may suggest protection from social isolation although we did not measure recipient social isolation in this study. About a third of caregivers did indicate some change in the amount of time they interacted with their recipient during the pandemic.

Considering the turmoil of the pandemic, this is less change than we expected. Interaction with others is an established protective mechanism for dementia progression (Ying et al., 2020). There is great concern about the 15% of recipients with AD/ADRD that may have been underserved during the pandemic.

There were not many significant differences in the analysis between NHWs and MAs. The significant findings included that NHWs with limited incomes felt more caregiving strain than MAs. This is likely due to MAs already taking on greater caregiving roles than NHWs prior to the pandemic, and therefore feeling less of a difference upon the start of the pandemic (Crist et al., 2009). Familism may also explain the increased impact felt by NHW compared with MA caregivers. Similarly, NHWs report more increased work hours and lack of a paying job prior to the pandemic, however, we did not consider retirement and unemployment in that question. One possible explanation is that NHW caregivers are more likely to be spouses and thus more likely to be retired. Further, while our data does not show an exacerbation of ethnic disparities, MA caregivers have previously reported higher need for financial resources (Mehdipanah et al., 2022). This in conjunction with the literature supporting that minority populations have suffered a disproportionate impact during the pandemic indicates that two mechanisms of inequity could be working at once (Tai et al., 2021).

There are limitations to this study. One limitation is that recipient’s demographics cannot be paired directly to their caregiver’s demographics. It is also not possible to determine whether recipients and caregivers who lived together during the pandemic were living together prior to the pandemic. Further, the results of the COVID survey have a range of explanations and must be considered in the context of the sample answering the questions. None of the COVID-19 survey questions assessed emotional bonds or burden. Additionally, we did not further analyze the time the survey was completed. It is possible that those who answered the survey in the beginning months of the pandemic may have had more of an impact on caregiving than those who answered in later stages.

Our study overall found 20% of caregivers indicated that the COVID-19 pandemic had a “very” or “extreme” impact on their role as a caregiver. Few other differences in caregiving pre- and post-COVID-19 pandemic onset were noted, compared to what was expected. This may be explained by a number of factors, including cultural values, recipients living in the home with their caregivers, a lack of change in employment status, or imprecision in self-reports from caregivers. These findings indicate that there were less changes to the informal caregiving role than predicted. This is not to disregard those who did report extreme changes in caregiving ability and less time spent with their recipient. Adequate resources and aid should be provided to those whose abilities to provide care were upended by the pandemic.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by NIH grant R01NS100687.

Ethical Statement

ORCID iD

Noreen Khan

References

Altieri

Santangelo

(2021). The psychological impact of COVID-19 pandemic and lockdown on caregivers of people with dementia. American Journal of Geriatric Psychiatry, 29(1), 27–34. https://doi.org/10.1016/j.jagp.2020.10.009

Blackwell

C. K.

Wakschlag

L. S.

Gershon

R. C.

Cella

with the ECHO PRO Core . (2018). Measurement framework for the environmental influences on child health outcomes research program. Current Opinion in Pediatrics, 30(2), 276–284. https://doi.org/10.1097/MOP.0000000000000606

Briceño

E. M.

Mehdipanah

Gonzales

Heeringa

Levine

D. A.

Langa

K. M.

Garcia

Longoria

Morgenstern

L. B.

Morgenstern

L. B.

(2020). Methods and early recruitment of a community-based study of cognitive impairment among Mexican Americans and non-Hispanic whites: The BASIC-cognitive study. Journal of Alzheimer’s Disease, 73(1), 185–196. https://doi.org/10.3233/JAD-190761

Cardona

Andrés

(2023). Are social isolation and loneliness associated with cognitive decline in ageing? Frontiers in Aging Neuroscience, 15(e-Collection 2023), 1075563. https://doi.org/10.3389/fnagi.2023.1075563

Chen

Dong

Qiu

Xin

ChenLin

L.G.

Yang

2021. AutoDebias: Learning to Debias for Recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '21). Association for Computing Machinery, New York, NY, USA, 21–30. https://doi.org/10.1145/3404835.3462919

Crist

J. D.

McEwen

M. M.

Herrera

A. P.

Kim

S. S.

Pasvogel

Hepworth

J. T.

(2009). Caregiving burden, acculturation, familism, and Mexican American elders’ use of home care services. Research and Theory for Nursing Practice, 23(3), 165–180. https://doi.org/10.1891/1541-6577.23.3.165

Elliott

M. R.

Valliant

(2017). Inference for nonprobability samples. Statistical Science, 32(2), 249–264. http://www.jstor.org/stable/26408228

Falzarano

Francesca

Moxley

Jerad

Pillemer

Karl

Czaja

Sara

(2022) (In press). Family Matters: Cross-Cultural Differences in Familism and Caregiving Outcomes. The Journals of Gerontology: Series B, 77(7), 1269–1279. DOI:10.1093/geronb/gbab160.

Goyer

(2021). Family matters: Multigenerational living is on the rise and here to stay. Generations United. https://www.gu.org/resources/multigenerational-families/

10.

Katz

M. J.

Wang

Nester

C. O.

Derby

C. A.

Zimmerman

M. E.

Lipton

R. B.

Sliwinski

M. J.

Rabin

L. A.

Rabin

L. A.

(2021). T‐MoCA: A valid phone screen for cognitive impairment in diverse community samples. Alzheimer’s and Dementia, 13(1), Article e12144. https://doi.org/10.1002/dad2.12144

11.

Langa

K. M.

Ryan

L. H.

McCammon

R. J.

Jones

R. N.

Manly

J. J.

Levine

D. A.

Sonnega

Farron

Weir

D. R.

Weir

D. R.

(2020). The health and retirement study harmonized cognitive assessment protocol project: Study design and methods. Neuroepidemiology, 54(1), 64–74. https://doi.org/10.1159/000503004

12.

Maximiano-Barreto

M. A.

Alves

L. C. d.

Monteiro

D. Q.

Gratão

A. C. M.

Pavarini

S. C. I.

Luchesi

B. M.

Chagas

M. H. N.

(2022). Cultural factors associated with burden in unpaid caregivers of older adults: A systematic review. Health and Social Care in the Community, 30(6), e3629–e3642. https://doi.org/10.1111/hsc.14003

13.

McGonagle

K. A.

Schoeni

R. F.

Sastry

Freedman

V. A.

(2012). The panel study of income dynamics: Overview, recent innovations, and potential for life course research. Longitudinal and Life Course Studies: International Journal, 3(2), 188. https://doi.org/10.14301/llcs.v3i2.188

14.

Mehdipanah

Briceño

E. M.

Gonzales

X. F.

Heeringa

S. G.

Levine

D. A.

Langa

K. M.

Garcia

Longoria

Morgenstern

L. B.

Morgenstern

L. B.

(2022). Dementia care needs for individuals and caregivers among Mexican Americans and non-Hispanic whites. Aging & Mental Health, 26(8), 1630–1641. https://doi.org/10.1080/13607863.2021.1925222

15.

Nasreddine

Z. S.

Phillips

N. A.

Bédirian

Charbonneau

Whitehead

Collin

Cummings

J. L.

Chertkow

(2005). The Montreal cognitive assessment, MoCA: A brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 53(4), 695–699. https://doi.org/10.1111/j.1532-5415.2005.53221.x

16.

Paplikar

Rajagopalan

Alladi

(2022). Care for dementia patients and caregivers amid COVID-19 pandemic. Cerebral Circulation - Cognition and Behavior, 3(3), 100040. https://doi.org/10.1016/j.cccb.2022.100040

17.

Pongan

Dorey

J. M.

Borg

Getenet

J. C.

Bachelet

Lourioux

Laurent

COVCARE Group Rey

Rouch

(2021). COVID-19: Association between increase of behavioral and psychological symptoms of dementia during lockdown and caregivers’ poor mental health. Journal of Alzheimer’s Disease, 80(4), 1713–1721. https://doi.org/10.3233/JAD-201396

18.

Soysal

Smith

Trott

Alexopoulos

Barbagallo

Tan

S. G.

Koyanagi

Shenkin

Veronese

European Society of Geriatric Medicine Special Interest Group in Dementia and Systematic Reviews and Meta-Analyses . (2022). the effects of COVID-19 lockdown on neuropsychiatric symptoms in patients with dementia or mild cognitive impairment: A systematic review and meta-analysis. Psychogeriatrics, 22(3), 402–412. https://doi.org/10.1111/psyg.12810

19.

Tai

D. B. G.

Shah

Doubeni

C. A.

Sia

I. G.

Wieland

M. L.

(2021). The disproportionate impact of COVID-19 on racial and ethnic minorities in the United States. Clinical Infectious Diseases, 72(4), 703–706. https://doi.org/10.1093/cid/ciaa815

20.

Tian

Xiong

Wang

Tao

Zhou

Yan

(2022). Association between familism and mental health in dementia caregivers: A systematic review and meta-analysis. Psychogeriatrics, 22(4), 469–477. https://doi.org/10.1111/psyg.12843

21.

Umaña-Taylor

A. J.

Updegraff

K. A.

Gonzales-Backen

M. A.

(2011). Mexican-origin adolescent mothers’ stressors and psychosocial functioning: Examining ethnic identity affirmation and familism as moderators. Journal of Youth and Adolescence, 40(2), 140–157. https://doi.org/10.1007/s10964-010-9511-z

22.

U.S. Census Bureau . (2022). QuickFacts: Nueces county, Texas. U.S. Census Bureau. https://www.census.gov/quickfacts/nuecescountytexas

23.

Ying

Vonk

J. M. J.

Sol

Brickman

A. M.

Manly

J. J.

Zahodne

L. B.

(2020). Family ties and aging in a multiethnic cohort. Journal of Aging and Health, 32(10), 1464–1474. https://doi.org/10.1177/0898264320935238