Multi-Site Study of Provider Self-Efficacy and Beliefs in Explaining Judgments About Need and Responsibility for Advance Care Planning

Abstract

Background and Objectives:

We examined the impact of advance care planning (ACP) self-efficacy and beliefs in explaining skilled nursing facility (SNF) provider judgments about resident need and provider responsibility for initiating ACP conversations.

Research Design and Methods:

This observational multi-site study of 348 registered nurses, licensed practical nurses, and social workers within 29 SNFs used an anonymous survey in which providers judged vignettes with assigned situational features of a typical SNF resident. Mixed modeling was used to analyze the vignette responses.

Results:

Providers who had more negative beliefs about ACP were less likely to judge residents in need of ACP and less likely to feel responsible for ensuring ACP took place. Self-efficacy did not have a significant impact on judgments of need, but did significantly increase judgments of responsibility for ensuring ACP conversations. Providers with the highest levels of ACP self-efficacy were most likely to feel responsible for ensuring ACP conversations. In an exploratory analysis, these relationships remained the same whether responding to high or low risk residents (i.e., based on risk of hospitalization, type of diagnosis, functional status, and rate of declining health).

Discussion and Implications:

Both negative beliefs about ACP and self-efficacy in one’s ability to conduct ACP discussions were associated with professional judgments regarding ACP. The findings illustrate the importance of addressing negative beliefs about ACP and increasing provider ACP self-efficacy through education and policies that empower nurses and social workers.

Keywords

advance care planning self-efficacy beliefs skilled nursing facilities factorial survey nurses social workers decision-making

Introduction

Advance care planning (ACP) is evolving, with practice moving from simple one-time documentation of the completion of advance directives, to complex ongoing conversations among patients, families, and providers along a continuum of settings and circumstances. It is an iterative process that helps adults of all ages, regardless of health status, recognize and share their beliefs, goals of life, and future health care preferences.¹

There is evidence that the use of ACP remains suboptimal for those older than 65,² specifically in nursing homes,^3,4 and that there is no single best practice for carrying out ACP effectively, efficiently and in a timely manner. There does, however, appear to be some evidence that ACP can be beneficial for patients, family, and healthcare providers. Most studies have emphasized the benefits of ACP near the end-of-life and include outcomes like improved satisfaction with care for patients and significant others, less intensive and unwanted treatments, and a better match between patients’ care preferences and the actual care received.^5
-7 Thus, there remains much work to be done in understanding this complex process.⁸

ACP is influenced by a myriad of interrelated factors within residents, their families, facilities, providers, and the multiple characteristics of each of these groups all occurring within the greater milieu of culture and policy, locally, nationally, and globally. This complexity makes it easy to see that much works remains to fully develop and understand the state of ACP evidence. Untangling these variations and their contributing factors is an ongoing daunting task.^3,9

Complexity of ACP Process

Beyond some of the factors noted above, the definition of ACP itself highlights its complexity, emphasizing that it is an iterative process. To carry off this process it is not a simple tick box exercise asking people on care admission, “Do you have a living will?” Helping those in the field sort and weigh the evidence on ACP has been aided by the growth of systematic reviews in this field in the past decade. Much of this work remains grouped in the literature specific to palliative or end of life care and to select disease processes. Key to the process is communication in the form of discussions or conversations, a critical skill for providers. Related to or underpinning communication are other intertangled factors that impact ACP that researchers often refer to as barriers. Some of these barriers include uncertainty of providers with medical prognosis^10,11; documentation issues^12,13; culture^10,14; lack of resources^15,16; and knowledge and attitudes.^17
-19

Specific to nursing homes, Gilissen and others have outlined 17 necessary preconditions for ACP organized into the following 5 domains: adequate knowledge and skills; readiness and capacity to share in ACP planning; good relations; administrative infrastructure; and facility resources.³ Provider knowledge and skills are explicitly identified in each of the domains. For example, in the domain of readiness and capacity a precondition for providers is ACP self-efficacy, one’s confidence and comfort in engaging in ACP.

In a previous study of community care managers for older adults living in their homes we used vignettes to explore provider judgments of ACP need and comfort level in initiating ACP conversations along with select provider characteristics.⁹ These results showed that the use of the vignettes as basis for anchoring ACP judgments was helpful in demonstrating that while clinical patient data was significant in predicting judgments, provider characteristics explained a larger part of the variance in ACP judgments. This past work, as well as the current study, uses the Transdisciplinary Model of Evidence-Based Practices as a framework for focusing our research.^20,21 This model predicts clinical judgments based on the 3 pillars of evidence-based practices (best evidence, patient characteristics, and provider characteristics) in addition to possible facility factors.

Present Study

In this study within skilled nursing facilities (SNFs) we move beyond patient characteristics and focus on provider characteristics. We include nurses, both registered nurses (RNs) and licensed practical nurses (LPNs), and social workers, who are key staff that have consistent close interactions with residents and often at the time of critical changes in the resident’s condition. We did not include physicians since they generally spend less time with residents. According to the Federal Nursing Home Reform Law (42 CFR §483) interactions with residents are expected to emphasize resident self-determination and individualized plan of care as part of residents’ rights.²² We examine how self-efficacy, one’s confidence in his or her ability to effectively discuss ACP with residents, and beliefs about ACP relate to judgments in response to vignettes. The vignettes describe typical SNF residents with varying degrees of health problems, activity limitations, risk for hospitalization, and prognosis, and require the rating of need for ACP and responsibility for ensuring ACP conversations.

Research Objectives and Hypotheses

Our main objective was to examine the impact of provider characteristics on judgments about initiating ACP conversations within SNFs. Our first 2 hypotheses involved the provider’s self-efficacy in providing ACP:

Providers with higher self-efficacy will be more likely to judge a resident in need of ACP than providers with lower self-efficacy.

Providers with higher self-efficacy will be more likely to feel responsible for ensuring an ACP conversation takes place than providers with lower self-efficacy.

Similarly, our other hypotheses involved the provider’s negative beliefs about ACP:

Providers with less negative beliefs about ACP will be more likely to judge a resident in need of ACP than providers with more negative beliefs.

Providers with less negative beliefs about ACP will be more likely to feel responsible for ensuring an ACP conversation takes place than providers with more negative beliefs.

Lastly, in an exploratory analysis without formal hypotheses, we examined interactions between self-efficacy and negative beliefs with the resident’s health risk level in each vignette. We speculated that self-efficacy and beliefs would have the weakest effect when there was a clear need for ACP (i.e., a high-risk resident). As we previously reported, residents at high risk for hospitalization, with rapidly declining health, a diagnosis of cancer vs. diabetes, and those needing more assistance with activities of daily living were rated as in more need of ACP (p < .001 for each variable).²¹ In addition, these disease trajectory variables also were significantly associated with providers feeling responsible for ensuring ACP conversations took place (p < .001 for each variable). Since most providers indicated a strong need and responsibility for ACP under these conditions, we speculated that self-efficacy and negative beliefs would have a weaker effect under these conditions.

Design and Methods

Sample and Setting

This observational survey study examines the impact of SNF provider’s ACP self-efficacy and beliefs on professional judgments in response to vignettes about typical SNF residents. After obtaining IRB approval from Northeast Ohio Medical University (#14-019) and permission from SNF administrators, we collected data from providers in 29 urban SNFs in 7 counties in a Midwest state in the United States. The response rate for SNFs was 31% (29 out of a possible 95) and an average of 67% of providers at each participating SNF completed a survey.²¹ Surveys were returned anonymously so no written consent was necessary. The sample included 348 registered nurses, licensed practical nurses and social workers, who were chosen as they are the licensed personnel with the most direct contact with residents.

Measures

The 2 main independent variables included ACP self-efficacy and negative beliefs toward ACP. These scales were developed using exploratory factor analysis with varimax rotation from 11 questions about ACP. Both were rated on a scale from 1 = strongly disagree to 9 = strongly agree.²³ Two factors emerged with eigenvalues greater than one: self-efficacy had 3 items (Cronbach’s alpha = 0.73) and negative beliefs had 8 items (Cronbach’s alpha = 0.77).

We also examined how the gender, race, age, discipline, and years of service at the SNF might influence the judgments of need for ACP and feeling responsible. Age was measured by the following groups: less than 30 years, 30 to 39, 40 to 49, and 50 or more years old. Disciplines included registered nurses, licensed practical nurses, and social workers or other disciplines. Years of service was measured as follows: less than 1 year, 1 to 4 years, and 5 or more years of service.

Other independent variables included 5 variables within vignettes describing a typical resident that we previously examined.²¹ Each vignette provided a set of common resident sociodemographic and medical details to anchor judgments in real world scenarios and not rely solely on self-reported ACP behaviors. The vignettes included the following 5 resident variables: race (Black or White), diagnosis (cancer or diabetes), rate of health decline, functional level, and risk for hospitalization. The following paragraph shows the vignette structure:

The resident is an (African American/white) male with (cancer/diabetes), whose health has been (slowly/rapidly) declining. The resident needs assistance with (eating and all other activities of daily living/only bathing and dressing). The resident is at (low/high) risk for future hospitalizations.

Since we had 5 variables within the vignette, there were 32 possible vignettes. Each participant responded to questions on 8 vignettes using a balanced design.²¹

In this extension of our previous work we added provider characteristics to our original statistical models of ACP judgments. We had 2 main dependent variables in response to the vignettes described above: a) was the resident in need of ACP, “I think an advance care planning conversation is needed with the resident at this time,” and b) feelings of responsibility for ensuring ACP takes place, “I would feel responsible for ensuring that an advance care planning discussion took place with the resident.” Both items were scaled from 1 to 9 with 9 indicating strongly agree.

Analytic Strategy

Mixed models were used to examine the effects of the provider variables on judgments of need and responsibility. The 2 mixed models included vignette-level variables (i.e., resident’s race, risk of hospitalization, prognosis, diagnosis, and functioning), the order of the vignettes, and providers nested within SNFs. The mixed modeling was needed to avoid underestimating error variance estimates for hypothesis tests. Although modeling was complex for this reason, the analysis can be described more simply as the relationship between providers’ scores on self-efficacy or negative beliefs and the providers’ average ACP ratings of need or responsibility across the 8 vignettes. Earlier models also included the provider demographic variables (gender, race, discipline, age, and years of service); however, these variables were not statistically significant so they were excluded from the final models.

For the exploratory analysis examining interactions, the vignettes were grouped into 3 levels. The low-risk vignettes included none or one resident risk factor (cancer, rapid decline, needs assistance with all activities, and high risk of hospitalization); the medium-risk vignettes included 2 risk factors; and the high-risk vignettes included 3 or 4 risk factors. To examine the impact of self-efficacy and negative beliefs under different resident risk levels in the vignettes, we examined first-order interactions between self-efficacy and risk level, and between negative beliefs and resident risk level. Interactions tell us if relationships between variables are true for one group (low-risk residents in this example) but not another group (high-risk residents).

Results

Most providers were female (91%) and white (76%) despite oversampling SNFs in African American neighborhoods (See Table 1). Over half (52%) had worked at the current SNF for less than 5 years and 40% were over the age of 50 years. The largest group (51%) was licensed practical nurses, followed by registered nurses (39%), and social workers (11%).

Table 1.

Participant Characteristics.

Demographic variables	Frequency (%)
Male	32 (9.12%)
Race/Ethnicity
White	267 (76.29%)
African American	62 (17.71%)
Other race/ethnicity	21 (6%)
Discipline
Licensed Practical Nurse	178 (50.57%)
Registered Nurse	136 (38.64%)
Social Worker or other	38 (10.80%)
Age
Less than 30 years	38 (10.83%)
30-39	75 (21.37%)
40-49	99 (28.21%)
50+	139 (39.60%)
Years at current SNF
Less than 1 year	75 (21.19%)
1 to 4 years	110 (31.07%)
5 or more years	169 (47.47%)
ACP Self-efficacy and Beliefs	Mean (SD)
Self-efficacy for ACP; range 1-9	6.58 (2.06)
Negative beliefs toward ACP; range 1-9	2.62 (1.39)

Three of our 4 hypotheses were supported. Table 2 shows the results of the mixed models. At the provider level, self-efficacy did not significantly impact judgments of ACP need (p = 0.054) but was associated with judgments of responsibility (p < 0.001). As predicted, participants who had more negative beliefs about ACP were less likely to judge residents in need of ACP (p = 0.01) and less likely to feel responsible for ensuring an ACP conversation with the resident (p = 0.07).

Table 2.

The Effect of Self-Efficacy and Negative Beliefs on Vignette Judgments by Skilled Nursing Home Providers That a Resident Is in Need of Advance Care Planning (ACP) and the Provider Feels Responsible for Ensuring ACP Takes Place.

	Model 1: Need for ACP			Model 2: Responsible for ensuring ACP
	Estimate (95% CI)	SE	p	Estimate (95% CI)	SE	p
Provider Variables
Self-efficacy	0.09 (-0.00, 0.18)	0.05	0.054	0.28 (0.18, 0.38)	0.05	<0.001
Negative ACP Beliefs	-0.17 (-0.31, -0.04)	0.07	0.01	-0.17 (-0.32, -0.02)	0.07	0.02
Resident Characteristics in Vignette
High risk for hospitalization	0.84 (0.69, 1.00)	0.07	<0.001	0.59 (0.46, 0.73)	0.06	<0.001
Diagnosis of cancer vs. diabetes	0.35 (0.20, 0.51)	0.07	<0.001	0.37 (0.24, 0.51)	0.06	<0.001
Needs high level of assistance	0.55 (0.40, 0.71)	0.07	<0.001	0.48 (0.34, 0.61)	0.06	<0.001
Rapid vs. slow decline in health	0.33 (0.17, 0.48)	0.07	<0.001	0.30 (0.17, 0.44)	0.06	<0.001
Vignette resident is Black vs. White	-0.00 (-0.16, 0.15)	0.07	0.99	0.01 (-0.12, 0.15)	0.06	0.85

CI = confidence interval, SE = standard error, p = probability level.

In the exploratory analysis we found that self-efficacy and negative beliefs did not have the weakest effect when there was a clear need for ACP (i.e., high risk residents) as we had speculated. The interaction between resident risk level and self-efficacy was not statistically significant in predicting need for ACP (p = 0.70) nor in predicting responsibility for ACP conversations (p = 0.57). The same was true for the interactions between negative beliefs and resident risk level in predicting need for ACP (p = 0.82) and responsibility (p = 0.85). The level of health risk for the resident did not change the relationships between the provider variables (self-efficacy and negative beliefs) and the outcome variables (need for ACP and feelings of responsibility).

Discussion

Self-efficacy significantly increased the likelihood of feeling responsible for ensuring an ACP conversation takes place (Hypothesis 2), but did not affect whether or not the provider felt there was a need for ACP (Hypothesis 1). We think self-efficacy had no effect on need because all providers are able to determine who is in need of ACP regardless of their confidence in their abilities, but high self-efficacy is needed to feel responsible for ensuring an ACP conversation takes place. These are professionals with extensive professional education and training so one would expect that they would all understand need, but not all would feel responsible for ACP conversations. This finding is congruent with the current, widely accepted definition of ACP as inclusive of all adults “at any age or stage of health” in determining their need for ACP.^1(p14) Negative beliefs influenced both judgments of the need (Hypothesis 3) and feeling responsible for ACP (Hypothesis 4). The higher the negative beliefs, the less likely providers saw the need for ACP when they read the vignettes and were less likely to feel responsible for ensuring an ACP conversation. As negative beliefs about ACP decreased providers were more likely to see a need for ACP in a wider variety of situations as presented in the vignettes. The findings show the impact of negative beliefs more clearly than the impact of self-efficacy on need and responsibility. Other researchers from several countries have found that the negative beliefs by nurses impact the ACP process.^24,25

Much current and past literature reiterates that ACP is a complex ongoing process that includes multiple actors and components and can vary greatly by setting and state of health or disease of residents. Providers are one of the critical actors in the ACP process and as such their willingness and education represent essential elements or preconditions for effective ACP.³ Past studies have examined how experience, perceived difficulty of ACP conversations, and gender impact confidence or self-efficacy in ACP.^26,27 However, few studies have examined how ACP self-efficacy can impact professional judgments related to ACP and the resulting ACP practices.²³ Similarly, past studies have identified that negative beliefs about ACP are a stated barrier to ACP,^17,19 but few have examined how negative beliefs impact decisions to implement ACP.

This study begins to examine the link between self-efficacy and beliefs to actions as an important addition to the literature. Our findings that showed negative beliefs and self-efficacy both had a role in influencing ACP judgments and are an important step forward in disentangling the complexity of provider characteristics. Educational programs on ACP have been shown to improve confidence levels in providers^28
-30 and perhaps are one way to bridge the connection between self-efficacy and negative beliefs.

Limitations

An often-noted critique is that vignettes are only hypothetical and may not reflect behaviors, even though there are many advantages to the design such as increasing internal validity³¹ and addressing sensitive subjects.³² Other limitations include low response rate from some facilities due to high turnover and lack of staff time to participate. In addition, some facilities may have been more willing to participate (e.g., because of a greater commitment to ACP) and thus leading to possible selection bias. Since we sampled from urban areas as they had a higher number of African American providers and residents, the results may not be generalizable to non-urban SNFs. We used a simple 3-item self-efficacy scale in the current study, but continued development and validation of a stronger self-efficacy scale is needed.³³

Implications

Numerous implications emerge from this study for practice, policy, and research. In clinical settings, consider assessing staff ACP self-efficacy and beliefs as a first step in determining educational needs, or to determine if staff with high self-efficacy can be used as champions or role models for ACP. In a corollary manner, assessing for negative attitudes about ACP among staff, and subsequent education about the positive impact of ACP may provide opportunities for improving ACP. As with any practice, staff need the resources and policies that support ACP at the individual and organizational level. Policy makers at these levels may consider re-examining old policies for effectiveness, shaping and writing new policies, and especially getting staff input in all stages of policy review and implementation. Changes should be supported with guidelines and resources that facilitate providers actively engaging in ACP and include active quality improvement monitoring. Further research into self-efficacy and negative beliefs is also warranted. In this study, we saw that both variables impacted the rating of vignettes, but stronger measures of actual behavior are needed in future studies.

Footnotes

Acknowledgments

A version of this paper was presented at the 21st IAGG World Congress of Gerontology and Geriatrics in 2017, San Francisco, CA.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article This work was supported by the Agency for Healthcare Research & Quality (AHRQ), [R03-HS022162].

ORCID iD

Kristin R. Baughman

References

Sudore

Lum

You

, et al. Defining advance care planning for adults: a consensus definition from a multidisciplinary Delphi panel. J Pain Symptom Manag. 2017;53(5):821–832.

Weathers

O’Caoimh

Cornally

, et al. Advance care planning: a systematic review of randomised controlled trials conducted with older adults. Maturitas. 2016;91:101–109. doi:10.1016/j.maturitas.2016.06.016

Gilissen

Pivodic

Smets

, et al. Preconditions for successful advance care planning in nursing homes: a systematic review. Int J Nurs Stud. 2017;66:47–59. doi:10.1016/j.ijnurstu.2016.12.003

Jones

Moss

Harris-Kojetin

. Use of advance directives in long-term care populations. National Center for Health Statistics Data Brief, No 54. National Center for Health Statistics. January 2011. https://www.cdc.gov/nchs/data/databriefs/db54.pdf

Baidoobonso

Patient care planning discussions for patients at the end of life: an evidence-based analysis. Ont Health Technol Assess Ser. 2014;14(19):1–72. PMID: 26357528; PMCID: PMC4561366

Brinkman-Stoppelenburg

Rietjens

van der Heide

. The effects of advance care planning on end-of-life care: a systematic review. Palliat Med. 2014;28(8):1000–1025.

Khandelwal

Kross

Engelberg

Coe

Long

Randall Curtis

Estimating the effect of palliative care interventions and advance care planning on ICU utilization: a systematic review. Crit Care Med. 2015;43(5):1102–1111.

McMahan

Tellez

Sudore

. Deconstructing the complexities of advance care planning outcomes: what do we know and where do we go? A scoping review. J Am Geriatr Soc. September 7, 2020. doi:10.1111/jgs.16801

Baughman

Ludwick

Merolla

, et al. Disentangling consumer and provider predictors of advance care planning. Am J Hosp Palliat Med. 2013;30(7):717–725. doi:10.1177/1049909112464692

10.

Baughman

Aultman

Hazelett

, et al. Managing in the trenches of consumer care: the challenges of understanding and initiating the advance care planning process. J Gerontol Soc Work. 2012;55(8):721–737.

11.

Lovell

Yates

. Advance care planning in palliative care: a systematic literature review of the contextual factors influencing its uptake 2008–2012. Palliat Med. 2014;28(8):1026–1035.

12.

Hemsley

Meredith

Bryant

, et al. An integrative review of stakeholder views on advance care directives (ACD): barriers and facilitators to initiation, documentation, storage, and implementation. Patient Educ Couns. 2019;102(6):1067–1079.

13.

Kastbom

Milberg

Karlsson

. ‘We have no crystal ball’—advance care planning at nursing homes from the perspective of nurses and physicians. Scand J Prim Health Care. 2019;37(2):191–199.

14.

Llewellyn

Jaye

Egan

Cunningham

Young

Radue

. Living into death: a case for an iterative, fortified and cross-sector approach to advance care planning. Anthropol Med. 2017;24(3):350–365.

15.

Johnson

Attree

Jones

Al Gamal

Garbutt

. Diagnosis, prognosis and awareness of dying in nursing homes: towards the Gold Standard? Int J Older People Nurs. 2014;9(2):95–105.

16.

Aultman

Baughman

Ludwick

. A broader understanding of care managers’ attitudes of advance care planning: a concurrent nested design. J Clin Nurs. 2018;27(19-20):3572–3582. Accessed October 1, 2020. https://onlinelibrary-wiley-com-443.web.bisu.edu.cn/doi/abs/10.1111/jocn.14531

17.

Nedjat-Haiem

Cadet

Amatya

Mishra

Healthcare providers’ attitudes, knowledge, and practice behaviors for educating patients about advance directives: a national survey. Am J Hosp Palliat Med. 2019;36(5):387–395.

18.

Hagen

Howlett

Sharma

, et al. Advance care planning: identifying system-specific barriers and facilitators. Curr Oncol. 2015;22(4):e237–e245.

19.

Jimenez

Tan

Virk

Low

Car

. State of advance care planning research: a descriptive overview of systematic reviews. Palliat Support Care. 2019;17(2):234–244.

20.

Satterfield

Spring

Brownson

, et al. Toward a transdisciplinary model of evidence-based practice. Milbank Q. 2009;87(2):368–390.

21.

Baughman

Ludwick

Jarjoura

Kropp

Shenoy

. Advance care planning in skilled nursing facilities: a multisite examination of professional judgments. Gerontologist. 2019;59(2):338–346.

22.

National Consumer Voice for Quality Long-Term Care. Nursing home residents’ rights. Published 2019. Updated 2020. Accessed August 7, 2019. https://ltcombudsman.org/uploads/files/issues/residents-rights-factsheet.pdf

23.

Ludwick

Baughman

Jarjoura

Kropp

. CE: original research: advance care planning: an exploration of the beliefs, self-efficacy, education, and practices of RNs and LPNs. AJN Am J Nurs. 2018;118(12):26–32.

24.

Beck

McIlfatrick

Hasson

Leavey

. Nursing home manager’s knowledge, attitudes and beliefs about advance care planning for people with dementia in long-term care settings: a cross-sectional survey. J Clin Nurs. 2017;26(17-18):2633–2645.

25.

Hsieh

Huang

Tung

Chen

Beaton

Jane

SW.

The exploration of the knowledge, attitudes and practice behaviors of advanced care planning and its related predictors among Taiwanese nurses. BMC Palliat Care. 2019;18(1):99.

26.

Coffey

McCarthy

Weathers

, et al. Nurses’ knowledge of advance directives and perceived confidence in end-of-life care: a cross-sectional study in five countries. Int J Nurs Pract. 2016;22(3):247–257.

27.

Sulmasy

Sood

Ury

. Physicians’ confidence in discussing do not resuscitate orders with patients and surrogates. J Med Ethics 2008;34(2):96–101.

28.

Alderman

Nair

Fox

. Residency training in advance care planning: can it be done in the outpatient clinic? Am J Hosp Palliat Med. 2008;25(3):190–194.

29.

Cotter

Hasan

Ahn

Budhathoki

. A practice improvement project to increase advance care planning in a dementia specialty practice. Am J Hosp Palliat Med. 2019;36(9):831–835.

30.

Chan

Ward

Lapin

, et al. Outpatient advance care planning internal medicine resident curriculum: valuing our patients’ wishes. J Palliat Med. 2016;19(7):734–745.

31.

Auspurg

Hinz

. Factorial Survey Experiments. Sage Publications; Published 2014. Accessed October 1, 2020. https://books.google.com/books?id=1jieBQAAQBAJ&lpg=PP1&ots=Xzxh9NICO5&dq=auspurg%20hinz%20factorial%20survey%20experiments&lr&pg=PR7#v=onepage&q=auspurg%20hinz%20factorial%20survey%20experiments&f=false

32.

Ludwick

Wright

Zeller

Dowding

Lauder

Winchell

An improved methodology for advancing nursing research: factorial surveys. Adv Nurs Sci. 2004;27(3):224–238.

33.

Baughman

Ludwick

Fischbein

, et al. Development of a scale to assess physician advance care planning self-efficacy. Am J Hospice Palliat Med. 2017;34(5):435–441.