Evaluating the Psychometric Properties of the Evidence-Based Practice Attitude and Utilization Survey

Abstract

Objective:

Most health professions recognize the value of evidence-based practice (EBP), yet the uptake of EBP across most health disciplines has been suboptimal. To improve EBP uptake, it is important to first understand the many dimensions that affect EBP use. The Evidence-Based practice Attitude and utilization SurvEy (EBASE) was designed to measure the attitudes, skills, and use of EBP among practitioners of complementary and alternative medicine (CAM); however, the dimensionality of the instrument is not well understood. The aim of the current research was to examine the psychometric properties of the attitudes, skills, and use subscales of EBASE.

Design:

This was a secondary analysis of data obtained from the administration of EBASE. Data were examined using principal components analyses and confirmatory methods. Internal consistency reliabilities of resultant subscales were also computed.

Participants:

1314 U.S. chiropractors and 554 Canadian chiropractors.

Results:

A unidimensional structure best fit the attitudes and use subscales. Skills subscale items were best represented by subscales with a multidimensional structure. Specifically, the skills construct was best modeled with three dimensions (identification of the research question, locating research, and application of EBP). All subscales had acceptable internal consistency reliability estimates.

Conclusions:

The findings support the modification of the scoring guidelines for the original EBASE. These changes are likely to result in a more accurate measure of EBP attitudes, skills, and use among chiropractors, and possibly CAM providers more generally.

Introduction

Evidence-based practice (EBP) refers to the integration of the best available evidence from research with clinical expertise, patient preference, and existing resources to inform decision making about the healthcare of individual patients.^1,2 While there are many purported benefits of EBP, including greater consistency of practice, reduced treatment costs, and improved patient outcomes,^1,3 the application of EBP in practice is less than optimal.^4

–7 Understanding the extent to which health professionals engage in EBP, as well as the factors that facilitate and hinder EBP uptake, is necessary to inform the design of appropriate interventions ensuring that the best available evidence underpins clinical practice.

Despite the abundance of different measures of evidence-based practice, most instruments to date have failed to capture the complexity of EBP by examining only two or three constructs relevant to EBP utilization.⁸ An exception to this is the Evidence-Based practice Attitude and utilization SurvEy (EBASE), which is a multidimensional instrument designed to measure several factors affecting EBP uptake. Informed by an extensive review of the EBP literature and existing tools, as well as expert opinion, the 84-item self-report instrument comprises three subscales (attitudes, skills, and use of EBP), with the remaining sections of the survey capturing the facilitators and barriers of EBP uptake, EBP training, and participant demographic characteristics.⁸ By providing a snapshot of a profession's attitudes, skills, and use of EBP, results from the EBASE can inform the design of tailored knowledge translation strategies to improve evidence-based clinical decision-making, which, in turn, may increase the quality of care and patient health outcomes.

Another strength of the EBASE relative to other measures of EBP is that the content validity, convergent validity, test–retest, and internal consistency reliability have been previously examined.⁸ Original scoring guidelines for the generation of attitudes, skills, and use subscores are available for the EBASE based on this prior psychometric investigation.⁸ In brief, the attitudes subscale consisted of nine items, with the subscore generated as a sum of the first eight items. The skills subscale comprised 13 items, with the sum of all skills items making up the skills subscore. The use subscale consisted of eight items; the sum of the first six items generated the use subscore.

Although the scoring guidelines were established, the internal structure of the subscales was not investigated. Therefore, further exploration of the EBASE's internal validity was needed to determine how well each individual factor aligns with the constructs of the EBASE. Hence, the goal of the current investigation was to use factor analytic methods to gain insight into the psychometric properties of the EBASE subscales: attitudes, skills, and use. Identification and confirmation of the subscales' dimensionality will increase measurement accuracy and inform scoring guidelines for future research.

Materials and Methods

Design

The current investigation was a secondary analysis of baseline EBASE data from two descriptive cross-sectional surveys conducted online between November 17, 2012, and March 5, 2013 (U.S. survey) and December 13, 2013. and June 5, 2014 (Canadian survey).

Sample

Two convenience samples were used to perform this analysis. The U.S. study recruited participants from a potential pool of approximately 30,000 chiropractors by using the membership rosters of nine organizations (American Chiropractic Association, Council on Chiropractic Guidelines and Practice Parameters, Congress of Chiropractic State Associations, Sacro Occipital Research Society International, U.S. ChiroDirectory, Activator Methods, International College of Applied Kinesiology, the Pediatric Councils of the American Chiropractic Association, and International Chiropractors Association). The Canadian study recruited participants from a potential pool of exactly 7200 doctors of chiropractic (DCs) with the support of the Canadian Chiropractic Association and all 10 provincial chiropractic associations. The surveys were open to all practicing DCs in North America who had Internet access and a valid e-mail address.

Recruitment

Invitations to participate in each survey were sent via e-mail by the above-mentioned organizations through their respective membership lists. E-mail recipients were encouraged to forward the message onto their colleagues. Three national chiropractic publications (i.e., Dynamic Chiropractic, The American Chiropractor, and the Journal of the Canadian Chiropractic Association) published advertisements, which provided an overview of the study and invited readers to participate in the online survey. Recruitment also involved the placing of advertisements in quarterly newsletters of the Canadian Chiropractic Association and provincial associations.

Ethics

Human ethics approval for the two studies was granted through the institutional review boards of the University of Pittsburgh (PRO12060417) and McGill University (A07-E62-13A) in June 2012 and July 2013, respectively. Informed consent was secured from all participants on the homepage of the research website before participation in the surveys.

Survey administration and data collection

The surveys were administered electronically through the University of Pittsburgh (U Pitt), Pennsylvania, using the U Pitt web platform. DCs were invited to follow a link to the U Pitt website (http://www.chirostudy.pitt.edu) to register for the study. Participants were subsequently e-mailed a password in order to complete the online survey. Anonymity was ensured by assigning a unique identification number to each registered participant. Responses were captured through a secure data-capturing feature/system, Web Data Xpress, an interface used by U Pitt that allows for direct entry and storage of data within a designated SQL server database. This method of data capture is resource-efficient and minimizes human error by avoiding the need for manual data entry.

Statistical analyses

The statistical analyses were performed in three steps. Step 1 was an exploratory analysis using approximately half of the U.S. sample (n = 662), and step 2 was a confirmatory analysis using the other half of the U.S. sample (n = 652). Step 3 of the analysis was a confirmatory analysis using the Canadian sample (n = 554). The U.S. sample was randomly split into two sets to obtain exploratory and confirmatory data. This process allowed examination of the internal validity of the structure. Separate confirmatory analyses were performed on the U.S. and Canadian data to determine whether the findings were upheld in a different sample, thus providing evidence of the external validity of the structure.⁹

The sample sizes were appropriate for the analyses. The recommended sample size for factor analysis based on a subjects-to-variable ratio is a minimum of 10 participants for each item being used in the analysis.¹⁰ Other recommendations include a “rule of 500,” which requires a total sample size of 500 or more.¹¹ The sample sizes of 662, 652, and 554 exceeded suggested minimums. For example, the recommended minimum number of participants needed to perform a factor analysis on the 13-items skills subscale would be 130 using the subjects-to-variable ratio.

Principal component analyses

The first step of the primary analysis was to conduct principal component analyses (PCA) using Statistical Packages for the Social Sciences, version 22 (IBM, Chicago, IL). The EBASE items were measured using an ordinal scale; therefore, using a Pearson correlation matrix would be inappropriate. Thus, a correlation matrix was created by using the Spearman rho, a nonparametric alternative, that was used in the PCA. This matrix is available from the first author upon request. Three separate PCAs were performed as part of the exploratory analysis using the responses of 662 U.S. chiropractors on the attitudes, skills, and use subscales. The purpose of PCA was to identify the dimensionality of the dataset by forming item sets, or components, that explained the variability within the data.¹² For example, if a subscale is unidimensional (i.e., it measures one underlying construct), then one component explains most of the variability in the data. However, more than one component may emerge if subscale items are not measuring the intended underlying construct.^12,13

The structure of each subscale was examined using both orthogonal (Varimax) and oblique (Promax) rotation; however, only the Promax solutions are reported. Promax rotation considers the correlation among components, and any resultant components within the attitudes, skills, or use subscales were expected to be related.¹⁴ Each subscale (i.e., attitudes, skills, and use) were analyzed separately, and the amount of variability explained by the extracted components was examined. The item loadings, which represent the correlation between the item and the underlying construct, were also examined. Item loadings of 0.5 or above were considered strong; that is, the item was strongly correlated with the construct.¹³ Through the examination of the variability explained by the components and the item loadings, sets of items to form attitudes, skills, and use subscales were identified.

Confirmatory analyses

Next, responses from 652 U.S. chiropractors and 554 Canadian chiropractors in separate confirmatory factor analyses (CFA) using the PROC CALIS method with SAS software (SAS Institute, Inc., Cary, NC) with items constrained to load on one factor. For this analysis, the generalized least-squares estimation method was used, which is recommended for ordinal data.¹⁵ Latent variables were fixed to a nonzero loading of 1. The results of the confirmatory analyses were used to compare the PCA solution to the original EBASE scoring guidelines.

The purpose of the CFA was to determine the best measurement model for the data.¹⁵ Two models were compared: Model 1 represented the PCA solution, and model 2 represented the original scoring. CFA model fit was evaluated using fit indices with cutoff values recommended by Hu and Bentler.¹⁶ Root mean square approximation (RMSEA) values of .06 or below; standardized root mean square residual (SRMSR) values of 0.08 or below; and comparative fit index (CFI), normed fit index (NFI), and non-normed fit index (NNFI) values of 0.90 or above were considered representative of good model fit. The final “best” models were selected for each subscale (attitudes, skills, use) within each sample (U.S., Canadian) after comparing the fit indices of the PCA solution and original scoring.

Internal consistency reliability

Internal consistency reliability coefficients were computed for the attitudes, skills, and use subscales formed from the factor analyses. A combined sample of 1868 (1314 from the U.S. sample and 554 from the Canadian sample) was used to compute reliability coefficients as well as descriptives for the new subscales. Internal consistency reliability was used to quantify how well the items within a subscale consistently measured the same construct. Coefficients near 0.80 were considered acceptable values for good reliability.¹⁰ Additionally, item statistics were computed to determine whether floor or ceiling effects were present, and to examine item discrimination indices. According to Nunnally and Bernstein,¹⁰ items with corrected item-total correlation coefficients greater than 0.3 have more common variance and thus provide good item discrimination.

Results

Demographic characteristics

The U.S. and Canadian samples are described in Table 1. Most of the sample was male (U.S., 75%; Canadian, 66%), with a mean age (± standard deviation) of 47 ± 11.6 years in the U.S. sample and 42.1 ± 11.4 years in the Canadian sample. Eighty-two percent of the U.S. chiropractors and 75% of the Canadian chiropractors reported a bachelor's degree qualification or higher. Two thirds (66%) of the U.S. and Canadian chiropractors reported a practice with a musculoskeletal focus. The Canadian chiropractors' practices were mostly located in a city setting (60%); the U.S. practices were primarily located in suburban areas (48%).

Table 1.

Demographic Characteristics

Variable	United States (n = 1314)	Canada (n = 554)
Mean age ± SD (yr)	46.7 ± 11.6	42.1 ± 11.4
Sex, n (%)
Male	989 (75.3)	363 (65.5)
Female	325 (24.7)	191 (34.5)
Highest education, n (%)
High school	17 (1.3)	102 (18.4)
Associate degree/Some college	214 (16.3)	36 (6.5)
Bachelor's degree	821 (62.5)	352 (63.5)
Master's degree/some graduate work	226 (17.2)	53 (9.6)
Doctorate	36 (2.7)	11 (2.0)
Practice setting, n (%)
City	449 (34.2)	337 (60.8)
Suburban	629 (47.8)	137 (24.7)
Rural	236 (18.0)	80 (14.4)
Focus of practice, n (%)
Musculoskeletal Focus	868 (66.1)	367 (66.2)
Non-musculoskeletal focus	446 (33.9)	187 (33.8)

SD, standard deviation.

PCA

Results of the PCA are reported in Tables 2 –5. For the attitudes subscale, one component explained 50.71% of the variability in the data. Three items, “takes into account my clinical experience,” “takes into account patient preference,” and “lack of evidence from clinical trials,” did not have item loadings of 0.5 or greater on the first component. Although the PCA solution revealed three potential components, the item content was evaluated, and it was determined that a reduced unidimensional scale without the three items was the best representation of the attitudes construct (Table 2).

Table 2.

Results of Principal Components Analyses for Attitudes

	C1		C2		C3
Item	P	S	P	S	P	S	Communalities
EBP is necessary^a	0.752	0.805	0.309	0.506	0.002	0.041	0.661
Literature and research findings are useful^a	0.763	0.742	0.019	0.230	0.048	0.013	0.585
I am interested in improving EBP skills^a	0.844	0.853	0.149	0.378	0.025	0.018	0.735
EBP improves quality of patient care^a	0.842	0.883	0.270	0.495	0.028	0.74	0.783
EBP assists me in decision making^a	0.841	0.887	0.287	0.511	0.040	0.086	0.791
EBP places an unreasonable demand on my practice^a	0.629	0.686	0.292	0.459	0.149	0.185	0.503
EBP takes into account my clinical experience	0.301	0.505	0.841	0.893	0.050	0.088	0.801
EBP takes into account patient preference	0.182	0.398	0.887	0.901	0.054	0.020	0.822
Lack of evidence from clinical trials to support treatment	0.018	0.038	0.009	0.013	0.992	0.991	0.984
Variance explained (%)	50.71		12.13		11.20

Boldface represents item loadings greater than 0.5.

Item is included as part of the recommended scoring for the attitudes scale; sum across all items with “a” footnote to form attitudes variable.

C1, component 1 (attitudes); C2, component 2 (unnamed, items not used in score); C3, component 3 (unnamed, items not used in score); P, pattern coefficient; S, structure coefficient; EBP, evidence-based practice.

Table 3.

Results of Principal Components Analyses for Skills

	C1		C2		C3
Item	P	S	P	S	P	P	Communalities
Identifying knowledge gaps in practice^a	0.032	0.424	0.029	0.395	0.776	0.856	0.753
Identifying answerable clinical questions^a	0.057	0.457	0.007	0.470	0.838	0.846	0.722
Locating literature^b	0.026	0.552	0.851	0.916	0.054	0.523	0.842
Online database searching^b	0.005	0.562	0.931	0.924	0.069	0.458	0.862
Retrieving evidence^b	0.079	0.635	0.857	0.929	0.002	0.548	0.866
Critical appraisal of evidence^c	0.521	0.660	0.034	0.537	0.226	0.657	0.541
Synthesis of research evidence^c	0.573	0.745	0.129	0.650	0.147	0.641	0.635
Applying research to patient cases^c	0.641	0.768	0.016	0.587	0.182	0.655	0.648
Sharing evidence with colleagues^c	0.569	0.712	0.061	0.482	0.052	0.504	0.515
Conducting clinical research^c	0.740	0.755	0.019	0.433	0.136	0.332	0.597
Using clinical research findings^c	0.678	0.763	0.008	0.536	0.104	0.564	0.599
Conducting systematic reviews^c	0.839	0.787	0.050	0.412	0.125	0.348	0.654
Using findings from systematic reviews^c	0.826	0.797	0.088	0.555	0.067	0.465	0.639
Variance explained (%)	49.61		6.65		4.58

Boldface represents item loadings greater than 0.5.

Item is included as part of the identification of the research question subscale; sum across items to generate subscore.

Item is included as part of the locating research subscale; sum across items to generate subscore.

Item is included as part of the application of EBP subscale; sum across items to generate subscore.

Table 4.

Results of Principal Components Analyses for Use

Item	C1	Communalities
Reviewed professional literature^a	0.866	0.750
Reviewed clinical research^a	0.904	0.817
Used professional literature to assist^a	0.901	0.812
Used professional literature to change^a	0.867	0.752
Used an online database^a	0.820	0.673
Used an online search engine^a	0.731	0.535
Consulted a colleague for decision-making	0.714	0.510
Referred to magazines, etc., to inform decision-making	0.479	0.229
Variance explained (%)	63.47

Boldface represents item loadings greater than 0.5.

Item used to create use subscale; sum across items to create use variable.

Table 5.

Results of Confirmatory Factor Analysis

Model	Chi-square ^a	SRMSR	RMSEA (95% CI)	CFI	NFI	NNFI
U.S. attitudes
PCA solution (6 items)	93.51, df = 14, p < 0.001	0.03	0.09 (0.07–0.12)	0.96	0.96	0.95
Original scoring (8 items)	409.43, df = 27, p < 0.001	0.07	0.14 (0.14–0.16)	0.88	0.88	0.84
Canadian attitudes
PCA solution (6 items)	40.32, df = 14, p = 0.002	0.02	0.05 (0.03–0.07)	0.98	0.98	0.98
Original scoring (8 items)	202.87, df = 27, p < 0.001	0.05	0.11 (0.09–0.12)	0.93	0.93	0.91
U.S. skills
PCA solution (3 factors)	391.20, df = 62, p < 0.001	0.03	0.09 (0.08–0.10)	0.93	0.92	0.92
Original scoring (1 factor)	1256.79, df = 65, p < 0.001	0.07	0.16 (0.15–0.17)	0.77	0.77	0.73
Canadian skills
PCA solution (3 factors)	435.12, df = 62, p < 0.001	0.04	0.10 (0.09–0.11)	0.91	0.90	0.89
Original scoring (1 factor)	1085.78, df = 65, p < 0.001	0.07	0.16 (0.16–0.18)	0.76	0.75	0.71
U.S. use
PCA solution (7 items)	295.87, df = 14, p < 0.001	0.04	0.17 (0.15–0.19)	0.91	0.91	0.87
Original scoring (6 items)	240.80, df = 9, p < 0.001	0.03	0.20 (0.18–0.22)	0.92	0.92	0.88
Canadian use
PCA solution (7 items)	239.84, df = 14, p < 0.001	0.04	0.17 (0.15–0.19)	0.93	0.92	0.89
Original scoring (6 items)	183.11, df = 9, p < 0.001	0.03	0.18 (0.16,0.21)	0.94	0.94	0.90

The chi-square test is a measure of model fit. Smaller values indicate better model fit.

SRMSR, standardized root mean square residual; RMSEA, root mean square approximation; CI, confidence interval; CFI, comparative fit index; NFI, normed fit index; NNFI, non-normed fit index; PCA, principal components analysis.

The skills PCA solution revealed three potential subscales. This solution was sensible on the basis of the item content; hence, the three components were labeled accordingly. The first two items, “identifying knowledge gaps in practice” and “identifying clinical questions,” formed the first component, which was labeled “identification of the research question.” The next three items, “locating literature,” “online database searching,” and “retrieving evidence,” formed the second component, which was labeled “locating evidence.” The remaining eight items were considered a representation of the “application of EBP” (Table 3).

The use PCA solution was examined by using all eight use items. The structure was unidimensional; however, one item, “I have referred to magazines,” had a factor loading less than 0.5 (Table 4). This item was reviewed, and the authors decided to drop it from the use subscale, thereby creating a unidimensional scale with seven items (Table 4).

Confirmatory analyses

The model fit indices for the comparison of the subscales formed by the PCA versus the original scoring can be found in Table 5. For the attitudes subscale, the PCA solution was a better fit for the data than the EBASE scoring in the U.S. and Canadian data sets. The SRMSR and RMSEA values were lower and the CFI, NFI, and NNFI were all larger for the PCA solution, indicating better model fit.

For the skills models, the three-factor solution from PCA was compared with the original one-factor scoring recommendation. The three-factor solution had lower SRMSR and RMSEA and higher CFI, NFI, and NNFI values than the one-factor model in the U.S. and Canadian data sets, providing evidence that a three-factor representation of skills was a better measurement model.

In relation to the use subscale, the PCA solution of seven items was similar to the six items used in the original scoring method. Upon comparison of fit indices, the PCA solution had better (smaller) RMSEA values than the original scoring model; however, the original model had better (higher) CFI, NFI, and NNFI values than the PCA solution. Because RMSEA values tend to inflate in simple models,¹⁷ the choice of the better model was based on CFI, NFI, and NNFI values. Therefore, the original six-item scoring was a better representation of the use construct.

Internal consistency reliability and item statistics

Internal consistency reliability estimates and item statistics were computed by using the U.S. and Canadian samples combined (n = 1868). Table 6 reports the internal consistency estimates and descriptive statistics for the new subscales. The internal consistency reliability coefficients for the new subscales met or exceeded the recommendations of Nunnally and Bernstein.¹⁰ Item reliability and discrimination statistics are reported in Table 7. No ceiling or floor effects were noted, and the standard deviations for all items was around 1 point. All corrected item-total correlations were greater than 0.5, and the squared multiple correlations ranged from 0.32 to 0.81, providing evidence that all items were satisfactorily discriminating.¹⁰ An examination of the Cronbach α if items were deleted showed that all items were contributing to the reliability for each subscale, and removal of one item would not increase the internal consistency estimate.

Table 6.

New Subscale Internal Consistency Estimates and Descriptive Statistics

Subscale	No. of items	Mean ± SD	Minimum	Maximum	Range	Cronbach α	Guttman split half
Attitudes	6	24.6 ± 4.2	6	30	24	.89	.88
Identifying the research question	2	7.7 ± 1.4	2	10	8	.75	–^a
Locating research	3	10.8 ± 2.9	3	15	12	.92	.82
Application of evidence-based practice	8	25.3 ± 6.1	8	40	32	.89	.86
Use	6	9.9 ± 6.5	0	24	24	.93	.91

Could not be computed because of small number of items on subscale.

Table 7.

Item Reliability/Discrimination Statistics for Revised EBP (n = 1868)

Subscale	Mean ± SD	Corrected item-total correlation	Squared multiple correlation	Cronbach α if item deleted
Attitudes^a
1. EBP is necessary	4.3 ± 0.9	0.72	0.53	0.86
2. Literature and research findings are useful	4.1 ± 0.8	0.60	0.39	0.88
3. I am interested in improving EBP skills	4.3 ± 0.8	0.74	0.57	0.86
4. EBP improves quality of patient care	4.1 ± 0.9	0.83	0.72	0.85
5. EBP assists me in decision making	4.2 ± 0.9	0.82	0.70	0.85
6. EBP places an unreasonable demand on my practice	3.6 ± 1.0	0.56	0.32	0.89
Skills: Identification of the research question^b
7. Identifying knowledge gaps in practice	3.7 ± 0.7	0.60	0.36	–^c
8. Identifying answerable clinical questions	3.9 ± 0.8	0.60	0.36	–^c
Skills: Locating research^d
9. Locating literature	3.8 ± 1.0	0.82	0.67	0.89
10. Online database searching	3.6 ± 1.1	0.85	0.71	0.88
11. Retrieving evidence	3.5 ± 1.0	0.85	0.73	0.87
Skills: Application of EBP^e
12. Critical appraisal of evidence	3.5 ± 0.9	0.67	0.55	0.88
13. Synthesis of research evidence	3.3 ± 0.9	0.74	0.63	0.87
14. Applying research to patient cases	3.7 ± 0.9	0.73	0.63	0.88
15. Sharing evidence with colleagues	3.3 ± 1.0	0.61	0.43	0.89
16. Conducting clinical research	2.1 ± 1.0	0.62	0.48	0.89
17. Using clinical research findings	3.7 ± .87	0.70	0.58	0.89
18. Conducting systematic reviews	2.5 ± 1.1	0.65	0.52	0.88
19. Using findings from systematic reviews	3.3 ± 1.1	0.71	0.57	0.88
Use^f
20. Reviewed professional literature	2.0 ± 1.3	0.83	0.77	0.91
21. Reviewed clinical research	1.8 ± 1.3	0.86	0.81	0.90
22. Used professional literature to assist	1.5 ± 1.3	0.85	0.77	0.91
23. Used professional literature to change	1.3 ± 1.2	0.81	0.72	0.91
24. Used an online database	1.4 ± 1.4	0.76	0.59	0.92
25. Used an online search engine	1.9 (1.3)	0.63	0.40	0.93

To compute the attitudes subscore, reverse-code 6 and sum items 1–6.

To compute the identification of the research question subscore, sum items 7 and 8.

Could not be computed because of small number of items on subscale.

To compute the locating research subscore, sum items 9–11.

To compute the application of EBP subscore, sum items 12–19. To compute an overall skills subscore, sum items 7–19.

To compute the use subscore, sum items 20–25.

Discussion

The current investigation examined the psychometric properties of the attitudes, skills, and use subscales of the EBASE. The findings build upon a previous psychometric evaluation of the EBASE⁸ by providing evidence of the construct validity for the survey. A unidimensional structure best fit the attitudes and use subscales, while the skills items were best represented by subscales with a multidimensional structure. The skills construct was best modeled with three dimensions: (1) skills for identification of the research question, (2) skills for locating research, and (3) skills for application of EBP. All subscales had acceptable internal consistency reliability estimates, establishing that the items were consistently measuring a single construct.

The authors recommend new scoring methods for the EBASE attitudes and skills subscales as a more accurate measure of each construct. Summing across six (rather than eight) “attitudes” items to form one attitudes subscale should be done. For the skills subscale, two possible scoring methods can be adopted depending on the research question. A global skills score can be examined by summing across all 13 items to form a single variable. A more in-depth picture of skills can be obtained by using three subscales. To examine the practitioners' skills in identifying the research question, the authors recommend summing the first two items of the “skills” subscale. Similarly, skills in locating research can be examined by summing across the next three items. Lastly, the application of EBP can be examined by summing the last eight “skills” items. Specific skills items are listed in Table 3. The final recommendation is to apply the original scoring method for the use subscale, that is, sum across the first six “use” items.

Reducing the gap between appreciation of EBP and the actual uptake and application of EBP in clinical settings is challenging^18

–21 for all health professions, including complementary and alternative medicine (CAM) practitioners. Improvements will require multilevel efforts by professional, educational, and organizational stakeholders who understand practitioner needs.^18,22 The EBASE instrument measures attitudes, skills, and use of EBP among CAM audiences, which may help inform the development of tailored knowledge translation strategies to improve clinical decision-making and to narrow the knowledge-practice gap. For example, the authors are using the EBASE to examine the effectiveness of an online educational program in shaping the attitudes, skills, and use of EBP among US chiropractors (ongoing study, National Institutes of Health/National Center for Complementary and Integrative Health R21 AT007547).

The EBASE can also be employed by educators to measure the attitudes, skills, and use of EBP among students commencing their educational programs, or at various time points within or beyond the program. Such work is being conducted by one author on a sample of South Australian final-year nursing students, who will be tracked for 12 months after graduation.²³ EBASE results can be used formatively to tailor course curriculum and learning objectives to better meet learner needs. CAM professional associations may also consider surveying their members with an online version of the EBASE to assess the need for EBP continuing education courses. Lastly, commercial insurers and healthcare policy organizations may wish to use the EBASE to survey enlisted network providers and use the results to develop EBP educational interventions to improve provider knowledge of evidence-based practice.

This investigation had several limitations. First, the data were not collected for the purposes of conducting a psychometric analysis, limiting the type of analyses that could be performed. For example, a more thorough psychometric analysis would have included examination of test–retest reliability, convergent validity, and discriminant validity using the chiropractic sample. Second, survey data are prone to responder bias; in this case it is likely that survey participants experienced different levels of skill, use, and attitudes toward EBP than nonresponders. However, this could not be determined because both studies used convenience samples. Third, these findings were based on responses from chiropractors only; hence, they should be confirmed with other CAM providers. Notwithstanding these limitations, the study had several strengths, including a large sample size, homogeneous population, and the use of multinational data.

Conclusion

This appears to be the first known study to evaluate the internal validity of the attitudes, skills, and use subscales of the EBASE. The findings provided evidence of construct validity and internal consistency reliability of the EBASE as a measure of at least three constructs of EBP. The subscale scoring guidelines should be modified to increase the accuracy of the measure. Corroborating these findings with other health disciplines is now warranted.

Footnotes

Acknowledgment

Funding for the primary study was through NIH/NCCIH Grant #R21AT007547.

Author Disclosure Statement

No competing financial interests exist.

References

Leach

. Evidence-based practice: a framework for clinical practice and research design. Int J Nurs Pract. 2006; 12:248–251.

Sackett

, Rosenberg

, Muir Gray

, et al. Evidence based medicine: what it is and what it isn't. BMJ, 1996; 312:71–72.

Bick

, Graham

. Evaluating the Impact of Implementing Evidence-based Practice. Ames, Iowa: Wiley-Blackwell; 2010.

Filbay

, Hayes

, Holland

. Physiotherapy for patients following coronary artery bypass graft (CABG) surgery: limited uptake of evidence into practice. Physiother Theory Pract, 2012; 28:178–187.

Leach

, Gillham

. Are complementary medicine practitioners implementing evidence based practice?. Complement Ther Med, 2011; 19:128–136.

McCurtin

, Roddam

. Evidence-based practice: SLTs under siege or opportunity for growth? The use and nature of research evidence in the profession. Int J Lang Comm Dis, 2012; 47:11–26.

Walker

, Stomski

, Hebert

, et al. A survey of Australian chiropractors' attitudes and beliefs about evidence-based practice and their use of research literature and clinical practice guidelines. Chiropr Man Therap, 2013; 21:44.

Leach

, Gillham

. Evaluation of the Evidence-Based practice Attitude and utilization SurvEy (EBASE) for complementary and alternative medicine practitioners. J Eval Clin Pract, 2008; 14:792–798.

American Educational Research Association, American Psychological Association & The National Council on Measurement in Education. Standards for Educational and Psychological Testing. 2nd ed). Washington, DC; 1999.

10.

Nunnally

, Bernstein

. Psychometric Theory. 3rd ed. New York: McGraw-Hill; 1994.

11.

Comrey

, Lee

. A First Course in Factor Analysis. Hillsdale, NJ: Erlbaum; 1992.

12.

Costello

, Osbourne

. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Res Evaluat, 2005; 10(7).

13.

Pett

, Lakey

, Sullivan

. Making Sense of Factor Analysis: The Use of Factor Analysis for Instrument Development in Health Care Research. Thousand Oaks, CA: Sage; 2003.

14.

Darton

. Rotation in factor analysis. Statistician, 1980; 29:167–194.

15.

Brown

. Confirmatory Factor Analysis for Applied Research. New York: Guilford Press; 2006.

16.

, Bentler

. Cutoff criteria for fit indices in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model. 1999; 6:1–55.

17.

Keith

. Multiple Regression and Beyond: An Introduction to Multiple Regression and Structural Equation Modeling. 2nd ed. New York: Routledge; 2015.

18.

Ubbink

, Guyatt

, Vermeulen

. Framework of policy recommendations for implementation of evidence-based practice: a systematic scoping review. BMJ Open, 2013. doi: 10.1136/bmjopen-2012-001881.

19.

Walker

, Stomski

, Hebert

20.

Suter

, Vanderheyden

, Trojan

, et al. How important is research-based practice to chiropractors and massage therapists?. J Manipulative Physiol Ther, 2007; 30:109–115.

21.

Hall

. Attitudes of chiropractors to evidence-based practice and how this compares to other healthcare professionals: a qualitative study. Clin Chiropr, 2011; 14:106–111.

22.

Kawchuk

, Newton

, Srbely

, et al. Knowledge transfer within the Canadian Chiropractic community. Part 2: narrowing the evidence-practice gap. J Can Chiropr Assoc, 2014; 58:206.

23.

Leach

, Hofmeyer

, Bobridge

. The impact of research education on student nurse attitude, skill and uptake of evidence-based practice: a descriptive longitudinal survey. J Clin Nurs, 2016; 25:194–203.