Abstract
Peer-reviewed publications are one way for researchers to communicate information about service delivery to rehabilitation counselors. However, little is known about rehabilitation counselors’ actual use of peer-reviewed publications. Thus, the current research explored the extent to which rehabilitation counselors (N = 334) read and use peer-reviewed publications, factors associated with their reading and using peer-reviewed publications, and several items assessing their judgments about peer-reviewed publications. Overall, we found that rehabilitation counselors used peer-reviewed publications a moderate amount, held generally positive judgments about peer-reviewed publications, and reported wanting to use information from peer-reviewed publications with a greater percentage of clients than they actually did. We also found that counselors’ reading and using of peer-reviewed publications increased as counselor opinions of peer-reviewed publications were more positive and as counselors perceived their professional organizations, universities, agencies, and supervisors encouraged reading and using peer-reviewed publication. Collectively, we believe the current study is useful for researchers to generate and communicate research to rehabilitation counselors.
Keywords
Rehabilitation counselors are responsible for assisting individuals with disabilities to become as personally, socially, and economically independent as possible (Commission on Rehabilitation Counselor Certification [CRCC], 2009). To achieve that responsibility, rehabilitation counselors should use the best available information to serve clients (American Counseling Association [ACA], 2014; CRCC, 2009). Although there are several sources that rehabilitation counselors can use to inform service delivery (e.g., continuing education, webinars, presentations), the current study focuses on rehabilitation counselors’ use of one possible source of information: peer-reviewed publications. Peer-reviewed publications have the potential to provide rehabilitation counselors with research-based information that could improve service delivery (Chan et al., 2012).
By providing rehabilitation counselors with up-to-date information about effective interventions, peer-reviewed publications can play an important role in rehabilitation counseling service delivery (Chan et al., 2012). Peer-reviewed publications are sources of information that have been vetted by experts prior to being made available. Within the field of rehabilitation counseling, peer-reviewed publications should provide counselors with theoretical or empirical recommendations for ways in which they can best deliver services to clients (Chan et al., 2012). These recommendations can be examined for whether they had the intended effect on service delivery and then new information, in the form of new peer-reviewed publications, can be produced with new recommendations. Ideally, when rehabilitation counselors receive information from peer-reviewed publications, when authors provide counselors with useful information, and when authors are responsive to counselors’ needs, peer-reviewed publications can help counselors progressively and iteratively improve their service delivery. This process should result in rehabilitation counselors providing clients with sound and empirically based services.
Although several studies have investigated factors associated with the implementation of evidenced-based practices in rehabilitation counseling (see Bezyak, Kubota, & Rosenthal, 2010), general counseling (see Mobley & Wester, 2007), and clinical psychology (Baker, McFall, & Shoham, 2008), research examining whether rehabilitation counselors read or use information from peer-reviewed publications is needed. Therefore, the current study examined the extent to which rehabilitation counselors read and use peer-reviewed publications. More specifically, the study examined how much rehabilitation counselors read and use peer-reviewed publications, factors associated with counselors’ reading and using peer-reviewed publications, and counselors’ judgments about peer-reviewed publications.
With these ideas in mind, the following sections present the major ideas that guided the design of the current study. First, we explicitly couch peer-reviewed publications as a conduit for knowledge translation. Second, we discuss factors that may be associated with whether rehabilitation counselors read and use peer-reviewed publications.
Peer-Reviewed Publications as Knowledge Translation
Knowledge translation is defined as “the multidimensional, active process of ensuring that new knowledge gained through the course of research ultimately improves the lives of people with disabilities and furthers their participation in society” (U.S. Department of Education, 2006, p. 8195). Thus, knowledge translation is one way for rehabilitation counselors to uphold standard A.1.b of the code of ethics for rehabilitation counselors (CRCC, 2009). This standard mandates that counselors create a plan that offers a reasonable promise of success, which requires that the plan is informed by something other than the counselor’s intuitions. Rehabilitation counselors are an essential component of knowledge translation. Specifically, rehabilitation counselors are typically in closest contact with the clients and, therefore, are the person who can most directly translate newly acquired information into service delivery.
Peer-reviewed publications are one way by which knowledge can be translated to counselors. Ideally, the information contained within peer-reviewed publications would be easily acquired by counselors, and this information could be used to benefit clients. However, this form of knowledge translation does not occur if rehabilitation counselors do not read peer-reviewed publications, if counselors do not have access to peer-reviewed publications, or if they do not have time to read peer-reviewed publications. Furthermore, once read, peer-reviewed publications may not be translated into service delivery because the information may be seen as irrelevant or be presented in a way that is unclear or unusable by counselors. All these factors impair the potential for peer-reviewed publications to impact service delivery. As a result, clients potentially may not receive the best services available. The framework of the current study provides insights into the knowledge translation process, which could inform practices that increase useable research information being consumed by rehabilitation counselors
Factors Affecting Counselors’ Reading and Using Peer-Reviewed Publications
The current research also examined whether the extent to which counselors read and use peer-reviewed publications were associated with several factors (Anderson, Matthews, Lui, & Nierenhausen, in press; Lui, Anderson, Matthews, Nierenhausen, & Schlegelmilch, 2014; Murray, 2009). We conceptualized factors that may be associated with counselors’ reading and using of peer-reviewed publications existing at different levels: organizational factors (e.g., supervisors, organizational culture, caseload demands), professional identity (e.g., professional organization affiliation, professional’s perceptions of their professional role), and individual counselors’ judgments or opinions of peer-reviewed publications.
First, research has demonstrated that rehabilitation counselors employed in work environments in which they have easy access to evidenced-based information and workplaces that encourage applying evidenced-based practices are more likely to implement evidenced-based practices (e.g., Anderson et al., in press; Graham et al., 2013; Lui et al., 2014). In the same vein, we reasoned that work environments have the potential to affect counselors’ use of peer-reviewed publications. Factors such as supervisors who encourage the use of peer-reviewed publications, considerations of counselors’ work demands, and work environments wherein counselors have ready access to peer-reviewed publications all should be positively associated with counselors’ reading and using peer-reviewed publications. In comparison, factors such as counselors’ perceptions that their organization and supervisors are discouraging the use of peer-reviewed publications, who do not have time to read peer-reviewed publications or who do not have access to peer-reviewed publications, should be negatively associated with counselors’ reading and using peer-reviewed publications.
Second, professional identity and training of a rehabilitation counselor are related to applying evidenced-based practices (Anderson et al., in press; Murray, 2009). Research fundamentals are mandated in accreditation standards for rehabilitation counselor education programs. For example, Section C.8 of the Council on Rehabilitation Education (CORE) standards covers research and program evaluation. The standards require training programs to cover basic statistics and psychometric concepts, basic research methods, effectiveness of rehabilitation counseling services, and ethical, legal, and cultural issues related to research and program evaluation. Programs are also required to train graduate students on how to “apply research literature to practice” (CORE, 2008, p. 12); and the 2016 Council for Accreditation of Counseling & Related Educational Programs (CACREP) standards require programs to train students on “the importance of research in advancing the counseling profession, including how to critique research to inform counseling practice” and “identification of evidence-based counseling practices” (CACREP, 2016, p. 13). Therefore, the extent to which graduate-level training programs successfully promote peer-reviewed publications also should be associated with the extent to which graduates of these programs read and use peer-reviewed publications.
In addition, rehabilitation counseling professional organizations often encourage their membership to read and use peer-reviewed publications. In fact, several professional organizations manage their own peer-reviewed publications with their membership being their intended reading audience. Counselors who believe the professional organizations to which they belong encourage them to use peer-reviewed publications also may be likely to read peer-reviewed publications and use information from such publications in their service delivery.
Finally, rehabilitation counselors’ reading and using peer-reviewed publications are likely to be related to their judgments about those publications. Related research has shown that counselors who hold negative judgments of research may be less likely to implement evidenced-based practices (Anderson et al., in press). Likewise, we believe that counselors who hold negative judgments about peer-reviewed publications also would be unlikely to read and use those publications. These negative judgments can take many forms, such as beliefs that peer-reviewed publications are irrelevant, difficult to read, or are uninteresting.
The current study was designed to test several research questions.
We hypothesize that counselors read and use peer-reviewed publications less than they would ideally like to.
We hypothesize positive associations between perceptions of support from universities, professional organizations, agencies, and supervisors and counselors’ reading and using peer-reviewed publications. We also hypothesize positive associations between counselors’ positive judgments about peer-reviewed publications and their reading and using peer-reviewed publications. Finally, we explored several demographic variables (e.g., age, years working) and professional practice setting characteristics (e.g., private vs. public rehabilitation) that may be associated with rehabilitation counselors’ reading and using peer-reviewed publications. These analyses will be exploratory, so no hypotheses were created.
Method
For the current study, we report how we determined our sample size, all data exclusions (if any), and all measures in the study (Simmons, Nelson, & Simonsohn, 2011, 2012). All hypotheses, planned analyses, and sample size decision criteria were specified prior to data collection and can be accessed on the Open Science Framework. All stimuli needed to replicate the current study and data needed to replicate the reported analyses can be acquired by contacting the authors or by visiting the project site on the Open Science Framework (https://osf.io/dsmyf/?view_only=f9162880d67346e9a57ab76ff0534b38). Finally, the current research was approved by the university’s human subjects review board prior to data collection.
Sample Size Decision Criteria
Most of our a priori analyses were planned to be tested with bivariate correlations. We determined our smallest effect size of interest for the current study to be r = .33, which would be a correlation in which one variable accounts for approximately 10% of the variance in the other. Correlations smaller than r = .33 were deemed to be too small to be of interest or too resource-expensive to reliably detect for the purposes of the current study. To have high statistical power (i.e., 1 − β = 0.99) to detect a correlation of r = .33 using a Type I error rate of α = .05, a power analysis suggested a target sample size of at least 186 participants. Furthermore, to maximize our final sample, we did not want to terminate data collection while new participants were still completing the survey. Thus, we terminated data collection once at least 186 participants completed the survey, and we did not receive any new responses for 7 days.
Participants
At the time that data collection was terminated, 359 potential participants visited the survey. Four potential participants did not consent, three were not currently employed as a rehabilitation counselor, and 18 were not currently working with clients. Thus, our final sample included 334 rehabilitation counselors who were currently working with clients.
The final sample was mostly female (67%), about 45 years old (M = 44.97, SD = 11.59), and reported an average of 14.54 years of work experience (SD = 10.97). Most (78%) rehabilitation counselors reported working at a public vocational rehabilitation setting (8% private for-profit, 2% not-for-profit, 12% other setting/missing) and reported having an average of 95.82 clients on their current caseload (SD = 58.98). Slightly less than half of rehabilitation counselors reported holding a CRC (Certified Rehabilitation Counselor; 44%) and only 8% reported having previously authored a peer-reviewed publication.
Procedure
A 30-item survey was created. Rehabilitation counselors from the American Rehabilitation Counseling Association, the International Association of Rehabilitation Professionals, and the Council of State Administrators of Vocational Rehabilitation were recruited for participation. We provided each professional organization with a recruitment email and a link to the survey, and requested the organization to forward the email to their members.
After providing consent, participants completed two items that were used to identify those who met our inclusion criteria: whether the participant was currently employed as a rehabilitation counselor and whether the participant currently worked with clients. Next, counselors read a description of peer-reviewed research: A peer-reviewed publication is a publication (digital or paper) that has been vetted by scholars in the field for quality and importance. Peer-reviewed publications could be conceptual in nature (e.g., review of literature) or contain original data (quantitative or qualitative). Peer-reviewed publications are published in academic journals. Editorials, newspaper or magazine articles, websites, or other nonprofessional publications are not considered to be peer-reviewed publications.
The survey also included three items to assess how much counselors read/use peer-reviewed publications, five items about factors that may encourage the use of peer-reviewed publications, five items about counselors’ opinions/judgments of peer-reviewed publications, two items on whether the counselors would rely on their own experience/judgment regardless of what peer-reviewed publications say, and four open-ended items. Finally, counselors reported demographic items and were thanked for their participation.
Results
The sample size for individual analyses may vary slightly due to missing responses to individual items. For qualitative interpretations of effect sizes as being “small,” “medium,” or “large,” we relied on the benchmarks from Cohen (1988).
Descriptive Statistics
Several items were used to test hypotheses of interest. Descriptive statistics for these items are presented in Table 1.
Descriptive Statistics.
Note. CI = confidence intervals; LL = lower limit; UL = upper limit.
Responses made on 1 to 7 (strongly disagree to strongly agree) scale. bResponses made on a 0% to 100% scale.
Rehabilitation counselors indicated their agreement with the statements “I often read peer-reviewed publications” and “I often use peer-reviewed publications to inform my work” on a 7-point scale ranging from 1 = strongly disagree to 7 = strongly agree. These items will be referred to as counselors’ reading and using of peer-reviewed publications herein. Counselors’ self-reported reading of peer-reviewed publications (M = 4.16, SD = 1.92) and using of peer-reviewed publications (M = 4.00, SD = 1.87) were both near the midpoint of the scale and were highly correlated with one another (r = .83, p < .001). As a complementary measure, we asked counselors to report the percentage of clients with whom they use peer-reviewed information on a 0% to 100% scale. On average, counselors reported using information from peer-reviewed publications with 31.04% (SD = 26.91) of their clients. Finally, using the same 0% to 100% scale, counselors reported the percentage of clients with whom they would ideally use such information (M = 53.94%, SD = 29.24).
Next, rehabilitation counselors indicated their perceptions of whether different entities encouraged the use of peer-reviewed publications on a 7-point scale ranging from 1 = strongly disagree to 7 = strongly agree. Counselors agreement with the statements that the college they attended (M = 5.33, SD = 1.67) and the professional organizations (M = 5.37, SD = 1.98) they belong to encouraged the use of peer-reviewed publications was above the midpoint of the scale. In comparison, counselors’ agreement with the statements that their current agency (M = 3.86, SD = 1.86) and their current supervisor (M = 3.81, SD = 1.82) encouraged the use of peer-reviewed publications was slightly below the midpoint of the scale. These items will collectively be referred to as encouraging factors herein. We also asked rehabilitation counselors to report yes or no whether they “had access to peer-reviewed publications.” Of the 313 counselors who responded to this item, 226 (72.2%) reported having access.
Next, rehabilitation counselors indicated their agreement with several statements about their judgments of peer-reviewed publications on a 7-point scale ranging from 1 = strongly disagree to 7 = strongly agree. These items will collectively be referred to as counselors’ judgments of peer-reviewed publications herein. Counselors agreed with the statement that peer-reviewed publications benefit clients (M = 4.99, SD = 1.54), are interesting to read (M = 4.86, SD = 1.45), are easily understood by the typical counselor (M = 4.75, SD = 1.36), and are useful to the typical counselor (M = 4.82, SD = 1.36). However, counselors also indicated they would rely on their own judgment (M = 4.26, SD = 1.44) and experiences (M = 4.20, SD = 1.42) regardless of what peer-reviewed publications said. That these two items were above the midpoint of the scale suggests that counselors were somewhat dismissive of peer-reviewed publications in comparison with their personal judgment and experiences. We also asked counselors to indicate their level of agreement with the following statement: “Researchers should consult with rehabilitation counselors about the research that would be most helpful to rehabilitation counselors.” On average, counselors strongly agreed with this statement (M = 5.98, SD = 1.11).
Prespecified Analyses
Discrepancies between actual use and ideal use of peer-reviewed publications
We compared counselors’ ideal percentage of clients with whom counselors would use information from peer-reviewed publications to their actual percentage of clients with whom they reported using such information. This difference represents a discrepancy between counselors’ actual use and ideal use of information from peer-reviewed publications. This discrepancy was statistically significant and was four fifth of a standard deviation, which is conventionally considered a large effect (e.g., Cohen, 1988), t(263) = 15.72, p < .001, d = 0.81. Also notable is that 7.1% of counselors reported actually using information from peer-reviewed publications with 0% of their clients, whereas only 2.6% reported actually using such information with 100% of their clients. Finally, only 13.6% of counselors reported using such information as much or more than their ideal amount, which means the vast majority of counselors do not use information from peer-reviewed publications as much as their ideal amount.
Factors associated with counselors’ reading and using peer-reviewed publications
We computed a variable that was the difference between the percentage of clients with whom a counselor actually used peer-reviewed publications and the ideal percentage of clients with whom a counselor would use peer-reviewed publications (i.e., ideal use minus actual use). We refer to this newly computed variable as counselors’ “use deficit” because a higher score represents a greater discrepancy between ideal use and actual use. Rehabilitation counselors’ use deficit was negatively associated with both their reading (r = −.18, p = .004) and using (r = −.15, p = .02) peer-reviewed publications. As counselors’ use deficit increased, so did their reading and using peer-reviewed publications. These effects are conventionally considered “small.”
We next examined whether rehabilitation counselors’ reading and using of peer-reviewed publications were associated with encouraging factors (i.e., counselors’ perceptions that other entities encouraged the use of peer-reviewed publications). Indeed, counselors’ reading and using of peer-reviewed publications were positively and significantly associated with their perceptions of whether the college/university they attended (r = .28, p < .001, and r = .26, p < .001), their current agency (r = .41, p < .001, and r = .50, p < .001), their current supervisor (r = .40, p < .001, and r = .49, p < .001), and their professional organizations (r = .17, p < .001, and r = .24, p < .001) encouraged using peer-reviewed publications (see Table 2). Counselors’ reading and using peer-reviewed publications increased as counselors’ perceptions of these encouraging factors increased.
Correlations With Counselors’ Reading and Using Peer-Reviewed Publications.
Responses made on 1 to 7 (strongly disagree to strongly agree) scale. bResponses made on a 0% to 100% scale.
p < .05. **p < .01. ***p < .001.
Parallel multiple regression analyses were conducted to examine these variables collectively. These analyses allowed us to examine whether counselors’ perceptions of the encouraging factors predicted their reading and using peer-reviewed publication independently of one another. These four variables (i.e., the college/university they attended, their current agency, their current supervisor, their professional organizations) accounted for 24% of the variance in counselors’ reading of peer-reviewed research, F(4, 302) = 24.65, p < .001, R2 = .24. When considered simultaneously, the college/university they attended (b = 0.27, p < .001), their current agency (b = 0.25, p = .001), and their current supervisor (b = 0.22, p = .008) each remained significant; however, their professional organization did not (b = −0.03, p = .60). Similarly, these four variables accounted for 31% of the variance in counselors’ use of peer-reviewed publications, F(4, 302) = 34.72, p < .001, R2 = .31. And the college/university they attended (b = 0.20, p < .001), their current agency (b = 0.30, p < .001), and their current supervisor (b = 0.26, p = .001) each remained significant whereas their professional organization did not (b = 0.01, p = .93).
These encouraging factors also were associated with the percentage of clients with whom counselors reported using information from peer-reviewed publications (see Table 2). The four encouraging factors accounted for 25% of the variance in counselors’ reading of peer-reviewed research, F(4, 256) = 21.96, p < .001, R2 = .25. When considered simultaneously, the college/university they attended (b = 1.96, p = .04), their current agency (b = 5.02, p < .001), and their professional organization (b = 2.15, p = .02) each remained significant whereas their current supervisor did not (b = 1.82, p = .14).
We also examined whether counselors’ self-reported access to peer-reviewed publications was associated with their reading and using of peer-reviewed publications. Relative to counselors who reported not having access to peer-reviewed publications, counselors who reported having access reported more reading (M = 4.59, SD = 1.85, vs. M = 3.01, SD = 1.62), more using (M = 4.36, SD = 1.84, vs. M = 3.03, SD = 1.65), and using peer-reviewed publications with a greater percentage of clients (M = 35.42, SD = 27.75, vs. M = 17.49, SD = 19.44): reading, t(311) = 7.01, p < .001, d = 0.88; using, t(309) = 5.87, p < .001, d = 0.74; percentage of clients, t(258) = 4.76, p < .001, d = 0.69. All these effects are conventionally considered “medium” or “large” in magnitude.
We next examined whether rehabilitation counselors’ reading and using of peer-reviewed publications increased as their opinions of peer-reviewed research were more positive. As seen in Table 2, counselors’ reading and using peer-reviewed publications and the percentage of clients with whom peer-reviewed publications were used were positively and significantly associated with their perception that peer-reviewed publications benefit clients (r = .44, p < .001; r = .50, p < .001; r = .41, p < .001), are interesting to read (r = .42, p < .001; r = .41, p < .001; r = .35, p < .001), are easily understood by the typical counselor (r = .31, p < .001; r = .25, p < .001; r = .20, p < .01), and are useful to the typical counselor (r = .35, p < .001; r = .35, p < .001; r = .32, p < .001). Counselors’ reading and using of peer-reviewed publications increase as their opinions are more positive. These associations are conventionally considered “medium” or “large.”
As with the encouraging factors, we conducted parallel multiple regression analyses wherein we simultaneously entered counselors’ opinions of peer-reviewed publications as predictors. These analyses allowed us to examine whether counselors’ opinions of peer-reviewed publications predicted their reading and using peer-reviewed publication independently of one another. Counselors’ opinions collectively accounted for 28% of the variance in counselors’ reading of peer-reviewed publications, F(4, 302) = 30.89, p < .001, R2 = .28. Perceptions that peer-reviewed publications benefit clients (b = 0.41, p < .001) and are interesting to read (b = 0.25, p = .004) remained significant, whereas perceptions that peer-reviewed publications are easily understood by the typical counselor (b = 0.15, p = .06) and are useful to the typical counselor (b = 0.13, p = .15) did not. Similarly, counselors’ opinions collectively accounted for 30% of the variance in counselors’ reading of peer-reviewed publications, F(4, 304) = 33.17, p < .001, R2 = .30. Perceptions that peer-reviewed publications benefit clients (b = 0.48, p < .001) and are interesting to read (b = 0.24, p = .006) remained significant, whereas perceptions that peer-reviewed publications are easily understood by the typical counselor (b = 0.04, p = .62) and are useful to the typical counselor (b = 0.12, p = .15) did not. Finally, counselors’ opinions collectively accounted for 21% of the variance in the percentage of clients with whom counselors use peer-reviewed publications, F(4, 255) = 17.39, p < .001, R2 = .21. Perceptions that peer-reviewed publications benefit clients (b = 6.40, p < .001) and are useful to the typical counselor (b = 3.10, p = .03) remained significant while perceptions that peer-reviewed publications are easily understood by the typical counselor (b = 0.001, p = .99) and are interesting to read (b = 2.55, p = .09) did not.
Finally, the extent to which rehabilitation counselors reported they would rely on their own judgment regardless of what peer-reviewed publications say was unassociated with their reading (r = −.04, p = .45) and using (r = −.10, p = .07) peer-reviewed publications, and negatively associated with the percentage of clients with whom they actually used peer-reviewed publications (r = −.16, p = .01). In comparison, the extent to which rehabilitation counselors reported they would rely on their own experiences regardless of what peer-reviewed publications say was negatively associated with their reading (r = −.12, p = .04) and using (r = −.16, p = .006) peer-reviewed publications, and negatively associated with the percentage of clients with whom they actually used peer-reviewed publications (r = −.15, p = .01). These associations are conventionally considered “small.”
Exploratory Analyses
The following analyses were not specified a priori. Therefore, confidence in these results should be tempered until they are properly replicated.
Relationships between counselors’ responses and counselor demographics
In general, demographic variables were unassociated with counselors’ responses. Counselors’ age was unassociated with how often they read (r = −.07, p = .26) and use peer-reviewed publications (r = .01, p = .81), and unassociated with the percentage of clients with whom they reported using peer-reviewed publications (r = .06, p = .40). Similarly, counselors’ years of work experience also was unassociated with how often they read peer-reviewed publications (r = −.09, p = .14), how often they use peer-reviewed publications (r = −.10, p = .11), and the percentage of clients with whom they reported using peer-reviewed publications (r = .02, p = .74).
Finally, we explored three demographic variables that could arguably be related to the extent to which counselors read and use peer-reviewed publications. First, there were no differences between counselors who had authored and those who had not authored a peer-reviewed publication in the extent to which they read (M = 4.46, SD = 2.27, vs. M = 4.13, SD = 1.89) and use (M = 4.39, SD = 2.27, vs. M = 3.97, SD = 1.86) peer-reviewed publications: reading, t(300) = 0.87, p = .37, d = 0.17; using, t(297) = 1.13, p = .26, d = 0.22. However, there was a difference between authors and non-authors in the percentage of clients with whom counselors reported using peer-reviewed publications (M = 48.04, SD = 32.29, vs. M = 29.36, SD = 29.84), t(249) = 3.34, p = .001, d = 0.70. Counselors who had authored a peer-reviewed publication used peer-reviewed publications significantly more than counselors who had not.
Second, there were no differences between counselors who currently had a CRC and those who did not currently have a CRC in the extent to which they read (M = 4.24, SD = 1.86, vs. M = 4.09, SD = 2.00) and use (M = 4.15, SD = 1.91, vs. M = 3.85, SD = 1.88) peer-reviewed publications, and with the percentage of clients with whom they used peer-reviewed publications (M = 32.07, SD = 28.84, vs. M = 29.89, SD = 25.19): reading, t(298) = 0.65, p = .52, d = 0.08; using, t(295) = 1.35, p = .18, d = 0.16; percentage of clients, t(250) = 0.64, p = .53, d = 0.08. Finally, size of counselors’ current caseload was negatively related to how often they read peer-reviewed publications (r = −.13, p = .03), how often they use peer-reviewed publications (r = −.13, p = .03), and the percentage of clients with whom they reported using peer-reviewed publications (r = −.11, p = .09). However, these latter associations would be considered “small.”
Differences in counselors’ responses between employment settings
Rehabilitation counselors reported being employed in a public vocational rehabilitation setting, a for-profit rehabilitation setting, a not-for-profit rehabilitation setting, or an “other” employment setting. Because these employment settings may differ in the type of clients served, type of counselors employed, and/or the mission of the organization, it is possible there are systematic differences in how those counselors employed in these settings responded to the survey items. For each of the following analyses, a one-way ANOVA was conducted with counselors’ work setting included as the between-participants’ factor. Because we had no a priori expectations about which settings differed, we followed up any significant omnibus effects with post hoc tests that controlled for the familywise Type 1 error rate.
We first explored whether the extent to which rehabilitation counselors read peer-reviewed publications differed by their employment setting. The overall ANOVA was significant, with employment setting accounting for 4% of the variance in counselors’ reading of peer-reviewed publication, F(3, 299) = 4.38, p = .005, η2 = .04. Counselors who worked in for-profit settings reported the highest levels of reading (M = 5.41, SD = 1.82), followed by those working in not-for-profit settings (M = 4.50, SD = 1.38), other settings (M = 4.10, SD = 2.81), and public rehabilitation settings (M = 4.03, SD = 1.88). We used Tukey’s HSD (honest significant difference) to explore which work settings differed. The only significant difference was between those working in for-profit settings and those working in public rehabilitation settings, p = .002.
Similarly, we explored whether the extent to which rehabilitation counselors use peer-reviewed publications differed by their employment setting. The overall ANOVA was significant, with employment setting accounted for 5% of the variance in counselors’ use of peer-reviewed publication, F(3, 296) = 5.58, p = .001, η2 = .05. Counselors who worked in for-profit settings reported the highest use (M = 5.26, SD = 2.01), followed by those working in other settings (M = 4.70, SD = 2.63), not-for-profit settings (M = 4.60, SD = 1.52), and public rehabilitation settings (M = 3.83, SD = 1.81). We again used Tukey’s HSD to explore for differences between work settings. As with reading, the only significant difference was between those working in for-profit settings and those working in public rehabilitation settings, p = .001.
Finally, we explored whether the percentage of clients with whom rehabilitation counselors use peer-reviewed publications differed by their employment setting. The overall ANOVA was significant, with employment setting accounting for 14% of the variance in counselors’ use of peer-reviewed publication, F(3, 249) = 14.05, p < .001, η2 = .14. Counselors who worked in for-profit settings reported the highest use (M = 58.48, SD = 30.55), followed by those working in other settings (M = 51.89, SD = 35.62), public rehabilitation settings (M = 27.24, SD = 23.86), and not-for-profit rehabilitation settings (M = 18.67, SD = 30.43). We used Tukey’s HSD to explore for differences between work settings. Counselors in for-profit settings reported a higher percentage than those in public rehabilitation settings, p < .001, and not-for-profit settings, p = .003, but not for those in other settings, p = .91. Counselors in public rehabilitation settings reported a lower percentage than counselors in for-profit settings, p < .001, and in other settings, p = .02, but did not differ from those in not-for-profit settings, p = .84.
Coding of Open-Ended Responses
Rehabilitation counselors were given the opportunity to answer four open-ended questions. The text from each participants’ responses were individually coded for emerging themes by the two authors and a research assistant. The identified themes were cross-checked among the two authors and any discrepancies with the coding were resolved through a discussion. The complete list of counselors’ de-identified responses and the codes applied to those responses can be accessed on the project’s Open Science Framework page.
First, rehabilitation counselors reported which topics they would like to see researchers focus on. The most common responses were general topics such as public awareness, advocacy, client participation, assistive technology, and ethics. Second, rehabilitation counselors discussed what, if any, problems they perceive with peer-reviewed publications. Counselors most often cited availability in terms of distribution, accessibility in terms of language and clarity, relevance of the research topics, and the perceived strength of the research as problems with peer-reviewed publications. Third, rehabilitation counselors provided useful ways for researchers to share research findings with rehabilitation counselors, besides traditional peer-reviewed journal publications. The most common response was for free, online resources that summarize research findings to be made available to counselors. Finally, rehabilitation counselors explained the resources they primarily rely on, instead of, or in addition to peer-reviewed publications, to inform their work with clients. Rehabilitation counselors most often cited using supervisors or co-workers, webinars, websites, and agency-specific manuals to inform their work.
Discussion
The current study examined the extent to which rehabilitation counselors read and use peer-reviewed publications, factors associated with their reading and using peer-reviewed publications, and their judgments of peer-reviewed publications. Overall, counselors reported reading and using peer-reviewed publications a moderate amount. Furthermore, counselors’ judgments of peer-reviewed publications were generally positive: Counselors believed peer-reviewed publications were beneficial for clients, interesting to read, easy to read, and written on useful topics. On the contrary, counselors also indicated they would rely on their own judgment and experiences regardless of the content of peer-reviewed publications.
A major finding was that rehabilitation counselors reported wanting to use information from peer-reviewed publications on a greater percentage of clients than they actually did. Even though some counselors’ ideal may be to use information from peer-reviewed publications rarely whereas other counselors’ ideal may be to use that information often, counselors consistently indicated a desire to use information from peer-reviewed publications more than they actually do. This finding supports the idea that some factors impair knowledge translation. Relatedly, rehabilitation counselors who reported a greater discrepancy between their ideal use and actual use of peer-reviewed publications also reported reading and using peer-reviewed publications less. Although this association cannot demonstrate a causal relationship, future research should study whether increasing counselors’ reading and use of peer-reviewed publications cause a reduction in this discrepancy (i.e., counselors whose actual use is closer to their ideal use).
We also examined factors associated with rehabilitation counselors’ reading and using peer-reviewed publications. Counselors’ perceptions that their organizations, supervisors, and professional organizations encourage the use of peer-reviewed publications were positively associated with the extent to which they reported reading and using peer-reviewed publications. Similarly, counselors’ positive judgments of peer-reviewed publications were positively associated with the extent to which they read and used peer-reviewed publications. These findings are similar to those from Tansey, Bezyak, Chan, Leahy, and Lui (2014). Specifically, the Tansey et al. study found that as confidence for engaging with evidence-based practices increased, counselors were more likely to incorporate evidence-based practices in their work. In the current study associations between positive judgments, reading, and using could reflect that counselors who read and use peer-reviewed publications are generally pleased with what is published. Or that counselors who already hold positive judgments are those who also read and use peer-reviewed publications. Whereas those counselors who already hold negative judgments tend to avoid reading and using peer-reviewed publications. We believe that both explanations are likely to be partially responsible for this observed association. Again, future research should use alternative methods to better understand these associations.
Although exploratory, an interesting finding had to do with the relationship between employment setting and reading and using information from peer-reviewed publications. Specifically, counselors employed in the private sector reported reading and using peer-reviewed publications at a higher rate than counselors employed in the public setting. Although existing literature is lacking to fully explain this finding, perhaps counselors employed in the private sector, such as personal injury litigation, are required to cite peer-reviewed publications. In comparison, although counselors employed in the public system are required to justify their actions with clients, perhaps they use other sources such as agency policies.
Limitations
There are three notable limitations of the current study: First, the current research included a sample of rehabilitation counselors, which means that we only examined our research questions from one perspective. Future research should conceptually replicate this study so that counselors’ use of peer-reviewed publications is considered from several other perspectives (e.g., researchers, supervisors, funding agencies).
Second, the current research did not examine whether rehabilitation counselors’ reading and using peer-reviewed publications were associated with better service delivery or with better client outcomes. Although we strongly believe that service delivery should ideally be based on solid empirical evidence, and that can be encouraged through the reading and using of peer-reviewed publications, it is possible that merely increasing the extent to which counselors read and use peer-reviewed publications will not improve service delivery or client outcomes.
Finally, although we focused the current study on the extent to which rehabilitation counselors personally read and used peer-reviewed publications, we acknowledge there are several ways counselors can acquire the necessary information other than personally reading peer-reviewed publications. For example, certifications may require continuing education which often includes staying up to date on the current research. Or counselors may be accessing high-quality information that is not a peer-reviewed publication (e.g., websites, training materials, etc.). At this point, we do not want to imply that acquiring information by personally reading peer-reviewed publications is superior to other methods of acquiring the same information. Future research should examine whether certain forms of information acquisition leads to better service delivery or client outcomes than other forms.
Concluding Thoughts and Recommendations
We believe that the current study has implications for counselors, supervisors, rehabilitation counselor educators, and rehabilitation researchers.
First, participants strongly believed that researchers should consider consulting practicing rehabilitation counselors when designing studies. Simply put, practicing rehabilitation counselors are likely to have valuable insight into what information they need on a day-to-day basis. We believe it would be excellent for researchers to routinely base their research questions and studies on counselors’ expressed interests and needs. And we believe it would be less than ideal for research to be conducted and manuscripts be published without considering the counselors who are in a position to act on those publications’ recommendations. Consultation with counselors could occur in a variety of ways. For example, professional organizations could host sessions at conferences designed to get counselors and researchers talking about research endeavors. Researchers could also reach out to counselors directly to consult during research endeavors.
Second, access is an obvious barrier to counselors being able to read and use peer-reviewed publications. In the open-ended responses, counselors reported access as being a major barrier and indicated that access to free, web-based information would be their preferred method of consuming peer-reviewed publications. Disseminators of peer-reviewed information should take rehabilitation counselors’ preferences into consideration when deciding on the method in which that information is shared. Professional organizations should consider providing free, web-based information for counselors as a benefit of membership.
Third, the current study has implications for the way in which future rehabilitation counselors are trained. Although additional research is needed to develop evidenced-based training procedures on how to best present research competencies in rehabilitation counselor education programs, the current study found that counselors’ reading and using of peer-reviewed publications were associated with the extent to which they perceived the university where they were trained encouraged the use of those publications. Future research should identify what characteristics of rehabilitation counselors’ universities or rehabilitation counselors’ training programs are responsible for the perception that those entities encourage the use of peer-reviewed publications (e.g., incorporating publications into the curriculum, faculty who explicitly encourage reading and using peer-reviewed publications, etc.).
Fourth, counselors and supervisors should be aware of the gap between desired and ideal use of information from peer-reviewed publications with clients. Most importantly, counselors and supervisors should assess the encouraging factors in their own agency and consider taking action to increase these factors (e.g., How can the agency and/or supervisor encourage the use of peer-reviewed publications? How can supervisors encourage counselors to apply their professional experiences along with information from peer-reviewed publications?). Taking action to increase the encouraging factors may increase the rates at which counselors are using information from peer-reviewing publications with clients. Relatedly, future research should examine whether the amount of counselors’ reading and using peer-reviewed publications have any discernable effect on their service delivery or on their clients’ outcomes. Future research should also examine whether certain types of peer-reviewed publications are more likely to affect counselors’ service delivery or client outcomes. This would allow recommendations to be made about both the amount and type of peer-reviewed publications that counselors should consume to achieve the best client outcomes.
Finally, we believe the current study has implications for how publications are used to evaluate job candidates and researchers in rehabilitation counseling. There are several incentives to publish research. For example, researchers’ publications are routinely used to evaluate job applicants and job performance, so there are professional incentives for authors to publish. Publications also should improve rehabilitation counselors’ service delivery and client outcomes, so there also are beneficent incentives to publish. Although publications will ideally achieve both of these goals, it is possible that researchers’ publications achieve the former goal but not the latter: Researchers may “get published” but those publications may not actually affect client outcomes. Incentives should be in place to evaluate and reward researchers’ contributions with their publications above and beyond the quantity of their publications. Specifically, incentives should ensure that researchers study topics that are relevant to rehabilitation counselors and service delivery, editors and reviewers select publications based on the merits of the research question and rigor of the methods rather than the results obtained, and the results of that research should be easily acquired and used by rehabilitation counselors.
Collectively, we believe these aforementioned actions have the potential to greatly improve rehabilitation counselors’ service delivery and, ultimately, client outcomes.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.