Abstract
We conducted a meta-study of research articles published in Rehabilitation Counseling Bulletin from 2001 to 2012. Variables associated with authors (biological sex, setting, domicile) and study characteristics (sample and research methodology) were coded and analyzed across groups of 4-year publication periods. Our findings and related implications for the rehabilitation counseling profession are discussed.
The American Rehabilitation Counseling Association (ARCA) was established as a professional community for counselors working with individuals who have disabilities across their life span (ARCA, 2013). Since its inception, ARCA has been associated with stimulating the legislative advocacy, educational experiences, and counseling practices that promote excellence within the rehabilitation counseling profession. Organizations such as ARCA have been in the forefront for bridging the gap between science and practice through professional development supports including advocacy, leadership training, continuing education, and scholarly publications. Given that the anticipated growth rate for knowledgeable and skilled rehabilitation counselors between 2010 and 2020 (28%; Bureau of Labor Statistics, 2012) is faster than average, it is a prudent task for counselors and counselor educators to be familiar with trends in the empirical activities that inform best practices with clients.
The Rehabilitation Counseling Bulletin (RCB) was first published in 1956 and today is a quarterly, peer-reviewed periodical that disseminates empirical research to ARCA members and the professional community at large. Issues of RCB contain original empirical research, theoretical essays, comprehensive literature reviews, intensive case studies, research critiques, and media reviews that inform best practices for rehabilitation counselors. The RCB editorial staff has historically fulfilled this task through illustrating current trends, and the current editor Douglas Strohmer has continued this legacy while overseeing the publication of four special issues to promote the utility of topics for the RCB readership. Several scholars (Charkow & Juhnke, 2001; B. T. Erford et al., 2011; Southern, Gomez, Smith, & Devlin, 2010; Weber, 1990; Williams & Buboltz, 1999) have suggested that given the role of counseling journals for propagating information that informs best practices, it is necessary to periodically conduct systematic analyses of trends, prevailing topics, and identify current needs to guide contemporary scholars. Precedent for these reviews has been well-established within American Counseling Association division journals, including the Journal of Addictions and Offender Counseling (JAOC; Juhnke, Bordeau, & Evanoff, 2005), Measurement and Evaluation in Counseling and Development (MECD; B. T. Erford, Miller, Duncan, & Erford, 2010), The Journal of Humanistic Counseling, Education, and Development (JHC; B. E. Erford, Erford, & Broglie, 2012), The Family Journal (Southern, 2006), Journal of Counseling and Development (JCD; B. T. Erford et al., 2011), Journal of Mental Health Counseling (JMHC; Streicher & Gerstein, 1994), and Journal of College Counseling (JCC; Byrd, Crocket, & Erford, 2012). To date, systematic analysis of publication patterns has not been conducted for RCB; the findings of a meta-study of publication patterns related to RCB may provide a resource for scientist-practitioners and counselor educators to develop research agendas that implement understudied populations, topics, and methodologies.
The purpose of this publication trend analysis was to answer two research questions:
Furthermore, we were interested in identifying the degree that these author and article characteristics have changed during a 12-year publication range from 2001 to 2012. We will provide an explanation of our methodology for conducting this meta-study of publication patterns, report results, identify trends detected, and discuss the implications of our findings as they relate to scholarly activities within the field of rehabilitation counseling.
Method
We examined all articles published in RCB during a 12-year period between 2001 and 2012. Three master’s-level students enrolled in a Council for the Accreditation of Counseling and Related Educational Programs (CACREP) clinical rehabilitation counseling program (second through fourth authors) independently coded articles within the review period. Each of the reviewers had completed master’s-level coursework in research methods and statistics, and received training from the first author on identifying relevant article characteristics and distinguishing between related categories of interest; during independent coding, the first author was available for consultation as needed to help identify the correct code for article characteristics. Primary author characteristics of interest included biological sex, author setting (university or nonacademic), and domicile (national or international).
The coding of research article characteristics followed the precedents established by Erford and colleagues (Byrd et al., 2012; B. T. Erford et al., 2010; B. T. Erford et al., 2011), and was conceptualized through a bipartite formulation of variables related to (a) the sample and (b) the methodology implemented by the researchers. Sample characteristics coded included sampling procedure (i.e., convenience vs. community recruitment), sample size (i.e., small, N < 30; medium, N = 30–99; large, N = 100–499; very large, N = 500+), ethnic identity (i.e., African American, Asian American, Caucasian, Hispanic/Latino, Other), sample type (i.e., children and adolescents, undergraduate students, graduate students, nonstudent adults), and disability status of participants (i.e., no disability, physical disability, cognitive-developmental disability, co-occurring disabilities). Article characteristics related to methodology included type of research conducted (i.e., quantitative, qualitative, mixed methodology), study focus (e.g., job placement and training, academic achievement, instrument development), research design (descriptive, comparative, correlational, quasi-experimental, experimental, case study), and type of primary statistical analyses used (descriptive, t-test, analysis of variance [ANOVA], correlation, multiple analysis of variance [MANOVA], nonparametric procedures). Although some characteristics were more objectively overt (e.g., data analysis and demographic characteristics), the more subjective item of study focus was determined by identifying the overarching, primary focus of the article. In instances where some ambiguity was present, consultation and consensus was pursued through the authors and confirmed by the first author. Coded data were entered into a Microsoft Excel spreadsheet and grouped into 4-year intervals (i.e., 2001–2004, 2005–2008, 2009–2012) for analysis, and analysis of trend data was completed using Statistical Package for the Social Sciences, Version 20 (SPSS; IBM Corp, 2011).
Data Analysis
Articles included in our analysis included only research articles published in RCB from 2001 to 2012; articles excluded from analysis included theoretical and conceptual articles, as well as, brief non-research-related contributions such as editorials, reactions to publications, and acknowledgments to individuals within the rehabilitation counseling field. We computed descriptive statistics for study variables across the 4-year intervals to identify the frequency of each variable occurring within distinct year ranges. Following is the evaluation of the trends between study variables between the 4-year intervals using the one-way ANOVA procedure with weighted proportions. Several authors (Byrd et al., 2012; B. T. Erford et al., 2010; B. T. Erford et al., 2011) have regarded this procedure as more favorable than repeated measures approaches when completing meta-studies given that the assumption of differences between the same individuals/units over time is violated. Post hoc explorations of significant findings were conducted using Scheffé’s test with an established alpha of .05 to test Type 1 error. Gravetter and Wallnau (2007) suggested that Scheffé’s test is an extremely conservative measure for assessing Type 1 error and is indicated for use with unequal sample sizes and, in our case, when group means are weighted. Finally, effect size was computed using eta squared (η2) and interpreted using the range of .01 = small effect, .09 = medium effect, and .25 = large effect as a guide.
Results
Three hundred twenty articles were coded for the publication period from 2001 to 2012. Of these articles, a notable proportion (n = 201; 62%) were empirical investigations that implemented quantitative, qualitative, or mixed methodologies to study variables of interest. Over time, RCB has predominately published quantitative studies (n = 167; 83%) to a greater degree than qualitative or mixed methodology investigations with no significant variation in this trend across publication periods, F(2, 198) = 0.24, p = .78, η2 = .002.
Author Characteristics
Frequencies and percentages for author characteristics across date ranges are presented in Table 1. Analysis of the biological sex for primary authors was completed to determine the proportion of men and women contributing to RCB over time. During the publication period from 2001 to 2012, sex of the primary author has trended toward statistical significance from being predominately associated with men to women, F(2, 196) = 2.36, p = .09, η2 = .02. From 2001 to 2004, 42 of 69 (61%) primary authors were men, whereas during the more recent period from 2009 to 2012, this proportion had decreased to 31 of 65 authors (48%).
Biological Sex, Employment Setting, and Domicile for Primary Authors Publishing in Rehabilitation Counseling Bulletin From 2001 to 2012.
Although the prevalence of women as primary authors within RCB has increased across publication periods, authors have continued to be predominately associated with a university rather than nonacademic institutions, F(2, 196) = 0.24, p = .78, η2 = .002. During the publication period from 2005 to 2008, nonacademic authors represented 9% of all research articles published; however, this was the height of this trend and represented a marginal change when compared with the 6% of articles during the years from 2001 to 2004 and 8% from 2009 to 2012. At this point in the publication history of RCB nearly all published research articles are authored by individuals residing within the United States. Whereas the prevalence of international authors has increased over time, this trend has not been significant across publication periods, F(2, 196) = 1.52, p = .22, η2 = .01.
Study Characteristics
Sample characteristics
Frequencies and percentages for sample characteristics across date ranges are presented in Table 2. Among published studies reporting sample size (n = 187 of 201; 93%) the majority of them have evaluated participant samples regarded as large (i.e., N = 100–499). This trend has remained stable between publication periods with reporting of findings associated with very large samples being secondary in prevalence, F(2, 185) = 0.31, p = .73, η2 = .003. Analysis of sample population type indicated that the overwhelming majority of participants across the publication periods reviewed have been associated with community-based, nonacademic affiliated adults, F(2, 182) = 0.38, p = .68, η2 = .004. Table 2 reveals that this population represented 81% of participants during the period from 2001 to 2008, with this percentage decreasing only slightly to 77% from 2009 to 2012. Furthermore, RCB has remained consistent over time in publishing research with individuals who have physical and/or co-occurring disabilities, F(2, 184) = 1.27, p = .28, η2 = .01. Although some articles published have utilized samples of individuals without a disability, these two population characteristics represent 58% of studies during 2001 to 2004, 54% of studies from 2005 to 2008, and 66% of those published within the 2009 to 2012 publication period.
Frequencies and Percentages for Sample Size, Population Type, Ethnic Identity, and Identified Disability for Research Articles Published in Rehabilitation Counseling Bulletin From 2001 to 2012.
One participant characteristic that has not remained constant over time is the ethnic identity, F(2, 160) = 7.83, p < .01, η2 = .08, indicative of a medium effect size. Review of Table 2 illustrates the transition from primarily Caucasian participant samples from 2001 to 2004 (85%), to more diverse populations in recent years. Post hoc comparisons of publication periods using Scheffé’s test revealed that the date range from 2001 to 2004 differed significantly from the most recent publication period 2009 to 2012, but that median period from 2005 to 2008 did not differ significantly from either publication range.
Research methodology
Analysis of changing trends in research methodology implemented within studies published in RCB from 2001 to 2012 variables related to sampling procedure, study topic, research design, and statistical analysis procedure. Our analysis detected a stable trend indicating that the overwhelming majority of studies across publication periods utilized a community recruitment sampling procedures in favor of convenience sampling, F(2, 191) = 1.41, p = .24, η2 = .01. Furthermore, Table 3 illustrates the consistent and longstanding history for RCB to include research articles that address interventions associated with wellness as well as job placement and training. Although some other topics such as academic achievement have demonstrated an increased prevalence, these priority topics have remained constant across publication periods, F(2, 197) = 0.34, p = .70, η2 = .003.
Frequencies and Percentages of Research Study Topics for Articles Published in Rehabilitation Counseling Bulletin From 2001 to 2012.
Analysis of the research study design type indicates that despite topics remaining consistent, the strategies that researchers are implementing in their investigations have changed significantly over time, F(2, 195) = 9.15, p < .01, η2 = .08, indicative of medium effect size. Post hoc comparisons of publication periods using Scheffé’s test revealed that although methodologies used by researchers during the date ranges from 2001 to 2004 and 2005 to 2008 did not differ significantly from one another, they both differed significantly (p < .05) from the 2009 to 2012 period. Table 4 illustrates that the detected trend is associated with an increasing trend for articles to report use of descriptive approaches to research as predominate research methodology. Despite this noted change in methodologies that have been implemented during the review periods, a stable trend for researchers to rely on descriptive and correlational statistical procedures across review periods was detected F(2, 192) = 1.20, p = .30, η2 = .01. Review of Table 4 illustrates that these two statistical analysis procedures were represented in 71% of studies during 2001 to 2004, 74% of studies from 2005 to 2008, and 89% of those published within the 2009 to 2012 publication period.
Characteristics of Research Designs and Primary Statistical Procedures for Research Articles Published in Rehabilitation Counseling Bulletin From 2001 to 2012.
Note. ANOVA = Analysis of variance.
Discussion
The results of this meta-study indicate that while several characteristics of research articles published in RCB from 2001 to 2012 have remained constant over time, others have changed along with the focus and scope of the counseling profession. This development has been noted by authors of other meta-studies completed with American Counseling Association (ACA) division journals such as MECD (B. T. Erford et al., 2010), JCD (B. T. Erford et al., 2011), and JCC (Byrd et al., 2012). Discussion of the implications associated with our results is presented herein first by addressing author characteristics and then features of studies themselves.
Author Characteristics
The results of this meta-study revealed that the proportion of women publishing research as the primary author within RCB has continued to increase across publication periods. Although this change was not statistically significant, a 13% proportional increase over a 12-year period represents a trend that reflects congruence with the increased presence of women within the counseling profession and counselor education programs. The finding by Michalski, Kohout, Wicherski, and Hart (2011) that approximately 75% of individuals receiving doctoral degrees across helping professions were women may explain the recent change in this proportion; as more women enter the field at the doctoral level, it is reasonable to conjecture that these individuals may be engaging in scholarship associated with rehabilitation counseling populations. Given that many assistant professor faculty positions continue to be evaluated along domains of research, teaching, and service, it is plausible that this trend may persist.
Our analysis also detected that the overwhelming majority of primary authors of research articles published in RCB from 2001 to 2012 were affiliated with academic institutions rather than other nonacademic organizations. As mentioned in relation to the prevalence of women as primary authors, this finding may also be associated with the premium placed on scholarly activity within academia. This finding is curious given that the preponderance of rehabilitation counseling is provided by community-based and government-affiliated groups rather than in higher education settings. It can be argued that academicians frequently pursue large external funding opportunities that lead to service provision to individuals in the community; however, the fact remains that the majority of rehabilitation counseling services are provided in venues such as schools, government-funded institutions, independent-living facilities, and community-based rehabilitation agencies. It may be prudent for RCB to solicit articles associated with program evaluation, best practices, or professional development issues from practitioners in nonacademic settings to reflect a more complete representation of the professional landscape. An increase in scholarly contributions from nonacademically affiliated professionals may be promoted through workshops, trainings, or other collaborative activities initiated by groups such as ARCA. With this perspective about the state of the profession counselor educators may be able to accommodate training curricula during the 28% expansion of rehabilitation counseling jobs from 2010 to 2020 predicted by the Bureau of Labor Statistics (2012).
Finally, primary authors submitting to RCB have consistently been located within the United States rather than international domiciles. Although the proportion of authors from international domiciles increased from 2% during the 2001 to 2004 publication period to 8% from 2009 to 2012, this development does not reflect the emphasis on international perspectives currently being espoused by the counseling profession as discussed by Byrd et al. (2012). RCB is certainly not the only ACA division journal that has remained represented by a domestic authorship; however, some division journals such as MECD and JCD have actively pursued international submission which has resulted in a greater diversity of client populations and professional issues represented with their publications. Given that the preponderance of RCB readership is located in the United States, the degree that similar solicitations and special sections may be useful is not certain. One possibility would be to establish a special issue of RCB that addresses international issues within rehabilitation counseling and survey the interest of readership for similar sections following its inclusion.
Study Characteristics
Sample characteristics
The results of this meta-study indicated that studies published in RCB have historically utilized large population samples (i.e., 100 to 499) to study variables of interest. In general, large samples are regarded as a methodological strength given the greater ability to generalize findings to the population being investigated (Sheperis, Daniels, & Young, 2010). It may be speculated that this link with the advantage of generalizability to client populations makes RCB a practical resource for its readership and other counseling professionals. An associated contention may be postulated given our findings that participant samples have consistently been drawn from nonacademic, community-based client samples. This finding challenges the objections by authors such as Henrich, Heine, and Norenzayan (2010), who have continued to cite the overuse of college undergraduates as research participants as a major limitation to social sciences research. In addition, the relevance of study findings to the related client population may be inferred from our finding that the disability status reported by study participants across publication periods has been physical and/or co-occurring disabilities. Finally, we detected an emerging trend indicating that RCB authors are increasingly accessing more diverse population samples which is consistent with the ACA and ARCA predilection for conceptualizing client issues through a perspective that acknowledges pluralistic worldviews and providing interventions that reflect multicultural sensitivity. Although the majority of participants continue to have a Caucasian ethnic identity, the use of large, community-based samples that report disabilities that counselors may encounter in applied settings appears to be self-justifying factors for the relevance of research findings published in RCB.
Research methodology
Across publication periods authors publishing in RCB relied on community recruitment of participants in favor of convenience sampling. This finding is expected considering the stable trends noted in the analysis of participant characteristics. In a related manner, community recruitment practices may be preferable to convenience sampling with the exception of studies intended to evaluate domains such as attitudes toward individuals with disabilities, counselor education/training practices, and student access to disability services. It is plausible that if rehabilitation counseling services continued to be provided in community-based settings this trend will continue within research articles published in RCB.
Analysis of the topics included within RCB research articles revealed an established trend over time for evaluating practices related to job placement/training and wellness. We submit that these topical areas are well within the scope of relevant practices for rehabilitation counselors given occupational descriptions provided by ARCA (2013), the Bureau of Labor Statistics (2012), and the Council on Rehabilitation Education (2013). Although job placement/training and wellness are acceptable topics for investigation, several areas that may also be relevant to rehabilitation counselors have been largely understudied. For example, innovative practices for individuals and group interventions targeting a number of relevant counseling outcomes may provide a useful reference for practitioners. In addition, our analysis detected a minority of articles addressing the initial or secondary development of assessment instruments that may facilitate client diagnosis, treatment planning, access to services, or other aspects of service coordination. Given that many popular assessment instruments lack normative data for populations of individuals that have a disability, submissions that provide this information for readers to reference may contribute to the diversity and utility of RCB to other researchers. Other topics that may contribute to more comprehensive topical content include psychological factors associated with partner/family resiliency, crisis intervention/postvention practices, and client perceptions and attitudes associated with social justice/advocacy issues and public policy issues. We conjecture that studies evaluating these aspects of the rehabilitation counseling profession may provide a more holistic and accurate representation of practitioner activities and broaden the knowledge base available for reference.
Our analysis indicated that the research designs used to investigate topics of interest have become disproportionately descriptive in nature. Descriptive studies have an important role within developing institutional and public policies that may be associated with improvements in clients’ quality of life. This utility of descriptive research designs has particular importance to rehabilitation counseling professionals who are often called to advocate for clients to third parties; however, descriptive research designs are generally regarded as providing only modest contributions to evidence-supported and evidence-based practices within helping professions (Rubin & Bellamy, 2012). Several authors (Chambless et al., 1998; Chambless et al., 1996; Lenz, 2013; Rubin & Bellamy, 2012) have suggested that the use of research designs that promote causal inferences is a prudent practice within a professional culture that is increasingly emphasizing accountability through credible interventions associated with measurable outcomes. Therefore, inclusion of research designs such as randomized clinical trials, experimental/quasi-experimental designs, and single-case research designs that allow researchers to make causal inferences regarding the effectiveness of counseling interventions may be a valuable support to the field. Because the methodological design selected to answer research questions often informs statistical procedure, we were not surprised to find that the preponderance of statistical analyses detected in this meta-study were associated with reporting descriptive data and correlation strategies (bivariate, regression, etc.). As with research design, statistical procedures can be organized along a continuum from modest to strong. Researchers are encouraged to implement more rigorous correlation procedures such as canonical correlation, regression analyses, and structural equation modeling as appropriate to their needs.
Limitations and Recommendations for Future Research
Some important limitations inherent within meta-study procedures are noted within our application of the methodology. Foremost, meta-studies provide descriptive information about the authors and types of studies being published in a journal and are not suited for causal inferences related to the degree that a publication is addressing societal and/or professional issues. Also, this study coded 14 variables across 201 research articles for a total of 2,814 data points; we acknowledge that coding errors may be present despite the coders’ educational preparation, orientation and training prior to coding, cross-checking by the lead author, and a system for consultation to remediate any uncertainty regarding target variables. Given that we implemented broad categories (e.g., wellness) for coding rather than more specific ones (quality of life, physical activity, spirituality), it is possible that other research teams may have subjectively identified variables differently from how we did. Future researchers may want to consider implementing a more specific set of coding criteria and an auditing system facilitated by a third party not associated with the research team.
Although a decision was made a priori to limit the discussion of author characteristics to primary authors, further exploration of author characteristics such as specific role within their organization/university and ethnic identity may have provided an additional perspective for consideration. The inclusion of junior authors in future studies would also allow for the identification of additional outcomes such as the most represented institutions and individuals within the journals history. Our findings were also limited by the underreporting of participant demographic data such as ethnic identity, mean age, and frequency of men and women within some participant samples. The American Psychological Association (2010) recommended that authors promote transparency of research practices and promote generalizability of findings through reporting of some standardized elements of a study; the lack of this data may have skewed our results. Also, our analysis only included the primary analysis implemented by researcher to answer research questions. Future investigators may consider evaluating additional approaches including those related to preliminary evaluation (e.g., dealing with missing values, assessing multicollinearity, etc.), statistical power analyses (e.g., a priori, sensitivity, etc.), post hoc methodologies (Scheffé’s. Tukey’s, etc.), and gender differences between data trends. Finally, it is possible that different data range intervals such as 3- or 5-year intervals may yield different trends across variables. To account for this fact, researchers such as Juhnke et al. (2005) have conducted follow-up analyses of journals after a reasonable amount of time has passed to allow for the development of trends.
Conclusion
The periodic review of publication patterns provides a snapshot of professional values through the evaluation of who is completing research, what topics are of interest, and how empirical practices are being implemented (B. T. Erford et al., 2010). Our goal was to provide an initial evaluation of author and article characteristics associated with RCB in an effort to support scientist-practitioners and counselor educators in developing research agendas that implement understudied populations, topics, and methodologies. Our findings are intended to support researchers, editorial board members, and the training of future rehabilitation counselors as they make strides toward increasing the functioning of clients and advancing the rehabilitation counselors’ role within the helping professions.
Footnotes
Acknowledgements
The authors acknowledge Douglas Strohmer, PhD, editor of Rehabilitation Counseling Bulletin for his support during the preparation of this article.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.