Abstract
Purpose:
Following up an Research on Social Work Practice article published a decade ago, this study aimed to examine reference error rates among five, widely circulated social work journals.
Methods:
A stratified random sample of references was selected from the year 2013 (N = 500, 100/journal). Each was verified against the original work to detect errors among author name(s), publication year, article title, journal title, volume number, and page numbers. Interrater consistency was 0.88.
Results:
In the sample, 163 (33%) references contained at least 1 error, producing 258 total errors. Author names held the highest error rate (0.26) and the volume number held the lowest (0.04). The highest error rate was found in Social Service Review (0.48), statistically significantly higher than the remaining journals.
Discussion:
Reference accuracy in social work journal articles has increased marginally. Substantial reference errors in articles among widely circulated journals may portray an aggregate lack of polished, scholarly writing/editing skills within the profession.
Reference list errors are barriers to several areas of research such as the ability to retrieve background information (Fenton, Brazier, De Souza, Hughes, & McShane, 2000). Fenton et al. further stated that reference list errors may cast doubt on the capability and trustworthiness of the researcher. Drs. Christina A. Spivey and Scott E. Wilks (2004) published an article in Research on Social Work Practice that exposed such errors—namely, types of errors and error rates—in prominent, social work journal articles. Until now, it has remained the lone article focused on reference accuracy within the social work discipline. The purpose of the current study was to follow-up on the Spivey and Wilks’ study, deferring to their methodology within a current time frame of the same journals. We begin with a review of literature relative to reference list error rates.
Background
According to the sixth edition of the Publication Manual of the American Psychological Association (APA; 2010), references must be correct and complete, with the best way to ensure accuracy being checking each reference against the original publication. Referencing sources with citations is essential in serving as a link between the source being cited and the citing document (Waytowich, Onwuegbuzie, & Jiao, 2006). Incorrectly citing references can have serious consequences including inhibiting the retrieval of articles, raising doubts about the author’s credibility, and reflecting poorly on the publishing journal (Faunce & Soames Job, 2001; Onwuegbuzie, Frels, & Slate, 2010; Spivey & Wilks, 2004). Reference errors are not a current phenomenon and the ramifications of what appear as simple mistakes in citations can take years to correct, if they are ever corrected (Faunce & Soames Job, 2001).
Several studies have sought to examine citation errors in various disciplines. Most examinations of error rates, the percentage of referenced citations containing at least one error in comparison to the total number of references reviewed, have been in medical journals (Spivey & Wilks, 2004). Citation errors in medical journals are an ongoing problem, and error rates in previous studies ranged from 10% to 41% in disciplines from general medicine to dentistry (Adhikari & Bhandari, 2011; Ngan Kee, Roach, & Lau, 1997). Fenton, Brazier, De Souza, Hughes, and McShane (2000) noted reference errors as high as 56% in medical journals.
Regarding the field of social work, Spivey and Wilks (2004) examined a stratified random sample of 500 references from the most circulated social work journals in the year 2000. They found that 206 (41.2%) of the referenced citations contained at least one substantial error—that is, error that undermines the ability to find the original source—with percentages for the five journals ranging from 29% to 57%. Spivey and Wilks further noted that the overall error rate for the examined, social work journals was relatively high in comparison to journals from other professions.
Some studies that have recognized the importance and severity of reference errors have also made recommendations to alleviate the frequency of said errors. For example, Onwuegbuzie, Frels, and Slate (2010) noted that authors should offer a more concerted effort to check their citations and always locate the original source whenever possible. Other suggestions include allowing only a limited number of references in submitted manuscripts, inclusion of the first page of a cited reference, and an editor-appointed individual to focus solely on reference lists (Adhikari & Bhandari, 2011; Fenton et al., 2000).
Citation errors are often deemed the sole responsibility of authors; yet, authors can collaborate with article reviewers and/or editorial staff to assist them in eliminating errors in their own works (De Jong, 2004; Onwuegbuzie et al., 2010). De Jong (2004) suggested that relevant parties compare the in-text spelling of the authors’ name to the corresponding citation, ensure that the cited year corresponds with the year noted in the text, and check the page numbers against original articles. In a later study to their original work on reference errors, Wilks and Spivey (2004) concurred with the notion reference list accuracy is a shared responsibility between authors and journals, with primary responsibility falling upon authors.
Hypotheses
In an effort to explore reference list accuracy in more recent, social work journal articles, the current study conducted a follow-up to the Spivey and Wilks’ (2004) work with the similar hypotheses. For Hypothesis 1, we posit that error rates are significantly different among popular, social work journals. These were the same journals from the original Spivey and Wilks’ work. For Hypothesis 2, we posit that the likelihood of error within a reference pre-2004 is significantly different than that from a reference on/post-2004. This 10-year threshold followed a similar time threshold set by Spivey and Wilks. For Hypothesis 3, we posit that error rates of references with multiple authors are significantly different than those references with a single author. To paraphrase Spivey and Wilks (2004, p. 283), too many authorial chefs may spoil the broth.
Method
Methods in the current study are replicated faithfully to the Spivey and Wilks’ (2004) study. We wished to review if accuracy has improved, if any, a decade after the aforementioned, original study was published. The same social work journals were selected as those in the original study (Spivey & Wilks, 2004): Journal of Social Work Education (JSWE), Clinical Social Work Journal (CSWJ), Social Service Review (SSR), Social Work (SW), and Social Work Research (SWR). Spivey and Wilks based their selection of these five journals on the following criteria: “(a) they are exclusively social work-focused, and (b) they are highly circulated compared to other social work journals” (p. 283). Volumes of each journal for the year 2013 were selected for the current study. The sampling frame consisted of all journal article references in each of the five journals (4,168 references). Journal article denotes published research manuscripts in scholarly journals and did not include book reviews, books/textbooks, editorials, or letters to the editors. A computer-generated stratified random sample was obtained from the five journals. From each journal, 100 references were selected, with a final N of 500. When a reference’s original source could not be located, a replacement reference was selected using a random numbers generator.
Each reference was verified against the original work for accuracy in the following six fields: (a) article title, (b) author name(s), (c) journal title, (d) pagination, (e) volume, and (f) year. Issue numbers were not included as a field as APA (2013) does not require the use of an issue number in the reference. Article title and journal title field errors included misspellings, omitted words, and omitted grammatical notation, such as colons and dashes. The author field errors included omitted names, misspellings, inaccurate or omitted initials, and incorrect order of authors.
To verify the congruence of findings among the current study’s three raters, interrater reliability was determined using a computer-generated, random stratified sample of 50 references, that is, 10% of the full sample and 10 references per journal. Each rater, separate from the other raters, reviewed all 50 references and marked errors in the six aforementioned fields. After such, each rater compared her or his error markings among the remaining group of raters. Unanimity of error markings among all raters was noted as interrater reliability for this study. Said unanimity of errors was observed at 88%, above the acceptable threshold of 75% (Fleiss, 1981).
Results
Descriptive Statistics
Quantile-quantile (QQ) plots were used to check the assumption of normality; data were distributed normally with no outliers. A Durbin–Watson test of independence supported data independence. A Levene’s test supported homogeneity of variances within the sample.
In examining the total sample (N = 500 references), 163 (32.6%) references contained at least one error. The overall number of errors, 258 in total, indicated a mere four error decrease from the original Spivey and Wilks’ (2004) study. The number of errors per category was as follows: author field, 26% (18.3% lower than the original study); article title, 23.6% (4.9% higher than the original study); journal title, 14.3% (8.6% higher than the original study); pagination, 11.2% (7.5% lower than the original study); year, 4.2% (0.8% higher than the original study); and volume, 3.9% (5.3% lower than the original study).
The number of errors per reference from each journal’s subsample (n = 100 per journal) were observed as follows:
SSR, 48%;
SW, 36%;
SWE, 30%;
CSWJ, 29%; and
SWR, 20%.
SSR contained the highest average number of errors (0.87) while SWR contained the lowest (0.30). Table 1 illustrates central tendency and dispersion for the number of errors per reference per journal.
Means and Standard Deviations of Errors per Reference.
aNote that 0.52 refers to the grand mean, the average of errors per reference in the entire sample (N = 500).
Hypothesis 1 Results
A one-way analysis of variance (ANOVA) was conducted to evaluate the relationship between journal and number of errors in its reference lists. The overall factor, journal, contained five attributes, representing each journal in this study. The outcome was the number of errors among the six error categories (previously discussed). The ANOVA was significant, F(4, 495) = 7.770, p = .000, η2 = .05. Bonferonni post hoc analysis distinguished significant, pairwise differences among reference error means as follows (in order, beginning with greatest mean difference): SSR and SWR, mean difference = 0.280 (p < .001); SSR and JSWE, mean difference = 0.210 (p < .05); SSR and CSWJ, mean difference = 0.200 (p < .05); and SSR and SW, mean difference = 0.200 (p < .05).
Hypothesis 2 Results
A Pearson χ2 test was utilized to evaluate the relationship between age of the reference (pre 2004 vs. 2004+) and existence of error (yes/no) for said reference. The result of the test was not significant, that is, χ2 (1, N = 500) = 0.698, p = .127. Data on Hypothesis 2 are shown in Table 2.
Cross-Tabulation: Age of Reference and Existence of Error in Reference.
Hypothesis 3 Results
A Pearson χ2 test was also utilized to evaluate the relationship between the number of authors in a reference (single vs. multiple) and the existence of error in said reference. Again, the result was not significant, that is, χ2 (1, N = 500) = 0.186, p = .357. Data on Hypothesis 3 are illustrated in Table 3.
Cross-Tabulation: Number of Authors in a Reference and Existence of Error in Reference.
Discussion and Application to Practice
Summary of Findings
The purpose of this study was to follow-up on the Spivey and Wilks’ study (2004) with similar methodology and examine reference errors among five, widely circulated social work journals. The percentage of references in the sample with at least one error was 32.6%, which is 8.6% lower than the corresponding result from the original study (Spivey & Wilks, 2004). Descriptive statistics showed that SSR contained the highest percentage of references with at least one error while SWR contained the lowest. These findings contrast the original Spivey and Wilks’ study (2004), which showed CSWJ and SSR to contain the highest and lowest percentages of reference errors, respectively.
Based on ANOVA, Hypothesis 1 was empirically supported. A significant difference in reference list error rates was observed among the selected social work journals. Post hoc analysis showed that SSR contained the highest level of errors, significantly different than the other journals. A similar hypothesis in the original Spivey and Wilks’ work (2004) showed CSWJ to be in the position of SSR as noted in the current study.
Hypothesis 2 was not supported. No significant relationship existed between reference age and existence of error in said reference. Hypothesis 3 was also unsupported. No significant relationship existed between a reference’s number of authors and existence of reference error. The findings for these two hypotheses were comparable to the findings of similar hypotheses in the original Spivey and Wilks’ study (2004).
Implications
Although the responsibility of proofing a reference list prior to publication appears to lie with the author, a uniform system of citations should be developed to allow for continual checks against the original reference (Faunce & Soames Job, 2001). In order to maintain reference integrity in peer-reviewed journals (including those in social work), we believe that a system of checks and balances should be developed with regard to reference lists. Editors can help to decrease error rates by conducting random audits of articles or by requesting the submission of primary references (Gosling, Cameron, & Gibbions, 2004). Authors and editors may also want to consider limiting the number of references for an article prior to submission (Adhikari & Bhandari, 2011).
Strategies that can used by authors to guard against reference errors include, but are not limited, looking for errors in author names (e.g., ensuring a last name and first initial), comparing author names in the reference list with names in text citations, and monitoring the spelling of authors names that appear most often (De Jong, 2004). An additional strategy is the use of writing style software (e.g., PERRLA for APA and StyleEase version 2), which many authors use to reinforce structural templates for in-text and reference citations (PERRLA, Inc., 2015; StyleEase Software, 2013). While accuracy of writing style within software programs has improved since the original Spivey and Wilks’ study (2004), no program is infallible; thus, the exclusion of errors in references remains authorial responsibility. In addition to the above-mentioned suggestions, the original study (Spivey & Wilks, 2004) mentioned an additional, possible solution—one that remains relevant and feasible a decade later—for editorial staff to increase reference list accuracy. They suggested a requirement for all manuscript submissions to include the first page of all sources referenced in said manuscripts.
A number of reasons exist to explain the plethora of reference errors in published articles. One, in particular, can be addressed in the classroom and virtually eliminated by social work educators. A reliance of secondary sources citations, rather than a commitment to finding the original source, could result in reference errors. Reflecting the sentiments of Froese, Boswell, Garcia, Koehn, and Nelson (1995, p. 236), “Citing secondary sources … precludes students from exercising discernment expected of professionals.” If the secondary source contains citation/reference errors, then reliance on the secondary source will lead to the same errors. We suggest a simple but effective solution to this matter, that is, a prohibition of any secondary source reference in paper assignments. This would eliminate a number of poor writing habits, including laziness (in finding the original source), unintentional plagiarism (Froese, Boswell, Garcia, Koehn, & Nelson, 1995), and potential reference errors.
A final note on the importance of reference list accuracy concerns systematic reviews. Increasing in frequency among health literature (Harris, Quatman, Manring, Siston, & Flanigan, 2013), systematic reviews, including statistical meta-analyses, offer a synthesis of scholarly information to inform evidence-based practice and policy. Authors of these reviews rely heavily on accuracy of reference lists to guide their searches for relevant studies. Consequently, reference errors undermine the methodology of this valuable research.
Limitations and Future Research
Although this study demonstrated the commonality of reference list error rates among selected social work journals, it is not generalizable to all social work journals. The current study is limited as it evaluated five of the several social work journals in print and online. Also, the current study did not examine all references, only journal article references. We cannot assume that inclusion of other works published in journals (e.g., editorials, letters to the editor, book reviews, etc.) would reveal similar numbers of reference errors. Lastly, the focus of this study was substantial error without delving deeper into statistics per error. Future research analyzing percentages per substantial error may be interesting to determine which errors are committed egregiously.
Conclusion
We concur with Spivey and Wilks’ (2004) assessment that reference list accuracy is not a trivial matter. Although not as essential as evidence-based research that improves practice competencies with vulnerable populations, scholarly inquiry into reference accuracy, including prevalence of lack thereof, holds merit. Articles housed in widely circulated, peer-reviewed journals reflect, at least on the surface, their respective disciplines’ abilities to educate, comprehend, and achieve rigorous writing protocols. The social work profession holds its practices into account via empirical scrutiny. Its scholarly writing standards should not be immune to similar scrutiny.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.