The More You Know

Abstract

The New School Psychology Bulletin (NSPB) is a peer-reviewed journal operated by clinical psychology graduate students. Forty-four members of the editorial board and 27 authors were surveyed before and after working with NSPB. Results of the survey demonstrated that experience with the publication process resulted in quantitative decreases in confidence, proficiency, and interest in publishing in the future. Despite these drops, respondents reported that NSPB was relevant and effective training, and they rated NSPB and their experience favorably. They also reported that it provided them with unique gains not available in their training programs. A follow-up study 2 months after termination explored the decreases in confidence, proficiency, and interest. Qualitative data suggest that after working with NSPB, about half of respondents felt more optimistic about the publication process, with the remaining half feeling less optimistic, experiencing no change, or reporting a mixed perception (e.g., feeling less confident but believing that they had a more realistic perspective). Taken together, the data suggest that early experience with publication and peer review disparately affects students’ attitudes toward publishing.

Keywords

New School Psychology Bulletin publication and peer-review training graduate student perceptions graduate student journal

The American Psychological Association (APA) has recognized the importance of training clinical psychology graduate students in both clinical and academic skills. The existence of scientist-practitioner or practitioner-scholar models reflect the ideology that clinical training and research practicum should be balanced in graduate school training, but very few programs emphasize both domains equally (Addis & Jacob, 2000; Bernstein & Kerr, 1993; Chang, Lee, & Hargreaves, 2008; Hunt & Wisocki, 2008; Maher, 1999). Despite the expectation of competence in academic skills, a gap remains between students’ training and the demands they face after graduation in the areas of scientific writing, manuscript preparation, peer review, and academic publishing (Chang et al., 2008). For example, the peer-review process has received limited attention in doctoral training programs (Hopwood, 2010; Hopwood & Stocks, 2010) despite calls for it in the field (Mischel, 2008; Roediger, 2007). Psychology graduate students also perceive gaps in their training (Brown, Antonius, & Hirst, 2009) and have expressed wanting more scientific training (Merlo, Collins, & Bernstein, 2008). A meta-analytic review on this subject concluded that graduate training programs are not effectively preparing students for the realities of the academic profession (Chang et al., 2008).

Evidence suggests that graduate students stand to benefit from training and practice in academic skills. Hopwood (2010) found that students who participated on the editorial boards of student-operated journals reported learning the culture of editing and peer review, academic writing skills, and professional collaboration. Less research addresses the impact of hands-on experience with publishing or reviewing for psychology graduate students, specifically. A survey by Brown, Antonius, and Hirst (2009) showed that psychology graduate students perceived student journals as relevant to their training and effective at enhancing their skills, but the majority of these students had not worked with student journals directly.

Despite identified gaps in training, very few outlets are specifically designed to train psychology graduate students in publishing-related skills. Two student-operated psychological journals exist—the Graduate Student Journal of Psychology at Columbia University, and the New School Psychology Bulletin (NSPB) at the New School for Social Research. NSPB was designed to fill the gaps in training mentioned previously (for a review of NSPB’s history and operations see Antonius, Brown, Todman, & Safran, 2007), and the editors instituted a formal training program in 2010. A number of indicators suggest that participation in scientific journals supplements graduate student training (Dunlap, 2006). Less is known about the impact of graduate student journals or the effectiveness of the training model approach. Some data suggest that psychology faculty members are skeptical about the utility of graduate student journals. A survey of 38 faculty members and training directors found that only about half would endorse a student journal like NSPB and encourage their students to submit to it (Doran et al., 2012). Respondents who were familiar with the journal, or who perceived participation as relevant to internship applications and postdoctoral positions, had higher endorsement rates (Doran et al., 2012). If student journals purport to address training gaps for psychology graduate students, an evaluation of the extent to which they achieve this goal is particularly important.

The aim of the present study was to assess clinical psychology graduate student perceptions and experiences in their training programs, their experiences with NSPB, and the efficacy of NSPB’s training model. We hypothesized that peer reviewers would increase their ratings of confidence, interest, and proficiency in publishing-related activities following their tenure with NSPB, lending support to the utility of NSPB’s training model approach. We also hypothesized that authors would show similar gains, with the caveat that the ultimate publication decision of their manuscripts might impact their perceptions (e.g., authors who had a paper rejected would feel less confident, interested, or proficient in publishing). Finally, we explored students’ experiences in more depth—what they felt their biggest gains were, if they felt that NSPB provided them with experiences not available in their training programs, and whether they changed their perceptions about peer review and publication overall.

Study 1

Participants

Peer reviewers

Out of an applicant pool of 122 clinical psychology graduate students, 48 were offered and accepted positions on the editorial board for the 2010–2011 academic year. Following their acceptance, reviewers were invited to participate in optional research on the efficacy of NSPB’s new training program. All 48 reviewers provided data at intake (August 2010) and 44 reviewers provided data at termination (June 2011). Only participants with data for both time periods were included in final analyses (N = 44). Reviewers ranged in age from 21 to 38 (M = 27, SD = 3.5) and were primarily Caucasian (77%) and women (77%). Of the participants who were not Caucasian, five identified as Asian American (11%), two as African American (4%), and three as “other” (7%). Reviewers came from 33 universities, with the majority from clinical psychology doctoral programs (72%), and the remainder enrolled in a clinical or general master's level program (27%). The majority of the sample (73%) had no previous publications at the beginning of their tenure with NSPB.

Authors

Participants were clinical psychology graduate student authors or coauthors who had submitted a manuscript to NSPB for publication consideration between June 2010 and March 2011. Following their initial submission, authors were invited to participate voluntarily in research on the efficacy of NSPB’s new training program. From the 51 articles received by the journal in the time frame for the study, 73 authors and coauthors were invited to participate in the intake survey. Of these authors, 38 completed the intake survey, and 27 of these participants also completed the termination survey. Data from these 27 participants was used in the final analyses (N = 27). Similar to the peer reviewer sample, authors ranged in age from 22 to 50 (M = 28, SD = 6) and were primarily Caucasian (82%) and women (71%). The remaining participants identified as Asian American (13%) or “other” (5%). Participants came from 29 universities and were enrolled in clinical psychology programs at either the doctoral (76%) or master’s level (24%). The majority of the sample had coauthored at least one publication for another journal (79%). Of note, 11 peer reviewers (22%) also submitted manuscripts in the course of the year. For the purposes of the survey, they were considered both authors and peer reviewers and completed individual surveys separately at the appropriate times.

Procedure

Peer reviewers

Peer reviewers joined the editorial board and took the intake survey in August 2010. Then, they received training materials that included a detailed peer-review training presentation created by the editors of NSPB, a sample peer review, a copy of two articles on peer reviewing (Brazeau, DiPiro, Fincham, Boucher, & Tracy, 2008; Roediger, 2007), and a sample manuscript formatted in APA style. We asked reviewers to familiarize themselves with APA style and encouraged them to purchase a copy of the APA’s Publication Manual (APA, 2009). In addition, one editor audited every first review, critiquing it and providing detailed feedback and suggestions for improvement. Editors gave additional guidance and feedback on an as-needed basis. Editors sent reviewers a copy of the other two reviews for each article they reviewed. Reviewers completed three or four initial reviews during their term, and most provided comments on resubmitted manuscripts as well. In June 2011, reviewers were thanked for their participation in NSPB and invited to complete the termination survey.

Authors

Graduate students may submit a manuscript to NSPB through the website’s online portal. The website directed authors to a resource page, which included writing tips by Roediger (2007), Bem (2003), and Baumeister and Leary (1997). Resources also included a sample manuscript formatted in APA style and a submission checklist. After the initial submission, authors were invited to complete the intake survey. The invitation clearly indicated that their decision to participate would not impact the editorial decision. Approximately 2 weeks after initial submission, authors received NSPB’s training presentation for authors, entitled “Preparing for the Editorial Decision.”¹ This presentation is intended to help authors psychologically prepare for the editorial action letter and the rigors of the revision process. Authors received editorial decisions approximately 4 to 6 weeks after initial submission. Similar to the process used with peer reviewers, authors interacted with the editors on an as-needed basis and received detailed feedback throughout the revision process. Authors were invited to complete the termination survey after a final decision was made on their paper, or when they informed the editors that they would not be resubmitting their manuscript.

Measures

The following questionnaires were created to assess perceptions and specific experiences with NSPB. The majority of questions were quantitative (5-point Likert-type scale ratings, ranging from 1 = not at all or definitely not to 5 = very or definitely yes), but all surveys also included open-ended questions.

Intake surveys

Both surveys included 46 questions and assessed basic demographic information and information about the participants’ educational experiences and career goals. Additional questions examined why participants wanted to work with NSPB or why they submitted a manuscript, what they hoped to gain from this experience, whether anything had stopped them from trying to publish work in the past, and what barriers they perceived to publishing future work. General questions probed the perceived relevance and efficacy of NSPB and its training model. Next, they rated their confidence, interest, and proficiency in peer reviewing, writing, and publishing academic articles as well as their interest in submitting work to peer-reviewed journals in the future.

Termination surveys

Both surveys included 38 questions, many repeated from the intake surveys. Some minor wording changes aimed for more parsimony. Additionally, the number of outcome variables differed for reviewers and authors (e.g., authors did not answer outcome variables regarding proficiency or confidence in reviewing skills). Both reviewers and authors reported what they believed they gained from their tenure with the journal and rated their overall experience with NSPB.

Qualitative Data Analysis

The consensual qualitative research paradigm (CQR; see Hill, Thompson, & Williams, 1997) was used to analyze open-ended responses. CQR is a rigorous iterative process that first divides open-ended responses into domains (topic areas), then constructs core ideas for the material within each domain, and finally creates categories to describe consistencies in the data. Frequencies and percentages were calculated for each coded domain (e.g., what percentage of the sample felt more vs. less optimistic about publishing in the future).

Results

General Perceptions and Experiences

Reviewers and authors reported significant gaps in their academic training. In response to the question, Do you feel that any of the following are missing from your graduate program/training? a large proportion of reviewers (R) and authors (A) responded yes to the following: information on professional journals and their standards/requirements (R = 53% and A = 64%); information about how to publish an article (R = 51% and A = 64%); information about the peer-review and revision process (R = 57% and A = 49%); and information about how to review, critique, or edit academic work (R = 55% and A = 31%). Approximately one fourth of reviewers and authors also indicated that information about how to write scientifically was missing from their programs (R = 22% and A = 26%).

Overall, reviewers gave positive ratings of their experience with NSPB at termination. They reported that the journal is both relevant to psychology graduate students (M = 4.53, SD = 0.74) and effective at providing valuable training experiences (M = 4.71, SD = 0.64); they gave high ratings of the quality of NSPB (M = 4.12, SD = 0.79); they expressed interest in working with the journal in the future (M = 4.33, SD = 0.97); and they indicated that they would recommend the journal to their peers (M = 4.49, SD = 0.91). When asked to give an overall rating of their experience with NSPB on a 1 to 10 scale (1 = worst it could have been and 10 = best it could have been), reviewers gave a mean rating of 8.21 (SD = 1.00). The majority of reviewers stated that NSPB provided them with training not available in their training program (82% answered yes); believed that the training and feedback from the editors was sufficient and helpful (M = 4.44, SD = 0.76); perceived NSPB as important for their chosen career path (M = 4.02, SD = 0.96); and believed that NSPB provided them with training and skills they would not have otherwise received (M = 4.42, SD = 0.82). Additionally, reviewers stated that there should be more opportunities for students to peer review and serve on editorial boards (93% answered yes); felt that they gained insight into the peer-review and publication process (M = 4.19, SD = 0.88); and expressed an interest in peer-reviewing in the future (M = 4.63, SD = 0.71).

Authors provided more moderate ratings overall. In general, they reported that they received quality feedback from both the editors (M = 3.6, SD = 0.06) and the peer reviewers (M = 3.3, SD = 0.80) assigned to their manuscript. Authors also provided relatively high ratings of their experience with NSPB on a 10-point Likert scale (M = 7.71, SD = 2.25). For both reviewers and authors, χ² analyses revealed no statistically significant effects of program type (i.e., MA, PhD, PsyD) or of years in graduate school on perceptions of NSPB or perceived training gaps (all ps > .05).

Change From Intake to Termination

Table 1 provides descriptive statistics and the results of paired sample t-tests for reviewers’ and authors’ confidence, interest, and proficiency in publishing-related skills assessed at intake and termination. For reviewers, only one variable changed significantly at the .05 level. Respondents reported being less interested in submitting articles for publication in the future. Given the large number of tests that were run simultaneously, this result should be interpreted with caution (utilizing a Bonferroni correction produced a recommended α level of .00625 for this analysis). Of note, and in contrast with our first hypothesis, all outcome variable means decreased; that is, reviewers reported less confidence, interest, and proficiency at termination.

Table 1.

Differences in Reviewers’ and Authors’ Publishing-Related Attitudes From Intake to Termination.

	Reviewers				Authors
	N	Δ	T	p	N	Δ	t	p
Confidence
Peer reviewing	43	−0.09	0.89	.38
Research/publishing	43	−0.28	1.70	.09	25	−0.46	1.93	.06^ᵻ
Interest
Research/publishing	43	−0.14	1.17	.25
Submitting articles	43	−0.30	2.38	.02*	24	−0.33	1.32	.20
Writing articles	41	−0.12	1.09	.28	25	−0.76	2.86	.01*
Research					25	−0.56	2.49	.02*
Publishing					25	−0.64	2.03	.05**^ᵻ
Proficiency
Peer reviewing	43	−0.16	1.48	.15
Writing articles	42	−0.19	1.31	.20	25	−0.36	1.74	.09

*Statistically significant at .05 level.

**Trend-level statistical significance.

The data showed primarily nonsignificant changes in authors’ degree of confidence, interest, and proficiency in publishing-related skills. Consistent with reviewer data, all means decreased from intake to termination. At a Bonferroni-corrected α of .00833, the changes in authors’ level of interest in scientific writing and conducting research approached significance (see Table 1). Additionally, there were trend-level decreases in author confidence in their research/publishing skills and their interest in submitting articles for publication in the future. The status of the author’s manuscript was unrelated to these results. Independent samples t-tests revealed no statistically significant differences between authors who had manuscripts accepted for publication and those who had manuscripts declined (all ps > .05).

Reviewers and authors were also asked whether the following barriers were likely to prevent them from conducting research or publishing findings in the future: anxiety/fear about the publication process, criticism or rejection from peers, or capabilities as a researcher; and lack of information/guidance about conducting research or writing, or the peer-review/publication process. These items were answered either yes or no. To determine levels of change between intake and termination, McNemar’s test for within-subject dichotomous variables was used. No statistically significant differences were found for reviewers (all ps > .05); however, authors indicated that anticipated criticism or rejection from peers became more of a barrier to future research or publishing efforts after their experience with NSPB (p = .016). See Table 2 for frequency data of reviewer and author endorsement across the two times.

Table 2.

Reviewer and Author Perceived Barriers to Research and Publishing at Intake and Termination.

	Reviewer		Author		Combined
Barriers to research/publishing	Intake Yes (%)	Termination Yes (%)	Intake Yes (%)	Termination Yes (%)	Intake Yes (%)	Termination Yes (%)
Anxiety/fear about publication process	11 (22%)	9 (18%)	6 (23%)	9 (35%)	17 (23%)	18 (24%)
Anxiety/fear of criticism or rejection from peers	7 (14%)	8 (16%)	2 (8%)*	9 (35%)*	9 (12%)	17 (23%)
Anxiety/fear about capabilities as a researcher	8 (16%)	10 (20%)	5 (19%)	10 (38%)	13 (17%)	20 (27%)
Lack of information/guidance about conducting research or writing	11 (22%)	13 (27%)	4 (15%)	5 (19%)	15 (20%)	18 (24%)
Lack of information/guidance about peer review or publication process	13 (27%)	10 (20%)	9 (35%)	9 (35%)	22 (29%)	19 (25%)

Qualitative data

At intake, authors and reviewers were asked what they expected to gain from their work with NSPB. Seven categories of gains were created based on CQR coding. Respondents reported expecting a publication (R = 5% and A = 89%), experience with manuscript preparation and publication (R = 68% and A = 52%), experience with the peer-review process (R = 93% and A = 15%), critical feedback of their work (R = 2% and A = 33%), knowledge of current student research (R = 66% and A = 19%), and networking and professional advancement (R = 48% and A = 7%). At termination, respondents were asked what they gained from their work with NSPB that they would not have otherwise gained in their training program. Respondents reported several unique gains: a publication (R = 0% and A = 11%); experience with manuscript preparation and publication (R = 25% and A = 30%); experience with the peer-review process (R = 48% and A = 4%); and critical feedback of their work (R = 0% and A = 22%). Although both reviewers and authors indicated that they expected to gain knowledge of current student research and networking/professional advancement at intake, these gains were reported with low or no frequency in the termination survey.

While a proportion of the data from Study 1 suggested that reviewers and authors had positive learning experiences and a favorable impression of NSPB, the results also suggested that respondents experienced less confidence, interest, and proficiency in publishing-related skills following their tenure with the journal. In light of this finding, which was counterintuitive and in contrast to our first hypothesis, we conducted a follow-up study to better understand the data.

Study 2

Study 2 was designed to address two potential confounds in the previous study: (a) the time of year of the intake and termination surveys was different (before and at the end of the academic year), so termination responses may have been negatively biased by end-of-the-year fatigue or burnout and (b) the recency of exposure to criticism or rejection may have resulted in transient decreases in confidence, proficiency, and interest. Thus, for the second study, we explored whether a more positive picture would emerge if the follow-up survey was taken at the same time of year as the original intake survey and after participants had time to gain perspective and integrate their experience and gains from NSPB.

Participants

A total of 49 participants from Study 1 completed the follow-up survey. Approximately half of the respondents were peer reviewers only (n = 26), 14 were authors only, and 11 participated in both capacities. Chi-square tests and independent samples t-tests revealed no statistically significant differences across variables between participants in Study 1 and Study 2, including type of program, years in graduate school, publication decision, or any of the target outcome variables (all ps > .05).

Procedure

Peer reviewers and authors were contacted in August 2011 and invited to complete a brief follow-up survey. As an incentive to participate, respondents were entered into a raffle to win one of three $20 gift cards. The follow-up survey consisted of six questions similar to Study 1: they rated their interest, confidence, and proficiency in peer reviewing, writing, and publishing academic articles, and their interest in submitting work to peer-reviewed journals in the future.

They then read about the results from the previous study (i.e., that ratings of confidence, interest, and proficiency decreased rather than increased) and responded to four open-ended questions: (1) Do you feel more or less optimistic about the publication process than when you started? (2) Do you think your perception of how difficult it is to publish research has changed based on your experience with NSPB? (3) Do you feel less confident and proficient and less interested in publishing in the future? and (4) Do the group results describe your experience with NSPB? We again used the CQR procedure for qualitative data analysis.

Results

Quantitative Data

Repeated-measures analyses of variance (ANOVAs) were conducted to examine changes in mean differences for reviewer variables at intake, termination, and follow-up. Reviewers’ interest in submitting articles continued to decrease from termination to follow-up, resulting in a statistically significant overall change across the three times, F(2, 58) = 3.757, p = .029. Least significant difference (LSD) pairwise comparisons indicated a statistically significant change from intake (M = 4.93, SD = 0.25) to follow-up (M = 4.53, SD = 0.82), p = .021, but not from intake to termination (M = 4.63, SD = 0.85) or termination to follow-up. None of the remaining attitudes and skills changed significantly, although for most variables, the change from termination to follow-up was in a positive rather than negative direction.

Results suggested that authors’ interest in research and publishing increased somewhat between termination and follow-up. Overall differences in authors’ interest in research were found across the three times, F(2,26) = 4.33, p = .024. However, LSD pairwise comparisons revealed no statistically significant difference between intake and follow-up means, M _diff = .429, p = .082, indicating authors’ interest in research decreased significantly immediately following their tenure with NSPB, then returned to levels that were similar to their original attitudes. Interest in writing showed a similar U-shaped pattern, with means dropping significantly at termination and then approaching levels close to baseline at follow-up, F(2, 26) = 4.333, p = .024. LSD pairwise comparisons of interest in writing showed a statistically significant decrease from intake to termination (M _diff = −.929, p = .031) followed by a near-significant increase from termination to follow-up (M _diff = .643, p = .057), resulting in nonsignificant differences from intake to follow-up (M _diff = −.286, p = .302). Interest in submitting articles was not significantly different across the three times, F(2, 24) = 2.293, p = .123; however, pairwise comparisons indicated a significant decrease from intake to follow-up (M _diff = .692, p = .013). Repeated-measures ANOVAs on confidence in research ability and proficiency in writing revealed no significant effects of time on these variables.

Qualitative Data

Forty-three respondents gave qualitative responses to the first question. More than half of reviewers and authors described feeling more optimistic about the publication process after their experience with NSPB (53%). A number of respondents perceived no change in their perception (23%), and a minority reported feeling less optimistic overall (14%). A small subset of respondents reported that their perception was mixed (4%) or gave a response that was coded as other (4%). When asked to give specific feedback about their perception of Study 1 results, 41 respondents provided qualitative responses. Approximately one third of respondents stated that the results did not fit their experience and that they did feel more confident/interested/proficient (29%). Another third of the sample stated that the results were a good fit and that they felt less confident/interested/proficient (29%). A smaller proportion of respondents reported having mixed feelings, typically describing their perceptions as more negative but also more informed and realistic following their experience with NSPB (20%). The remainder of respondents perceived no change (10%) or gave a response that was coded as other, such as having more negative feelings but stating that these were due to outside factors and not NSPB (12%).

Discussion

The aims of the current study were to examine clinical graduate students’ perceptions of their training programs, to evaluate peer reviewers’ and authors’ perceptions of and experiences with NSPB, and to evaluate the efficacy of NSPB’s training model. Consistent with previous research (Brown et al., 2009; Merlo et al., 2008), the present study demonstrated important training gaps in clinical psychology graduate education. Across degree programs, graduate students believed they were not getting enough training and experience in publication-related skills. They also perceived important barriers to future publication, even after completing a term with NSPB. Participants largely felt that a training model like NSPB was relevant to their training and an effective approach, with the majority of reviewers recommending more opportunities for students to serve on editorial boards and practice peer reviewing. Qualitative analyses of open-ended responses lend further support to the effectiveness of the NSPB training model and demonstrate that graduate students believed that NSPB provided them with training not otherwise available to them. The most frequently cited unique gains included experience with manuscript preparation and publication and experience with the peer-review process.

Despite relatively high rankings of participants’ experiences and positive qualitative feedback in Study 1, an analysis of the change between intake and termination yielded conflicting results. Observed changes in variables assessing confidence, interest, and proficiency unexpectedly decreased, although not significantly, across time points. The decrease in authors’ interest in submitting articles in the future was statistically significant, suggesting that the publication process may have a more negative impact on authors than reviewers and that exposure to peer review may be a deterrent to publishing in the future. Regardless of manuscript acceptance, the difficulties inherent in the publication process appeared to discourage some authors from pursuing future publication endeavors. The qualitative data demonstrated that, while some authors were discouraged after their experience with NSPB, this was not true for all authors in the sample, nor was it true for all authors who had a manuscript rejected. This suggests important individual differences in the way new authors react to editorial feedback. A third variable such as psychological resilience (Block & Block, 1980; Lazarus, 1993) may moderate this relationship, permitting certain authors to recover and benefit from a negative publication experience while others remain discouraged.

Author data generally showed a U-shaped pattern across time for three of the five outcome variables, with decreases between intake and termination, followed by an increase from termination to follow-up. It is possible that a ceiling effect occurred in the intake data, with high means and minimal variance leaving little room for growth. Perhaps reviewers and authors began their training with inflated expectations or naive optimism, and the small observed decreases reflect the acquisition of a more tempered and realistic perspective. This is somewhat supported by the qualitative data, which demonstrated variable trajectories for graduate students who are exposed to the publication and peer-review process. Students’ endorsed increased optimism about the publication process (≈50% of the sample) or no change in perception (≈25%), with the remaining 25% of the sample reporting feeling less optimistic overall or having mixed feelings (stating less optimism but a more realistic and educated perspective). It is also possible that the time between termination and follow-up provided a consolidation of gains, reflected in the increase seen in some of the target variables. Future research could evaluate these possibilities.

The current studies are limited in several ways. Whereas both the reviewer and author samples were diverse in terms of program type and university affiliation, the sample size was relatively small and it is possible that students who sought out experience with NSPB did so because their programs particularly lacked academic skills training. The results of the current study, which examined clinical psychology graduate students, cannot generalize to graduate students in all universities or who are from other types of graduate programs. It may be the case that students who do receive adequate exposure to publication and peer review do not seek additional training through NSPB. Because our surveys had not been previously validated, psychometric problems with item wording or the sensitivity of the measures are possible. Also, multiple aspects of the training experience may have helped or hindered reviewers and authors, including the didactic information provided during initial training. We do not know how many participants read and utilized these materials, or what impact this information may have had. Finally, despite our using an established method for qualitative data analysis, it contains an unavoidable degree of subjectivity. Nevertheless, the mixed-methods approach and the use of a clarifying follow-up study lend support to the validity of the results.

Overall, it appears as though working with a graduate student journal for 1 year provided students with important training that was largely unavailable to them in their training program. Despite the constructive and positive feedback approach adopted by NSPB, for some students, the publication process proved a frustrating and discouraging experience. Although the quantitative data revealed a general lack of movement on outcome variables, qualitative feedback demonstrated that students made important gains that they attributed to working with NSPB. These data suggest that NSPB provides important education about publishing and increases students’ academic skills. For students who pursue publication in the future, these gains have the potential to increase their likelihood of academic success. Aside from providing concrete training in publishing-related skills, the greatest utility of the journal possibly lies in exposing students to the difficult realities of the publication and peer-review process early on, allowing them to make a choice about whether to pursue future publication of their work.

Early exposure to publishing may encourage students in their own publication endeavors, providing them with experiential knowledge on how to move from research to publication and contribute to the literature in a meaningful way. In light of the importance of publishing-related skills, training models like NSPB should continue to be explored and evaluated as potential avenues for academic training for psychology graduate students. The current studies offer an important first step in validating the utility of graduate student psychology journals and NSPB’s training model. They also provide important data on graduate student perceptions and on different possible outcome trajectories from early exposure to publication and peer review. Future research should continue to explore the outcomes of graduate students who participate in training journals like NSPB, to help determine if graduate journals are capable of bridging the training gap. Research should also investigate potentially related variables, such as psychological resilience or openness to feedback and criticism, that may correspond to or predict the different outcomes. Finally, it will be important to assess long-term outcomes of such training experiences, such as publication success and other career indicators, to determine if graduate student journals play a role in preparing students to effectively meet the demands of the field.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Note

References

Addis

M. E.

Jacob

(2000). On the process and outcome of graduate training in clinical psychology: Where do we want to go, how will we get there, and who will join us? Clinical Psychology: Science and Practice, 7, 364–367. doi:10.1093/clipsy/7.4.364.

American Psychological Association. (2009). Publication manual of the American psychological association (6th ed.). Washington, DC: Author.

Antonius

Brown

A. D.

Todman

Safran

J. D.

(2007). Integrating science in applied psychology programs: A student-operated journal. Teaching of Psychology, 34, 31–34. doi:10.1207/s15328023top3401_7

Baumeister

R. F.

Leary

M. R.

(1997). Writing narrative literature reviews. Review of General Psychology, 1, 311–320.

Bem

D. B.

(2003). Writing the empirical journal article. In Darley

J. M.

Zanna

M. P.

Roediger

H. L.

(Eds.), The complete academic: A practical guide for the beginning social scientist (2nd ed., pp. 171–201). Washington, DC: American Psychological Association.

Bernstein

B. L.

Kerr

(1993). Counseling psychology and the scientist-practitioner model: Implementation and implications. The Counseling Psychologist, 21, 136–151. doi:10.1177/0011000093211008.

Block

J. H.

Block

(1980). The role of ego-control and ego-resiliency in the origination of behavior. In Collins

W.A.

(Ed.), The Minnesota symposia on child psychology (Vol. 13, pp. 39–101). Hillsdale, NJ: Erlbaum.

Brazeau

G. B.

DiPiro

J. T.

Fincham

J. E.

Boucher

B. A.

Tracy

T. S.

(2008). Your role and responsibilities in the manuscript peer review process. American Journal of Pharmaceutical Education, 72, 69.

Brown

A. D.

Antonius

Hirst

(2009). Filling the gaps of graduate school training: Perceived relevance and efficacy of the New School Psychology Bulletin. Psychology Learning & Teaching, 8, 46–52.

10.

Chang

Lee

I. L.

Hargreaves

T. A.

(2008). Scientist versus practitioner: An abridged meta-analysis of the changing role of psychologists. Counseling Psychology Quarterly, 21, 267–291. doi:10.1080/09515070802479859.

11.

Doran

J. M.

Antonius

A. D.

Brown

A. D.

Kriss

Lehr

E. Y. C.

Evans

Steele

(2012). The perceived relevance and efficacy of a graduate school journal among graduate faculty and training directors. Teaching of Psychology, 39, 62–66. doi:10.1177/0098628311430316.

12.

Dunlap

(2006). The effect of a problem-centered, enculturating experience on doctoral students’ self-efficacy. Interdisciplinary Journal of Problem-based Learning, 1, 19–48. Retrieved from http://docs.lib.purdue.edu/ijpbl/vol1/iss2/4/.

13.

Ferreira

V. S.

(2010, August). Writing papers for publication. Paper presentation at the 118th APA Annual Convention, San Diego, CA.

14.

Hill

C. E.

Thompson

B. J.

Williams

E. N.

(1997). A guide to conducting consensual qualitative research. The Counseling Psychologist, 25, 517–572.

15.

Hopwood

(2010). Doctoral students as journal editors: Non-formal learning through academic work. Higher Education Research and Development, 29, 319–331. doi:10.1080 /0729436090353 2032.

16.

Hopwood

Stocks

(2010). Bringing the backstage into the light? Thoughts on shadowy academic practices. International Journal for Academic Development, 15, 89–91. doi:10.1080/13601440903529968.

17.

Hunt

S. L.

Wisocki

P. A.

(2008). Balancing science and practice in clinical psychology training programs: A survey of training directors. Behavior Therapist, 5, 91–96.

18.

Lazarus

R. S.

(1993). From psychological stress to emotions: A history of changing outlooks. Annual Review of Psychology, 44, 1–21.

19.

Maher

B. A.

(1999). Changing trends in doctoral training programs in psychology: A comparative analysis of research-oriented versus professional-applied programs. Psychological Science, 10, 475–481. doi:10.1111/1467-9280.00192.

20.

Merlo

L. J.

Collins

Bernstein

(2008). CUDCP affiliated clinical psychology student views of their science training. Training and Education in Professional Psychology, 2, 58–65. doi:10.1037/1931-3918.2.1.58.

21.

Mischel

(2008). Our urban legends: Journal reviews. Observer, 21.

22.

Roediger

H. L.,

III . (2007). Twelve tips for reviewers. APS Observer, 20. Retrieved from https://www.psychologicalscience.org/observer/getArticle.cfm?id=2157.