Abstract
Advanced scientific communication skills are highly desired for academic and nonacademic career professionals but are insufficiently developed in many graduate students. Because students with undergraduate degrees have inconsistent training and variable skill levels in advanced scientific communication, first-year graduate students can benefit from intentional training of these skills through coursework. This case study provides an educational framework to implement a research proposal assignment modeled after the National Science Foundation Graduate Research Fellowship Program (NSF GRFP) application. We detail the assignment implementation in a first-year graduate-level technical course (class size of around 20 students), including project rubrics, timelines, and explanations of iterative peer review tasks. Project goals included training advanced scientific communication skills, collaborating across disciplines through iterative peer review, and providing opportunity for first-year students to engage in their original research work at an early stage. Self-reported student responses and outcomes collected over 4 years of project implementation implicated improvements in perceived competence due to the assignment for the following skills: communicating technical topics to a broad audience, developing testable hypotheses, and original scientific writing. The writing assignment also likely supported timely and high-quality applications to the NSF GRFP. Despite self-reported gains in critical skills and bolstering fellowship applications, student responses also indicated that more training in advanced scientific communication skills may be necessary. Thus, we suggest inclusion of writing projects across multiple courses in graduate curricula. Using the provided educational framework, instructors can design other projects that develop critical competencies for Science, Technology, Engineering, and Math graduate students and their future careers in and beyond an academic setting.
Introduction
National reports highlight effective science communication as a requisite transferrable skill for Science, Technology, Engineering, and Math (STEM) graduate students (Denecke et al., 2017; National Academies of Sciences, Engineering, and Medicine, 2018); specifically, advanced degree hires are expected to both communicate highly technical content and present such content in understandable terms to nonexperts. Nonetheless, industry representatives have identified writing and crossdisciplinary communication as areas that need improvement in graduate-level hires (Denecke et al., 2017). This need likely arises from undergraduate and graduate programs that lack—or provide insufficient—training in scientific communication. In undergraduate programs, scientific writing is often relegated to laboratory courses where the writing is highly structured or prescriptive (Moskovitz and Kellogg, 2011). Outside of coursework, students with undergraduate research opportunities often develop data analysis and experimentation skills (National Academies of Sciences, Engineering, and Medicine, 2017). Yet, students are not necessarily well trained in more advanced competencies like formulating/testing hypotheses, identifying research questions, and writing research publications (Kardash, 2000). Undergraduate research experiences can vary in duration and level of autonomy, which may influence gains in research skills; shorter durations and lower autonomy may limit advanced research skill development (Gilmore et al., 2015). Thus, original scientific writing remains underdeveloped both in and outside the classroom for most STEM undergraduates. Consequently, first-year graduate students may have limited experience in original scientific writing or formulating research questions.
Graduate students can be trained in scientific writing by integrating writing into required curricula. In this short communication, we demonstrate implementation of a writing project designed to better prepare first-year graduate students in scientific inquiry and communication. The project was implemented for 4 years in a required first-year microbiology and environmental chemistry core course in the Water and the Environment program within the Department of Civil and Environmental Engineering at the University of Iowa. Using the National Science Foundation Graduate Research Fellowship Program (NSF GRFP) as a basis for the project, graduate students were provided an early opportunity to identify a technical research problem/question, articulate the problem through writing, design a research strategy, and collaborate with other students from diverse disciplinary backgrounds. Throughout the assignment, we incorporated iterative peer feedback tasks. This exposed students to the scientific peer review process and emulated the process of feedback in professional scenarios within and outside academia. Iterative feedback has also been shown to enhance student learning through active engagement and dialog; it increases ability of self-assessment, provides opportunity for revision, and allows students to recognize strengths and areas of improvement (Nicol, 2010; Reinholz, 2016). Altogether, the project trains multiple critical skills and can encourage NSF GRFP applications. Given the positive outcomes we have observed, it is our intent to share our observations in hopes that other educators may be motivated to implement these strategies to enhance graduate student writing skills.
Implementation
Course context
Enrollment for the course in which the project was implemented ranged from 15 to 21 students, with students from a variety of academic backgrounds (e.g., environmental engineering, biochemistry, geology, etc.). The course's learning objectives were designed to be relevant and accessible to diverse academic backgrounds (abridged syllabus in Supplemental Information: Item S1). Similarly, the skills trained by the writing project are broadly important for graduate students in STEM fields; students developed the abilities to create testable hypotheses, design research experiments, and write in a clear and concise manner for technical audiences. These objectives can be targeted in other first-year technical graduate courses.
Project description
Each student prepared a two-page research statement, mirroring the specifications of the NSF GRFP application. The project timeline was such that students provided, received, and responded to feedback at various stages of project completion to train a more intentional and structured process of writing (Supplementary Fig. S2). Students emulated scientific peer review in their feedback format (Supplemental Information: Item S3). Peer review requirements included a summary, overall impressions for strengths and weaknesses, and line-by-line comments. Concurrently, the instructor presented a series of short lectures (Supplemental Information: Slides S4) on hypothesis generation/experimental design, the peer review process, and ways to structure/outline a research proposal. Successful GRFP proposals (names redacted) were provided as examples.
The rubric placed the most emphasis on developing testable hypotheses, experimental design, and statistical hypotheses tests (Supplementary Table S5). Rubrics were provided to students at the beginning of the project to guide their work. Students chose their own topics and were asked to consider current literature, understand hypothesis testing techniques, and communicate the scientific process and importance of their work. Students also had to consider limitations and alternative outcomes.
Survey data collection
The Institutional Review Board at the University of Iowa approved the use of previously collected and new survey data for this study. Previously collected survey data were used for course improvement purposes and included questions about intent to apply to the NSF GRFP, reasons for applying/not applying, and fellowship eligibility. Past students provided consent for their data to be used in the study. New survey data (self-reported perceptions of advanced writing skills) were collected in Fall 2020 and questions are detailed in the Supplemental Information (Survey S6). Given the small sample size for this study, survey validation was not deemed appropriate.
Project evolution
Based on instructor reflection and student feedback, the project was modified over the 4 years of implementation. Initially, the project included the personal statement portion of the GRFP application. This was dropped to focus more on the research statement. The review process was adjusted so the instructor could provide feedback after completion of peer reviews. This served two purposes: to strengthen research statements for those who wished to submit a GRFP application, and to provide an opportunity for students to revise research statements to regain points on the assignment.
Discussion
Student outcomes and responses
Over 4 years, the project helped facilitate NSF GRFP (highly competitive national graduate fellowship) applications (Table 1). The number of GRFP applications typically exceed 13,000 nationwide, with around 2,000 fellowships awarded annually. In the course surveys, some students specifically indicated they would not have applied if not for the assignment. Because the timeline of the project aligns with the GRFP application, it is reasonable to infer that the project helped students submit a timely application. Of the GRFP-eligible study participants, 76% submitted applications. Through the process of peer and instructor review, the project may have also increased quality of applications; however, it is difficult to measure direct effects of the project on applicant success given the small sample sizes. Additionally, the number of GRFP-eligible students decreased over the study period, in part from an increase in MS nonthesis students in the course. Nonetheless, 25% of participants in the study who submitted applications were awarded a fellowship or honorable mention. In the 4 years preceding this study, the department had three GRFP recipients—one recipient fewer than the equal-length study period. We posit the structure and timeline of the project help encourage GRFP applications, and iterative peer review paired with instructor feedback can bolster application quality. Future studies could track the number of applications and success rates over time with larger sample sizes to examine GRFP outcomes.
Summary of Graduate Research Fellowship Program Information Over 4 Years for the Course, with Numbers Reflecting Data for Students Consenting to Participate in the Study
For 2017–2019, response rate represents students who consented to have previously collected data used relative to the class size of that year.
In 2020, one consenting response was incomplete and not used in further analysis.
GRFP, Graduate Research Fellowship Program.
Differing survey responses from pregraduate experiences
We observed differences in Fall 2020 survey responses based on undergraduate discipline and prior research experience (Table 2). Considering both extracurriculars and coursework, students with undergraduate engineering degrees reported equal or higher levels of training for the three surveyed competencies: communicating technical topics to a broad audience, developing testable hypotheses, and original scientific writing. This may be because engineering programs are required to demonstrate training in communication to receive accreditation (e.g., ABET). Students with prior research experience reported equal or lower levels of training for these skills in their extracurriculars (including research) relative to those with no prior research experience. Although our sample size is small, this finding supports literature evidence that students can have little/no training in advanced scientific skills, such as designing experiments and developing testable hypotheses during undergraduate research experiences (Kardash, 2000). In addition to sample size, we recognize that interpretation of self-reported competence is nuanced. For example, self-assessments may not match other assessments of competence (e.g., grades or teacher ratings), and students with higher skill often underreport their competence (Lew et al., 2010; Brown et al., 2015; Andrade, 2019), which may contribute to lower reported values of competence in students with prior research relative to those with no prior research.
Median Values for Select Fall 2020 Survey Responses on Training Received During Undergraduate Coursework or Extracurricular Activities (Values Range from 1-No Training to 4-High Training), Categorized by Engineering Versus Nonengineering Disciplines and Prior Research Versus No Prior Research Experience
Project impact on self-reported competence
Fall 2020 participants felt the assignment improved both perceived importance and competence of all three skills (Fig. 1). Over half of the students viewed the skills as slightly or much more important after the assignment. Almost half of the students reported great improvement in original scientific writing competence due to the assignment. Yet, many still indicated low self-reported values of competence in certain skills; overall, students reported a “moderate” level of competency for “developing testable hypotheses” and “original scientific writing” (see Supplementary Table S7). These two skills were reported to be less trained in undergraduate coursework (“moderate” for original scientific writing and “little training” for developing testable hypotheses). Even the three respondents who previously attended graduate school reported little/moderate training for those skills. This implies that faculty should integrate projects that train these critical skills in multiple courses. Additionally, graduate programs could assess competency training throughout the curriculum.
Percentage distribution of student responses from Fall 2020 indicating self-reported competency, perceived improvement, and changes in perception for three critical competencies. Questions were five-point Likert-type items ranging from “Very low” to “Very high,” “Greatly lowered” to “Greatly improved,” and “Much less” to “Much more,” respectively.
Conclusion
The described writing project can be adapted to fit early graduate courses for many STEM areas; it works well in technical courses, as it can replace more traditional term assignments such as literature reviews without losing technical rigor. Early in their graduate program, students can develop advanced scientific communication skills like formulating and evaluating their original research. Students can also be better prepared and supported throughout the NSF GRFP application process. Our rubrics, timelines, and other project materials can act as an implementation framework for instructors who wish to adopt the assignment. More broadly, technical communication competencies must be evaluated in graduate programs to ensure that students are well prepared for postgraduate careers. STEM professionals are required to communicate technical content through proposals/reports, and they receive iterative feedback; specifically, proposal writing is extensive outside academia. Our exploratory study indicated that single projects may not provide the requisite training to fully develop key competencies needed for these tasks. More education research on graduate competencies is needed to understand best practices for improving graduate-level training. Graduate programs should consider competency training throughout the curriculum so students can build proficiency in advanced scientific communication.
Footnotes
Authors' Contributions
Contributor Roles Taxonomy (CRediT) Statement: E.A.W.: conceptualization, formal analysis, investigation, data curation, writing—original draft, writing—review and editing, visualization. G.H.L.: conceptualization, methodology, investigation, writing—review and editing, supervision, project administration.
Author Disclosure Statement
No competing financial interests exist.
Disclaimer
Any opinion, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Funding Information
This material is based upon work supported by the National Science Foundation (NSF) Graduate Research Fellowship under Grant No. 1546595. Additional support for this work was provided by the NSF through the NSF Division of Graduate Education under Grant No. 1633098 and by the NSF CBET CAREER under Grant No. 1844720.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
