Abstract
Writing performance of a complex recommendation report produced by student teams for an actual client during a 15-week semester was compared in a writing-intensive Agronomy 356 course and in paired Agronomy 356/ English 309 courses. The longitudinal study investigated differences that existed between reports produced for each learning environment in terms of argument effectiveness, document usability, and professionalism. Three agronomy and three professional communication raters ranked the 12 lengthy reports in the sample. The study found that all top-rated reports were generated in the paired courses and all lowest-rated reports were generated in the stand-alone agronomy course. Four pedagogical factors appear influential in this result: working in dual problem-solving spaces, pushing the boundaries on problem solving, incorporating workplace realities, and using just-in-time teaching.
Keywords
During the last 20 years, the number of writing-across-the-curriculum (WAC) and writing-in-the-disciplines (WID) initiatives at universities and colleges across the country has increased by approximately one third (Thaiss & Porter, 2010). Of the 1,126 institutions with WAC/WID initiatives, 27% include writing courses “attached to other courses in the disciplines” (p. 549). In such paired courses, students often coenroll in a cohort for a writing course that is linked to another course, in engineering, mathematics, the sciences, the humanities, or the like. Many of the pedagogical benefits and challenges specific to paired, integrated courses have been researched, especially at the intersection of WAC/WID and learning communities literatures (Heckelman & Dunn, 2003; Wardle, 2004; Watts, 2008; Wojahn, Dyke, Riley, Hensel, & Brown, 2001; Zawacki & Williams, 2001). However, more work needs to be done to understand the ways that such curricular initiatives affect students’ writing performance.
Our longitudinal study of the paired courses, Agronomy 356 (Soil, Water, and Fertilizer Management) and English 309 (Report and Proposal Writing), provides an opportunity to compare versions of a complex recommendation report produced by student teams for an actual client during a 15-week semester. Reports are drawn from a 10-year period of Agronomy 356 as a stand-alone, communication-intensive course as well as from 18 months of a paired-courses WAC/WID initiative in which a cohort of students coenrolled for Agronomy 356 and English 309. In this latter initiative, the agronomy and English instructors integrated these two upper-level courses in substantive ways: coassigning and coassessing selected assignments, collaborating to provide feedback on selected assignments, sharing and exchanging class times when necessary to meet integration goals, and attending each other’s classes.
Our study focuses on the major assignment in these integrated courses: the farm management recommendation report. We use this document for our analysis of student writing performance for two reasons. First, the farm management report was assigned in Agronomy 356 during the 10 years that it was offered as a stand-alone course. Later, when Agronomy 356 was paired with English 309, the farm management report was coassigned and coassessed in Agronomy 356/English 309 (hereafter, 356/309). The assignment parameters of the report remained largely unchanged throughout its use in Agronomy 356 and in 356/309. Second, the agronomy instructors who team-taught Agronomy 356 for 10 years also taught the course when it was paired with English 309. Table 1 offers information about the instructors’ academic and industry experience.
Agronomy 356/English 309 Instructors
Note: The instructors granted permission for their names to be used, a feature of the study approved by the Institutional Review Board.
Linking or integrating pairs or even clusters of courses across all the disciplines is the modus operandi at only a few colleges and universities (e.g., The Evergreen State College, New Century College at George Mason University). Instead, most institutions offer students selected course integration opportunities during certain years of study, which might include opportunities for first-year students, honors students, or students who have specific career goals or interests. For example, University of Maryland–College Park offers its College Park Scholars programs, which are “two-year living-learning programs for first- and second-year students” in which students select a theme (e.g., “Art,” “Business, Science, and the Economy,” “Media, Self, and Society”) and enroll in courses specific to that theme (see http://www.scholars.umd.edu/programs/). In this example, clusters of courses are integrated around a common interdisciplinary theme, which students explore throughout their first two academic years. These themes then feed into or augment programs of study that students may wish to pursue.
Online resource guides such as Washington Center’s Learning Community Commons, as well as its long-running National Summer Institute on Learning Communities, offer faculty and staff valuable resources pertaining to recruiting faculty and students to course integration opportunities, successfully assembling and marketing these integrations, implementing them, and assessing their effectiveness and effect on student learning and community building. Edited collections and other resources offer similar guidance (Laufgraben & Shapiro, 2004; B. L. Smith, MacGregor, Matthews, & Gabelnick, 2004). Early works in this area still resonate today—especially, Gabelnick, MacGregor, Matthews, and Smith (1990), Graham (1992), and Shapiro and Levine (1999).
Situating the Study
In terms of the process of linking or clustering courses in the discipline with writing courses and the ensuing effects on participating students and instructors, research indicates a number of important issues relevant to our study that concern faculty collaboration, assignment construction and assessment, and perceived effects specific to community building, learning, and writing performance.
Process and Effects of Course Integration
Collaborating to teach integrated courses is challenging: Studies show that to successfully leverage the cross- or multidisciplinary nature of the learning environment, participating instructors must be supported and rewarded for “genuinely [modeling] deep learning” for those students who participate in the integrated courses (Tagg, 2003, p. 263). As learning community pioneers Barbara Leigh Smith, Jean MacGregor, Roberta S. Matthews, and Faith Gabelnick (2004) rightly note,
because teaching in a learning community [i.e., an integrated courses experience] may require new ways of taking up the teaching role and new opportunities to work with colleagues from different parts of the campus, faculty and staff development is essential. (p. 282)
Even when faculty development is part of the process of preparing for teaching integrated courses, writing faculty often are still challenged by these collaborations. Writing instructors can experience a disciplinary “culture shock” when working with instructors across the disciplines (Brammer, Amare, & Campbell, 2008). Others may experience resistance from their in-the-disciplines colleagues pertaining to rhetorical and writerly expertise (Mahala & Swilkey, 1994), or these colleagues may simply be ignorant about the importance of writing and rhetoric in the academy and in the workplace and about the ways in which writing affects learning and disciplinary knowledge (Norgaard, 1999; Samuels, 2004).
While many writing instructors face an uphill battle to meaningfully collaborate in integrated courses, the focus of much pedagogical effort and investigative study recently has shifted toward assignment construction. That is, teachers of integrated courses have begun to investigate in earnest the benefits of the carefully crafted and assessed coassignment (i.e., an assignment coassigned by instructors in each pair of courses). In fact, an entire recent issue of the Journal of Learning Community Research (Cuevas, Campbell, Lowery-Hart, Mallard, & Andersen, 2008/2009) showcased the processes of constructing, implementing, and assessing “integrative assignments” to promote meaningful across-the-disciplines faculty learning and integrative student learning. The studies in this issue showed that the process of assignment design not only helped tease out ways to identify and integrate different disciplinary perspectives and issues and highlight and clarify individual instructor’s areas of expertise but also established (well in advance—even before instructors entered the classroom) what integrative learning they wanted students to achieve.
This focus on integrative assignment design speaks to challenges that writing instructors have faced specific to genre and cross-disciplinary collaboration. In a study of learning communities in which first-year composition courses were paired with introductory-level courses in the disciplines, Elizabeth Wardle (2004) found that writing instructors had little knowledge of academic disciplinary genres and were challenged to teach them to students enrolled in courses in their learning communities. In fact, other studies have shown the difficulties that writing instructors experience in learning highly specialized genres and communication strategies of the disciplines (Bayer, Curto, & Kriley, 2005; Cross & Wills, 2005). Indeed, Terry Myers Zawacki and Ashley Taliaferro Williams (2001) argue that integrated courses using a WAC approach are unique and particularly challenging teaching environments: “The coordinated/integrated studies structure creates an intensive learning environment and a changed dynamic among students and teachers” (p. 15). Furthermore, the complexity of these learning environments has been shown to complicate assessment: “In order to design effective assessment, it is crucial to consider how writing may be different—and more complex—in these settings” (p. 27).
Despite the challenges specific to the pedagogy of the integrated course experience, research shows that such curricular structures offer students much in the way of community building and peer support (Phillips & Kim, 2009). Studies also show that linking a college success course (in which students learn study, reading, and other skills and strategies) to a challenging disciplinary course such as those in the sciences or mathematics can improve performance in the disciplinary course (Hansen, Meshulam, & Watson, 2010) and GPA performance and retention (Cuevas et al., 2008/2009).
Writing Performance in Integrated WAC/WID Classrooms
For instructors teaching writing and for scholars studying it, the most compelling effects of the integrated courses approach concern its usefulness specific to improving writing performance. Early work in this area shows that an integrated courses approach can positively affect student writing performance (Messina & White, 1992), and more recent studies help to support this perspective. In a study of student performance, retention of course concepts, and students’ perceptions of satisfaction, Cargill and Kalikoff (2007) compared students enrolled in linked psychology and writing courses with students not enrolled in an integrated courses experience. Students in the linked courses performed better, were retained in the course more frequently, and expressed “higher satisfaction, engagement, and feelings of academic belongingness” than their counterparts (p. 90).
As we mentioned, assignment design that helps to thoughtfully draw out instructors’ areas of expertise and facilitate students’ integrative learning is becoming a common practice in integrated courses. In one instance, Manahan and English (2002) analyzed the use of a cross-disciplinary assignment, which asked genetics and astronomy students to correspond with each other about their laboratory experiments. This writing-to-learn experience is meant to help students better understand their lab experiences by conveying these to an audience other than their instructors. Overall, the assignment was a success: “It transformed the reporting of scientific results from the traditional dry lab report style to an interactive format that emphasized communication and inquiry” (p. 69).
To foster a successful integration between a health sciences course and a writing class, Clark and Fischbach (2008) “used rhetoric, with particular emphasis on the concepts of identity, genre, and performance . . . to understand what it means to be a Public Health professional” (p. 15). Students were asked to complete a problem-based argumentative writing assignment as a means to educate citizens about public health issues, which was seen as a way to specify the genre and tasks that students were to complete. In this case, the design of this assignment was crucial, enabling students to “assume a more nuanced writer identity” (p. 16). In assuming a writerly identity, students learned more about what they needed in order to become a professional in the health sciences field. Clark and Fischbach underscored the importance of the “dramatistic or rhetorical element” of the public health educator who must draft “a persuasive argument in order to motivate individual behavior change and in doing so alter the collective health to the community” (p. 24).
This notion of the ways that disciplinary expertise and professionalism relate to a writing-intensive, integrated courses approach is particularly cogent to our study. Scholars who theorize about writing performance and disciplinary expertise focus on a variety of issues, yet early WAC researchers quickly understood that the relationship between writing and disciplinary expertise was a complex and intriguing one, worth investigation and critique (Jones & Comprone, 1993; Langer, 1992; Petraglia, 1995, 1998). Following Spinuzzi (1996) and others (e.g., Dannels, 2003; Paretti, 2008; Russell, 1997; Wardle, 2004), activity theory is now a useful theoretical construct with which to examine communication practices and performances and their impact on disciplinary expertise.
Such WAC research examines the ways that writing enables students to think critically about how knowledge is constructed in their disciplines. In this sense, student writing performance is bound to issues of disciplinarity and epistemology (Beaufort, 2004; Hilgers, Hussey, & Stitt-Bergh, 1999; Kelly, Chen, & Prothero, 2000). One study of a writing-intensive first-year engineering design course cotaught by writing and engineering instructors showed that improvement occurred in students’ abilities to synthesize and argue, on the basis of key findings in the literature after instructors redesigned the assignment so that “students could more easily imagine themselves in a professional role” and engage more actively in practicing argument (Yalvac, Smith, Troy, & Hirsch, 2007, p. 119). In an earlier study, Haswell (2000) examined the writing performance of 64 randomly selected students, which indicated “a rise in holistic scores longitudinally from first-year to junior, and movement along a number of rhetorical fronts toward competent occupational writing” (p. 337). Yet, as Haswell himself noted, his study leaves much unsaid about what exactly constitutes writing improvement.
As students learn to see writing in more nuanced ways, they are asked to analyze professional genres (Bazerman, 1994) or genres that constitute academic discourse (Bacon, 2000; Greene, 2001; Walker, 1999). Scholarship shows ways in which genre analysis (Luzón, 2005) or activity theory (Kain & Wardle, 2005) can be helpful in teaching technical communication. Using situated learning and activity theories, Paretti (2008) showed that assignment genres need to be carefully structured in courses meant to help students practice workplace genres; otherwise, students may experience frustration in the “mismatch between the standards expected of them in the course and the standards expected at work” (p. 499). Other studies describe similar student experiences with academic genres (Artemeva, Logie, & St-Martin, 1999; Craig, Lerner, & Poe, 2008; Dannels, 2003; Freedman, Adam, & Smart, 1994; Paretti, 2006). These studies about genre address ways that writing in school and in professional contexts differ and the effect that different contexts have on writing performance (Dannels, 2000, 2003; Dias, Freedman, Medway, & Paré, 1999; Dias & Paré, 2000; Freedman & Adam, 2000; Wickliff, 1997).
Our study extends two important areas of learning community and WAC/WID scholarship—empirical research about cross-disciplinary course integration and student writing performance. Specifically, we investigate what happens to student writing performance on a complex, collaboratively written document when the learning environment in which that document is situated moves from a course in the disciplines to one that pairs the course in the disciplines with a writing class.
Research Question
We are interested in whether pairing Agronomy 356 and English 309 enabled the three instructors to help students create reports that contain usable, professional arguments. Specifically, we ask the following research question:
What differences—in terms of argument effectiveness, document usability, and professionalism—exist between reports produced for the stand-alone disciplinary course and the integrated WAC/WID courses?
Our focus on these three critical areas is the result of analyzing extended discussion among the three instructors who explored and eventually agreed that these areas mattered most in assessing the quality of the farm management reports. Initially, during these instructor discussions, we (in our roles as researchers) listened to their energetic interactions during their regular weekly meetings and took careful notes. The instructors were explicit in identifying areas affecting the quality of the farm management reports since these areas shaped both the focus of the teaching and the assessment criteria that the instructors used. These three areas are also important to the farmer clients. Though the farmers were not involved in discussions with the instructors to identity areas to assess the farm management reports, the instructors certainly considered the farmer clients’ reactions to the farm management reports. Finally, the instructors considered the competencies they wanted their students to exhibit as outcomes to the courses, outcomes as entry-level professionals who need to be skillful in communicating technical agronomic information.
Our study comparing writing performance from one learning environment to another is one way to build on existing literature, especially in terms of students’ understanding of and their ability to use agronomic and rhetorical principles in planning, writing, revising, and presenting recommendation reports—in this project, a farm management report. As part of preparing these reports, students learn about the ways that various agronomic, social, environmental, and economic issues apply to and affect specific problems and opportunities found at the client’s farm operation. When students prepare their reports for the client, as they work through revising their drafts, they gradually learn that getting the science right is not enough. Students must learn to present their recommendations rhetorically to clients and in ways that account not only for agronomic issues but also for social, environmental, and economic issues as well.
Explaining the Background
Our study examines recommendation reports to ascertain the ways that writing performance is affected by an integrated courses learning environment. Agronomy 356 and English 309 existed as stand-alone courses for many years before they were offered as 356/309. In this section, we describe each course and discuss the course integration strategies used to create the integrated 356/309 courses.
Agronomy 356 and English 309
Agronomy 356 (Soil, Water and Fertilizer Management) is a four-credit elective course (three classroom credits and one lab credit) offered in the College of Agriculture. Because of the range of topics discussed, the course is cotaught by two agronomy professors. Students learn about the management of tillage and soil nutrients, highly erodable soil (soil prone to water or wind erosion), and hydric soil (soil prone to water saturation, ponding, or flooding) as well as new operational and management technologies available to farmers. Students are evaluated by their performance on weekly essay quizzes, the oral final exam, the team-written farm management report, and the team-delivered oral presentation to the client. John Schafer and Tom Polito cotaught Agronomy 356 as a stand-alone course for 10 years. Prompted largely by the active communication-in-the-disciplines program in their college, the Agronomy 356 instructors approached faculty in the Department of English to work with them to help Agronomy 356 students to better achieve communication outcomes. One year later, with funding from the university’s learning community initiative, Agronomy 356 was paired with English 309, which was taught by a veteran writing/rhetoric professor, David Roberts.
English 309 (Report and Proposal Writing) is a three-credit course offered in multiple sections per semester by the Department of English in the College of Arts and Sciences. The course attracts English majors and technical communication majors as well as students outside the Department of English who use the course to fulfill required upper-level writing course credits. English 309 focuses on rhetorical principles and practices: organizing effective proposals and reports and drafting recommendations for problems in complex contexts. Students complete a major proposal (often a simulation case study rather than a project with an actual client) and several shorter related documents and presentations.
Course Integration Strategy
The decision to pair Agronomy 356 with a section of English 309 initially came near the beginning of the university’s learning community initiative, in which support was available to coenroll a cohort of students in Agronomy 356 and English 309. Enrollment requirements for 356/309 were dictated by Agronomy 356: Students were juniors or seniors in the College of Agriculture; most were agronomy or agricultural business majors; and enrollment in 356/309 was capped at 15 students. Because of the required coenrollment in English 309, many of the students also saw 356/309 as an opportunity to complete their upper-level writing requirement. In general, more males than females enrolled for 356/309; few had industry or internship experience in their field of study; and most grew up on their family farms.
Prior to offering the integrated 356/309 courses, faculty discussed learning outcomes and course integration strategies. The instructors identified the following four learning outcomes for 356/309 students:
Understand the responsibilities of professional agronomists and the ways that agronomic decision making relates to environmental, social, and economic concerns
Communicate effectively in a variety of professional settings and show improvement in communication skills
Understand how to capitalize on the career options as agronomy professionals
Work in teams to complete complex tasks
While the 356/309 learning outcomes and content remained much the same as each stand-alone course, the instructors did make minor changes in the scheduling of presentations and assignments to better accommodate the pairing of the courses.
The core of the stand-alone Agronomy 356 course was the consulting relationship with the farmer client, which served as the basis for much of the students’ work. Each year in the stand-alone Agronomy 356 course, student teams worked with one farmer client, usually with different clients from one year to the next. The consulting relationship with an actual farmer client and the farm management recommendation reports submitted by student teams to the farmer client were hallmarks of Agronomy 356. This consulting relationship and the accompanying recommendation reports from each student team remain critical in the integrated version of 356 and 309 and serve as an organizing feature for the courses.
During week one of the semester, the three instructors for 356/309 assign students to teams of three to four, and students work in these teams to complete the semester-long consulting project with the farmer client. For the project, student teams are employed by a fictional agricultural consulting firm with the 356/309 instructors in the roles of project managers. Teams complete a series of tasks throughout the semester for an actual farmer client, tasks that reflect the work of professional agronomists. The teams first interview their farmer clients and then collect a variety of data from the farm: soil and manure samples as well as observations concerning tillage practices, pest infestation on standing crops, and soil erosion. Teams use these data to write one prospective client report, one proposal, two progress reports, and one recommendation report and complete two presentations. Table 2 describes each assignment, indicates which faculty members provided students with feedback about each assignment, and specifies the course for which students earn grades for each assignment. The recommendation report and oral presentations include farm management strategies concerning nutrient and manufactured fertilizers, cultivation practices, crop residue and erosion management, available new technologies (equipment or services) suitable for the farm operation, and any other topics requested by the client, which the client could (and would) use to alter farm practices.
Consulting Project Assignments Coassigned and Coassessed in Agronomy 356 and English 309
Students developed these recommendations in ways that tested their knowledge of not only agronomy but also applied rhetoric and communication. Technical accuracy was insufficient; students could not simply interpret their data without also placing the test results into the context of the client’s farm operation and considering the client’s unique needs and interests. Students were obliged to develop rhetorically situated recommendations that were agronomically sound, environmentally friendly, economically viable, socially acceptable, and persuasive to their farmer clients.
Method
In this section, we identify the data we collected, the criteria we used to assess that data, the selection and training of our six raters, the analysis of our data, and, finally, the limitations of the study. Our study was approved by the university’s institutional review board for human subjects’ research.
Data Collected: Selection of Recommendation Reports
The farm management report was an important assignment for student teams to complete, both when Agronomy 356 was taught as a stand-alone course and when Agronomy 356 was integrated with English 309. Regardless of the nature of the course or the year in which a report was written, each report was lengthy (20 or more pages) and typically included extensive visuals (maps, tables, and occasionally photographs) and appendices. The reports written for the stand-alone Agronomy 356 course were written in teams, just like the reports written for the 356/309 integrated courses. The parameters of the report assignment remained relatively unchanged during its use in the stand-alone course as well as when the assignment was adopted for use in the 356/309 integrated courses. An archive of farm management reports existed, including
18 farm management reports collaboratively written for Agronomy 356 and
6 farm management reports collaboratively written for 356/309.
To respond to our research question, we selected 6 of the 18 available recommendation reports from the stand-alone Agronomy 356 course and included all six of the reports written by the six student teams (19 students total) enrolled in the paired 356/309 during its first 2 years. Thus, we had 12 reports to be assessed by outside raters.
We used a stratified random sample of the reports from Agronomy 356 as a stand-alone course. We stratified the sample of reports to represent every year and every level of quality; then, we randomly selected reports from each level of the stratification. Using report grades to select a representative sample of six recommendation reports from the stand-alone Agronomy 356 course was not possible because many report grades were no longer available, particularly for the older reports. To select the reports from Agronomy 356 as a stand-alone course according to their chronology and rhetorical strengths and weaknesses, we conducted a holistic assessment of each of the 18 reports, using the following criteria:
articulated purpose (i.e., clear statement of the report’s objectives),
consideration of context and consultant’s role,
adaptation of audience,
effective organization,
content completeness and accuracy,
useful visuals,
appropriate signaling (e.g., headings, subheadings, topic sentences),
sentence clarity and conciseness, and
compliance with mechanical/grammatical conventions.
Each of us independently assessed the 18 reports on a scale of 1 to 5. Then we combined our ratings to select 1 excellent report, 4 adequate reports, and 1 weak report while balancing the selections chronologically. We agreed exactly on the rating for 5 of the 6 reports selected; for the 1 report about which we disagreed, one of us rated a report as a 2 and the other, as a 3—that report was counted as one of the weak reports.
The overall sample of 12 reports for assessment by outside raters comprised 6 reports from Agronomy 356 as a stand-alone course and 6 from the integrated 356/309. Once the 12 reports were chosen, all student names, dates, and place names (e.g., farm names, field names, and county or town locations) were blacked out to protect student and client privacy and to prevent raters from recognizing the dates that the reports had been written and the farm locations used during particular semesters.
Assessment Criteria
Once the report sample was selected, the three 356/309 instructors collaboratively developed the assessment criteria used in this study. While the instructors spent approximately 3 hours articulating and defining criteria (argument effectiveness, document usability, and professional), in reality, this task synthesized dozens of hours of meetings since the beginning of 356/309 that explored issues related to assessing student writing. All three instructors created these categories collaboratively, not in discipline-specific ways. The criteria that the instructors developed to assess differences among the reports are shown in Table 3.
Assessment Criteria for Farm Management Report Project Drafts
Articulating the assessment criteria was seen as an instructor task, a critical part of the study but one that had to be completed by instructors, not researchers. While we as the researchers were present for the discussions to identify and define the assessment criteria that raters would use, the instructors completed this task without direct input from us, drawing on their own assignment evaluation criteria as well as nearly a year and a half of discussions. We made a conscious effort not to contribute content categories/definitions or suggest ways to organize these categories. The instructors’ aim was to formulate criteria that could be used to assess the reports effectively.
Interestingly, instructors never considered stereotypes often associated with technical disciplines. For example, no instructor considered having disciplinary distinctions, with one category as “agronomy” and the other as “professional communication,” and no one suggested categories such as “content accuracy” or “grammar and mechanics.” Instead, the instructors were equally involved and committed to articulating categories that represented the concepts emphasized in all the work that the students completed, which meant that agronomy and professional communication were inexorably integrated. The 356/309 instructors’ articulation of the assessment criteria speaks somewhat to research comparing technical writing and engineering instructors’ feedback, which shows that each group’s “standards” for evaluating undergraduate students’ feasibility studies were “not as wide as is generally assumed” (S. Smith, 2003). We should note that the two agronomy instructors had each been involved in WAC/WID professional development workshops for more than 5 years and that the English instructor had a strong technical background, making the three of them unlikely to see assessment criteria as disciplinary.
Rater Selection and Training
To include raters from both disciplines, three agronomists and three professional communicators were invited to serve as raters. As Table 4 shows, the raters had a range of academic and workplace experience suitable for assessing the argument effectiveness, document usability, and professionalism of the reports. Two agronomy raters were male and one was female; two professional communication raters were male and one was female. The raters participated in a 2-hour rater training session and volunteered the extensive time needed to assess 12 lengthy reports (in exchange for our eternal thanks and some snacks during training).
Rater Areas and Professional Experience
We must note other important characteristics about the raters: All of the agronomy raters were not only highly regarded agronomic researchers or practitioners but were explicit in their dedication to education. All believed that effective written, oral, and visual communication are critical skills for agronomists; they themselves were experienced and effective communicators. Similarly, all of the professional communication raters were highly regarded researchers or experienced practitioners who were explicit in their dedication to education. The professional communication raters also had a modest knowledge of agriculture (from growing up on an Iowa farm to working in the Center for Rural Development, a think tank devoted to agricultural innovation)—a knowledge beyond the average person on the street but certainly not expert. So while none of the raters had anything resembling or approaching expertise in the other area, they all had some experience, as well as a positive attitude about the other area.
The training session, conducted by us (as the two researchers), was meant to enable raters to understand the assessment criteria, to allow them to apply the criteria using concrete examples from the training materials, and to calibrate the raters. Once the rater training was complete, we distributed a packet to each rater containing photocopies of the 12 farm management reports. We used paper copies of the reports for assessment because the final copies that students submitted were print, not electronic. Each report was assigned a code number that had no correlation to the year in which it was written. Each rater was also given paper and electronic versions of a worksheet to rank-order the reports, as well as 12 additional worksheets for them to describe the rationale behind ranking.
Due to the length of the reports, the raters were allowed 5 weeks to complete their task. Five of the six raters completed both assessment tasks electronically (i.e., rank-ordering the reports and then writing a rationale of each rank); one rater chose to submit the assessment tasks in a hard copy. All raters submitted their assessment during Week 5.
Data Analysis
Once the raters completed their task, we determined the report ranking and whether differences in the raters’ rankings were statistically significant. To further analyze the results, we looked at agreements and disjunctures between the statistical data and the rationale statements written by each rater. Analyzing the data in this way allowed us to identify distinctions that we may have otherwise neglected, and doing so illustrates the value of drawing on methods both quantitative and qualitative.
Study Limitations
While the study affords us the opportunity to analyze writing performance in stand-alone and integrated courses, it has certain limitations. The enrollment cap in Agronomy 356, as well as the fact that student teams (rather than individual students) were assigned to draft the recommendation reports, precluded us from including a larger sample size of reports from the 356/309 integrated courses. Students in both environments—the stand-alone and integrated courses—self-selected their enrollment in the courses; thus, they were not randomly assigned. This study could not control for reactivity and, in particular, the Hawthorne effect, as we did not use a blind experiment design.
Results and Discussion
The three agronomy raters and three professional communication raters assessed 12 reports—6 representative reports from the stand-alone course and 6 from the integrated courses—for argument effectiveness, document usability, and professionalism (Table 5). No statistically significant difference (<.0001) was found between the ratings done by the agronomy raters and the professional communication raters, which indicates that the two categories of raters responded similarly when they assessed the reports, despite their disciplinary differences.
Mean Rank for Reports Written in the Integrated and Stand-Alone Courses
The rank orderings of the 12 reports are displayed in Table 6, which shows the rank that each rater assigned to each report and which indicates whether the report was generated by a student team in a stand-alone section of Agronomy 356 or in the integrated 356/309 courses. All of the highest-rated reports were produced in integrated courses, and all of the lowest-rated reports were written in stand-alone sections of 356. Raters disagreed only slightly as to which were the top- or bottom-rated reports. We found the raters’ comments on the reports to be relatively similar in demonstrating a clear understanding of argument effectiveness, document usability, and professionalism, whether they were commenting on top-, middle-, or low-rated reports.
Raters’ Rank Order of the 12 Reports
1 = top ranked, 12 = lowest ranked.
SA = stand-alone; IC = integrated courses.
Unlike their counterparts in the stand-alone courses, student teams in the integrated courses who produced top-rated reports drafted documents that had effectively crafted arguments, usable content and design, and appropriately professional tone and appearance. Interestingly, the raters’ comments indicate that high-rated reports displayed a relatively sophisticated understanding of the client as user while low-rated reports did not. Farmer clients are users of these reports, as they are likely to read the report in a number of ways: initially skimming to learn the key recommendations, reading carefully to understand the analyses and rationales, referencing specific sections to plan implementation of recommendations, and periodically returning to the report to reread and recheck information. Explicit attention to the client as user clearly differentiated high-rated and low-rated reports.
Top-rated reports displayed accurate arguments grounded in “sound” agronomic principles (Agronomy Rater 3). Top-rated reports also tailored their agronomic argument and resulting recommendations to the client’s unique farm operation. In speaking to this audience adaptation, one rater indicated that the recommendations in one top-rated report were not only “organized very effectively” but also “very usefully focused on client needs and uses” (Professional Communication Rater 1). The arguments made in low-rated reports, then, not only lacked agronomic rigor but also failed to connect with or sometimes even acknowledge the client. For instance, one rater concluded that “this [low-rated report] report is written like an essay—not very useful for the client. There are not clear guides for the reader. . . . This is a report written for a teacher, not a usable document written for a farmer client” (Professional Communication Rater 1).
Part of this inadequacy involved identifying and explaining the problems and opportunities specific to the client and his farm operation, which students in both the stand-alone and the integrated courses were able to learn about in several ways: participating in a lengthy group interview with the client, studying topographic maps, participating on a farm tour during which they were encouraged to make observations and take notes, taking soil and manure samples for testing at a state laboratory, and analyzing the results of soil and manure testing. Without exception, though, the raters said that low-rated reports exhibited difficulties stating the problems and opportunities clearly—or sometimes even at all. Neglecting the problems and opportunities had a spillover effect in that these low-rated reports also had, not surprisingly, inadequate recommendations. As one of the agronomists stated, “recommendations [in this low-ranked report] run from reasonable to ridiculous” (Agronomy Rater 3).
Our study’s results echo work done about writing performance in paired courses, which also showed this type of learning environment’s positive impact on student writing (Cargill & Kalikoff, 2007; Messina & White, 1992). In particular, our study demonstrates the usefulness of coassigning a writing assignment to help students practice addressing an audience other than the instructor (Manahan & English, 2002) and using rhetorical strategies to recommend or persuade (Clark & Fischbach, 2008; Yalvac et al., 2007).
Our study’s results extend work about the benefits of the “integrative assignment” to student learning, in which instructors from two disciplinary courses collaborate to coassign a project or assignment to students in the paired classes (Boix-Mansilla & Dawes, 2004). In their study of writing performance in an integrated pair of courses (one in political science, the second in film studies), Dunlap and Sult (2008/2009) describe how they carefully revised their integrative assignment—and the ways they taught it—to enable students to move beyond “rudimentary integration” toward more sophisticated written analyses of politics and film (p. 30). In their study, they noted the importance of giving students “opportunities to practice, to falter” and of providing “scaffolding,” especially in the form of substantive feedback as they worked on the assignment (p. 35). In 356/309, collecting and analyzing data, talking with the farmer client, and writing drafts were opportunities for students to practice engaging in a consulting relationship within the protective environment of the classroom. A reiterative cycle of report drafting, feedback, and revisions—described below—was the primary method by which instructors prompted students to “falter.” As one of the agronomy professors noted, “if [students] are going to make mistakes, let’s make them here [in 356/309]. . . . Let them fail here so they can succeed later.” This notion of the classroom as a safe environment for students to try new skills and strategies speaks to the differences between expectations for performance at work and school. Unlike those in the workplace, errors in school are often “valued as opportunities to clarify a particular concept or teach a relevant skill” (Dias et al., 1999, p. 68).
In 356/309, instructors developed a cycle of writing, feedback, and revision for the reports. Student teams received written feedback on penultimate drafts of the tillage progress and nutrient progress reports from each 356/309 instructor. Teams then revised these drafts and submitted them for a grade. Since these progress reports were written early in the semester, students could practice engaging with the farmer client using a report structure, thereby enabling them to be more prepared to tackle the lengthier and more involved recommendation report. Student teams also received written and oral feedback on a penultimate draft of the recommendation report. Two weeks before recommendation report drafts were due to the farmer client, teams submitted a penultimate draft to each instructor. The instructors provided written feedback individually on these drafts and then met as a group with each team for about 20 minutes, discussing with students the issues that needed improvement in their drafts. Lab time in Agronomy 356 was used for these feedback sessions. When Agronomy 356 was taught as a stand-alone course, students did not receive feedback like this on their progress or recommendation report drafts. Students could use this 356/309 instructor feedback not only to improve their drafts but also to reflect about the consulting relationship with the client and ultimately help them to achieve the 356/309 learning outcomes.
Our study showed that this intentional approach to crafting and providing feedback about the integrative assignment helped the 356/309 instructors to move away from simply requiring that students mimic workplace genres in the classroom (Dias et al., 1999; Freedman & Adam, 2000; Paretti, 2008). Students received feedback from instructors in both disciplines, encouraging them to practice integrating agronomic and rhetorical knowledge. As evidenced by their performance on the reports, many students were able to do so, reaching out to the clients and making recommendations that were not only agronomically sound but also appropriate, given the client’s particular farm operation. Thus, not only was the report assignment the lynchpin between 356 and 309, but the report also enabled the students to practice integrating agronomic and rhetorical knowledge in substantive and effective ways. In the concluding section, we speculate about what pedagogical factors specific to these integrated courses helped students accomplish this integration.
Conclusions
Four factors appear to be influential in enabling students to produce far better recommendation reports in 356/309: working in dual problem-solving spaces, pushing the boundaries on problem solving, incorporating workplace realities, and using just-in-time teaching.
First, the integrated nature of the courses encouraged students to work in dual problem-solving spaces, both in agronomy and in professional communication, each of which has distinct disciplinary approaches and conventions. In 356/309, these spaces were mediated by students’ written work on their team’s recommendation report. The dual problem-solving spaces operated simultaneously—a result of instructors coassigning and coassessing assignments, sharing time and facilities for both classes, and providing feedback to students from all three instructors. The instructors all pointed explicitly to the problem-solving nature of students’ agronomic and communication tasks—the parallels, the overlaps, and the differences.
And were students eager—or even comfortable—working collaboratively in these dual problem-solving spaces? No. They were often resistant and reluctant, but their success depended on two critical factors: becoming a cooperative member of their own team and functioning in these dual spaces. The focus for student inquiry and response was the recommendation report, which remained a consistent assignment and writing product across both spaces. Through the semester-long focus on the client relationship and report, students were introduced to appropriate, efficient strategies that moved considerably beyond formulas and data. The instructors encouraged students to interpret data in light of various agronomic, social, environmental, and economic issues so that they critiqued reasoning and processes, not just outcomes. The process itself promoted self-reflection and self-analysis.
The key to success? Tasks were done by teams in dual problem-solving spaces, not individually in isolated classes. The duality of the problem-solving spaces was reinforced by the shared physical spaces since the integrated classes and lab all used the same room. In English 309, students learned the importance of audience as well as the use of rhetorical strategy to connect with that audience, focusing especially on workplace contexts. Clearly, the intentional instruction in these areas helped students to produce more usable reports. The 356/309 instructors’ commitment to integration and collaboration was key in the success of this instruction. Without some type of support or professional development, writing instructors often are challenged to help students improve their performance on complex, discipline-specific genres (Bayer et al., 2005; Cross & Wills, 2005; Wardle, 2004).
In 356/309, the instructors collaborated with one another to provide that support. To do so, instructors held weekly 1-hour meetings to coordinate seminar and lab activities and to schedule and discuss assignments and assignment deadlines; in addition, instructors often attended one another’s courses. These faculty meetings and class visits provided useful opportunities for instructors to hear their colleagues discuss teaching strategy, course concepts, and notions about the clients, consulting project, and student performance. Instructor collaboration also moved beyond the classroom. During a 3-year period, instructors delivered six presentations as a teaching team about 356/309 at national and university-sponsored conferences. To do so, they together critiqued 356/309, reflected on and examined their roles, and discussed pedagogy. These presentations also prompted the instructors to develop a coherent narrative about the integrated courses, the collaboration, and their teaching practice.
Second, students were encouraged to push the boundaries on problem solving. Because of the complexity of the task, they learned that problem solving is recursive rather than linear and that the simple, narrow path of problem solving may not yield any productive results (Watts, 2008). This boundary pushing was seldom welcomed by the students. In fact, because the 356/309 courses were one of the first in their curriculum to present problem solving as a complex, long-term, recursive activity, students often resisted engaging in it. An important reason why 356/309 succeeded in pushing students toward recursive problem solving was the integrative nature of 356/309—integrating a core philosophy, the curriculum, and teaching, as well as sharing the same physical space. Both the agronomy instructors and the English instructors reinforced the importance of recursive problem solving by assigning a number of ongoing activities that occurred in both courses throughout the semester (thus also reinforcing the criticality of dual problem-solving spaces):
Check and refine predictions: Students recorded preliminary predictions about their observations and then revised and refined those predictions when they received new and more specific data. Students knew generalizations about the effect of nutrients, then they refined these generalizations based on the laboratory analyses of the manure and soil sampling.
Analyze data and recommend farm management practices: Students were encouraged to interpret rather than simply repeat the laboratory test results from manure sampling and soil sampling. They were scaffolded as they began the difficult task of looking for meaningful patterns in the test results, their interview with the farmer, and their observations of this particular farm. They needed to be frequently reminded that their recommendations had to be based on their interpretation of test results, the farmer’s needs, and their own observations.
Represent data visually: Students needed to learn when and how to represent data visually in maps, tables, diagrams, and graphs. Too often, they depended on the default parameters of their software rather than making informed decisions about appropriate ways to represent data visually.
These repeated activities were designed to discourage students from using simplistic or erroneous methods to assess complex factors and instead encourage them to engage in recursive problem solving that reinforced the integrative nature of the tasks.
Third, the integrated courses and the activities and assignments in the courses incorporated workplace realities. The courses were not the workplace, but these courses provided a space for the classroom and the workplace to intersect. The classroom—with three teachers—provided a space for students to learn new information in two disciplines and practice using it in a single profession. The workplace—working for a farmer client who wanted data-driven recommendations—gave students opportunities to observe what was typical and to spot anomalies, also enabling instructors to provide students with feedback about ways to respond to such situations. Instructor feedback was critical for encouraging students to connect with and learn from the client and their consulting relationship.
Fourth, the instructors used what they called just-in-time teaching, which enabled them to create a responsive learning environment that included more problem solving and more links to the world outside the classroom than what is typical in many project-based classes. The instructors were able to provide scaffolding as appropriate and to encourage students to use productive models (e.g., models of proposal and report drafts) and heuristics (e.g., key questions to ask farmer clients). The instructors also consciously built in what, for students, were unexpected situations needing workarounds, recalculations, and reinterpretations. And, finally, all the instructors believed that the regular and detailed feedback they provided to students for the review and revision of their work was enormously important.
Given the results of this study, we believe that integrating content and communication courses can improve student writing performance. Discovering which features helped to improve writing performance—multiple revisions, feedback from different disciplines, student teams—is the next challenge.
Footnotes
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The author(s) received no financial support for the research, authorship, and/or publication of this article.
