Student and Faculty Perceptions of Generative Artificial Intelligence in Student Writing

Abstract

Background

Psychology instructors frequently assign writing-to-learn exercises that include personal reflection. Generative Artificial Intelligence (GenAI) can write text that passes for humans in other domains.

Objective

Do students and faculty rate a reflection written by GenAI differently than reflections written by students? Do students and faculty agree about the appropriateness of using GenAI for college-level writing?

Method

Eighty-three students and 82 faculty read four reflections (three written by undergraduate students and one by GenAI). After rating the quality of each, they chose which one they thought was AI-generated. Participants then rated the ethicality of nine potential ways to use GenAI in college-level writing and the potential of each to compromise learning.

Results

Participants rated the AI-generated reflection similarly to the student-generated reflections and failed to reliably detect AI-generated writing. Faculty and students agreed that using GenAI to produce the final text for a student likely compromises learning more than using it to generate ideas.

Conclusion

AI-generated reflections blend in with student-written reflections, and students and faculty agree about the potential detriments to learning.

Teaching Implications

GenAI can be hard to detect in the psychology classroom. Rather than implementing one-size-fits-all policies, instructors might focus classroom conversations on how GenAI could compromise learning.

Keywords

student writing,generative artificial intelligence plagiarism

Get full access to this article

View all access options for this article.

References

American Psychological Association (2023). APA guidelines for the undergraduate psychology major, (Version 3.0). www.apa.org/about/policy/undergraduate-psychology-major.pdf

Anderson

L. W.

Krathwohl

D. R.

Bloom

B. S.

(2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives (Complete ed.). Longman.

Burton

J. W.

Stein

M.-K.

Jensen

T. B.

(2020). A systematic review of algorithm aversion in augmented decision making. Journal of Behavioral Decision Making, 33(2), 220–239. https://doi.org/10.1002/bdm.2155

Chang

T.-S.

Huang

H.-W.

Whitfield

(2021). Exploring EFL students’ writing performance and their acceptance of AI-based automated writing feedback [Conference session]. 2nd International conference on education development and studies, pp. 31–35. https://doi.org/10.1145/3459043.3459065

Clerwall

(2014). Enter the robot journalist: Users’ perceptions of automated content. Journalism Practice, 8(5), 519–531. https://doi.org/10.1080/17512786.2014.883116

Colman

A. M.

(2014). A dictionary of psychology (3rd ed.). Oxford University Press. https://doi.org/10.1093/acref/9780199534067.001.0001

Dietvorst

Simmons

J. P.

Massey

(2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology, 144(1), 114–126. https://doi.org/10.1037/xge0000033

Dunn

D. S.

Saville

B. K.

Baker

S. C.

Marek

(2013). Evidence-based teaching: Tools and techniques that promote learning in the psychology classroom. Australian Journal of Psychology, 65(1), 5–13. https://doi.org/10.1111/ajpy.12004

Fahmi

M. A.

Cahyono

B. Y.

(2021). EFL students’ perception on the use of grammarly and teacher feedback. Journal of English Educators Society, 6(1), 18–25. https://doi.org/10.21070/jees.v6i1.849

10.

Figueredo

Varnhagen

C. K.

(2005). Didn't you run the spell checker? Effects of type of spelling error and use of a spell checker on perceptions of the author. Reading Psychology, 26(4–5), 441–458. https://doi.org/10.1080/02702710500400495

11.

Graefe

Haim

Haarmann

Brosius

(2018). Readers’ perception of computer-generated news: Credibility, expertise, and readability. Journalism, 19(5), 595–610. https://doi.org/10.1177/1464884916641269

12.

Hostetter

A. B.

Call

Frazier

James

Linnertz

Nestle

Tucci

(2024, July 24). Student and faculty perceptions of generative artificial intelligence in student writing. https://doi.org/10.17605/OSF.IO/PHFTJ

13.

Kellogg

R. T.

Raulerson

B. A.

(2007). Improving the writing skills of college students. Psychonomic Bulletin & Review, 14(2), 237–242. https://doi.org/10.3758/BF03194058

14.

Kim

N. J.

Kim

M. K.

(2022). Teacher’s perceptions of using an artificial intelligence-based educational tool for scientific writing. Frontiers in Education, 7, 1–13. https://doi.org/10.3389/feduc.2022.755914

15.

Kobis

Mossink

L. D.

(2021). Artificial intelligence versus Maya Angelou: Experimental evidence that people cannot differentiate AI-generated from human-written poetry. Computers in Human Behavior, 114(2), 10653. https://doi.org/10.1016/j.chb.2020.106553

16.

Marche

(2022, December 6). The college essay is dead. The Atlantic. https://www.theatlantic.com/technology/archive/2022/12/chatgpt-ai-writing-college-student-essays/672371/

17.

McDaniel

M. A.

(2014). Active processing via write-to-learn assignments: Learning and retention in introductory psychology. Teaching of Psychology, 41(4), 303–308. https://doi.org/10.1177/0098628314549701

18.

McGuire

Lay

Peters

(2009). Pedagogy of reflective writing in professional education. Journal of the Scholarship of Teaching and Learning, 9(1), 93–107.

19.

National Association of Colleges and Employers (2022). The attributes employers want to see on college students’ resumes. https://www.naceweb.org/about-us/press/the-attributes-employers-want-to-see-on-college-students-resumes/

20.

Nevid

J. S.

Pastva

Mcclelland

(2012). Writing-to-learn assignments in introductory psychology: Is there a learning benefit? Teaching of Psychology, 39(4), 272–275. https://doi.org/10.1177/0098628312456622

21.

OpenAI (2023). ChatGPT (Jan 15 Version) [Large language model]. https://chat.openai.com/chat

22.

SERPninja (2024). Flesch Kincaid calculator. https://serpninja.io/tools/flesch-kincaid-calculator/

23.

Spirgel

A. S.

Delaney

P. F.

(2016). Does writing summaries improve memory for text? Educational Psychology Review, 28(1), 171–196. https://doi.org/10.1007/s10648-014-9290-2

24.

Stewart

T. L.

Myers

A. C.

Culley

M. R.

(2010). Enhanced learning and retention through “Writing to Learn” in the psychology classroom. Teaching of Psychology, 37(1), 46–49. https://doi.org/10.1080/00986280903425813

25.

Tian

Cui

(2023). Towards detection of AI-generated text using zero-shot and supervised methods. https://gptzero.me

26.

Undetectable LLC (2024). Undetectable AI. https://Undetectable.ai

27.

Writer (2024). AI content detector. https://Writer.com/ai-content-detector

28.

Zimmerman

(2006). Why some teachers resist change and what principals can do about it. NASSP Bulletin, 90(3), 238–249. https://doi.org/10.1177/0192636506291521