AI-generated content tools and students’ critical thinking: Insights from a Chinese university

Abstract

This study examines the impact of artificial-intelligence-generated content tools on students’ critical thinking skills and their attitudes towards these tools. A survey of 851 students from a Chinese university was conducted to investigate their usage patterns, motivations, perceived benefits and risk awareness, and the importance of critical thinking skills. The study also explores how libraries and librarians can help students develop critical thinking skills. The findings reveal that male and non-binary students utilized artificial intelligence tools more frequently, highlighting a research gap. The study uncovers motivations such as saving time and effort. It also demonstrates students’ awareness of the risks and limitations, emphasizing the need for critical thinking skills in navigating artificial-intelligence-generated content tools. These findings have significant implications for academic libraries, as students expressed a desire for education and training in artificial intelligence literacy and critical thinking. Libraries can provide resources, workshops and guidance to empower students in informed and critical thinking practices.

Keywords

AI-generated content tools critical thinking libraries ChatGPT era student perspectives

Introduction

The pervasive influence of artificial-intelligence-generated (AI-generated) content on students’ critical thinking is a pressing concern in today’s society. While AI-generated content holds the potential to enhance efficiency and productivity, there is apprehension that it may impede students’ critical thinking abilities (Wu et al., 2023). Numerous studies have expressed concerns that the use of ChatGPT as a search engine could lead to a decline in critical thinking skills (Dans, 2023) and that AI tools might hamper creativity and originality in students’ writing (Spector and Ma, 2019).

Despite these concerns, university students exhibit positive attitudes toward AI-generated content tools and acknowledge their potential benefits, such as improved efficiency and accuracy (Hoyos, 2023). However, students also recognize the potential limitations of these tools in fostering critical thinking and creativity. To effectively integrate AI into education, it is crucial to comprehend its opportunities, challenges and limitations.

Libraries emerge as vital allies in this transformative landscape. Fernandez’s (2023) study underscores that libraries can provide essential information literacy education, enabling students to evaluate and employ AI-generated content tools effectively. Additionally, Adetayo’s (2023) research reveals that libraries can leverage ChatGPT for technical and reader services, albeit with the understanding that it cannot replace human librarians due to its inherent limitations, such as inaccurate query responses and limited comprehension. Adetayo (2023) stresses that, instead, ChatGPT should be viewed as a supplementary tool, complementing the expertise of human librarians.

In light of these considerations, this study embarks on crucial exploration. It aims to delve into students’ attitudes and perceptions regarding AI-generated content tools and their awareness of the vital role critical thinking skills play in the modern educational landscape. Through rigorous examination, this research seeks to uncover the intricacies of AI integration and critically analyse its impact on students’ intellectual growth.

To achieve this goal, the study employed a comprehensive survey to collect data from students across various age groups, genders, university years and academic programmes. The survey focused on questions concerning the frequency of using AI-generated content tools, reasons for use, and associated benefits and risks. Additionally, it investigated how libraries help students developing critical thinking skills.

Literature review

In recent years, the field of education, including higher education, has witnessed significant transformations due to the emergence of AI. One specific area where AI has found applications is academic libraries, where AI chatbots have gained prominence. Adetayo (2023) delves into the utilization of AI chatbots in academic libraries, with a focus on the rise of ChatGPT as a leading chatbot in this domain. Additionally, Hoyos (2023) examines the use of AI in classrooms and discusses its potential benefits and challenges. Hoyos found that the benefits of AI include its ability to level the playing field for students, providing equal opportunities for high-quality work. It can also help students stay organized and manage their time effectively, and offer personalized suggestions. Proper training in using AI tools is crucial for maximizing their benefits. The challenges, such as concerns around cheating, necessitate strategies such as oral examinations and handwritten assessments to combat AI-generated cheating. However, the focus should be on the opportunities AI offers in enhancing education.

While AI holds promise in higher education, it is imperative to consider students’ perceptions and perspectives. Despite limited research on students’ attitudes towards AI-generated content tools like ChatGPT, a study conducted by Chan and Hu (2023) focuses on students’ familiarity, willingness to engage, perceived benefits (such as personalized learning support and research capabilities) and challenges (including accuracy and ethical concerns). Conversely, Chesterman (2023) explores the threat posed to the economic foundations of creativity by advanced AI tools like Stable Diffusion and ChatGPT, which can produce artwork and text that are almost indistinguishable from human creations. Traditionally, knowledge workers earned their living by analysing and writing – skills that are now replicable by AI with minimal time and cost. Chesterman (2023) suggests emphasizing the need for clear guidelines on the ownership of content generated by AI and the requirement for such content to be distinguishable as AI-produced, addressing the challenges posed by these technologies.

Early career researchers can also benefit from leveraging ChatGPT to enhance their productivity – for example, by using ChatGPT to debug code or brainstorm research ideas, as highlighted in an article by Askham (2023). Uludag (2023) conducted an interview with ChatGPT to assess its creativity in the field of psychology. It was found that ChatGPT has creative potential, especially in certain contexts. However, further research is required to optimize ChatGPT’s design and explore its creative capabilities in diverse fields. Zawacki-Richter et al. (2019) conducted a systematic review of AI applications in higher education. They identified challenges such as a lack of critical reflection on risks, weak connections to pedagogical theories, and the need for further exploration of the ethical and educational aspects in AI applications in higher education.

Critical thinking skills are of paramount importance in education, and an increasing body of research is exploring strategies for fostering these skills. Onen (2021) discusses approaches to cultivating critical thinking among graduate students. The study suggests the formal inclusion of critical thinking in the curricula of graduate programmes to enhance its cultivation among students, despite other existing challenges. Huber and Kuncel (2016) present research on developing critical thinking among college students. They found that educators consider critical thinking to be vital, but its effectiveness in college teaching is unclear. The study raises questions about the methods used to teach critical thinking in college and emphasizes the need for more effective approaches to ensure lasting improvements in critical thinking skills and attitudes among students. Todd et al. (2021) go beyond theoretical concepts and standards. Their article provides practical applications through case studies, demonstrating a model and framework for student engagement. This model aims to create an online course environment and strategies specifically geared towards developing critical thinking skills. Omariba (2021) explores the use of technology-enhanced classrooms to enhance critical thinking skills. This study found that technological tools can aid in collecting, analysing, displaying and communicating information.

The impact of ChatGPT on education has become an area of burgeoning interest. Paredes Fuentes (2023) shares initial lessons learned from employing ChatGPT in teaching macroeconomics. This study concludes that ChatGPT and similar technologies are here to stay and might become as prevalent as research engines in academia. Despite the current limitations, academia must engage with these technologies. Teaching and assessments need to adapt to accommodate them effectively. Lo (2023) presents a literature review on the influence of ChatGPT on education. The article suggests immediate actions, including updating assessment methods and institutional policies in schools and universities. It emphasizes the importance of instructor training and student education to effectively respond to ChatGPT’s impact on the educational environment. Bitzenbauer (2023) reports on a pilot study investigating ChatGPT’s application in physics education to foster critical thinking skills among secondary school students. The article provides two practical examples of implementing ChatGPT in teaching, demonstrating its easy integration into the classroom setting. Rusandi et al. (2023) discuss how ChatGPT can bridge the gap between AI and education. They emphasize that during AI integration, critical thinking and academic integrity should remain central concerns to ensure the responsible and meaningful use of AI technologies in education and research.

However, concerns have also been raised regarding the use of ChatGPT in education and research. Plebani (2023) points out that there is a debate in the scientific community regarding the use of AI tools like ChatGPT in publications. Policies have been developed by journals, including Nature, Science and JAMA: Journal of the American Medical Association, which prohibit naming such tools as credited authors due to the lack of accountability AI tools have for the work. Anders (2023) questions whether employing ChatGPT could be deemed cheating or plagiarism. Educational institutions need to future-proof their curricula to handle the forthcoming wave of AI-assisted assignments, ensuring that ethical guidelines and boundaries are clearly defined. Dai et al. (2023) reconceptualize ChatGPT and generative AI as innovations driven by students in higher education. Empowering students through ChatGPT requires collaborative efforts among stakeholders to address challenges related to student training, curricula, assessment, technology development and governance. McCarthy (2023) examines how the use of ChatGPT could reshape views on academic misconduct. The study found that ChatGPT’s introduction threatens traditional concepts such as plagiarism, authorship, ownership and learning in higher education.

Collectively, these studies demonstrate that while the integration of AI in education offers opportunities and challenges, an increasing body of research is investigating the potential of ChatGPT and other AI applications in education, as well as the best practices for cultivating critical thinking skills. This research study aims to address gaps in the existing literature by exploring university students’ attitudes towards AI-generated content tools, such as ChatGPT, DALL-E and the new Bing. It also investigates their usage frequency and motivations, and associated benefits and risks, as well as the significance of developing critical thinking skills in relation to text- or data-mining tools, recommendation systems, image or object recognition, automated content summarization, sentiment analysis, and speech recognition or synthesis. Additionally, the study investigates the role of libraries in enhancing students’ critical thinking abilities in the ChatGPT era.

Method

In this study, we conducted a survey to explore college students’ attitudes and behaviours regarding AI-generated content tools and their perspectives on critical thinking skills. The survey comprehensively covered various aspects, including the use of AI-generated content tools, reasons for utilizing these tools, perceived benefits, awareness of risks and limitations, the importance of critical thinking skills for future career development, and exposure to AI literacy and critical thinking education from libraries or other sources. We aimed to assess the students’ confidence in distinguishing between AI-generated and human-generated content, their perspectives on university resources, and libraries' support for fostering critical thinking skills in the ChatGPT era. Open-ended questions were included to delve into the ethical dilemmas associated with the use of AI-generated content in academic work.

The survey questionnaire consisted of four sections with a total of 20 items utilizing closed-ended, multiple-choice and open-ended formats. These questions were specifically crafted to probe students’ usage patterns of AI-generated content tools, their perceptions of the benefits and risks of AI, their understanding of the significance of critical thinking skills, and their exposure to AI education and training. The survey was administered online, and the participants were allotted approximately 20 minutes to complete it.

The analysis of the survey results addresses the following research questions:

Are there any differences in the usage of AI content-generation tools in the ChatGPT era based on college students’ age, gender, academic year and field of study?

What are the primary motivations for college students to utilize AI-content tools?

What differences exist in the motivation to utilize AI-generated content tools based on students’ age, academic year, major and gender?

What benefits can students receive from utilizing AI-generated content tools?

How aware are college students of the potential risks and limitations associated with AI-generated content tools?

What are the perceptions of college students regarding critical thinking skills and the significance of critical thinking in academic research and daily life, and their views on AI-generated content tools?

What resources and training do college students expect libraries to provide in order to help them enhance their critical thinking skills?

Participants

In addition to the previously mentioned information, it should be noted that the study adhered to the ethics rules and procedures in China. The survey was conducted anonymously to ensure confidentiality and protect the privacy of the participants. All of the participants signed a consent form. We also received consent from the parents or guardians of the participants who were under 18.

The study focused on examining the characteristics of the 869 participants who completed the survey through social media platforms at a Chinese university. The participants were selected using a convenience sampling method to ensure a representative sample. The sample size was determined based on a total population of 34,813, which included 19,061 undergraduate students, 12,304 Master’s-level students and 2712 doctoral-level students. The study aimed for a confidence level of 95% and a margin of error of 5%.

Here are some of the key participant characteristics identified in the study. These participant characteristics provide insights into the demographic composition of the survey respondents at the Chinese university:

Age range. The age group with the highest percentage was 18–20, accounting for 36.66% of the participants. This was followed by the 21–23 age group, accounting for 35.37%. The lowest percentage was observed in the under 18 age group, accounting for .59%.

Gender. Most of the respondents (63.57%) were male. Females accounted for 31.96% of the participants, while those who preferred not to disclose their gender or identified as non-binary accounted for 4.47%.

Academic year. The difference between graduate and undergraduate students was not significant. Undergraduate students represented 54.18% of the participants, while graduate students accounted for 45.82%.

Field of study. Engineering disciplines had the highest percentage, with 62.98% of the participants belonging to this field. Humanities and social sciences accounted for 13.28%, natural sciences for 8.7%, business and economics for 3.06%, and other fields for 11.99%.

Data collection and processing

The study adopted a survey tool equivalent to SurveyMonkey and Qualtrics – Wenjuanxing – to distribute the survey through university social media platforms over the duration of two weeks. During this period, we received a total of 869 responses.

To ensure the integrity of the survey sample, we implemented data-cleaning steps. Initially, we eliminated records with response times below 60 seconds, taking into account the length of the questionnaire and conducting a preliminary assessment of response times on a limited scale. Moreover, we eliminated records in which the same option was chosen for all questions except for the essential demographic information. These measures yielded a total of 851 valid responses, which were subsequently subjected to further analysis.

To assess the reliability of the questionnaire, we conducted an internal consistency analysis on specific sections of the survey: Section 2 (Usage of AI-Generated Content Tools), Section 3 (Critical Thinking Skills) and Section 4 (Library Strategies). The reliability coefficients obtained were .675, .671 and .671, respectively. These coefficients meet the required standards, indicating satisfactory reliability for the questions in these sections.

Data analysis

For the purpose of data analysis, we utilized the Statistical Package for the Social Sciences software. We also assessed the normality of the data using the Kolmogorov–Smirnov test. The results indicated a deviation from normal distribution for the scale items in the questionnaire. Consequently, non-parametric tests were employed to investigate differences among the independent samples.

For the scale-type items, we employed the Kruskal–Wallis H test to examine differences among the samples. If the Kruskal–Wallis test yielded statistically significant results, we proceeded to conduct the Mann–Whitney test to determine specific pairwise differences between the groups. On the other hand, the analysis of non-scale-type items involved the use of the Pearson chi-square test. In cases where the Pearson chi-square test showed significance, we performed pairwise comparisons utilizing the Bonferroni adjustment method to explore specific differences between the two groups. We also employed cross-tabulation to visually illustrate the differences in the proportions.

Results

Differences among students in the utilization of AI content-generation tools based on age, gender, grade level and major

The study investigated whether there are differences in the utilization of AI content-generation tools in the ChatGPT era based on students’ age, gender, grade level and major. The survey received 869 valid responses regarding the frequency of using AI-generated content tools. The responses were distributed as follows: 10.01% strongly agreed, 36.36% agreed, 43.84% were neutral, 7.25% disagreed and 2.53% strongly disagreed. As shown in Figure 1, a substantial proportion of the respondents (46.37%) either agreed or strongly agreed with using AI-generated content tools more frequently than before. However, the majority of the respondents (51.62%) expressed a neutral or dissenting stance towards increased usage of AI-generated content tools.

Figure 1.

Attitudes towards increased frequency of using AI-generated content tools.

Additionally, we conducted a Kruskal–Wallis test to explore the variations in AI content-generation tool utilization among the students based on age, gender, grade level and major. The question asked was whether individuals use AI-generated content tools more frequently than before (see Table 1). The findings indicate that age does not have a significant impact on the frequency of AI-generated content tool usage among students. Furthermore, no notable differences were observed in AI-generated content tool utilization based on the students’ grade level and major.

Table 1.

Kruskal–Wallis test results.

Variable	H	p
Age group	6.961	.138
Academic year	4.505	.479
Major	6.635	.156
Gender	25.208	.000***

***p < .01.

However, gender-related disparities in AI-generated content tool usage were identified. Specifically, male students and students identifying as non-binary tend to utilize AI-generated content tools more frequently than before. Moreover, non-binary students exhibit a stronger inclination towards the usage of AI-generated content tools compared to other gender identities. According to the p < .01 value in Table 1, there is variability in gender regarding the frequency of using AI-generated content tools in the ChatGPT era.

Additionally, we utilized the Mann–Whitney U-test to determine specific pairwise differences between groups based on gender (see Table 2). Based on Table 2, it was found that there were group differences between females (F) and males (M), females and non-binary (N), males and non-binary, males and prefer not to disclose (P), and non-binary and prefer not to disclose.

Table 2.

Mann–Whitney U-test results.

Gender group		Number of respondents (n)		Mdn		U	p	Cohen’s d
A	B	A	B	A	B	U	p	Cohen’s d
F	M	272	541	3	2	84242.5	.001***	0.243
F	N	272	17	3	2	3335	.002***	0.960
F	P	272	21	3	3	2365.5	.307	0.449
M	N	541	17	2	2	6085	.030**	0.626
M	P	541	21	2	3	4016	.029**	0.637
N	P	17	21	2	3	82	.006***	1.078

**p < .05. ***p < .01.

Finally, we performed a cross-examination to assess the disparities between genders in their attitudes towards the frequency of AI-generated content tool utilization. The results are presented in Table 3. For each gender identification, the first row displays the percentages of attitudes based on gender and the second displays the percentages pertaining to the frequency of usage of AI-generated content tools in the ChatGPT era.

Table 3.

Cross-tabulation displaying disparities between genders in attitudes towards frequency of usage of AI-generated content tools.

			Attitudes					Total
			Strongly agree	Agree	Neutral	Disagree	Strongly disagree	Total
Genders	M	n	63	210	224	29	15	541
		%	11.6	38.8	41.4	5.4	2.8	100
		%	74.1	67.5	60.2	46.0	75.0	63.6
	F	n	14	91	134	31	2	272
		%	5.1	33.5	49.3	11.4	0.7	100
		%	16.5	29.3	36.0	49.2	10.0	32.0
	N	n	7	5	4	1	0	17
		%	41.2	29.4	23.5	5.9	0.0	100
		%	8.2	1.6	1.1	1.6	0.0	2.0
	P	n	1	5	10	2	3	21
		%	4.8	23.8	47.6	9.5	14.3	100
		%	1.2	1.6	2.7	3.2	15.0	2.5
Total		n	85	311	372	63	20	851
		%	10.0	36.5	43.7	7.4	2.4	100
		%	100	100	100	100	100	100

Table 3 shows that the differences between males and females primarily manifest in their attitudes. Males exhibit the highest proportion of agreement and strong agreement, while females have the highest proportion of neutrality. Regarding the differences between males and non-binary individuals, non-binary college students show an agreement and strong agreement proportion exceeding 70%, with less than a quarter expressing neutrality. In contrast, among male students, the proportion of agreement and strong agreement exceeds 50%, while neutrality accounts for over 40%.

The differences between males and females compared to the differences between males and students who prefer not to disclose their gender are of a similar nature. However, the magnitude of the differences varies. Based on the Cohen’s d value, it can be observed that the difference between males and females is of a smaller magnitude, while the difference between males and students who prefer not to disclose their gender is of a moderate magnitude.

The main distinction between females and non-binary individuals lies in the distribution of attitudes. Among non-binary individuals, the majority express agreement and strong agreement, while among females the largest proportion leans towards neutrality. Significant differences are found among non-binary individuals, as well as students who prefer not to disclose their gender. Non-binary individuals display the highest proportion of agreement and strong agreement, followed by neutrality, and the lowest proportion of disagreement. Conversely, students who prefer not to disclose their gender have the highest proportion of neutrality, followed by agreement and strong agreement, and the lowest proportion of disagreement and strong disagreement. However, the disparity in proportions between agreement and disagreement is not substantial, amounting to less than 5%.

Primary motivations to use AI content-generation tools

The research findings unveiled the primary motivations driving college students to employ AI content-generation tools, which were identified as time and effort saving (see Figure 2).

Figure 2.

Main reasons for using AI-generated content tools.

Differences in motivation to use AI-generated content tools based on students’ age, academic year, major and gender

We employed a chi-square test to examine the differences in motivation for using AI-generated content tools based on students’ age, academic year, major and gender (see Table 4). The note **p < .05 signifies that there is a statistically significant difference in the academic year regarding the motivation to use AI-generated content tools.

Table 4.

Chi-square test results.

Dimension	χ²	p
Age group	9.683	.644
Academic year	30.511	.010**
Major	15.046	.239
Gender	14.333	.111

**p < .05.

We also performed cross-tabulations to analyse and identify significant differences in academic years concerning the primary motivations for utilizing AI-generated content tools. Table 5 shows that the cross-tabulations were performed to examine the primary motivations based on academic years. The table presents the results of the cross-tabulations, showing the relationship between academic years and the primary motivations.

Table 5.

Results of cross-tabulations showing the relationship between academic year and primary motivations.

Academic year		Primary motivations				Total
Academic year		Saving time and effort	Improving quality of work	Enhancing creativity	Other	Total
First year	n	178	18	14	4	214
First year	%	83.2	8.4	6.5	1.9	100
Second year and above	n	138	41	17	3	199
Second year and above	%	69.3	20.6	8.5	1.5	100
Total	n	316	59	31	7	413
Total	%	76.5	14.3	7.5	1.7	100

Table 5 shows that distinctions in motivations were observed across different academic years. Among first-year undergraduate students, the majority (over 80%) identified time and effort saving as their primary motivation, while less than 10% considered improving quality of work. However, at the second-year-and-above level, the proportion of students motivated by time and effort savings dropped below 70%, while over 20% were motivated by improving work quality, indicating a significant difference.

Benefits of using AI-generated content tools

Approximately 59.7% of the college students strongly agreed or agreed that AI content-generation tools would bring benefits, while only 2.59% held opposing views, and 37.72% remained neutral. Gender differences were observed on this matter, with over half of males expressing agreement and approximately one-third maintaining a neutral position, and females having the highest proportion of agreement, closely followed by neutrality, with both exceeding 40% with minimal differences. Non-binary individuals exhibited the highest proportion of strong agreement, followed by agreement, with both exceeding 40% and showing slight differences. Students who preferred not to disclose their gender had a higher proportion of neutrality, followed by strong agreement and agreement, with each accounting for approximately one-fifth (see Figure 3).

Figure 3.

Perceptions of college students regarding benefits of using AI-generated content tools.

Potential risks and limitations of AI-generated content tools

The majority of the students (82.84%) demonstrated a noteworthy level of awareness regarding the potential risks and limitations associated with AI-generated content tools. Interestingly, gender differences were observed in the responses to potential risks and limitations, with females displaying a higher level of awareness compared to males. Also in comparison to females, non-binary individuals were more likely to respond negatively, indicating a lack of awareness, when asked about their understanding of the potential risks and limitations.

Among the surveyed students, the top-three risks and limitations identified were a lack of originality and creativity, plagiarism and inaccuracy. A significant proportion (78.01%) expressed concerns about the lack of originality and creativity associated with AI-generated content tools (see Figure 4).

Figure 4.

Top-three risks and limitations associated with AI-generated content tools.

Critical thinking skills

As shown in Figure 5, an overwhelming majority of the college students (88.96%) acknowledged the importance or high importance of fostering critical thinking skills for both academic research and daily life during their university studies. Notably, the responses to this inquiry did not reveal significant differences based on factors such as age, gender, grade level and major.

Figure 5.

Perceptions of college students regarding critical thinking skills.

As shown in Figure 6, among the college students, 86.02% considered the development of critical thinking skills to be very important or important for their future career progression. Similar to the previous question, no noteworthy distinctions were observed in the responses based on age, gender, grade level and major (p values > .05).

Figure 6.

Attitude towards developing critical thinking skills.

Regarding encounters with ethical dilemmas linked to the utilization of AI-generated content tools, approximately 96.83% of the participants reported not encountering any such dilemmas. However, on examining the descriptions provided by those who did face dilemmas, it became evident that they primarily arose in fields such as painting, drawing, cartoons and images. The keywords employed by the respondents to elucidate these ethical dilemmas included concepts of threat, conflict, issues of ownership and uncertainty.

Resources and training

A minority of the students (less than 10%) had received explicit education or training focused on AI literacy and critical thinking skills. Among the respondents who had received relevant training or education, a range of training formats was reported, with lectures on AI and information literacy being the most prevalent. Approximately 57.7% of the students expressed a desire for additional education or training concerning AI literacy and critical thinking skills. As shown in Figure 7, the students exhibited diverse preferences for various forms of education or training targeting AI literacy and critical thinking skills.

Figure 7.

Types of education or training preferred by the college students.

A significant majority of the students (over two-thirds of the participants) expressed a positive perception of the university’s capacity to provide sufficient resources and support for fostering critical thinking skills in the era of ChatGPT. Consensus on this matter was observed irrespective of age, gender, grade level or field of study, as these factors did not exhibit significant differences in the responses.

Regarding opinions and suggestions on how libraries can contribute to the promotion of critical thinking in the era of ChatGPT, the following points were identified: organizing lectures, training courses and other activities to enhance students’ critical thinking abilities; acquiring relevant books and journals aimed at augmenting critical thinking; offering credit courses focused on the topic; and implementing initiatives to foster critical thinking, such as organizing debates, creating educational games, producing promotional videos and utilizing customized micro-videos that align with students’ interests. Additionally, libraries can effectively disseminate and facilitate these efforts through LibGuides and other platforms by means of notifications.

Discussion

This discussion section aims to provide an analysis and interpretation of the study’s findings in relation to the information presented in the introduction and literature review. By examining the congruence and divergence between the results and previous research, a comprehensive understanding of the study’s implications can be achieved.

The present study focused on investigating the impact of AI-generated content tools on students’ critical thinking skills and their attitudes towards such tools. The introduction highlights concerns expressed by researchers regarding the potential limitations of AI-generated content on critical thinking, which are supported by previous findings, indicating a decrease in creativity and originality in writing when using AI tools (Dans, 2023; Spector and Ma, 2019; Wu et al., 2023). Conversely, positive attitudes towards AI-generated content tools have been identified among university students (Hoyos, 2023). These contrasting perspectives lay the foundations for exploring the consistency and inconsistency of the current study’s findings.

The literature review reveals the increasing utilization of AI chatbots in academic libraries, with ChatGPT emerging as a prominent tool in this field (Adetayo, 2023). The review also explores students’ perceptions of generative AI and underscores their recognition of the potential benefits (Chan and Hu, 2023). Additionally, the importance of cultivating critical thinking among college students is emphasized, providing a basis for examining the impact of AI-generated content tools on this essential skill (Huber and Kuncel, 2016). A review of the literature on ChatGPT’s influence on education further contextualizes the present study (Lo, 2023).

On analysing the results, several noteworthy consistencies and inconsistencies have been identified. In terms of usage patterns, the study found that male and non-binary students exhibited a higher frequency of utilizing AI-generated content tools compared to previous levels. This investigation fills a gap in the existing literature by specifically examining gender differences in technology adoption within the context of AI-generated content tools.

The study also examined the motivation behind using AI content-generation tools – namely, the desire to save time and effort. There is no specific mention of previous studies directly investigating the motivation behind using AI content-generation tools, and specifically the desire to save time and effort. Furthermore, the study demonstrated that the majority of the students were aware of the potential risks and limitations associated with AI-generated content tools, which corroborates the concerns raised in the literature (Dans, 2023; Spector and Ma, 2019). Dans (2023) expresses concerns about the decline in critical thinking skills when using ChatGPT as a search engine. Spector and Ma (2019) suggest that AI tools can decrease creativity and originality in students’ writing. This suggests that students recognize the drawbacks of solely relying on AI-generated content tools and underscores the importance of promoting critical thinking skills to navigate these tools effectively.

Notably, the finding that 88.96% of the college students acknowledged the importance or high importance of cultivating critical thinking skills resonates with the literature, which emphasizes the significance of critical thinking in academia and daily life (Huber and Kuncel, 2016). This reaffirms the value of integrating critical thinking education within the curriculum, particularly in the context of AI-generated content tools.

However, while the literature review highlights positive attitudes towards AI-generated content tools among university students, the present study did not explicitly assess student attitudes towards such tools. Future research should explore this aspect to provide a more comprehensive understanding of students’ perceptions.

Considering the implications of the findings, academic libraries play a significant role. With 57.7% of the students expressing a desire for additional education or training on AI literacy and critical thinking skills, there is an opportunity for libraries to support students in developing these essential competencies. By providing resources, workshops and guidance on critically evaluating and utilizing AI-generated content tools, libraries can contribute to fostering informed and critical thinking among students.

In conclusion, this study examined the consistency and inconsistency between the findings and the information presented in the introduction and literature review. While certain consistencies were observed, such as gender differences in usage patterns and the importance of critical thinking skills, further research is warranted to explore students’ attitudes towards AI-generated content tools. These findings offer valuable insights for academic libraries in enhancing students’ AI literacy and critical thinking skills, empowering them to navigate the digital landscape effectively.

One limitation of the study is that it was conducted solely at a Chinese university. Therefore, the results obtained from this study may not provide a comprehensive representation of the seven research questions for college students in general. It is important to acknowledge that cultural, educational and contextual factors specific to the Chinese university setting might have influenced the findings. Therefore, caution should be exercised when generalizing the results to a broader population of college students. Future research should aim to replicate the study in diverse settings and cultural contexts to obtain a more holistic understanding of the topic.

Conclusion

This study surveyed 851 students from a Chinese university to examine the impact of AI-generated content tools on students’ critical thinking skills and their attitudes towards these tools. The analysis aimed to explore the role of libraries in supporting students in navigating AI-generated content tools and fostering critical thinking. The findings show that male and non-binary students used AI tools more frequently, contributing to a research gap. Motivations such as time and effort saving were identified. The students demonstrated awareness of the risks and limitations associated with AI-generated content tools, highlighting the importance of critical thinking skills in utilizing these tools. While the study has focused on the impact on critical thinking skills, it also reveals the significance of developing these skills in the academic context.

Future research is needed to assess student attitudes towards AI-generated content tools. The study has implications for libraries, as the students expressed a desire for additional education and training in AI literacy and critical thinking. Libraries can provide resources, workshops and guidance to empower students in evaluating and utilizing AI-generated content. Overall, this study contributes to existing knowledge by examining the influence of AI-generated content on students’ critical thinking skills and attitudes, emphasizing gender differences and the need for further research in these areas, and that libraries and librarians play a vital role in helping students develop critical thinking skills.

Footnotes

Acknowledgements

Ping Fu and Xiaozhu Zou proposed the topic, designed the study, organized the experiment, supervised the project, and provided resources, guidance and mentoring. Ping Su and Lexing Li collected, processed and analysed the data. All of the authors contributed to writing the article and read and approved the final version. The authors thank the students who participated in the survey study and provided their input, insights and expertise.

Availability of data and materials

The data sets used and/or analysed during the study are available from the corresponding author.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The authors received no financial support for the research, authorship and/or publication of this article.

ORCID iD

Ping Fu

References

Adetayo

(2023) Artificial intelligence chatbots in academic libraries: The rise of ChatGPT. Library Hi Tech News 40(3): 18–21.

Anders

(2023) Is using ChatGPT cheating, plagiarism, both, neither, or forward thinking? Patterns 4(3): Article 100694.

Askham

(2023) Spectrum launch: How early-career researchers can use ChatGPT to boost productivity. Spectrum, 9 March. DOI: 10.53053/tych2095.

Bitzenbauer

(2023) ChatGPT in physics education: A pilot study on easy-to-implement activities. Contemporary Educational Technology 15(3): Article ep430.

Chan

CKY

(2023) Students’ voices on generative AI: Perceptions, benefits, and challenges in higher education. Available at: https://arxiv.org/pdf/2305.00290v1 (accessed 31 July 2023).

Chesterman

(2023) AI-generated content is taking over the world. But who owns it? NUS Law Working Paper no. 2023/002. DOI: 10.2139/ssrn.4321596.

Dai

Liu

Lim

(2023) Reconceptualizing ChatGPT and generative AI as a student-driven innovation in higher education. Procedia CIRP 119: 84–90.

Dans

(2023) ChatGPT and the decline of critical thinking. Available at: https://www.ie.edu/insights/articles/chatgpt-and-the-decline-of-critical-thinking/ (accessed 31 July 2023).

Fernandez

(2023) ‘Through the looking glass: Envisioning new library technologies’: AI-text generators as explained by ChatGPT. Library Hi Tech News 40(3): 11–14.

10.

Hoyos

(2023) Welcoming AI into the classroom. Available at: https://www.ie.edu/insights/articles/welcoming-ai-into-the-classroom/ (accessed 31 July 2023).

11.

Huber

Kuncel

(2016) Does college teach critical thinking? A meta-analysis. Review of Educational Research 86(2): 431–468.

12.

(2023) What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences 13(4): Article 410.

13.

McCarthy

(2023) ChatGPT use could change views on academic misconduct. Student Affairs Today 26(2): 3–6.

14.

Omariba

(2021) Technology-enhanced classroom to enhance critical thinking skills. In: Information Resources Management Association (eds) Research Anthology on Developing Critical Thinking Skills in Students. Hershey: IGI Global, pp 488–506. DOI: 10.4018/978-1-7998-3022 -1.ch025.

15.

Onen

(2021) Cultivating critical thinking amongst university graduate students. In: Information Resources Management Association (eds) Research Anthology on Developing Critical Thinking Skills in Students. Hershey: IGI Global, pp. 1104–1119. DOI: 10.4018/978-1-7998-3022 -1.ch057.

16.

Paredes Fuentes

(2023) Some initial lessons from using ChatGPT and what I will tell my macroeconomics students. Available at: https://economicsnetwork.ac.uk/showcase/fuentes_chatgpt (accessed 31 July 2023).

17.

Plebani

(2023) ChatGPT: Angel or demond? Critical thinking is still needed. Clinical Chemistry and Laboratory Medicine 61(7): 1131–1132.

18.

Rusandi

Ahman

Saripah I

, et al. (2023) No worries with ChatGPT: Building bridges between artificial intelligence and education with critical thinking soft skills. Journal of Public Health 45(3): e602–e603.

19.

Spector

(2019) Inquiry and critical thinking skills for the next generation: From artificial intelligence back to human intelligence. Smart Learning Environments 6: Article 8.

20.

Todd

Ravi

McCray

(2021) Cultivating critical thinking skills in online course environments. In: Information Resources Management Association (eds) Research Anthology on Developing Critical Thinking Skills in Students. Hershey: IGI Global, pp 837–858. DOI: 10.4018/978-1-7998-3022 -1.ch043.

21.

Uludag

(2023) Testing creativity of ChatGPT in psychology: Interview with ChatGPT. Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4390872

22.

Gan

Chen

, et al. (2023) AI-generated content (AIGC): A survey. Available at: https://arxiv.org/abs/2304.06632v1 (accessed 31 July 2023).

23.

Zawacki-Richter

Marín

Bond

, et al. (2019) Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education 16: Article 39.