Educational Implications of SEO Courses on Algorithm Awareness

Abstract

This study investigated how teaching search engine optimization (SEO) influences students’ algorithmic awareness, but also help them distinguish between organic and paid (SEA) results. Pre-course and post-course surveys were conducted with 150 students, and focus group discussions were conducted with five students from three disciplines: business, multimedia, and languages. The SEO courses enabled students to better distinguish between sponsored and organic search engine results and to understand the impact of algorithms. This has led to a more informed consumption of online information, with students becoming more skeptical of sponsored content. SEO skills have been shown to be relevant in each discipline, enhancing students’ algorithm awareness. This underlines the importance of teaching SEO to deal with the algorithmic information landscape in an informed way. This study makes an original contribution to the understanding of algorithm awareness, particularly in the context of SEO courses for students. While previous research has looked at the effects of algorithms on user behavior, few studies have looked at the influence of specific education on how students perceive and interact with search engines. This research fills a gap on the educational impact of SEO courses.

Keywords

algorithm awareness search engine optimization (SEO)information retrieval SEO courses search engine advertising (SEA)search engine algorithm

Introduction

Search engines are indispensable gateways to online information, and part of users’ daily lives. While easy to use, their complex algorithms for selecting information mean that they are regarded as “Black Boxes” (Lloyd, 2019). With a worldwide market share of 91.47% compared with other search engines in 2024 (Statista, 2024), Google remains the dominant actor. Such dominance is reinforced by users’ trust in Google’s algorithms to filter out the most pertinent information in response to their queries (Schultheiß & Lewandowski, 2023). This trust highlights why search engine optimization (SEO) and search engine advertising (SEA) are vital strategies. Seeking higher rankings in search results through SEO and SEA has become an important concern for any individual or organization striving to improve their online visibility (Sagot & Ben Arfa, 2023; Sagot et al., 2018). SEO involves adjusting the source code and environment of web pages to improve their ranking (Sagot et al., 2018), and SEA involves displaying targeted ads, usually above and to the side of the search engine results pages (Shankar et al., 2022). While SEO enhances a website’s organic visibility in search engine results, SEA involves buying ad placements to directly boost visibility. SEO helps to understand how the algorithms index and classify information.

The need to increase awareness among users of how search engines, algorithms and results pages work has already been highlighted by Schultheiß and Lewandowski (2023) “to create an understanding of the different interests of the actors on search results pages.” This is reinforced by Sundin and Haider’s concept of algorithm awareness, which is “the way individuals imagine, perceive, and experience algorithms through the concepts of algorithmic imaginaries and infrastructural meaning-making” (Haider & Sundin, 2021). By emphasizing the importance of developing an awareness of algorithmic issues and their implications, algorithm awareness enables individuals to understand which economic and strategic bias influence their information retrieval (Jylhä et al., 2024). Given these issues, universities and schools have an important role to play in developing algorithm awareness among students. Yet SEO education remains underdeveloped in academia (Atkins & Reilly, 2019; Moody & Bates, 2013). Moreover, many students, set to engage with digital strategies in their future careers, frequently underestimate or fail to grasp the profound impact of algorithms and SEO practices on the online information ecosystem (Lucas et al., 2022; Martzoukou et al., 2020). To date, no studies have examined the educational implications of integrating SEO courses into the curriculum and its potential to improve student algorithm awareness, as well as their ability to distinguish between SEO and SEA results. To answer this question, this study attempts to answer the three following research questions:

• RQ1: “How does teaching SEO affect students’ evaluation of search engines?”

• RQ2: “How does teaching SEO influence students’ information retrieval strategies?”

• RQ3: “What changes have been observed in students’ critical reasoning as a result of teaching SEO?”

This paper is structured as follows: The first section presents the literature review on algorithm awareness and SEO education. The second section explains the methodology, which consists of a mixed method with a survey and focus group discussions. The third section presents the results, and the fourth section discusses the educational implications of integrating SEO courses into academic curricula. The paper will conclude with a reflection on the findings, limitations, and future directions of research.

Literature Review

Algorithm Awareness

Essential to access information in a targeted and personalized way, algorithms filter content on search engines and social networks (Norström et al., 2020). Nevertheless, this raises a significant question: Does the widespread integration of search engines’ algorithms into everyday life translate into a deeper understanding of their mechanisms? Despite the increasing reliance on search engines, most users possess only a superficial or fragmentary comprehension of their algorithms, as demonstrated by Jylhä et al. (2024).

Algorithm awareness refers to users’ understanding of how algorithms work and the role they play in interacting with these systems (Shin et al., 2022). Algorithm awareness makes it possible to encompass the definition and operation of algorithmic systems, the collection and use of user data for their operation and their business and socio-technical context that shape them. With algorithm awareness, users can become aware of their role in algorithmic systems that work through their interactions and data to personalize and target information (Shin et al., 2022). Unlike digital literacy, which focuses on broader digital skills, algorithm awareness specifically focuses on the technical, operational, and critical skills needed to understand algorithms. Without these skills, users no longer have autonomy regarding the content offered to them by search algorithms and expose themselves to many risks. In this sense the study of Häußler et al. (2023) revealed that without understanding the mechanisms behind SEO tactics, users may unknowingly be exposed to deceptive practices aimed at manipulating search rankings for financial or ideological purposes. As a result, users may be misled by false information masquerading as credible sources, leading to uninformed decision-making and potential harm (Häußler et al., 2023).

Users’ perception of algorithms is often limited to basic knowledge, creating a major gap between what they believe they have mastered and the objective reality of their mastery (Dogruel et al., 2022). Although users are aware of algorithmic biases, very few take active measures to counter or mitigate them. This passive behavior, highlighted by Haider and Sundin (2021), could be due to resignation or excessive confidence in these technological tools. What is even more surprising is that despite the ongoing evolution of search engines, particularly those run by dominant players such as Google, users’ habits in searching and browsing search results remain remarkably stable (Schultheiß et al., 2018). Historically, users have tended to focus on the first results page to appear, giving inordinate importance to the algorithmic choices that determine these rankings (Roy et al., 2022). This almost unwavering fidelity to the first results reflects certain blind trust that users place in search engines. With the evolution of generative artificial intelligence, which refines and customizes results with a conversational interface within search engines, search results pages are reduced in importance. This in turn may further amplify the user’s passivity (Jylhä et al., 2024).

Bink (2023) warns against over-reliance on algorithms, especially when algorithms independently make decisions about what is or is not relevant to a user. It is the question of the user’s sovereignty over his or her search for information that is at stake. Ignorance of algorithmic functioning, sometimes chosen, sometimes suffered, impacts the quality of the information consumed, and by extension, the quality of our decisions and judgments (Schultheiß & Lewandowski, 2023). Meanwhile, as search engines mold our informational landscape, there are those who mold the engines themselves: content managers, online journalists and search engine practitioners (Schultheiß & Lewandowski, 2023). Unlike most of users, they understand how search algorithms work and can use them to their own benefit (Schultheiß & Lewandowski, 2020).

SEO Education

Search engine optimization (SEO) involves refining the structure and content of a website to improve its visibility and ranking in organic search engine results (Sagot et al., 2018). It is based on two pillars: technical and content improvement of webpages (Ziakis et al., 2019). A third pillar also exists that focuses on improving the website experience by improving its authority with internal and external links. This improves its loading speed and responsiveness to ensure that content is not only relevant and of high quality, but also easily accessible, durable and usable (Lacom & Sagot, 2022). The main aim of SEO is to help search engines index, categorize, and classify information more efficiently, making the search process more effective for users. In contrast, SEA involves the purchase of ad placements on the search engine results pages that appear above or next to organic results (Clarke et al., 2018). The difference between these techniques is that SEO focuses on aligning content with algorithmic criteria for sustainable visibility while SEA offers immediate but temporary visibility through paid ads.

In addition, unlike SEO, SEA could harm the user experience due to the intrusive nature of ads (Brajnik & Gabrielli, 2010). This explains why SEO has become a pillar of the company’s online communication strategy in recent years (Raikar et al., 2020; Sagot & Ben Arfa, 2023; Sagot et al., 2018; Spencer et al., 2023). Despite its preponderant role, SEO is often limited to its marketing dimension. The work of Giomelakis and Karyotakis on online journalism shows that its impact goes far beyond the field of marketing (Giomelakis et al., 2019; Giomelakis & Veglis, 2016; Karyotakis et al., 2019). Indeed, SEO has a major influence on the way information is structured, presented, and consumed by users. From this perspective, SEO is not only a strategy, but also a powerful lever for influencing communication strategies and structuring web content for optimal discoverability and access. Despite this, SEA is the only visible technique for users who know nothing about SEO, given that search engines must indicate sponsored content on their results page (Schultheiß & Lewandowski, 2023). This does not prevent Google from minimizing the visual difference between SEA and SEO, and over time this distinction has been made even more complex (Sharma et al., 2017).

This gap between the importance of SEO and its lack of notoriety highlights the complexity and opacity of the algorithms that search engines use. This raises the question of how to narrow the gap between SEO practitioners and search engine users. Following Lloyd’s teachings on information literacy, the knowledge needs to be taught in a holistic way, covering both practical skills and algorithmic culture (Lloyd, 2010; 2019). Appropriate teaching of SEO, integrated into the academic curriculum, could be a relevant way of promoting algorithm awareness. The idea is not so much to train future experts but to teach citizens to understand, if only partially, the workings of search engines, so that they can better decode the information presented to them.

Yet, teaching SEO also raises ethical issues for future practitioners. As Bogina and Gudivada point out, SEOs and other stakeholders in the algorithms, with their expertise, have a responsibility for the way in which information is brought to attention online (Bogina et al., 2022; Gudivada et al., 2015). With fake news and misinformation, SEO techniques can be misused to promote misleading content. In fact, teaching SEO must also instill an ethical approach, centered on responsibility and integrity. The technical aspect of SEO should not be neglected either: SEO encompasses multiple dimensions of skills. It requires knowledge of programming, marketing, content writing, user experience and accessibility (Atkins & Reilly, 2019).

Integrating SEO courses into the academic curriculum presents an opportunity to enhance users’ critical evaluation skills of online information, empowering them to discern between reliable and unreliable sources. SEO courses encompass a range of skills and knowledge fundamental to understanding how search engines work and optimizing content effectively. The structure of SEO courses is often designed to be experiential and competitive, with a focus on practical application and real-world strategies (Clarke & Clarke, 2014; Lui & Au, 2017, 2018, 2020). These key concepts are described in a similar study carried out by Mark Frydenberg and John S. Miko, highlighting both the technical adjustments required and the importance of practical experience in developing professional skills (Frydenberg & Miko, 2011). These include understanding search engine algorithms, keyword research and strategy, on-page and off-page criteria, SEO-optimized content, link building and performance monitoring (Sheffield, 2020). Thus, integrating SEO into educational curricula means offering cross-disciplinary teaching, at the crossroads of several disciplines.

Various studies show that introducing SEO into educational programs enables educational or informative websites to increase their visibility, and, therefore, their impact (Clarke et al., 2018; Kurniawan & Sanjaya, 2011). The effectiveness of SEO teaching has been discussed from a professionalization perspective (Frydenberg & Miko, 2011; Moody & Bates, 2013), and studied from the perspective of pedagogical innovation (Atkins & Reilly, 2019; Lui & Au, 2017, 2018, 2020). From the perspectives outlined by Atkins and Reilly (2019), Frydenberg and Miko (2011), and Moody and Bates (2013), the integration of SEO into the educational curriculum is considered a component of the broader concept of algorithm awareness. According to Gillespie, understanding the complex mechanisms of search engines that govern the creation and dissemination of online information is essential, given their role as arbiter of truth (Gillespie, 2014). SEO courses offer the opportunity to demystify these algorithms, by providing greater transparency on their mechanism and the ecosystem in which information is produced. SEO courses involves both on-page and off-page techniques based on a large literature base in computer science, marketing sciences, library and information sciences (Atkins & Reilly, 2019; Kumari et al., 2022; Kurniawan & Sanjaya, 2011; Lui & Au, 2017, 2020; Scott, 2015). On-page optimization refers to how content is structured and coded on a webpage to be SEO-friendly (Zhou et al., 2018). Key aspects include page title, meta description, heading tags, internal links, and images, among others. Off-page optimization focuses on activities outside the website such as link building, social media optimization, local citations, and reviews (Beier & Wagner, 2015).

SEO courses also cover “White Hat” techniques that follow search engine guidelines and help structure websites and “Black Hat” techniques that aim to manipulate search results but can lead to penalties (Malaga, 2010; Scott, 2015). Overall, learning SEO provides a better understanding of website architecture, search results pages and how search engines work. According to Deuff and Roumanos (2022), mastery of algorithm literacy constitutes a real counterweight to what is called “digital supremacy,” the hegemony of a few digital giants over the management and dissemination of online information. The importance of algorithmic literacy is now recognized for professional and academic success (Cotter & Reisdorf, 2020; Sparks et al., 2016). SEO could be an opportunity to build essential digital literacy skills, particularly in terms of algorithm awareness, through the concrete and pragmatic aspects of the sector. SEO can serve as a gateway to a deeper understanding of search engines and algorithms (Martzoukou et al., 2020). By analyzing the mechanisms that make information visible, SEO courses become a valuable tool for teaching students how search engines arbitrate content selection and the various optimizations and manipulations at play in the race for visibility.

To date, the impact of SEO education on algorithm awareness, in particular how it improves understanding of organic and paid search strategies, has not been explored. This study highlights the significant potential of SEO training, equipping students with the necessary skills to effectively evaluate search engines, optimize information retrieval, and critically differentiate between SEO and SEA.

Method

This study addresses three research questions examining how SEO education enhances students’ algorithm awareness. The first question (RQ1) investigates “How does teaching SEO affect students’ evaluation of search engines?”, addressing the potential evolution in students’ understanding of search engine mechanisms. The second question (RQ2) explores “How does teaching SEO influence students’ information retrieval strategies?”, considering that SEO knowledge might modify how students approach online searches. The third question (RQ3) analyzes “What changes have been observed in students’ critical reasoning as a result of teaching SEO?”, investigating the development of analytical skills regarding algorithmic systems.

To address these questions, this study employed a mixed-methods approach, integrating quantitative analysis through pre- and post-course surveys alongside qualitative exploration via focus group discussions. The analysis involved 150 students from three different institutions and academic disciplines who completed identical SEO courses, while the focus groups consisted of in-depth discussions with five students from the same institutions who participated in the surveys. SEO courses were developed using a rigorous methodology to comprehensively cover both theoretical principles and practical applications of SEO (Atkins & Reilly, 2019; Kumari et al., 2022; Kurniawan & Sanjaya, 2011; Lui & Au, 2017, 2020; Scott, 2015). Theoretical lessons inspired by Sheffield’s article (Sheffield, 2020), addressed topics such as search engine algorithms, keyword research strategies, on-page and off-page criteria, as well as the different practices used by professionals, whether “Black Hat” or “White Hat.” Practical exercises were facilitated through SEO-ELP platform (SEO-ELP, 2025) an immersive and interactive SEO e-learning platform allowing consistent content delivery and hands-on practice in a controlled environment (Domenget et al., 2022).

Survey

The survey took place during the 2023 academic year and was administered to three institutions in France covering three academic disciplines: business, multimedia, and applied foreign languages. This SEO course was mandatory in the students’ training program, for all institutions. The sample is composed of students’ groups with varying levels ranging from undergraduate to graduate who had never taken SEO courses:

(1) Business’ students, at École Supérieure des Technologies et des Affaires (ESTA) in Belfort pursue a sales and engineering program to become business engineers. Their initial training focuses on sales, business, science, computing, and technical fields. SEO is taught to raise the visibility of companies and carry out website audits. These include a total of 100 students.

(2) Multimedia’ students at University Institutes of Technology Nord Franche-Comté are trained to become professionals in the fields of communication, web development, multimedia content production, and project management. SEO is taught to help students understand how information is structured online and how to optimize content to improve visibility and reach in various digital platforms and contexts. These include a total of 43 students.

(3) Applied Foreign Languages’ students at the Training and Research Unit in Industry Sciences, Techniques and Management of University of Franche-Comté, pursue a program structured around three main pillars: foreign languages, web development, and communication. SEO is taught with the aim of understanding how information is structured on the web, making it easier to write content for international companies and improve their online visibility. These include a total of 7 students.

This sample reflects the interdisciplinary nature of SEO (Frydenberg & Miko, 2011). The study involved local lecturers to assess the impact of SEO courses on students’ algorithm awareness across three academic disciplines. Prior to starting the courses, students completed an online survey using the “Le Sphinx” survey tool (Sphinx Declic, 2007), accessed via an email and QR code provided by their lecturers. Responses were collected during the lessons, and a similar survey followed the course’s conclusion. At the end, students could opt-in to participate in focus groups by providing their email addresses. The pre-course survey, which collected demographic information with explicit consent, included safeguards to anonymize personal data after assigning unique identifiers.

The survey was pilot-tested with student representatives from each discipline to ensure clarity and effectiveness, leading to enhancements such as interactive Likert scales with thumbnail illustrations for better engagement (Leutner et al., 2017; Reynolds-Keefer & Johnson, 2011). Feedback refined the variables related to selecting and understanding search engines, as well as the elements encountered during queries. The survey was structured to explore the effects of SEO courses on three categories: (1) evaluation of search engines, (2) information retrieval strategies, and (3) critical reasoning toward SEO. Variables were selected based on course content and relevance to SEO. The pre-course survey comprised 13 questions around these categories, while the post-course survey excluded the demographics part.

The structure of the pre-course and post-course survey is detailed in Appendix A. In the demographics category, the survey gathered basic information about the students’ backgrounds, including gender, age, last obtained degree, current field of study, and previous major, which helped to characterize the academic diversity of the participants. For the evaluation of Search Engines, students evaluated the importance they place on various search engine criteria such as the visibility of sources, search functions, algorithm transparency, usability, design, privacy, security, efficiency, and environmental sustainability. They rated these aspects on a five-point Likert scale ranging from “Not Important” to “Very Important.” A five-point Likert scale was chosen for its balance of simplicity and depth (Joshi et al., 2015). This evaluation aimed to understand what factors students consider critical when choosing and using a search engine. The information retrieval strategies focused on how students evaluate specific elements during an information search, such as the page title, website address, description, images, authorship, and HTTPS/HTTP protocols. Similar to the previous section, a five-point Likert scale was used to gauge the importance students place on these elements, which are crucial for effective SEO practice. The critical reasoning toward SEO category addressed students’ abilities to differentiate between “advertising” and “organic” content on search engines like Google. It also assessed their views on the influence of SEO actions on internet users’ search behaviors, the search results they find, the quality of websites, and their opinions on whether SEO changes the nature of online search results. Students’ responses were measured on a five-point Likert scale from “Strongly Disagree” to “Strongly Agree,” providing insights into their critical understanding of SEO and its broader implications. Responses were collected during the lessons, and a similar survey was administered at the course’s conclusion to assess changes in understanding and attitudes.

Variables

The first category of variables included in the survey visibility of sources, search functions, search algorithms, usability, and design, reflecting evaluation criteria from Jansen and Pooch (2001), Dudek et al. (2007) and Shafiq and Rasool (2014). These assess perceptions of information source clarity, search functionality, algorithm transparency, and interface usability—key areas SEO courses aim to improve. Additional variables like privacy, security, efficiency, and environmental sustainability offer broader insights into students’ judgments, drawing on Palanisamy (2013) and Duka et al. (2023). The second category evaluated on-page factors such as page titles, addresses, descriptions, images, and authorship—important for SEO ranking—as well as off-page factors like page ratings and HTTPS/HTTP protocols, which influence trust and safety (Lewandowski, 2008; Palanisamy, 2013; Sharma et al., 2017; Wogalter & Mayhorn, 2008). As SEO courses instruct students on best practices for optimization as well as the importance of user experience and security, these variables provide insight into whether their consideration of such online reputation and protection factors has changed in the process of information gathering (Karyotakis et al., 2019; Palanisamy, 2013; Wogalter & Mayhorn, 2008). The survey’s final section assessed students’ critical reasoning toward SEO through six variables. Q1 examines the ability to identify commercially influenced results (Schultheiß & Lewandowski, 2023). Q2 and Q3 focus on understanding SEO’s influence, both on user behavior and search results (Haider & Sundin, 2021). Q4 addresses how SEO impacts website quality perception (Lloyd, 2010). Q5 explores students’ comprehension of SEO’s fundamental role in search engines through a hypothetical scenario (Gillespie, 2014). Q6 measures students’ overall critical perspective on SEO practices, reflecting recent work on algorithmic fairness (Bogina et al., 2022). All variables are detailed in Table 1.

Table 1

Variables

Category	Variable	Description
Evaluation of search engines	Visibility of sources	Measures how easily information sources are identifiable relating to computer credibility (Alloing, 2016)
	Search functions	Evaluates the diversity of search functionalities (Jansen & Pooch, 2001)
	Search algorithm	Analyzes the transparency and relevance of the search algorithm (Jansen & Pooch, 2001)
	Usability	Evaluates the ease users have in using the search engine (Dudek et al., 2007)
	Design	Evaluates the user-friendliness of the interface (Palanisamy, 2013; Shafiq & Rasool, 2014)
	Privacy	Measures the protection of users’ personal data (Palanisamy, 2013)
	Security	Evaluates protection against online threats (Palanisamy, 2013)
	Efficiency	Measures the speed and accuracy of search results (Dudek et al., 2007)
	Environmental sustainability	Evaluates the ecological impact (Duka et al., 2023)
Information retrieval strategies	Page title	The title of the webpage displayed in search results (Sharma et al., 2019)
	Website address (www.)	The URL of the website (Palanisamy, 2013; Wogalter & Mayhorn, 2008)
	Description of the webpage	A summary or brief description of the webpage (Lewandowski, 2008; Sharma et al., 2019)
	Image of the webpage	A representative image of the webpage (Wogalter & Mayhorn, 2008)
	Author of the webpage	The author or source of the webpage (Wogalter & Mayhorn, 2008)
	People’s rating of the webpage	User reviews or ratings of the webpage (Sharma et al., 2019)
	HTTPS/HTTP protocol	The security protocol used for connecting to the website (Callegati et al., 2009)
Critical reasoning toward SEO	Q1. Is it possible to differentiate between “advertising” and “organic” content on search engines?	Assesses users’ ability to distinguish between paid (advertising) and unpaid (organic) search results (Schultheiß & Lewandowski, 2023)
	Q2. Do you think that SEO actions have an influence on Internet users’ searches for information?	Measures perception of the influence of SEO actions on users’ search behavior (Haider & Sundin, 2021)
	Q3. Do you think SEO influences the search results you find online?	Evaluates perception of the impact of SEO on individual search results (Haider & Sundin, 2021)
	Q4. Do you think SEO can influence your decision about the quality of a website you visit?	Assesses perception of the influence of SEO on the perception of the quality of visited websites (Lloyd, 2010)
	Q5. Do you think online search results would be different if SEO didn’t exist?	Evaluates perception of the hypothetical impact of the absence of SEO on online search results (Gillespie, 2014)
	Q6. Do you have a favorable opinion of SEO actions?	Assesses participants’ overall attitude toward SEO actions, whether they view them positively, negatively, or neutrally (Bogina et al., 2022)

Focus Group

Following the survey, focus groups were organized after the SEO courses so that the students could interact about what they had just learned and compare their experiences (Fusch et al., 2022; Onwuegbuzie et al., 2009). After a preliminary text, participants were recruited by email following their expressed willingness to take part in the focus group (Boydell et al., 2014). The sessions were conducted via Microsoft Teams video conferencing and recorded with participants’ consent. The focus group session was structured around the same three main categories: (1) evaluation of search engines, (2) information retrieval strategies, and (3) critical reasoning toward SEO. The details and description of the three categories discussed in the focus group are given in Appendix B. Following the thematic discussions, a second segment of the session allowed students to react to the results of a questionnaire allowing them to articulate their perceptions and insights on the course content influencing their practical and theoretical understanding. Full transcripts of the discussions were generated, ensuring that every word spoken by the participants was accurately transcribed. Data analysis was carried out manually enabling complete immersion in the empirical material (Maher et al., 2018). The authors carefully read each verbatim, identifying and highlighting the salient elements in relation to the research objectives. This analysis stage involved a thematic classification of the data, where responses were coded according to categories, revealing significant trends and perceptions shared by the participants.

Results

Statistical analyses were carried out using XLSTAT statistical tool (XLSTAT, 2025). Paired-sample statistical tests were performed to compare pre- and post-course survey results on the same participants. Firstly, the Shapiro-Wilk normality test was used to assess whether the data collected through the pre- and post-course questionnaires followed a normal distribution (Öztuna et al., 2006). This statistical test verified the normality hypothesis, by comparing the deviations between the observed distribution of the data and a theoretical normal distribution (Razali & Yap, 2011). If a significant deviation was identified (p < .05), this meant that the data did not follow a normal distribution, and non-parametric analyses were then carried out using the Wilcoxon test (Wang et al., 2022). Conversely, if no significant differences were detected (p ≥ .05), the data could be considered normally distributed, and Student’s t-test was then used to check whether the training resulted in a significant increase in performance for the same participants tested before and after (Najmi et al., 2021). The results presented below concern only the analyses performed with the Wilcoxon test, consistent with the non-normal distribution of our data.

Demographics

Table 2 summarizes the demographics characteristics of the participants according to “gender,” “age,” “education,” “current field of study,” and “academic background.” Most of the participants were male (72%) in Business study (66.67%) with a Sciences background (70%). There were no significant differences with sample demographics subgroups.

Table 2.

Summary of Participant Demographic Information

Demographics (n = 150)	Variables	Respondents	Percentage
Gender	Male	108	72
Gender	Female	42	28
Age	18–19	50	33.33
	20–21	78	52
	22–23	16	10.67
	Over 24	6	4
Education	Master’s degree	98	65.33
Education	Bachelor’s degree	52	34.67
Current field of study	Business	100	66.66
	Applied Foreign Languages	7	4.67
	Multimedia	43	28.67
Academic background	Arts	4	2.67
	Business	2	1.33
	Sciences	105	70
	Law and Political Sciences	5	3.33
	Humanities and Social Sciences	15	10
	Literature and Languages	11	7.33
	Psychology	1	0.67
	Other	7	4.67

Survey Results

The results are described according to the three categories outlined in the methodology section: evaluation of search engines, information retrieval strategies, and critical reasoning toward SEO.

Evaluation of Search Engines

Table 3 presents the results of students’ evaluation of search engine attributes, before and after attending SEO courses. Results compare the means obtained for each variable on the converted 5-point Likert scale (ranging from 0 “Not Important” to 4 “Very Important”), as well as the p-values from Wilcoxon non-parametric test, indicating the statistical significance of the observed changes at significance level p < .05 (*) and significance level p < .01 (**). Among the nine variables examined (“Efficiency,” “Visibility of source,” “Search functions,” “Search algorithm,” “Usability,” “Design,” “Privacy,” “Security,” and “Environmental sustainability”), only “Efficiency” showed a statistically significant change, with the mean score decreasing from 2.973 before the course to 2.807 after the course (p = .0001). For all other eight variables, no statistically significant differences were observed between pre-course and post-course evaluations (ps > 0.05), indicating that there is no proof of change after SEO courses. Thus, students initially chose scores that remained stable after the course: “Security” and “Usability” were rated as “Important”; “Visibility of source,” “Search functions,” “Search algorithm,” “Design,” and “Privacy” were rated as “Moderately Important”; and “Environmental sustainability” were rated as “Not Very Important.”

Table 3.

Evaluation of Search Engines

Variable	Before Course		After Course		p-Value
Variable	Mean	Std. Dev	Mean	Std. Dev	p-Value
Visibility of sources	1.947	1.015	1.940	0.907	0.924
Search functions	2.373	0.807	2.240	0.808	0.153
Search algorithm	2.007	1.000	2.000	0.934	0.955
Usability	2.590	0.698	2.530	0.640	0.291
Design	1.880	0.996	1.907	0.814	0.907
Privacy	2.407	0.875	2.467	0.739	0.519
Security	2.560	0.755	2.647	0.557	0.392
Efficiency	2.973	0.258	2.807	0.413	0.0001**
Environmental sustainability	1.493	0.961	1.487	0.910	0.702

Note. Wilcoxon Signed Rank Test (** Correlation is significant at the 0.01 level. * Correlation is significant at the 0.05 level); n = 150.

Information Retrieval Strategies

The second part of the analysis delved into the importance attributed by students to various key elements of SEO when conducting information searches on Google. Table 4 presents the results regarding students’ evaluation of technical SEO elements during information searches using a five-point Likert scale (ranging from 0 “Not Important” to 4 “Very Important”). Two variables showed statistically significant changes after the course: the “HTTPS/HTTP Protocol” demonstrated an increase, rising from 1.460 to 1.940 (p = .001), shifting from “Not Very Important” to “Moderately Important,” and “People’s Rating of the Webpage” showed an increase, rising from 1.640 to 1.880 (p = .040). For the other five variables (“Page Title,” “Website Address,” “Description of the Webpage,” “Image of the Webpage,” and “Author of the Webpage”), no statistically significant differences were observed between pre-course and post-course evaluations (ps > 0.05), indicating no proof of change for these variables after SEO courses. Thus, students initially chose scores that remained stable after the course: “Page Title” was rated as “Important”; “Website Address,” “Description of the Webpage,” and “Image of the Webpage” were considered “Moderately Important”; and “Author of the Webpage” was rated as “Not Very Important.”

Table 4.

Information Retrieval Strategies

Variable	Before Course		After Course		p-Value
Variable	Mean	Std. Dev	Mean	Std. Dev	p-Value
Page title	2.587	0.667	2.587	0.604	0.977
Website address (www.)	1.847	1.041	1.833	0.930	0.876
Description of the webpage	2.100	0.888	2.027	0.882	0.461
Image of the webpage	1.527	1.021	1.593	0.935	0.621
Author of the webpage	0.933	1.028	1.013	1.030	0.379
People’s rating of the webpage	1.640	1.025	1.880	0.941	0.040*
HTTPS/HTTP protocol	1.460	1.240	1.940	1.101	0.001**

Note. Wilcoxon Signed Rank Test (** Correlation is significant at the 0.01 level. * Correlation is significant at the 0.05 level); n = 150.

Critical Reasoning Toward SEO

Table 5 presents the results of the evolution of students’ critical reasoning toward online information, specifically influenced by SEO. The questions were designed to assess students’ understanding of SEO and its potential to influence Google search results. For question 1, “Is it possible to differentiate between ‘advertising’ and ‘organic’ content on Google?”, there was a notable increase in the mean of responses, from 3.020 to 3.267, with a decrease in the standard deviation from 0.790 to 0.783 after the course (p = .10*). This convergence of student responses was statistically significant. Regarding question 2, “Do you think that SEO actions have an influence on Internet users’ searches for information?”, although there was a slight increase in the average responses after the course, this change was not statistically significant (p = .806). Significant changes were observed for question 3, “Do you think SEO influences the search results you find online?”, with an increase in the average response from 3.173 to 3.433 (p = .002). Question 4, “Do you think SEO can influence your decision about the quality of a website you visit?”, showed an increase in the average response from 2.860 to 3.093 (p = .012*), indicating a rising awareness of the potential influence of SEO on website quality perception. For question 5, “Do you think online search results would be different if SEO didn’t exist?”, there was a notable increase in the average response from 3.107 to 3.407, (p = .001**), suggesting a better understanding of the impact of SEO on search results. Finally, question 6, “Do you have a favorable opinion of SEO actions?”, showed an increase in the average response from 2.573 to 2.820, (p = .006**), indicating an improvement in students’ opinion of web SEO after the course.

Table 5.

Critical Reasoning Toward SEO

Question	Before Course		After Course		p-Value
Question	Mean	Std. Dev	Mean	Std. Dev	p-Value
1	3.020	0.790	3.267	0.783	0.010*
2	3.233	0.660	3.260	0.718	0.806
3	3.173	0.730	3.433	0.763	0.002**
4	2.860	0.859	3.093	0.789	0.012*
5	3.107	0.778	3.407	0.828	0.001**
6	2.573	0.780	2.820	0.828	0.006**

Note. Wilcoxon Signed Rank Test (** Correlation is significant at the 0.01 level. * Correlation is significant at the 0.05 level); n = 150.

Results From the Focus Group

Evaluation of Search Engines

Details about the demographics of focus group participants are given in Appendix C. When asked about search engine preferences after the course, most students identified Google as their primary search engine, with only one participant (P03) indicating a preference for Bing. Their preferences remained unchanged from before the course. In their responses, some participants (P02 and P03) also mentioned Safari alongside Google, demonstrating a persistent confusion between tools used to access to the Internet (web browsers) and tools used to search for information (search engines). This confusion between the two types of technologies suggests that while students use these tools daily, they may not fully distinguish their distinct roles in web navigation and information retrieval.

After this exchange on search tools, one student also expressed a new awareness of the mechanisms behind search rankings, marking a shift from passive to informed user behavior:

“I don’t think I was sufficiently aware of the extent to which the simple act of clicking on a website could be just as useful to another organization. And I think that discovering the mechanism behind the search engine helps you to use it better.” (P05)

These words exemplify a shift from seeing search engines as external tools to understanding oneself as an active participant in their algorithmic systems. The realization that “the simple act of clicking” might contribute to a wider algorithmic dynamic is a step away from being a passive consumer to becoming a conscious player in the search engine ecosystem.

Information Retrieval Strategies

SEO courses have notably impacted the students’ approach to information retrieval, shifting from broad, ambiguous queries to more strategic and keyword-focused searches. One student mentioned:

“I think I put more keywords, target words than big sentences… just the basic keywords that are useful.” (P01)

This transition highlights a refined search strategy, influenced by their understanding of how search algorithms operate. It demonstrates a departure from the casual approach to a more analytical and calculated method, influenced by SEO insights.

Subsequent discussions post-survey illustrated subtle shifts in students’ perspectives. They noted more strategic evaluations of search engines, particularly valuing secure transfer (HTTPS). One student highlighted an increased focus on secure browsing:

“Personally, I pay even more attention now to making sure I go on HTTPS and not HTTP. Because it’s much better? Secured?”. (P03)

Similarly, their growing consideration of webpage ratings was reflected in comments like:

“I prefer to refer to video tests or tests by professionals… I pay very close attention to the ratings.” (P04)

This was mentioned by a student, revealing a critical engagement with content beyond mere SEO tactics, an advancement from their pre-course queries and keyword-focused strategies.

Critical Reasoning Toward SEO

The transformation in students’ critical reasoning toward online information and search engine algorithms was profound toward SEO. Initially, most participants lacked a comprehensive understanding of SEO principles. As the sessions progressed, a noticeable change took place: students started to critically evaluate the reliability and credibility of online information, underscored by student’s reflection on the quality of websites:

“You don’t feel like going to the website that doesn’t look appealing… You’d rather go to another site that looks a bit more welcoming.” (P01)

The discussion among students during the focus groups revealed a nuanced understanding of SEO and SEA, particularly in distinguishing between organic and paid search results. For instance, one student pointed out the course’s impact on the understanding of why certain links appear first in search results:

“SEO has helped me personally in my search habits. In fact, when I used to see sponsored results and so on, I’d click on the first link, thinking it was the most useful.” (P02)

Similarly, another student humorously discussed the economic impact of clicking on sponsored links, showing a deeper comprehension of PPC pay-per-click (PPC) advertising dynamics:

“Basically, every time you see ‘sponsored’ or ‘ad’ on a result, it’s a paid listing. Brands pay for every click you make on it. So, if you want to make them pay, you make a lot of clicks. You need to hire people like that in companies to bring down the competition.” (P04)

While another student noted skepticism toward the authenticity of online reviews:

“In fact, reviews can be biased; anyone can write anything without it being verified, so it skews a lot.” (P05)

This skepticism represents a mature analytical approach toward online content, contrasting with earlier sessions where the focus was more on the structural aspects of SEO.

Furthermore, discussions around the ethical dimensions of SEO practices, like “Black Hat” and “White Hat” strategies, revealed an enhanced critical engagement with the material. The notion of being able to discern and evaluate the ethical implications of various SEO strategies illustrates a significant shift from mere content consumers to ethical and informed digital citizens. This enhanced critical attitude extends beyond academic boundaries to personal and professional realms, as evidenced by students’ willingness to discuss and apply SEO knowledge in their family and workplace contexts. For example, one participant shared SEO concepts with their mother, revealing a trickle-down effect of knowledge and a shift in discourse from not only understanding but also educating others outside the academic sphere. The shift in student attitudes toward SEO was markedly positive, as illustrated by the reflective comments post-survey:

“At first, it can always be intimidating when it’s a notion we don’t know… but then it was approached in a playful way, and we all realized the real added value it brought quite quickly.” (P03)

This sentiment reflects a broader acceptance and enthusiasm for SEO, contrasting with initial uncertainties and doubts. Moreover, the discussion around Google’s prospective changes and the integration of AI-like features in search engines catalyzed further critical reflection. While students expressed mixed feelings about the potential impacts on search behavior and advertising, they consistently showcased an informed and nuanced understanding. One student speculated that:

“It probably will be a danger to advertisers because if they use Google Gemini… they’ll get straight to the point without really proposing an ad.” (P04)

Overall, these verbatims underscore a significant transformation in students’ perspectives, from foundational SEO knowledge to a more critical, nuanced understanding of digital information retrieval and the ethical considerations surrounding it.

Discussion

Evaluation of Search Engines

This study used a mixed-methods approach, integrating pre- and post-course surveys alongside focus group discussions. The SEO course’s impact on students’ evaluation of search engines was mixed. The results demonstrated that among the nine variables examined, “Efficiency” score showed a statistically significant decrease after completing the SEO course. This finding adds nuance to the observations of Dudek et al. (2007) about the relationship between speed and search quality. While the authors found that users prioritize speed, they also noted that “query results submitted via search engine may be found in a matter of seconds, [but] users cannot always be guaranteed the best results.” Our study suggests that SEO education reinforces this understanding, leading students to develop a more complex appreciation of the trade-offs between search efficiency and quality.

The survey analysis also revealed that eight out of nine variables (“Visibility of sources,” “Search functions,” “Search algorithm,” “Usability,” “Design,” “Privacy,” “Security,” and “Environmental sustainability”) showed no significant change, but exhibited three distinct importance levels in their score. “Security” and “Usability” all stayed “Important” to the participants before and after the SEO course. Palanisamy (2013) identified security as significant evaluation criteria for search engines, though his study did not quantify its relative importance. Dudek et al. (2007) found that usability was a key factor in users’ search engine preferences, as it directly impacts their ability to accomplish tasks quickly and efficiently. Although we found no significant changes in these ratings after the SEO course, our findings extend the conceptual framework developed by Palanisamy and Dudek et al. by providing quantitative measurements of students’ perception of these criteria. This provides concrete metrics for future research on search engine evaluation and development, particularly in educational contexts.

Furthermore, while no statistically significant changes were observed after the course for eight of the nine variables, the survey analysis revealed distinct importance levels for algorithmic and interface elements. “Visibility of sources,” “Search functions,” “Search algorithm,” “Design,” and “Privacy” stayed as “Moderately Important” to the participants. These moderate scores for algorithmic elements could be explained the findings of Dogruel et al. (2020) that users’ awareness of algorithmic systems varies based on their perceived control; when users feel in control of their interactions, they are less conscious of algorithmic influence. This interpretation is further supported by the observations of Dogruel et al. that users demonstrate limited awareness of algorithmic decision-making, particularly in situations where they feel they are making autonomous choices. These importance scores reflect what Lloyd (2019) describes as the paradoxical relationship between users and algorithmic systems: while users regularly delegate authority to algorithms in their search practices, they maintain a simplified and somewhat detached understanding of these systems’ influence. As Lloyd notes, this can lead to a “narrowing of information landscapes” where users simultaneously rely on algorithmic systems while failing to fully appreciate or critically examine their impact on information access and evaluation. Our findings empirically support this analysis, showing how participants recognize algorithmic elements as relevant to their search experience but don’t prioritize understanding their underlying mechanisms and influences. Finally, “Environmental sustainability” remained at “Not Very Important” for participants, with no significant changes observed after the course. Arguing about Web sustainability, Duka et al (2023) assert that environmental considerations, including energy efficiency and green hosting solutions, have become critical challenges for Web environments. This gap between current sustainability priorities and our students’ evaluation criteria suggests the need to better integrate environmental awareness into SEO education.

Addressing our first research question (RQ1: “How does teaching SEO affect students’ evaluation of search engines?”), the survey results showed limited changes, with only “Efficiency” showing a statistically significant decrease. However, our focus group discussions provide evidence of a more nuanced transformation in how students understand search engines. This is particularly evident in one student’s reflection:

The persistence of established habits was particularly evident in students’ search engine preferences. The focus groups revealed that most participants continued to use Google as their primary search engine after the course, with only one student using Bing. Moreover, several students confused web browsers with search engines in their responses, suggesting that while technical knowledge improved, fundamental distinctions between different web technologies remained unclear. This observation further supports the general idea emerging from the analysis that rather than changing how students evaluate search engines technically, SEO education fostered what Haider and Sundin (2021) describe as “algo-rhythm awareness”: a critical understanding of how users are entangled with algorithmic systems through temporal tactics and anticipatory practices. As our findings show, students developed a more nuanced appreciation of their role within these systems, even if their technical evaluations remained relatively stable.

Information Retrieval Strategies

The SEO course’s impact on information retrieval was also mixed. First, the results of the survey demonstrated a substantial increase in students’ attention to security protocols, with “HTTPS/HTTP protocol” rising significantly from “Not Very Important” to “Moderately Important.” This finding is supported by our focus group discussions where students expressed heightened security awareness:

“Personally, I pay even more attention now to making sure I go on HTTPS and not HTTP. Because it’s much better? Secured?”. (P03)

These results align with the concept of “algo-rhythm awareness” discussed by Haider and Sundin (2021), though in our case focusing on security rather than temporal aspects. Like their participants who developed tactics to navigate algorithmic systems, our students showed increased attention to the infrastructural elements that shape their online interactions. The ambivalent relationship with digital privacy protection found in our participants’ evolved evaluation strategies echoes the findings of Katavić (2023) about Generation Z students, who demonstrate cautious attitudes toward data protection while showing gaps in their privacy protection knowledge and strategies. As in their study, our results suggest that students are increasingly aware of privacy concerns but would benefit from additional education in this area. Second, our analysis shows a significant increase in students’ attention to “People’s rating of the webpage.” The focus group provides deeper insight into this evolution:

“I prefer to refer to video tests or tests by professionals… I pay very close attention to the ratings.” (P04)

This growing attention to evaluation mechanisms aligns with findings of Shin et al. (2022) that users develop increased algorithmic awareness through critical assessment practices, particularly when they understand how their evaluations contribute to and are shaped by algorithmic systems.

For five out of seven variables (“Page Title,” “Website Address,” “Description of the Webpage,” “Image of the Webpage,” and “Author of the Webpage”), the results revealed three distinct importance levels, but no significant changes were observed. Among the highest-rated elements, “Page title” maintained its score of “Important.” These consistently high ratings for page titles align with the findings of Lewandowski (2008) that demonstrated the critical role of page titles in users’ evaluation process, where title precision significantly impacts selection behavior. This is further supported by later observation of Schultheiß et al. (2018) that users continue to carefully assess titles even as their trust in algorithmic rankings decreases. This suggests that while users’ relationship with search engines evolve, titles remain fundamental to their evaluation strategies.

At a moderate level of importance “Website address,” “Description of the webpage,” and “Image of the webpage” all scored as “Moderately Important.” This intermediate positioning aligns with findings of Wogalter and Mayhorn (2008) that while visual elements, descriptions and URLs contribute to users’ credibility assessments, they are secondary to more fundamental trust indicators like domain suffixes (.edu, .gov) and organizational reputation. Their research showed that users tend to rely on multiple cues when evaluating website trustworthiness, with descriptions, visual and URL elements serving as complementary rather than primary trust signals. Finally, at the lower end of the scale “Author of the webpage” remained as “Not Very Important.” This relative lack of attention to authorship echoes with observations of Palanisamy (2013) about users’ limited consideration of source attribution in search contexts, as web users believe that the quality and relevance of the information provided by the search engine is sufficient.

In response to our second research question (RQ2: “How does teaching SEO influence students’ information retrieval strategies?”), our findings revealed that the course had a selective impact on students’ evaluation behaviors. While significant improvements were observed in security awareness (HTTPS/HTTP protocol) and result evaluation (People’s ratings), most established search behaviors remained stable before and after the SEO courses. This suggests that SEO education can successfully enhance specific aspects of information retrieval strategies while having limited impact on other deeply ingrained search habits. Taken altogether, the survey and focus group results presented here demonstrate that students develop new evaluation practices while maintaining their pre-existing search habits. An interesting side finding was that students adopted more strategic approaches to searching, as evidenced in our focus groups:

“I think I put more keywords, target words than big sentences… just the basic keywords that are useful.” (P01)

These results yield additional evidence that the course has a greater influence on theoretical understanding than on practical information-seeking behavior.

Critical Reasoning Toward SEO

The SEO course had a profound impact on students’ critical reasoning abilities. The survey results provide preliminary evidence of significant improvements in five out of six variables further substantiated by rich focus group discussions. During these discussions, participants demonstrated their evolved critical reasoning through detailed exchanges about SEO mechanisms, ethical implications, and practical applications. First, our analysis demonstrates a significant increase in students’ ability to differentiate between advertising and organic content (Q1) showing significant improvement. The focus group results reinforced this evolution:

“SEO is free and I remember that you had visibility that was longer, more sustainable. And SEA is paid, and so you might have less sustainable visibility but you have immediate visibility because you’re directly first.” (P03)

This finding adds complexity to the research of Schultheiß and Lewandowski (2023) which found that while most users struggle to identify commercially influenced results, those with greater algorithmic knowledge tend to be more critical and show better discrimination abilities. The survey results reveal significant improvements in three interrelated aspects of SEO understanding, all showing significant statistical increases. First, students’ perception of SEO’s influence on search results (Q3) reflects what Haider and Sundin (2021) describe as “infrastructural meaning-making,” where users develop a deeper understanding of how algorithmic infrastructures shape information access. Our findings suggest that SEO education helps students recognize the role of SEO in determining search visibility and ranking. As one student’s observation:

“Basically, every time you see ‘sponsored’ or ‘ad’ on a result, it’s a paid listing. They pay for every click you make on it.” (P03)

This illustrates a critical understanding of how users become entangled with algorithmic systems through temporal tactics and anticipatory practices. Second, their evaluation of SEO’s impact on website quality (Q4) reflects what Lloyd (2010) describes as the development of “practical understanding” in information practices. Our survey results show that students developed an enhanced ability to evaluate how SEO practices affect website quality. This was evident in the focus groups where students demonstrated understanding that quality content is essential for effective SEO, as one participant noted:

“We understand that the algorithm detects scams […] you have to create real content.” (P04)

This shows how students began to appreciate that SEO effectiveness is intrinsically linked to content quality. Third, their recognition of SEO’s fundamental role in shaping search results (Q5) aligns with what Gillespie (2014) describes as the public role of algorithms in determining visibility. This increased understanding was clearly articulated in the focus groups:

“Nowadays without a website doing your business seems very complicated to me. And without having knowledge in SEO, well it’s a bit difficult to create a website that will be seen.” (P02)

This quote demonstrates students’ growing awareness of how SEO fundamentally shapes online visibility and access to information, moving beyond technical understanding to recognize its essential role in the digital economy. Students’ overall opinion of SEO practices (Q6) also showed significant improvement. Their enhanced critical awareness emerged in discussions of both ethical implications and content evaluation. One student noted manipulative practices:

“There are techniques that can get you evicted by Google if I put lots of keywords on a page with the same color as the background.” (P02)

While another demonstrated skepticism toward user-generated content:

“In fact, reviews can be biased; anyone can write anything without it being verified, so it skews a lot.” (P05)

This evolution reflects what Häußler et al. (2023) describe as the development of “risk perception”: where users become more aware of potential adverse consequences in web search, and aligns with the concept of “algorithmic awareness” provided by Shin et al. (2022), particularly their finding that increased technical knowledge leads to more sophisticated evaluation of algorithmic systems through both “heuristic” and “systematic” processing. Only SEO’s influence on users’ search behaviors (Q2) showed no significant change, remaining stable at the “Agree” level. This stability reflects the findings of Jylhä et al. (2024) that algorithmic recommendations tend to “enable not searching” and encourage passive rather than active search behaviors, as users delegate search responsibilities to systems. As Schultheiß and Lewandowski (2023) note, this reflects a broader pattern where users with limited algorithmic knowledge tend to maintain established search habits rather than developing new ones, even after training. The persistence of these behaviors suggests that changing search practices requires not just education about SEO, but explicit interventions targeting the human-algorithm interaction context.

In response to our third research question (RQ3: “What changes have been observed in students’ critical reasoning as a result of teaching SEO?”) our findings demonstrate that SEO education effectively enhanced students’ critical reasoning across five of our six operationalized dimensions. The most substantial improvements were observed in students’ ability to evaluate commercial content, understand algorithmic influence, and develop critical perspectives on SEO practices. While behavioral changes remained limited, students developed sophisticated analytical capabilities that enabled them to better understand and critically evaluate search engine mechanisms.

These results suggest a need to go further into the integration between technical SEO education and behavioral intervention strategies. While our results demonstrate that students can develop sophisticated critical understanding of algorithmic systems, the survey results also indicate that transforming this knowledge into new search practices remains challenging. This suggests that future SEO education should incorporate explicit behavioral components, combining theoretical knowledge with practical exercises that challenge established search habits. As demonstrated by the work of Lui and Au (2020) on educational game design for SEO teaching, interactive and experiential learning approaches can help bridge this theory-practice gap. This need for practical engagement aligns with the findings of Atkins and Reilly (2019) on the effectiveness of hands-on SEO curricula, while supporting the argument of Martzoukou et al. (2020) that digital competences must be developed through both theoretical understanding and practical application.

Conclusion

This study examined how SEO education impacts students’ algorithm awareness and search engine interactions through a mixed-methods approach combining quantitative measurements and qualitative focus groups. Our findings address three main research questions:

• RQ1: “How does teaching SEO affect students’ evaluation of search engines?”: Our results demonstrated that while most evaluation criteria remained stable, students showed a significant evolution in their perception of search engine efficiency. The focus groups revealed a deeper understanding of algorithmic mechanisms, even if technical evaluation practices remained largely unchanged.

• RQ2: “How does teaching SEO influence students’ information retrieval strategies?”: The findings indicated selective changes in information retrieval behaviors, with significant improvements in security awareness and result evaluation, while most established search behaviors remained stable. This suggests that SEO education can enhance specific aspects of information retrieval while having limited impact on deeply ingrained search habits.

• RQ3: “What changes have been observed in students’ critical reasoning as a result of teaching SEO?”: Our analysis revealed substantial improvements in students’ critical reasoning, with five out of six dimensions showing significant enhancement. Students demonstrated increased ability to distinguish between organic and paid results, better understanding of SEO’s influence on search results, and more sophisticated evaluation of website quality.

The results of our study provide a practical framework for developing critical thinking in SEO education. While our findings indicate limited changes in evaluation of search engines and information retrieval strategies, they also reveal a substantial transformation in students’ critical reasoning toward SEO. The significant evolution observed suggests that a progressive approach is effective, where technical concepts serve as anchor points for developing algorithmic awareness. For example, learning about SEO naturally enables students to strengthen critical thinking through concrete tasks such as distinguishing between organic and advertising results, or analyzing the mechanisms of algorithmic influence. The combination of practical platform exercises and reflective discussion, as illustrated in the focus groups, demonstrates how critical thinking can be developed through hands-on learning, particularly when students connect theory to the ethical implications of their actions. This methodology, validated by our quantitative and qualitative results, can be transposed to other technical courses to simultaneously develop students’ technical expertise and algorithmic awareness in practical settings.

The study’s sample, which included students from three French institutions, limits the generalizability of the results. Moreover, the contrast between significant changes in algorithm awareness and limited evolution in technical practices raises questions about the long-term impact of such education. The emergence of new search methods, such as generative AI, is disrupting traditional usage and reinforcing the need to address algorithmic issues to counterbalance the influence of digital platforms.

Future research should explore how to better influence students’ evaluation of search engines and information retrieval practices while further developing algorithm awareness, particularly as generative AI transforms search engines.

Statements and Declarations

Footnotes

Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Clémentine Fruchard Muller

Sylvain Sagot

Jean-Claude Domenget

Appendix

References

Alloing

(2016). La fabrique des réputations selon Google. Communication et Langages, 188(2), 101–122. https://doi.org/10.4074/s0336150016012060.

Atkins

A. T.

Reilly

C. A.

(2019). Pedagogical strategies for integrating SEO into technical communication curricula. Communication Design Quarterly, 6(3), 66–73. https://doi.org/10.1145/3309578.3309585.

Beier

Wagner

(2015). Crowdfunding success: A perspective from social media and E-commerce. In International conference on interaction sciences, Busan, August 16–18, 2011. https://www.semanticscholar.org/paper/Crowdfunding-Success%3A-A-Perspective-from-Social-and-Beier-Wagner/2af28a6964efe4b54d8e8a4f326472f0c6a231db.

Bink

Schwarz

Draws

Elsweiler

(2023). Investigating the influence of featured snippets on user attitudes. In Proceedings of the 2023 conference on human information interaction and retrieval (pp. 211–220). Association for Computing Machinery. https://doi.org/10.1145/3576840.3578323.

Bogina

Hartman

Kuflik

Shulner-Tal

(2022). Educating software and AI stakeholders about algorithmic fairness, accountability, transparency and ethics. International Journal of Artificial Intelligence in Education, 32(3), 808–833. https://doi.org/10.1007/s40593-021-00248-0.

Boydell

Fergie

McDaid

Hilton

(2014). Avoiding pitfalls and realising opportunities: Reflecting on issues of sampling and recruitment for online focus groups. International Journal of Qualitative Methods, 13(1), 206–223. https://doi.org/10.1177/160940691401300109.

Brajnik

Gabrielli

(2010). A review of online advertising effects on the user experience. International Journal of Human-Computer Interaction, 26(10), 971–997. https://doi.org/10.1080/10447318.2010.502100.

Callegati

Cerroni

Ramilli

(2009). Man-in-the-middle attack to the HTTPS protocol. IEEE Security and Privacy Magazine, 7(1), 78–81. https://doi.org/10.1109/MSP.2009.12.

Clarke

T. B.

Clarke

(2014). A competitive and experiential assignment in search engine optimization strategy. Marketing Education Review, 24(1), 25–30. https://doi.org/10.2753/MER1052-8008240104.

10.

Clarke

T. B.

Murphy

Wetsch

L. R.

Boeck

(2018). Teaching search engine marketing through the Google ad grants program. Marketing Education Review, 28(2), 136–147. https://doi.org/10.1080/10528008.2018.1448282.

11.

Cotter

Reisdorf

(2020). Algorithmic knowledge gaps: A new horizon of (digital) inequality. International Journal of Communication, 14, 745–765.

12.

Deuff

O. L.

Roumanos

(2022). Enjeux définitionnels et scientifiques de la littératie algorithmique: Entre mécanologie et rétro-ingénierie documentaire. Tic & Société, 15(2–3), 325–360. https://doi.org/10.4000/ticetsociete.7105.

13.

Dogruel

Facciorusso

Stark

(2022). ‘I’m still the master of the machine.’ Internet users’ awareness of algorithmic decision-making and their perception of its effect on their autonomy. Information, Communication & Society, 25(9), 1311–1332. https://doi.org/10.1080/1369118X.2020.1863999.

14.

Domenget

J.-C.

Fruchard Muller

Sagot

(2022). L’hybridation d’une formation en lien avec l’hybridité d’un métier : un enjeu de professionnalité pour les référenceurs web ?. University Panteion. Hybridation des formations : de la continuité à l’innovation pédagogique ?, pp. 345-360, 2022.

15.

Dudek

Mastora

Landoni

(2007). Is Google the answer? A study into usability of search engines. Library Review, 56(3), 224–233. https://doi.org/10.1108/00242530710736000.

16.

Duka

Sikora

Strzelecki

(2023). From web catalogs to Google: A retrospective study of web search engines sustainable development. Sustainability, 15(8), 6768. https://doi.org/10.3390/su15086768.

17.

Frydenberg

Miko

J. S.

(2011). Taking it to the Top: A Lesson in Search Engine Optimization. Information Systems Electronic Journal, 9(1), 24–40.

18.

Fusch

G. E.

Hall

J. A.

Walker

N. A.

Booker

J. M.

(2022). How to conduct a focus group interview: Tips, strategies, and examples for novice researchers. Education for Information, 38(2), 171–187. https://doi.org/10.3233/EFI-211520.

19.

Gillespie

(2014). The Relevance of Algorithms. In Gillespie

, et al., (Ed.) Media Technologies: Essays on Communication, Materiality and Society. The MIT Press. https://doi.org/10.7551/mitpress/9780262525374.003.0009.

20.

Giomelakis

Karypidou

Veglis

(2019). SEO inside newsrooms: Reports from the field. Future Internet, 11(12), 261. https://doi.org/10.3390/fi11120261.

21.

Giomelakis

Veglis

(2016). Investigating search engine optimization factors in media websites. Digital Journalism, 4(3), 379–400. https://doi.org/10.1080/21670811.2015.1046992.

22.

Gudivada

V. N.

Rao

Paris

(2015). Understanding search-engine optimization. Computer, 48(10), 43–52. https://doi.org/10.1109/MC.2015.297.

23.

Haider

Sundin

(2021). Information literacy as a site for anticipation: Temporal tactics for infrastructural meaning-making and algo-rhythm awareness. Journal of Documentation, 78(1), 129–143. https://doi.org/10.1108/JD-11-2020-0204.

24.

Häußler

Schultheiß

Lewandowski

(2023). Is googling risky? A study on risk perception and experiences of adverse consequences in web search. Journal of the Association for Information Science and Technology, 75(5), 567–580. https://doi.org/10.1002/asi.24802.

25.

Jansen

Pooch

(2001). A review of web searching studies and a framework for future research. JASIST, 52(3), 235–246. https://doi.org/10.1002/1097-4571(2000)9999:9999<::aid-asi1607>3.3.co;2-6.

26.

Joshi

Kale

Chandel

Pal

(2015). Likert scale: Explored and explained. British Journal of Applied Science & Technology, 7(4), 396–403. https://doi.org/10.9734/BJAST/2015/14975.

27.

Jylhä

Hirvonen

Haider

(2024). Algorithmic recommendations enabling and constraining information practices among young people. Journal of Documentation, 80(7), 25–42. https://doi.org/10.1108/JD-05-2023-0102.

28.

Karyotakis

M.-A.

Lamprou

Kiourexidou

Antonopoulos

(2019). SEO practices: A study about the way news websites allow the users to comment on their news articles. Future Internet, 11(9), 188. https://doi.org/10.3390/fi11090188.

29.

Katavić

S. S.

, et al. (2023). Digital privacy and data protection knowledge and skills of university students in Croatia: Preliminary findings. EFI, 39(4), 493–515. https://doi.org/10.3233/EFI-230056.

30.

Kumari

Patthey

Anute

(2022). Search Engine Optimization (SEO) And Social Media Marketing Techniques And Its Impact As A Marketing Tool With Special Reference To Luxury Brands. International Journal of Research Publication and Reviews, 3(7), 3913–3920. https://doi.org/10.55248/gengpi.2022.3.7.40.

31.

Kurniawan

Sanjaya

(2011). Search Engine Optimization (SEO) Implementation for Educational Purposes. Special Issue of the International Journal of the Computer, the Internet and Management, Volume 19 Issue 1, 181-184.

32.

Lacom

Sagot

(2022). A research framework for B2B green marketing innovation: The design of sustainable websites. In 2022 IEEE 28th international conference on engineering, Technology and innovation (ICE/ITMC) & 31st international association for management of Technology (IAMOT) joint conference (pp. 1–9). IEEE. https://ieeexplore.ieee.org/abstract/document/10033239/.

33.

Leutner

Yearsley

Codreanu

S.-C.

Borenstein

Ahmetoglu

(2017). From Likert scales to images: Validating a novel creativity measure with image based response scales. Personality and Individual Differences, 106, 36–40. https://doi.org/10.1016/j.paid.2016.10.007.

34.

Lewandowski

(2008). The retrieval effectiveness of web search engines: Considering results descriptions. Journal of Documentation, 64(6), 915–937. https://doi.org/10.1108/00220410810912451.

35.

Lloyd

(2010). Framing information literacy as information practice: Site ontology and practice theory. Journal of Documentation, 66(2), 245–258. https://doi.org/10.1108/00220411011023643.

36.

Lloyd

(2019). Chasing Frankenstein’s monster: Information literacy in the black box society. Journal of Documentation, 75(6), 1475–1485. https://doi.org/10.1108/JD-02-2019-0035.

37.

Lucas

Bem-haja

Santos

Figueiredo

Ferreira Dias

Amorim

(2022). Digital proficiency: Sorting real gaps from myths among higher education students. British Journal of Educational Technology, 53(6), 1885–1914. https://doi.org/10.1111/bjet.13220.

38.

Lui

R. W. C.

C. H.

(2017). Designing simulation games for information systems education—a case study in teaching for digital marketing. In 2017 IEEE 6th international conference on teaching, assessment, and learning for engineering (TALE) (pp. 290–295). IEEE. https://doi.org/10.1109/TALE.2017.8252349.

39.

Lui

R. W. C.

C. H.

(2018). Establishing an educational game development model: From the experience of teaching search engine optimization. International Journal of Game-Based Learning, 8(1), 52–73. https://doi.org/10.4018/IJGBL.2018010104.

40.

Lui

R. W. C.

C. H.

(2020). I.S. educational game: Adoption in teaching search engine optimization (SEO). Journal of Computer Information Systems, 60(3), 233–243. https://doi.org/10.1080/08874417.2018.1461034.

41.

Maher

Hadfield

Hutchings

de Eyto

(2018). Ensuring rigor in qualitative data analysis: A design research approach to coding combining NVivo with traditional material methods. International Journal of Qualitative Methods, 17(1), 1609406918786362. https://doi.org/10.1177/1609406918786362.

42.

Malaga

R. A.

(2010). Search engine optimization—black and white Hat approaches. In Advances in computers (Vol. 78, pp. 1–39). Elsevier. https://doi.org/10.1016/S0065-2458(10)78001-3.

43.

Martzoukou

Fulton

Kostagiolas

Lavranos

(2020). A study of higher education students’ self-perceived digital competences for learning and everyday life online participation. Journal of Documentation, 76(6), 1413–1458. https://doi.org/10.1108/JD-03-2020-0041.

44.

Moody

Bates

(2013). PR students’ perceptions and readiness for using search engine optimization. Journalism and Mass Communication Educator, 68(2), 166–180. https://doi.org/10.1177/1077695813478182.

45.

Najmi

Sadasivam

Ray

(2021). How to choose and interpret a statistical test? An update for budding researchers. Journal of Family Medicine and Primary Care, 10(8), 2763–2767. https://doi.org/10.4103/jfmpc.jfmpc_433_21.

46.

Norström

Islind

A. S.

Snis

U. L.

(2020). Algorithmic Work: The Impact of Algorithms on Work with Social Media. European Conference on Information Systems.

47.

Onwuegbuzie

A. J.

Dickinson

W. B.

Leech

N. L.

Zoran

A. G.

(2009). A qualitative framework for collecting and analyzing data in focus group research. International Journal of Qualitative Methods, 8(3), 1–21. https://doi.org/10.1177/160940690900800301.

48.

Öztuna

Elhan

Tüccar

(2006). Investigation of four different normality tests in terms of type 1 error rate and power under different distributions. Turkish Journal of Medical Sciences, 36(3), 171–176.

49.

Palanisamy

(2013). Evaluation of search engines: A conceptual model and research issues. International Journal of Business and Management, 8(6), 6. https://doi.org/10.5539/ijbm.v8n6p1.

50.

Raikar

P. V.

Byatnal

S. A.

T N

(2020). Review of search engine optimization. International Journal of Recent Engineering Science, 7(3), 30–35. https://doi.org/10.14445/23497157/IJRES-V7I3P106.

51.

Razali

N. M.

Yap

B. W.

(2011). Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-darling tests. Journal of Statistical Modelling & Analytics, 2(1), 21–33.

52.

Reynolds-Keefer

Johnson

(2011). Is a Picture Worth a Thousand Words? Creating Effective Questionnaires with Pictures. Practical Assessment, Research & Evaluation, 16(8), 1–7.

53.

Roy

Maxwell

Hauff

(2022). Users and contemporary SERPs: A (Re-)Investigation. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval (pp. 2765–2775). ACM. https://doi.org/10.1145/3477495.3531719.

54.

Sagot

Ben Arfa

(2023). Enhancing efficiency of crowdfunding campaign financing: The role of search engine optimization and social media. International Journal of Strategic Decision Sciences, 14(1), 1–24. https://doi.org/10.4018/IJSDS.327790.

55.

Sagot

Fougères

A.-J.

Ostrosi

(2018). Collaborative engineering decision-making for building information channels and improving Web visibility of product manufacturers. Advanced Engineering Informatics, 38, 264–276. https://doi.org/10.1016/j.aei.2018.07.003.

56.

Schultheiß

Lewandowski

(2020). “Outside the industry, nobody knows what we do” SEO as seen by search engine optimizers and content providers. Journal of Documentation, 77(2), 542–557. https://doi.org/10.1108/JD-07-2020-0127.

57.

Schultheiß

Lewandowski

(2023). Misplaced trust? The relationship between trust, ability to identify commercially influenced results and search engine preference. Journal of Information Science, 49(3), 609–623. https://doi.org/10.1177/01655515211014157.

58.

Schultheiß

Sünkler

Lewandowski

(2018). We still trust in Google, but less than 10 years ago: An eye-tracking study. Information Research, 23(3).

59.

Scott

(2015). White Hat search engine optimization (SEO): Structured web data for libraries. Partnership: the Canadian Journal of Library and Information Practice and Research, 10(1), 1–21. https://doi.org/10.21083/partnership.v10i1.3328.

60.

SEO-ELP . (2025). SEO-ELP (search engine optimization e-learning platform). SEO-ELP. https://seo-elp.fr/.

61.

Shafiq

Rasool

(2014). Evaluation of search interfaces of Online Resources: A literature review. Journal of Advancements in Library Sciences, 1, 57–63.

62.

Shankar

Grewal

Sunder

Fossen

Peters

Agarwal

(2022). Digital marketing communication in global marketplaces: A review of extant research, future directions, and potential approaches. International Journal of Research in Marketing, 39(2), 541–565. https://doi.org/10.1016/j.ijresmar.2021.09.005.

63.

Sharma

Gupta

Mateen

Pratap

(2017). Making sense of the changing face of Google’s search engine results page: An advertiser’s perspective. Journal of Information, Communication and Ethics in Society, 16(1), 90–107. https://doi.org/10.1108/JICES-06-2017-0035.

64.

Sharma

Shukla

Giri

A. K.

Kumar

(2019). A brief review on search engine optimization. In 2019 9th international conference on cloud computing, data science & engineering (confluence) (pp. 687–692). IEEE. https://doi.org/10.1109/CONFLUENCE.2019.8776976.

65.

Sheffield

J. P.

(2020). Search engine optimization and business communication instruction: Interviews with experts. Business and Professional Communication Quarterly, 83(2), 153–183. https://doi.org/10.1177/2329490619890335.

66.

Shin

Kee

K. F.

Shin

E. Y.

(2022). Algorithm awareness: Why user awareness is critical for personal privacy in the adoption of algorithmic platforms? International Journal of Information Management, 65, 102494. https://doi.org/10.1016/j.ijinfomgt.2022.102494.

67.

Sparks

J. R.

Katz

I. R.

Beile

P. M.

(2016). Assessing digital information literacy in higher education: A review of existing frameworks and assessments with recommendations for next-generation assessment. ETS Research Report Series, 2016(2), 1–33. https://doi.org/10.1002/ets2.12118.

68.

Spencer

Enge

Stricchiola

(2023). The art of seo: Mastering search engine optimization (4th edn). O’Reilly Media.

69.

Sphinx Declic . (2007). Sphinx declic. Online Survey Software. https://www.lesphinx.es/sphinx-declic?lang=en.

70.

Statista . (2024). Global search engine market share 2024. Statista. https://www.statista.com/statistics/1381664/worldwide-all-devices-market-share-of-search-engines/.

71.

Wang

Chai

Ravishankar

(2022). Statistical analysis methods in engineering education research: A state-of-the-art review. In 2022 IEEE global engineering education conference (EDUCON) (pp. 438–444). IEEE. https://doi.org/10.1109/EDUCON52537.2022.9766391.

72.

Wogalter

M. S.

Mayhorn

C. B.

(2008). Trusting the internet: Cues affecting perceived credibility. International Journal of Technology and Human Interaction, 4(1), 75–93. https://doi.org/10.4018/jthi.2008010105.

73.

XLSTAT . (2025). XLSTAT|statistical software for excel. XLSTAT. https://www.xlstat.com/en/.

74.

Zhou (Jamie)

Fan (Patrick)

Wang

G. A.

(2018). Project description and crowdfunding success: An exploratory study. Information Systems Frontiers, 20(2), 259–274. https://doi.org/10.1007/s10796-016-9723-1.

75.

Ziakis

Vlachopoulou

Kyrkoudis

Karagkiozidou

(2019). Important factors for improving Google search rank. Future Internet, 11(2), 32. https://doi.org/10.3390/fi11020032.