Algorithmic media use and algorithm literacy: An integrative literature review

Data collection

The data collection for our literature review involved two distinct steps (Figure 1). In the first step, we used the Web of Science database to identify peer-reviewed academic papers in English published between January 1, 2000, and September 6, 2023, in the fields of social science and humanities (including the categories Communication, Political Science, Anthropology, Human Geography, Information Systems, Educational Science, Sociology). Drawing upon terms identified by Oeldorf-Hirsch and Neubaum (2023), we searched for articles in which the term “algorithm” appeared within two words (NEAR/2) of the terms “literacy,” “knowledge,” “competence,” “awareness,” “skill,” “education,” “belief,” “attitude,” “experience,” “folk theory,” or “imagination” in the abstract, title, and keywords. Our inclusion criteria in this step covered peer-reviewed empirical studies, reviews, theoretical works, and doctoral dissertations, while initially excluding preprints, unpublished work, reports, and conference proceedings due to variations in the quality of the peer-review processes associated with these products (Scherer and Saldanha, 2019; Paez, 2017).

OPEN IN VIEWER

Figure 1.

PRISMA flow diagram.

This search yielded a total of 729 publications. After applying formal criteria, including publication date, language, peer-review status, and alignment with humanities and social sciences, we excluded 286 items. The remaining 443 publications were further assessed for eligibility, using a sequence of exclusion criteria: 1) not related to communication on and through algorithmic platforms (e.g., the role of algorithmic mathematics in Wall Street’s financial system), 2) substantial content not related to algorithms (e.g., general media literacy pieces that do not address algorithms as object of inquiry), 3) not related to literacy relevant for using, evaluating, or navigating algorithmic media use (e.g., policy analysis of EU legislation on algorithms). All publications were double-coded for eligibility, which resulted in an acceptable Krippendorff’s alpha value (α = .71). After further discussing the differences in coding, 101 publications were selected for retrieval. After a full-text analysis, 91 publications (86 journal articles and five doctoral dissertations) met the substantive criteria for inclusion.

In the second step of data collection, we conducted forward searches (via Google Scholar) and backward searches (via reference lists and Connected Papers) using publications from our initial sample. In this step, we broadened our scope to include conference proceedings identified through these searches. This decision represented a compromise, as it involved a) acknowledging recommendations to incorporate conference proceedings in systematic literature reviews (Scherer and Saldanha, 2019); b) recognizing the interdisciplinary nature of AL research, spanning fields such as human-computer interaction at the intersection of social science and computer science/informatics, primarily published through conference proceedings; and c) acknowledging the impractical volume of proceedings beyond our resources. This search yielded 119 articles, which, after being subjected to our exclusion criteria sequence, resulted in the inclusion of 78 additional articles. Our final dataset, therefore, comprises 169 articles. (See the full sample list in Supplementary Materials or the dedicated Open Science Framework directory: https://osf.io/scxmu/).

Data analysis

We analyzed the sample using deductive and inductive coding. While we outlined these stages sequentially, our process was iterative, often involving simultaneous engagement with different phases and revisiting earlier stages as we incorporated new sources into our sample (Cronin and George, 2023). First, we coded the articles deductively based on key descriptors: the research paradigms (social constructivism, positivism), conceptual or empirical approach and methods (qualitative, quantitative, mixed methods), publication year, and national affiliation of the authors. Next, we performed an in-depth full-text examination. We randomly selected a subsample of articles, and each team member conducted open coding on this subsample guided by our RQs and theoretically defined focal categories (dimensions of AL, influencing factors, antecedents, and outcomes). This stage involved extensive note-taking. Through regular discussions, we inductively refined our focal categories. We also created subcategories based on our notes to identify patterns within these categories. For example, while we deductively identified the need to code for exogenous factors (based on DeVito et al., 2018), we further developed these categories inductively by reviewing the literature. Having developed a cohesive coding scheme, we systematically coded the remaining sample, distributing it among team members. To ensure rigor, two team members independently coded about 40% of the sample, resolving discrepancies through discussion. In the final stage, we revisited the focal categories of interest (e.g., dimensions of AL identified by Oeldorf-Hirsch and Neubaum, 2023) and used them to construct our organizing framework (Cronin and George, 2023). For a detailed overview of the category structure including references to exemplary publications, see Supplementary Material (SM-Tables 1, 2, and 3) or under the above provided OSF link.

Descriptive findings

Scholarly work on users’ perspectives toward algorithms and their practices has strongly increased since 2018 (Figure 2), presumably in parallel with an increased public awareness of algorithms and datafication, due, for example, to the Facebook Cambridge-Analytica scandal in 2018 (Hinds et al., 2020). Another contributor to the heightened interest was presumably the global launch of TikTok in 2016, which soon established itself as one of the most used apps worldwide (Bhandari and Bimo, 2022).

OPEN IN VIEWER

Figure 2.

Number of publications per year by empirical method (2010–2023).

Regarding methodological approaches, conceptual work (n = 18, 10.7%) is far less prevalent than empirical work (n = 151, 89.3%). Empirical articles favor qualitative methods, such as interviews or qualitative content analyses (57.4%), followed by quantitative methods, such as surveys and experiments (22.5%; Table 1).

OPEN IN VIEWER

Table 1.

Applied methods.

	Method	n	%
Qualitative	interviews	42	24.9%
	content analysis	14	8.3%
	ethnography	8	4.7%
	other (participatory methods, focus groups, literature reviews, media diaries)	8	4.7%
Quantitative	survey	25	14.8%
	experimental survey	10	5.9%
	content analysis	1	0.6%
Mixed Methods	qualitative	25	14.8%
	qualitative & quantitative	16	9.5%
	quantitative	2	1.2%
Conceptual		18	10.7%
	Total	169	100%

Defining and measuring algorithm literacy (RQ1)

Guided by the dimensions of AL conceptualized by Oeldorf-Hirsch and Neubaum (2023), we classified the sampled articles according to the cognitive, affective, and behavioral dimensions of AL. (For a summary, see Supplementary Materials, SM-Table 1). In addition, we assessed whether authors examined the users’ perceived, subjective qualifications of algorithmic functions or measured AL against a factual benchmark.

Developing algorithm literacy (RQ2)

In the following section, we outline the endogenous and exogenous factors of algorithmic media use and the user characteristics that shape AL. (For a summary and exemplary studies for each subcategory, see Supplementary Materials, SM-Table 2.)

Outcomes of algorithm literacy (RQ3)

Outcomes of AL highlight the practical achievements enabled through algorithmic media in everyday contexts. Although often implied and serving as the normative basis for calls for increased AL, these outcomes are rarely explicitly examined, thus remaining a relatively uncharted territory. In our review, we align with Helsper’s (2021) argument on the importance of differentiating various outcome domains, as success in one literacy-related outcome may not necessarily translate into success in others. (For a summary and exemplary publications for each subcategory, see Supplementary Materials, SM-Table 3.)

To begin with, AL is linked with notable consequences on the individual level. Our analysis identifies heightened elaboration as a key personal outcome of AL, with studies showing that individuals with elevated AL levels engage with algorithmic media more mindfully and intentionally (e.g., Adams-Grigorieff, 2023; Pronzato and Markham, 2023). Another presumed personal outcome is increased agency over algorithmic experiences, involving intentional interaction with algorithms to improve user experience (e.g., Adams-Grigorieff, 2023; DeVito, 2022). AL also enables users to achieve selective (in)visibility, protecting themselves and their communities from online harassment on algorithmic platforms (DeVito, 2022). Further, AL contributes to self-actualization, allowing individuals to pursue interests, shape identities, and gain visibility in their communities of interest (e.g., DeVito, 2022; Simpson et al., 2022).

Finally, a segment of the literature examines personal well-being as an outcome of AL, albeit rarely as a focal variable. One exception is studies on challenges faced by women using algorithmic media to cope with pregnancy loss stigma, as algorithms targeting these women assume all pregnancies proceed as expected, thus leading to a decrease in their well-being (Andalibi and Garcia, 2021; Bogers et al., 2020). Additionally, research indicates that engagement in the platform economy negatively impacts well-being, inducing stress irrespective of individuals’ AL levels (e.g., Bishop, 2018; Curchod et al., 2020). We derive two observations from the reviewed literature: 1) the surprising underemphasis on well-being as a focal AL outcome, despite established links between media literacy and well-being (Schreurs and Vandenbosch, 2021), and 2) the predominant focus on adverse well-being outcomes of insufficient AL. In contrast, research on general digital literacy prioritizes resilience, akin to well-being, defined as “learning from past positive and negative experiences online to avoid negative outcomes and exploit ICT benefits in the future” (Helsper, 2021: 80).

Community building stands out as a central social outcome of AL, fostering online communities based on shared interests and values (e.g., Avella, 2023). In this case, AL implies that individuals and communities can control their presence within the algorithmic flows and shape the narratives surrounding them. Similarly, AL allows users to accumulate social capital (e.g., Bhandari and Bimo, 2022) through affiliations with like-minded communities or by establishing themselves as knowledge brokers, as exemplified by those who disseminate algorithmic knowledge on platforms like YouTube (e.g., Bishop, 2020).

Regarding economic outcomes, the literature suggests a link between AL and the ability to achieve visibility on algorithmic platforms, which is central to platform workers in creative industries, as it opens opportunities for monetization, such as brand collaborations (e.g., Avella, 2023). Conversely, studies on gig workers, such as Uber drivers and delivery couriers, reveal that lacking AL can result in precarity and exploitation (e.g., Curchod et al., 2020; Sun, 2019). These workers face the constant dilemma of striving for visibility, reputation, and consistent clients by engaging in emotional labor while risking financial loss by undervaluing their work to avoid penalties (e.g., Bucher et al., 2021). Nevertheless, it remains an open question whether greater AL directly leads to greater visibility, or if those with higher AL are simply better positioned to achieve visibility due to other factors. For example, socioeconomic background and societal status may play a significant role in influencing both AL and visibility.

Political outcomes associated with AL encompass the augmented capacity to mobilize online communities for social and political causes. Although examples are limited, instances of AL contributing to political mobilization are evident across the political spectrum (e.g., Cotter, 2024; Maly, 2019). Moreover, a lack of AL increases the likelihood of polarization through personalized algorithmic news based on political affiliations (Calice et al., 2021). Conversely, AL is positively related to informed citizenship (e.g., Makady, 2023), enabling individuals to exercise information care and due diligence in everyday news practices, ensuring a more balanced approach compared to overreliance on algorithmic curation (Du, 2023; cf. Schaetz et al., 2023). Studies emphasize the importance of critical consciousness in contributing to ethical discussions on algorithmic systems and advocating for improved regulation, such as addressing content manipulation and the addictive nature of personalized content (Chung, 2023; Scalvini, 2023). Thus, individuals with AL are more prone to garner policy support for matters related to platform regulation (e.g., Chung, 2023). Conversely, a lack of AL may result in “algorithmic impotence,” where individuals lack the capacity to critique or influence perceived unfair algorithmic processes (e.g., Qadri and D’Ignazio, 2022; Sun, 2019).

The extent of AL within a population can significantly influence broader societal outcomes. Epistemic privilege refers to unequal access to knowledge about algorithms that bestows greater privilege upon certain actors, typically platforms, compared to others, such as content creators subject to those algorithms (Cotter, 2023). The overarching implication of epistemic privilege is that specific knowledge becomes more potent, shaping decision-making processes across various life domains and determining who controls these decision-making processes (Lloyd, 2019). Likewise, a lack of AL within segments of the population is likely to exacerbate existing digital divides (Cotter and Reisdorf, 2020; Gran et al., 2021; Zarouali et al., 2021b).

In summary, research on AL outcomes predominantly emphasizes the negative consequences of a lack of literacy, particularly for vulnerable groups. Indeed, the unequal adoption of AL is likely to exacerbate existing inequalities across diverse population strata since outcomes of AL are linked to individuals’ other resources, with disadvantaged individuals facing compound barriers in terms of access, competences, and norms in algorithmic media use (Helsper, 2021: 118). This points to the need to situate individuals within broader social and societal contexts to understand the dynamics of acquisition and outcome quality of AL.

Toward an experiential learning framework for algorithm literacy

As illustrated previously, scholarship tends to treat algorithms as experience technologies (Cotter and Reisdorf, 2020; Swart, 2021), and while this idea is echoed within our sample, the literature needs a theoretical framework to integrate these findings. Based on our reading of the literature, we suggest that users acquire AL through algorithmic media use in a process that can be classified as experiential learning. We contend that the Experiential Learning Theory (ELT) proposed by Kolb (1984, 2015) offers a suitable framework for further thinking about AL in algorithmic media use. The core tenet of ELT is that “learning is the process whereby knowledge is created through the transformation of experience” (Kolb, 1984: 38). Accordingly, ELT treats learning as an ongoing, cyclical adaptation to the world through interactions between the individual and the environment (Vince, 1998). While this theory has been applied in other disciplines to explain learning experiences and outcomes (e.g., Morris, 2020), it has not received much attention in media and communication science outside of media pedagogy (for an exception, e.g., Greenberg, 2007).

ELT outlines four stages representing an idealized learning cycle: concrete experience, reflection on thoughts and feelings, abstraction by drawing conclusions, and experimentation involving behavioral adaptation based on prior conclusions (Kolb, 1984; Vince, 1998). Concrete experiences are “highly contextualized, primary, experience[s] that involve hands-on learner experience in uncontrived real-world situations” (Morris, 2020: 1070). In the context of AL (see Figure 4), concrete experiences involve encounters with algorithms in daily life through algorithmic media use, such as algorithmically curated news feeds, search engine results, or personalized content recommendations on streaming platforms. Based on these experiences, individuals become aware of algorithms and (ideally) reflect on these encounters. Reflection plays a central role in the learning process and is vital for making sense of the experience. With heightened awareness, individuals engage in abstract conceptualization (abstraction). ELT further suggests that learning is incomplete without applying knowledge in new situations. This involves testing the fit of abstract conceptualizations formulated against new concrete experiences (Morris, 2020). Concrete experiences with algorithms necessarily entail affective responses that accompany all learning cycle stages. In the preceding sections, we highlighted that literature suggests that people who use algorithmic media strategically are more likely to have a higher level of AL. This aligns with ELT, underscoring the importance of purposeful learning within specific contexts and concrete problems (Morris, 2020: 1069). The reviewed studies reveal that individuals engaging with algorithms in practical tasks or with specific goals are more motivated to learn and encounter more tangible learning opportunities. In the context of AL, active experimentation is manifested as (more or less) conscious adaptation in users’ engagement with algorithms. This involves experimenting with different online behaviors and observing algorithmic responses, refining one’s understanding and strategies in algorithmic interactions. In the preceding section, we categorized these strategies as alignment, compliance, subversion, and resistance.

OPEN IN VIEWER

Figure 4.

Experiential learning cycle.

The stages of the learning cycle are not rigidly delineated but rather fluid in nature, meaning that the learner “touches all the bases” in a “recursive process that is sensitive to the learning situation and what is being learned” (Kolb, 2015: 51). Since conditions of the context may change across time and place, “all knowledge is provisional and needs testing in context” (Morris, 2020: 1072). This aligns with literature describing learning through experience in algorithmic media use (specifically, see DeVito, 2021, on adaptive folk theorization). At the same time, because learning is always context-specific, ELT requires people to be comfortable with ambiguity and uncertainty related to new learning experiences (Morris, 2020: 1071).

By employing an established framework and aligning review findings with it, we hope to provide a structured foundation for future research endeavors to understand the acquisition and cultivation of AL. For example, the reviewed literature suggests that individual characteristics might moderate the learning cycle, with factors like the level of elaboration and topic involvement fostering abstraction and adaptation. At the same time, a lack of general digital literacy or a history of marginalization may hinder development (e.g., Helsper, 2021). Thus, future research should investigate individual factors and the circumstances under which user experiences, reflection, abstraction, and adaptation lead to an increased AL. In addition, according to Morris (2020), a fundamental aspect of the experiential learning process involves recognizing that knowledge is situated within specific contexts and evolves over time and space. Given that learning is context-dependent and involves managing ambiguity and dissonance (Morris, 2020), it is crucial to foster critical reflection on algorithmic experiences amidst the uncertainty often associated with them (Vince, 1998).

Beyond experiential learning: defining benchmarks and standardizing algorithm literacy

While we argue for the usefulness of ELT (Kolb, 1984, 2015) as a conceptual umbrella, experiential learning alone may not provide a comprehensive understanding of algorithms and one’s behavioral options. In fact, experientially developed AL might be self-referential and confined since it might drive adaptation based on impressions rather than a thorough grasp of the underlying algorithmic processes (cf. Morris, 2020). If being algorithmically literate ultimately means acting in one’s best interest through algorithmic media use (Das, 2023), AL must extend beyond establishing folk theorization, and we must determine what this best interest might mean in different domains. Thus, we see several critical avenues for 1) standardizing AL measures (e.g., developing scales and measuring AL across populations, cf. Dogruel et al., 2022) and 2) further elaborating on the relationship between subjective vs. objective (or “factual”) aspects of AL.

Related to standardization, we hope to have shown in this review that descriptive studies of AL are aplenty. Drawing from prior studies (Dogruel et al., 2022; Zarouali et al., 2021a), additional efforts are necessary to establish valid and reliable AL measures. Further, in the next phase of the field development and in line with the aim that AL should enable people to act in their best interest through algorithmic media (Das, 2023), researchers should not stray away from defining desirable outcomes of AL across domains of study against which we can examine the level and content of competencies needed to reach these outcomes. Relatedly, future research must delineate domain-specific competencies indicative of AL encompassing cognitive, affective, and behavioral dimensions. Defining these is crucial for measuring AL, thus enabling a more rigorous examination of the influence and value of AL in various domains and across populations. Emphasizing the importance of cross-domain skills, we encourage research to develop transferable AL indicators, recognizing their enduring relevance in the context of evolving platforms (Helsper, 2021).

The relationship between the subjective and objective aspects of AL requires better integration. While establishing absolute truths about algorithms is challenging (or perhaps impossible) due to their proprietary and dynamic nature, it is worthwhile trying to define a minimal set of competences as a baseline on top of which users can build. Consider the analogy of driving a car. Central to driving literacy is the objectively ascertainable knowledge of traffic regulations and the ability to operate a vehicle, typically acquired through formal instruction (exogenous factor). However, the effectiveness of this learning typically depends on the learner’s involvement and self-efficacy (endogenous factors). After acquiring basic knowledge, becoming a proficient driver requires further practice in various settings—driving different vehicles, traversing roads in different conditions, and customizing the driving experience to personal needs. The starting competencies remain central, but true proficiency develops through habituation and experimentation, resulting in individualized driving styles and preferences. Regarding AL, establishing fundamental competencies should go hand-in-hand with the development of contextual and experiential knowledge, where users refine their understanding through personal interactions with algorithms. This is contrary to the current situation, where users often lack a foundational baseline. With a solid base of objective knowledge, users could better develop their specific preferences, ideas, and strategies. This is to say that while we do not favor any specific dimension, we do wish to underscore that AL involves a mix of objective and subjective elements, shaped by endogenous and exogenous learning processes. In this context, formal education (exogenous) can provide a foundational understanding and create awareness for ethical considerations of algorithms (comparable, e.g., to knowledge on the environmental impact of cars), while personal experience and adaptation (endogenous) refine and deepen that knowledge to meet individual needs.

Researching algorithm literacy comparatively

The literature consistently suggests that cross-platform use enhances algorithmic learning. However, a systematic examination is warranted. To this end, we encourage scholars to carefully consider designations such as “the TikTok algorithm” or “the Facebook algorithm,” which, while acknowledging the context-dependent nature of algorithms shaped by specific platform affordances and vernaculars, can also reinforce the mystification of algorithms. Instead, we should consider algorithms as a collection of mechanisms that can be disentangled and studied separately, much like how we have learned to approach “the internet” not as a monolithic entity but as a bundle of mechanisms (Farrell, 2012). This shift in perspective would allow for a more systematic examination of how algorithms function across different platforms, and how algorithm literacy develops in various contexts. To this aim, we propose two approaches to cross-platform comparisons of AL. First, a user-centric approach would involve specifying focal-use goals (e.g., parenting, political information) and comparing individual user engagement, issue involvement, and affective assessments across platforms. This approach could directly examine the sought-after AL outcomes, enabling a finer understanding of how users interact with and learn about different algorithms. Second, an algorithm-centric approach would compare algorithms that execute specific tasks across platforms, focusing on how users understand these functions. While proprietary mechanisms often obscure the inner workings of algorithms, it is known that they are designed to fulfill specific roles for platform users. By systematically disentangling these mechanisms, we can begin to demystify algorithms.