Into the black box: Laypeople's folk theories about generative artificial intelligence chatbots

Folk theory as a conceptual framework

Folk theories are defined as “intuitive causal explanatory theories that people construct to explain, interpret, and intervene in the world around them” (Gelman and Legare, 2011: 380). As intuitive “theories” constructed by laypeople, folk theory plays a crucial role in everyday cognitive processes, guiding interactions with the world and influencing perspectives across fields such as biology, physics, and psychology (Gopnik and Wellman, 1994). For instance, folk theories, such as “folk biology” and “folk physics,” offer intuitive understandings of complex systems, rooted in childhood experiences that prioritize appearance and behavior over scientific accuracy, like a child that would group bats with birds due to their wings (Gelman and Legare, 2011). More recently, Devito et al. (2018) broadened the definition of folk theories to encompass not only explanations but also assumptions and expectations that shape how users develop these theories. According to Devito et al., in HCI, “folk theory” can be expanded to the intuitive and informal theories that individuals develop to make sense of how technological systems operate and their potential outcomes or impacts. These theories, which guide users’ reactions and behaviors towards the technology, are socially constructed and highly adaptable (Devito et al., 2018).

Like scientific theories, folk theories emerge through “inductive-deductive reasoning processes.” These intuitive frameworks enable individuals to interpret and explain complex social phenomena, offering conceptual tools to “explain, understand and intervene in the world” (Gelman and Legare, 2011: 381). Both scientific theories and folk theories offer generalized perspectives on how a complicated system works and can be assessed through ontological, procedural, epistemological, and ethical questions. Specifically, it can be asked, “what is this system?”, “what can this system do?”, “what do people know about it?”, and “what qualifies as a good system?”. Based on this analytic framework, Nielsen (2016) developed the definition of folk theory of journalism as “actually existing popular beliefs about what journalism is, what it does, and what it ought to do” (Nielsen, 2016).

Unlike scientific theories, folk theories are more fluid, adaptable, and rooted in individual and societal contexts. Rather than being rigorously evaluated, folk theories rely on intuitive inference and are often influenced by cognitive biases that shape thought and behavior. Drawn from second-hand sources, cultural narratives, and personal experiences, they serve as interpretive frameworks for understanding the world, regardless of their accuracy (Nisbett and Ross, 1980; Rip, 2006). In other words, folk theories do not undergo any formalized scrutiny and collective assessment that are typical within scientific communities (Nielsen, 2020), and may “loose out” to scientific theories that offer testable explanations. Importantly, conflicting folk and scientific theories are not always competitive; they can coexist to provide coherent explanations for complex situations. For example, people might use evolutionary theory to explain human development while also using creationism to explain human origins (Gelman and Legare, 2011).

Several factors contribute to the gap between folk theories of complex opaque systems and their actual workings. One reason may be that humans rely on cognitive frameworks dating back to prehistoric periods, categorizing unfamiliar entities, such as AI, into known types like tools or animals (Rip, 2006). This tendency leads us to attribute intelligence and consciousness to objects that seem to have these qualities, echoing ancestral experiences (Epley et al., 2007). In the context of contemporary AI systems, these challenges are intensified because the internal working mechanism is not observable, making it harder for laypeople to infer how they operate from the surface.

Similar to the concept of folk theories, other related constructs have been proposed, such as “mental models,” which is described as simplified internal representations that help individuals “understand, explain, and predict phenomena” by “providing predictive and explanatory powers” for them (Johnson-Laird, 1983; Norman, 2014; Rezwana and Maher, 2025; Staggers and Norcio, 1993). Mental models and folk theories differ in their degree of adaptability and social grounding. Whereas mental models are conceptualized as relatively stable cognitive structures formed at the individual level, folk theories are more flexible in that they evolve through ongoing interactions (Devito et al., 2018). Moreover, unlike mental models, folk theories are socially constructed and collectively negotiated through shared experiences, cultural narratives, and interpersonal communication (French and Hancock, 2017). Metaphors are also cognitive tools that help lay users understand new technology by simplifying complex cognitions through drawing parallels with familiar experiences (French and Hancock, 2017; Khadpe et al., 2020). Metaphors and folk theories are closely related in that metaphors play an important role in the formation of folk theories, which acts as a cognitive tool to help individuals to draw analogies between the known and the unknown (Jamrozik et al., 2016; Khadpe et al., 2020). Yet, existing research often treats metaphors and folk theories as relatively stable cognitive constructs, which limits their ability to capture the dynamic and socially negotiated nature of users’ sensemaking about GAI chatbots.

Many studies on algorithmic systems’ perception use folk theory as a lens, primarily focusing on social media, news feeds, and recommendation systems in HCI (e.g. Bucher, 2017; Eslami et al., 2016; French and Hancock, 2017; Huang et al., 2022; Siles et al., 2019, 2020; Ytre-Arne and Moe, 2020). Devito et al. (2018) categorizes this research into two types: one treats folk theories as a diagnostic tool for exploring whether algorithms function in ways that users expect, and the other explores the various folk theories that exist around different social media recommendation algorithms and examines how these theories develop.

Although these studies focus on algorithmic systems that are not always AI-based, some recommendation systems might be still rule-based, these studies still remain theoretically relevant because they illustrate how users make sense of opaque systems. In summary, research exploring folk theories about AI recommendation systems shows three trends. First, many users are unaware of the algorithms behind social media, view them as either helpful assistants or intrusive corporate tools, and adjust perceptions when algorithms defy expectations (Eslami et al., 2016; French and Hancock, 2017). Second, users typically interpret AI systems as personalized agents with human-like qualities, as functional tools akin to other machines, or as ck box systems whose logic is revealed when expectations are violated (Jussupow et al., 2020; Siles et al., 2020). Perceptions are also split between “algorithmic appreciation,” where users trust AI advice in structured settings, and “algorithmic aversion,” a tendency to prefer human input, especially after errors (Dietvorst et al., 2015; Eslami et al., 2016; Jones-Jang and Park, 2022; Logg et al., 2019). Generally, people favor human judgment in moral or personal contexts but rely on algorithms for tasks with clear success metrics.

Unlike recommendation systems, where users often remain unaware of algorithms, GAI chatbots engage users directly, making their algorithmic nature apparent. Their multi-functionality also shapes perceptions of their role in daily life differently from specialized recommendation systems. As a result, existing research on folk theories of recommendation systems may not fully apply to GAI chatbots.

Taken together, these concepts illustrate the multi-layered nature of human sensemaking by showing how individuals draw on cognition, social negotiation, and metaphorical reasoning to understand complex systems. However, each construct tends to emphasize only one aspect of this process. Mental models foreground relatively stable individual cognition, whereas metaphors highlight the analogical reasoning people use to simplify complexity. By integrating insights from these adjacent concepts, folk theories function as a synthesizing lens that captures the full sensemaking process, which is intuitive, imperfect, and shaped by prior experience, cultural narratives, and everyday interactions. Building on this integrative understanding, we define the “folk theory of GAI chatbots” in this research as the intuitive causal beliefs about what a GAI chatbot is, how it operates, and what it ought to be.

Emerging perceptions of generative artificial intelligence tools

Research on people's perceptions of GAI chatbots has expanded rapidly. Thus far, research on perceptions of GAI chatbots focuses on three areas. The first stream focuses on the comparison between human-generated and AI-generated content (e.g.Cui and Zhang, 2025; Ragot et al., 2020). The second stream focuses on users’ attitudes, revealing general positivity towards GAI's role in creativity and coding (e.g. Haque et al., 2022) and usage patterns, showing diverse applications in creative writing, coding, and information retrieval (e.g. Floridi and Chiriatti, 2020; Taecharungroj, 2023). Recently, the third stream, which investigates users’ mental model of GAI systems is rising. A number of studies in this area focus specifically on the perceived consciousness of AI, showing that people attribute varying degrees of consciousness to LLMs, treating them as moral agents, and that these attributions tend to strengthen with increased usage (Colombatto and Fleming, 2024; Guingrich and Graziano, 2024; Manoli et al., 2025). Existing studies have also examined users’ perceptions of AI chatbots by conceptualizing them as part of broader, complex ecosystems (Wang et al., 2025). A growing body of research has begun to examine how users interpret specific AI behaviors, particularly unexpected system outputs. These studies show that when GAI chatbots produce errors or hallucinations, users often interpret such responses as signs of autonomous agency rather than as consequences of technical limitations or probabilistic failures (Lee et al., 2025; Rapp et al., 2025; Zhu and Lu, 2025). Finally, as noted earlier, metaphors often serve as cognitive bridges that help people make sense of unfamiliar technologies. A growing body of research uses metaphors as an analytical framework to examine how users construct mental models of GAI systems. These studies show that users rely on metaphorical reasoning—such as comparing AI to search engines, human assistants, or other familiar entities—to interpret how GAI operates and to understand its capabilities and limitations (An et al., 2024; Cheng et al., 2026; Ljadov et al., 2025; Van Es and Nguyen, 2025)

Yet, most existing research investigates only fragmented aspects of user interpretation, such as consciousness attribution, metaphorical reasoning, or reactions to unexpected system behaviors, and lacks an integrated account of how these elements jointly constitute laypeople's broader folk theories of GAI chatbots. While prior research sheds light on what users think about GAI chatbots, it offers limited insight into how laypeople develop, negotiate, and refine these understandings, and how these understandings influence their subsequent behaviors. Building on these gaps and guided by our definition of folk theories, we raise the following research questions:

RQ 1: What are laypeople's folk theories of GAI chatbots?

Participants and procedure

Data were collected through six FGDs conducted in Singapore, involving a total of 36 participants, with each group comprising six individuals. This method was chosen for its exploratory nature and its ability to uncover folk theories through interactive peer discussions. The FGDs were conducted until theoretical saturation was reached, ensuring comprehensive insights into our core research questions (Ling and Lai, 2016; Low, 2019).

Our participants were recruited using convenience sampling. We circulated a brief study advertisement through internal channels at a local university and via a participant recruitment group that regularly distributes calls for research participation. Individuals who expressed interest were directed to an online screening survey that collected basic demographic information and included an inclusion criterion regarding prior use of any GAI chatbot. Only those who reported previous interactions with GAI chatbots were invited to participate. To ensure that our sample comprised laypeople—defined as individuals who are affected by AI technologies but are not AI experts—we excluded respondents who self-identified as having expert-level knowledge or professional experience in the AI domain.

To ensure participants possessed experience interacting with GAI chatbots, participants were screened through pre-FGD surveys to confirm their experience with GAI chatbots. The screening question asked: “Have you been using any generative AI Chatbot (e.g. ChatGPT, Gemini, or Claude)?”. Only participants who indicated prior use were invited to participate in the FGDs, while those without such experience were excluded from recruitment. This pre-screening procedure aligns with prior research on users’ folk theories and algorithmic sensemaking (e.g. Devito et al., 2018; Eslami et al., 2016).

Usage frequency significantly shapes how people form and revise users’ perceptions of technical systems, where heavy users tend to exhibit deeper cognitive involvement and more developed folk theories (Goldsmith et al., 1994). Previous research shows that users’ perceptions of a system evolve through continuous interaction with AI systems. For instance, frequent usage has been found to correlate strongly with higher levels of perceived consciousness of AI (Kang et al., 2025; Norman, 2014). Prior research also identifies experience as a moderator of core predictors of technology use (Ayaz and Yanartaş, 2020). Furthermore, research on AI literacy suggests that users’ competencies and understanding of AI systems develop progressively with exposure (Long and Magerko, 2020). Therefore, we expected variation in perceptions of GAI chatbots across experience levels. Hence, we separated participants into light and heavy users based on their self-reported usage frequency to examine potential differences in folk theories and interaction strategies. Before the FGD discussions, participants were asked two screening questions: (1) “On average how often do you use generative AI chatbot (e.g. ChatGPT, Gemini) in a typical week ?” (response options: Never, Rarely, Sometimes, Every day, Very frequently) and (2) “How would you rate your frequency as a generative AI chatbot user on a scale from 1 to 5, where 1 indicates a light user and 5 indicates a heavy user?” Participants who selected every day or very frequently for the first question, or 4 or 5 for the second question, were categorized as heavy users. Conversely, those who selected never or rarely and rated themselves as 1 or 2 were classified as light users. To ensure a clear contrast between usage groups, participants who selected sometimes or 3, as well as those with inconsistent responses across the two items (e.g. selecting every day on the first item but rating themselves as 1 or 2 on the second), were excluded from further analysis.

The sample (12 females, 24 males) ranged from 22 to 60 years old, averaging 28, and included 50% Singaporeans, 40% Chinese, and 10% from other countries (Turkey, Malaysia, and Sri Lanka). The sample was recruited via snowball and convenience sampling (see Table 1). The FGDs, conducted in English, were audio- and video-recorded and guided by a trained moderator. Following the interview guide, a well-trained moderator independently led the discussions to explore participants’ understanding of how GAI chatbots operate, their usage patterns, concerns, and perspectives. Open-ended questions were employed to encourage in-depth responses (see Appendix FGD guide, Supplemental material). Each FGD lasted approximately 60 min, and after each FGD, a trained researcher manually transcribed the audio recordings for subsequent data analysis.

Data analysis

This study followed the three-step protocol proposed by grounded theory scholars (Glaser and Strauss, 2017). First, in the open coding stage, the transcriptions of the discussions were thoroughly reviewed by the first author to develop rough codes and themes through processes of comparison, summarization, and iterative rereading. Second, in the axial coding stage, the three authors categorized the initial codes into conceptual bins and identified relationships among them through constant comparison to identify patterns and shared ideas (Wicks, 2017). Then, these categories were collectively reviewed and refined to develop broader themes that captured how participants made sense of and evaluated their interactions with GAI chatbots (see Braun and Clarke, 2006). For example, the perceived characters of GAI chatbots can be summarized as people-pleaser, a social chameleon and a moral being. The analysis process was deliberately iterative to maintain rigor and responsiveness to the data. Finally, in the selective coding stage, the researchers aimed to refine the second-level categories to align them with the research questions. The final themes derived as a result of this process were thoroughly discussed to ensure agreement on their relevance and applicability to addressing each research question. Throughout this process, the participants’ actual comments were clearly demarcated and noted in a Word document for further analysis and reporting. Finally, it should be noted that the names of participants mentioned in the findings were kept anonymized to ensure privacy.

Folk theories of generative artificial intelligence chatbots (RQ1)

Users’ folk theories of GAI chatbots spans three areas: an understanding of how chatbots function (how they learn, how they work), perceived characteristics of chatbots (derived by attributing human-like traits), and expectations of their ideal roles and interactions.

How does a Chatbot work?

Perceived characteristics of generative artificial intelligence chatbots

Expectations towards generative artificial intelligence chatbots

The construction of folk theories: sources of understanding (RQ2)

Participants naturally shared their thoughts about how they construct their folk theories in the FGDs, including through the imagination of buzzwords related to GAI, interactions with GAI chatbots and comparisons with familiar objects.

New social scripts: strategies people use to interact with generative artificial intelligence chatbots (RQ3)

Social scripts, defined as “a predetermined, stereotyped sequence of actions that defines a well-known situation” (Schank and Abelson, 2013: 41), serve as cognitive shortcuts from past interactions. Users adapt new social scripts when interacting with GAI chatbots, influenced by their folk theories. As John (male, heavy user, group 4) noted, “The social scripts about how we communicate with each other don’t apply to talk to a machine…”. We observed that users develop new social scripts pattern like using commanding words and express more emotions and provide clear instructions when interacting with AI chatbots.

Lay people's folk theory (RQ1)

Three areas were identified with regard to laypeople's folk theories about GAI: working mechanisms, perceived characters, and user expectations of GAI. These respectively answer the three questions that permeate the folk theory literature (see Dietvorst et al., 2015; Eslami et al., 2016; Jones-Jang et al., 2022; Logg et al., 2019), “what is a GAI chatbot”, “how do they operate”, and “what it ought to be.”

In summary, users’ folk theory about GAI chatbots’ underlying mechanisms is vague, oscillating between two schemas: “human” and “machine.” This ambiguity is expected given the lack of technical knowledge about the workings of the human brain or the hidden algorithms behind search engines. In such instances, people rely on familiar schemas they already hold (Devito et al., 2018). Our findings similarly reveal that the majority of users’ folk theories are related to the representational characters and patterns they observe with GAI chatbots, rather than reflecting a deeper understanding of the underlying mechanisms. One possible explanation for this tendency is that when systems operate in opaque ways, users may rely more on intuitive, heuristic-based interpretations (Tversky and Kahneman, 1974).

This oscillation between human-like and machine-like explanations also aligns with the concept of mind perception, which comprises two primary dimensions: agency, referring to the capacity for intentional action and decision-making, and experience, referring to the ability to feel emotions or have subjective experiences. Previous studies have shown that people often attribute a moderate degree of agency to AI systems, acknowledging their capacity to act intentionally, while attributing little to no capacity for experiential states such as emotions or sensations (Jackson and Williams, 2021), while recent research reports that users attribute varying degrees of consciousness to LLM based system, This shift is partly explained by the presence of anthropomorphic cues, which have been shown to increase perceptions of AI as possessing a mind (Gray et al., 2007; Martini et al., 2016).

Our findings partially align with these observations but differ in an important way. Unlike specific-purpose AI applications, GAIs are versatile and lack a clearly defined usage context. We find that people attribute anthropomorphic features and mind perceptions based on their own interpretations, with their perceptions oscillating between human-like (high mind perception) and machine-like (low mind perception) characteristics. In line with CASA paradigm, users anthropomorphize GAI chatbots even when they recognize them as data-driven, non-human entities and with limited anthropomorphic cues. Our results support this idea. For example, Quinn, a heavy user, attributed AI's politeness to intentional design, revealing how anthropomorphic and machine-based views coexist.

Regarding the second question, how GAI chatbots operate, three perceived characteristics emerged from our data: GAI has the propensity to please people (people-pleaser), it has the tendency to mimic people (social chameleon) and is an object that adheres to moral principles (moral being). Although our FGDs were conducted in 2023, recent work has reported similar patterns, particularly the tendency to please (Naddaf, 2025). Traditional AI alignment work primarily focuses on ensuring that models obey human instructions or conform to human values through mechanisms such as reward optimization or behavioral constraints (Kirk et al., 2025). Yet, as participants like Romy pointed out, an overly accommodating system risks reinforcing existing biases by echoing users’ opinions rather than providing objective perspectives. This is especially concerning as GAI systems are increasingly treated as social actors rather than mere tools. When an AI becomes both morally constrained and socially obliging, its alignment to human preferences may magnify cognitive biases and encourage echo chambers. Hence, for future study, it is pressing to ask, how should we better align AI with human values rather than simply prioritizing user appeasement.

The two distinct themes of user expectations are in sync with the two fundamental dimensions of social perception—competence and warmth—commonly used to characterize other people (Fiske et al., 2002, 2007). The expectation of GAI chatbots as omniscient hidden laborers reflects users’ emphasis on the competence dimension, highlighting their assumptions about the chatbots’ intelligence, capability, and ability to perform tasks seamlessly in the background. Similarly, the expectation of GAI chatbots as human-like and creative aligns with the warmth dimension, emphasizing relational and emotional attributes such as approachability, empathy, and humanity. This finding corroborates previous studies, such as Hoffmann et al. (2020), which demonstrate that individuals tend to prefer warm and anthropomorphic language styles over colder, more impersonal ones.

How people construct folk theories (RQ2)

To address research question two on how folk theories of GAI are constructed, the FGD identified three key influences: imaginations of buzzwords, direct interaction with GAI chatbots, and comparisons with familiar concepts. Consistent with prior research, participants’ perceptions are shaped by media framing and social narratives as sources of buzzwords (Brauner et al., 2023). They often compare past and new experiences to guide their understanding (Siles et al., 2020) and frequently use metaphors to rationalize their interpretations (Lakoff and Johnson, 2008). It is important to note that most people don’t hold static but rather malleable folk theories, participants continuously updated their folk theories through ongoing interactions with GAI chatbots, as well as information gained from their social networks and personal imagination, as discussed in results part, participants adjusted their folk theories after debating with other participants during the FGD. These findings align with Devito et al.'s (2018) findings, which highlighting the dynamic and evolving nature of folk theories. Hence, this construction process is dynamic, interactive, and influenced by users’ unique approaches, though our current data does not fully explain why certain users rely more on specific methods. Future research could explore the factors influencing these preferences.

New social scripts (RQ3)

Another crucial finding from this research is that users develop new social scripts tailored to their understanding of GAI chatbots, moving beyond the automatic application of human-to-human social norms posited by CASA. While the CASA paradigm suggests that people apply human social heuristics to computers (Gilovich et al., 2002; Nass and Moon, 2000), we find that users adapt and redefine these scripts based on their folk theories of GAI. This aligns with Gambino et al. (2020), who argue that users do not simply apply human–human interaction norms but develop unique scripts for media entities based on their “knowledge and experiences with media agents” and the evolving nature of human–AI interactions (Gambino et al., 2020).

Participants in the FGD reported using more direct, commanding language with GAI chatbots, creating shorter, more structured interactions. This reflects a shift from human-like expectations to more practical, function-driven communication methods shaped by users’ folk theories. These new social scripts shape users’ interactions and the “imagined affordances” they attribute to GAI chatbots—meaning the potential actions users envision based on their expectations and prior experiences (Nagy and Neff, 2015).

These findings suggest that as users better understand chatbot capabilities, they adjust their communication to align with its perceived limitations and affordances. Interestingly, our findings suggest that some participants intuitively apply strategies akin to “prompt engineering,” which involve structured communication methods designed to optimize outputs from LLMs, such as “chain of thought prompting” and “role prompting,” without formal knowledge of the discipline (Gero et al., 2022; White et al., 2023).

Despite the study's insights, there are some limitations. Firstly, the sample consisted of participants with relatively high AI literacy, which may not represent the broader population. As Johnson and Gardner (2007) argue, CASA effects can be moderated by users’ literacy and prior experience with technology (Johnson and Gardner, 2007). Future research should include a more diverse sample with varying levels of experience to explore how different demographics construct their folk theories. Second, the inductive approach used in this study cannot fully verify the relationship between constructed folk theories and user behaviors. Future studies should adopt mixed methods or longitudinal approaches to further investigate this relationship.