ChatGPT for cultural heritage and the customization of generative AI: A talkthrough analysis of the Luigi Einaudi chatbot

Generative AI, LLMs and synthetic heritage

Despite their relatively recent introduction, generative AI has already been applied in a wide range of areas and sectors. Examples of its customization to specific areas and tasks include personalized versions of ChatGPT that users generate through tools available in the OpenAI platform so that they can address their individual needs (OpenAI, 2023), chatbots that are trained or prompted to impersonate historical characters (character.ai, 2024), versions of LLMs that are fine-tuned to be used for specific domains and tasks, such as academic research, healthcare or online searches (Maharjan et al., 2024), and the personalization of text-to-image models that makes it easier to produce specific types of images (Heaven, 2024). In this context, the cultural heritage sector has not been the exception. Institutions including museums, archives and libraries worldwide have made efforts to customize LLMs and other generative AI models as a pathway to facilitate the dissemination of knowledge and to introduce new modalities of access to cultural heritage that enhance users’ engagement. In particular, the customization of LLM-based chatbots to create conversational agents that impersonate historical characters or introduce other pathways for the public to engage with cultural heritage is being attempted by a growing number of institutions worldwide. Examples include the Rosicrucian Egyptian Museum’s customization of Ameca, an LLM robot, into Thoth, the Egyptian god of writing, in San José, California (Rosicrucian Egyptian Museum, 2025); the Metropolitan Museum’s featuring of a chatbot impersonating socialite Natalie Potter as part of the exhibition ‘Sleeping Beauties: Reawakening Fashion’ in New York City (MET Museum, 2024); and the avatars of naturalists Alfred Russel Wallace and Michele Lessona included in the permanent exhibition at the Regional Museum of Natural Sciences of Torino, Italy (Piemonteinforma, 2024). Even before the generative AI turn, moreover, chatbots had been used as tour guides (Kopp et al. 2005), and gamification tools to increase museum entertainment (Gaia et al., 2019). Chatbots impersonating historical figures had also been explored. These included, for example, the Anne Frank chatbot, which was launched in 2017 by the museum dedicated to her in Amsterdam and could be asked questions about her life via Facebook Messenger (Tzouganatou, 2018).

Beyond the cultural heritage field, characterization has often been used to improve chatbots’ credibility and storytelling, but critics have highlighted issues including the use of gender representations and stereotypes (Chen, 2025), the potential for manipulation (Natale, 2021) and gamification dynamics (Ge and Hu, 2025). Moreover, outputs of generative AI can give an impression of high specificity, but at the same time have been shown to exhibit significant levels of homogeneity (Westberg and Kvåle, 2024), which might jeopardize the capacity of LLM-based chatbots to deliver authentically original or sophisticated responses.

The customization of generative AI models is not only regulated by technical affordances, but also by aspects that are specific to the domain of application, its professional culture and institutional environment (Parry, 2010), thereby raising a range of specific challenges and problems. A key issue in this context concerns the uneasy relationship between authenticity and historical reconstruction. There is a long trajectory of cultural heritage institutions such as museums navigating the boundaries between original and copies by reproducing physical artefacts of the past (Agar, 1998) or creating digital environments and objects that simulate actual spaces and artefacts (Arvanitis and Pavlovskyte, 2023). For what concerns generative AI, Nieto McAvoy and Kidd (2024) have recently proposed the notion of ‘synthetic heritage’ to describe AI-generated media that blend authentic historical elements with artificial enhancements. Synthetic heritage activates computational processing supported by the strong resemblance to human features (e.g. in deepfakes and speech synthesis technologies) and requires an examination of the ‘processes and the socio-technical infrastructures that underpin them, as well as their mnemonic impacts’ (Nieto McAvoy and Kidd 2024: 2). Normative aspects of AI algorithms work retroactively to reshape past information as it is encoded in a defined knowledge and aligns with the categories in which the system or database is organized (Jacobsen and Beer, 2021). Such dynamics are informed by the extractive approach through which tech corporations such as OpenAI and Google employ historical information and materials (Grohmann, 2025). As observed by Schuh (2024), LLMs leverage fragments of the past – such as the texts that are used to train and fine-tune the models – to position themselves as primary mediators of information about history and heritage; in Schuh’s words, they are ‘agents that synthesise and reconfigure information about the past, interacting with humans (and other models and algorithms) to produce new interpretations and access to these traces’ (Schuh, 2024: 233). Scholars have argued that ChatGPT, for example, produces ‘answers that are pleasing to the largest possible group of users’, privileging hegemonic readings of the past: ‘the more people like what the system produces, the better it becomes at predicting and providing what most people will like as an answer’ (Smit et al., 2024: 19). Critics, indeed, underlined how the synthetic production of culture sparked by generative AI reproduces normative identities and narratives (Gillespie, 2024) and homogeneous representations (Laba, 2024), with potential repercussions on the diversity of perspectives through which the past is narrated and interrogated.

The customization of generative AI for cultural heritage also forces practitioners in this field to confront the problem of reliability and historical accuracy. This is related to the fact that the internal processes of generative AI are not transparent and can only be verified through inputs and outputs (Lieto, 2021). This opacity impairs the reliability of the systems, as it is difficult to trace back the sources that determined a result (Bathaee, 2018). Moreover, LLM’s tendency to hallucinate, producing inaccurate information, contrasts with cultural heritage’s commitment to evidence-based historical research (Matei, 2024). The problem is worsened by the fact that, as some have noted (e.g. Hicks et al., 2024), hallucinations are not a bug but an inherent feature of these systems, since LLMs are programmed to generate outputs that diverge from the data used to train them. While this may enhance the models’ creative potential (Celis Bueno et al., 2024), it also makes them more unreliable. Another potential issue derives from the fact that LLMs are usually designed to be as helpful as possible to users (Startari, 2025; Zou et al., 2024), to the point that the companies developing the models have been criticized for eliciting a form of ‘AI flattery’ (Piper, 2025). The effects of these characteristics on customized versions of chatbots that produce historical narratives are difficult to establish. Synthetic memory, therefore, poses significant challenges to established understandings of historical accuracy, knowledge and truth (Munn et al., 2024).

The extent to which these problems can be overcome is under discussion, posing significant questions to the cultural heritage field as generative AI models are increasingly applied with the expectation that they can enhance public engagement and access (Tiribelli et al., 2024). As Siffels and Sharon (2024) note, there is the risk that the availability of new technologies, such as generative AI, presses stakeholders towards adoption, while the problems that the technology should address remain badly defined and constructed. Exactly like institutions and groups in other sectors, cultural heritage institutions are vulnerable to technosolutionism, defined as the tendency to propose technology-based fixes for complex social, political and human problems in ways that oversimplify or ignore the deeper structural causes of those issues (Morozov, 2013). Technosolutionism also bears the risk of propagating Western assumptions and ideologies, subtracting space and potential for innovation emerging from diverse cultural contexts (Mühlhoff, 2025; Natale et al., 2025). There is a need, therefore, to interrogate what practices can be mobilized in the cultural sector to engage productively with these technologies, and what practices cannot (Thiel and Bernhardt, 2024). In this context, critical analyses of cases such as the analysis of the Einaudi chatbot conducted here can be of crucial importance to help identify potential benefits as well as the many challenges and risks raised by the application of generative AI in the cultural heritage sector.

Customization of LLMs: Fine-tuning, alignment, MyGPTs

Although customization practices are important also for other AI models, such as text-to-image models (Ferrari and McKelvey, 2023), our research examines the case of LLMs. An LLM such as OpenAI’s GPT can be fine-tuned and adapted so that it can complete very different tasks, for instance as a professional tool such as in ChatGPT, but also as a companion chatbot such as Replika (Depounti et al., 2023), an educational tool such as Duolingo Max (Duolingo, 2024), or as an avatar that helps explore the life and works of a notable character, as in the case of the Einaudi chatbot examined here. In this section, we provide a high-level explanation of the training process behind state-of-the-art LLMs and the different prompting mechanisms that can be adopted when using them, focusing on the example of GPT. This will clarify some terminology and allow us to showcase the technical approaches that can be employed for the customization of this technology in the cultural heritage sector.

Since the development of GPT-3 (Brown, 2020), an astonishing amount of research has been done in order to improve the training process for LLMs. These efforts resulted in a plethora of adjustments and variations, many of which are partially kept secret due to the commercial nature of the most successful LLMs. Nevertheless, a common pipeline has been established for training LLMs and is composed of three main steps. First, in the Pre-Training (self-supervised) phase, the model is subjected to vast amounts of text and is trained on the token prediction task, that is, the model learns the morphology, grammar, vocabulary and semantics of a language and is thus able to successfully generate proper sentences given a text sequence. Second, in the Instruction Tuning (also called Supervised Fine-Tuning) phase, the model is further trained on specific datasets for the resolution of specific tasks such as Q&A, summarization, translation and so on. In this phase, the model learns to follow instructions, understand them and retrieve knowledge based on them. These instructions can be either prepared by humans (like what OpenAI did with ChatGPT, see Ouyang et al., 2022) or filtered from Internet data (Rae et al., 2021). Third, in the Alignment phase, the model learns to reply with what are considered the best answers. Dialogue is dynamic and given a single prompt there are many possible answers that a model could produce. Purpose of this phase is to align the model towards the best answers. The notion of ‘answer A is better than answer B’ is given by a second model called reward model. There are a few approaches that can be adopted to implement this alignment phase, the most common are Reinforcement Learning with Human Feedback (Bai et al., 2022) and Proximal Policy Optimization (Schulman et al., 2017). The alignment of the model is also used to implement white-box safeguard techniques (Brown, 2020) in order to reduce the generation of harmful or inappropriate content. Sometimes the last two steps are referred to as a general ‘fine-tuning’ phase.

The training process changes the internal representation of the model, and so it constitutes an aspect that can be taken in consideration when attempting technocultural customization. However, fine-tuning an LLM is a costly approach that requires in-depth expertise and vast amounts of high-quality data. Therefore, the actors that perform techno-cultural customization work may opt for ‘lighter’ approaches that help prepare the models for more specific tasks and culturally-situated contexts. In particular, instead of changing the data in the training set, different prompting strategies can be adopted in order to achieve techno-cultural customization. These include Retrieval-Augmented Generation techniques (Gao et al., 2023) that improve LLMs responses by grounding the model on external sources of knowledge to supplement the LLM’s internal representation of information. The process involves transforming the external data into a representation that can be queried; common approaches include embedded vector spaces. Then, every user prompt is converted into a vector that can be matched against the external data vector space: in this way, only the relevant information is extracted from the knowledge base. This information is then added to the prompt so that the user request towards the LLM is actually augmented via external and reliable data. Another common strategy is few-shot prompting, by which the behaviour of LLMs can be partially steered by prepending a prompt with examples of the desired output. By instructing the LLM on the type of output expected, techno-cultural customization can be achieved, however, this method is not always effective and is very sensitive to attacks – that is, it is relatively simple for a user to bypass the adaptation (Shen et al., 2024).

This set of approaches is often made simpler by commercial LLMs compared to self-hosted solutions. For instance, OpenAI offers ‘MyGPTs’ (OpenAI, 2023) and ‘Assistant APIs’ that allow for the simple implementation of RAGs and few-shot prompting. Alternatively, OpenAI also offers APIs for the fine-tuning of their models. It is worth mentioning that the availability of these tools for users to customize the models signals the increasing platformization of generative AI, by which users and organizations are invited to experiment and personalize the product, while remaining however within the context of the infrastructures owned and dominated by big platform corporations (Nieborg and Poell, 2018).

As a final note, we want to point out that customizing an LLM can be particularly challenging due to the fact that all LLMs models are already culturally adapted in some way. The training set instils certain cultural traits that are furtherly adapted by the companies that release the LLMs. This ‘default’ adaptation is often opaque and as such it poses a complication for those who are interested in furtherly adapting the model. Metaphorically, in adapting an LLM we do not start from a blank canvas, but rather from a blurred image that is only partially understandable. Finally, a commercial LLM could simply not be entirely adaptable to specific needs. For instance, due to the fact that commercial LLMs such as GPT have been programmed to avoid potentially problematic topics, adapting an LLM to impersonate a controversial historical character may prove difficult if not impossible, as the original model might have been trained to avoid controversial (yet, historically accurate) sentences and ideas.

‘Luigi Einaudi, digital human’: the context and environment of use

A prominent intellectual and politician, Einaudi was the first president of the Italian republic elected by the Italian parliament after World War II, serving in this authority from 1948 to 1953. An influential intellectual and expert in economic theory, he was a critic of fascist economic policies and was forced to exile in Switzerland due to its stance. His presidency was marked by a pragmatic, non-partisan approach, prioritizing economic growth and democracy (Mariuzzo, 2016). The Einaudi Foundation was established in Turin, Italy in 1964, 3 years after his death, with the mission of conserving the collection initially donated by the family of Luigi Einaudi, of collecting further archival materials about Einaudi, of maintaining Einaudi’s legacy alive, and of promoting reflection, research and debates about Einaudi as a historical figure and more broadly on socio-economic, historical and political studies (Fondazione Einaudi, 2024a). The Foundation is now a lively cultural institution at national and international level, physically located in the historical Palazzo d'Azeglio in Turin, Italy. Its library and archives are open to the public and a range of archival materials, including Einaudi’s writings and papers, are searchable and made available through an online database. The Foundation’s activities include the funding of postdoctoral fellowships and scholarships, the organization of cultural and educational events and the publication of books and scientific journals.

The project of creating an avatar of Luigi Einaudi was developed by the Einaudi Foundation in collaboration with Reply, a multinational company offering consultancy services on digital solutions ranging from AI to virtual reality, cybersecurity and UX. The project’s objective, as stated in the Foundation’s website, was to allow members of the public to discover Einaudi’s liberal thought through an experience of dialogue attuned to contemporary times (Fondazione Einaudi, 2023a). Historian Paolo Soddu, moreover, mentioned his and the Foundation’s interest in experimenting with the creation of new pathways to access historical heritage, which could in the future complement more traditional pathways for study and research, such as archival research (Interview with Paolo Soddu, 19 June 2024). The project, therefore, was not only intended as a divulgation effort, but also as part of the wider mission to make the writings and thought of Einaudi accessible to the Foundation’s key public, that is, researchers and students.

As described by one of the project’s technical developers (Interview with Federico Pugliese, 9 July 2024), the creation of the chatbot entailed two main steps. The first step aimed to recreate a model of the ‘mind’ of Einaudi through the implementation of LLM technologies. A body of texts, which included around 500,000 words composed of Einaudi’s writings and a biography redacted by Soddu, was selected and made available to the model. The Retrieval-Augmented Generation (RAG) paradigm was implemented, directing the model to the most relevant parts of the text through the use of knowledge graphs that organized content in the corpus. The selection of portions of text relevant to specific themes and keywords was conducted manually by Soddu as well as through automation, that is, by asking ChatGPT to extract knowledge graphs from the texts. The developers, moreover, used prompts to adapt and improve the chatbot’s performances. This meant that customizing the chatbot did not require training the model with additional data, but the application of appropriate prompt design principles. Prompts were instrumental to giving indication on aspects such as the avatar’s speaking style, which needed to be accessible to a wide potential public but also to remain faithful to Einaudi’s intellectual stature, as well as the scope of the conversations that users would conduct with the chatbot. It was deemed important, in fact, to limit the potential subjects mainly to political and cultural themes, responding to the Foundation’s mission and to Einaudi’s reserved demeanour, as the former president would not easily share elements about his private life.

The second step involved creating the visual appearance, movements, voice and movements for the chatbot to mimic Einaudi. To this end, they used MetaHuman Creator, an interactive 3D environment developed as part of gaming company Epic Games’s Unreal Engine to provide users with user-friendly tools to create faces, clothing and gestures of digital characters with much detail. The Metahuman Creator engine provided the ground for developing a visually rich simulation of the character, complementing and framing the avatar’s linguistic ability made possible by LLMs.

The communication campaign through which the chatbot was launched underlined the Foundation’s successful efforts to align the AI model with the historical figure of Einaudi, as well as AI’s growing role in producing knowledge and making it available to large potential public in new, engaging ways (Fondazione Luigi Einaudi Onlus, 2023b). In our conversation, Paolo Soddu also mentioned the ambition to further develop the chatbot in ways that expand its synergy with more traditional paths to access cultural heritage, for instance by detailing relevant sources and linking to the online database of the Foundation’s archives and library for further research (Interview with Paolo Soddu, 19 June 2024).

It is worth mentioning, moreover, that external communications about the project described the avatar as a ‘digital human’ (Fondazione Luigi Einaudi Onlus, 2023b). Scholars have stressed the deep implications of choosing specific terms and metaphors to describe digital media and AI (Wyatt, 2021). The use of ‘digital human’ replicates the terminology adopted by Unreal Engine’s Metahuman Creator, which promises to create ‘Digital Humans’ by offering a visually compelling representation that can be animated and made even more realistic through computational tools and digital storytelling (Unreal Engine, 2024). Others, however, have opted for concepts such as digital characters or persona (Bucher, 2014), which stress more explicitly the cultural work involved in creating the avatar.

‘Talk to Einaudi’: entering the web application

The pathways through which users first access an app or a software are instrumental to set up expectations and rules of interaction and are therefore given specific attention in the walkthrough method (Light et al., 2018). When entering the website of the Einaudi Foundation, visitors find the option to access the Einaudi chatbot in the top menu, at the same hierarchical level of the main headings that introduce the site’s key resources and pages. The interface offers specific affordances that suggest how the user can interact with the virtual environment (Gibson, 2014; Norman, 2021) and that are aimed at human-like conversation through natural language processing. They echo existing approaches in chatbot interface (McTear et al., 2016; Mygland et al., 2021) by distilling information to facilitate user comprehension of large amounts of data and by enriching information through the visual appearance of the avatar. The invitation to ‘Talk to Einaudi’, without any specification of what this entails, recalls similar prompts used for the marketing of voice assistants. For instance, early communication promoting Apple’s voice assistant Siri invited users to ‘talk to Siri’, thereby naturalizing what was at the time an unfamiliar experience for the vast majority of people, that is, communicating with a machine through voice (MacArthur, 2014). Users are thus encouraged to take up at face value the possibility of holding a conversation with a piece of software – which, in this case, personifies a deceased historical figure. The same approach continues after the link is selected, when a loading spinner appears and the following text is visualized: ‘Loading in Progress. Just a moment longer, Luigi Einaudi will be here shortly’. The introduction of the character invites users to summon a character that despite not being human can be imagined and interacted with as one.

The humanization of the chatbot is promoted by visual and behavioural realism (Kang and Watt, 2013), where the former describes the resemblance to human features (i.e. body appearance) and the latter the non-verbal expressiveness (i.e. characters' gestures). Einaudi’s ‘digital human’ is depicted from the waist up, dressed in a jacket and tie, rendered in 3D. The chatbot’s appearance is faithful to photographs of the original character. He has a neutral expression, as if waiting, and moves in a loop composed by small head swings and a slightly vacant gaze. The only gesture that significantly differs from this trend is Einaudi adjusting his glasses by bowing his head (Figure 1). This recalls the so-called ‘idle animations’ that identify those actions made by video game characters while waiting for player input (Cooper, 2019, p. 133). Idle animations lend naturalness to the character; at the same time, they can imbue personality to the avatar and therefore serve as an indicator of the relevance attributed to its representation regardless of the user's actions. In the case of Einaudi, its visual and behavioural realism replicate a stereotypical connotation of an intellectual, thereby absolving a characterization function. The realistic portrayal overcomes the ‘uncanny valley effect’ – that is, the eeriness of relating to a character that is similar but not identical to a human – so that the anthropomorphization of the bot increases the user's engagement with the historical figure (Schuetzler et al., 2020).

OPEN IN VIEWER

Figure 1.

Animations in the visual interface of the Einaudi chatbot. Screenshot taken by the authors from https://www.fondazioneeinaudi.it/parla-con-einaudi.

The stereotypical representation persists also in the background, where a bookshelf with old-looking books completes a minimal diegetic contextualization. These elements, apparently scenic, constitute specific visual affordances that invite towards a precise type of use, namely conversation on intellectual matters. This impression is strengthened by the avatar’s formal dress, which signals its official standing. The application thus preconfigures, in this opening page, a pragmatic model of introduction through visual cues (i.e. the background and the clothing) that have normative implications (i.e. the type of behaviour that is usually to be maintained in such situations and environments). These elements do not add significant values to the interactive options, as users can only observe the avatar and the environment without any room for exploration.

While Einaudi’s face occupies the right half of the screen, in a dedicated box on the left, a range of textual instructions are provided. These offer a number of suggestions and precautions that users should consider when engaging in conversation with the chatbot. In the first sentence, the chatbot’s nature and expected use is clarified: ‘You are talking to an artificial intelligence that has been trained on the basis of Luigi Einaudi’s economics writings. It can only discuss topics found in these texts: monopoly, competition, monetary policy, currency, fiscal policy, budget, the market, banking, taxes, and inflation’ (Fondazione Einaudi, 2024b). Users are advised ‘to keep the conversation related to these topics’ and asked ‘to follow some rules’ such as using either Italian or English and avoiding offensive or inappropriate language. Users are also asked to abstain from sharing personal data, pointing to the risks associated with their improper dissemination. It is worth noting that, at this point, the instructions indicate that the data entered are ‘provided to Luigi Einaudi’, while at the beginning, it was made clear that the user is talking to an artificial intelligence, and in the final lines of the instructions, the users are told that ‘The answers you will receive are provided based on the information in Luigi Einaudi’s texts and make no claim to exhaustiveness’ (Fondazione Einaudi, 2024b). This shows a fundamental oscillation between the invitation to ‘Talk to Einaudi’ in the website’s menu, which encourages users to play along with the illusion generated by the talking software (Natale, 2021), and a more detached, scientific viewpoint that illustrates how the chatbot’s outputs are generated. Even though users are invited to talk to the chatbot as if it would be a real person, the presence of AI is made explicit, acknowledging not only the potentials but also the limitations of the system.

The talkthrough: chatting with Einaudi

The first phase of the talkthrough entails familiarizing with the chatbot through free conversations – exactly like for the walkthrough, researchers need first to familiarize with the functionality of the app or software under exam (Light et al., 2018). In this phase, we gained insights about the general functioning of the chatbot, its conversational style and interactive model. In terms of functioning, users have two options to communicate with the chatbot: typing or recording their voice, which is then transcribed by the system and appears in the chat log. The chatbot’s outputs are also provided as text as well as voice, generated through a speech synthesis software. The chatbot’s voice appears neutral, carrying no indication of emotional participation from the avatar. After each response by the chatbot, users can type their own prompts or select from a short list of automatically generated questions on topics related to economic and societal issues, which are provided as suggestions.

The chatbot’s tone is formal and intellectual. It maintains a cordial intonation and provides exhaustive answers, expanding the topic if necessary, addressing the user's prompts, or justifying and explaining its shortcomings. Overall, the chatbot’s conversational style remains anchored to the scholarly texts used to customize the chatbot and, even more clearly, to the language style of the original GPT model. Its lengthy replies and formal style may remind users of the experience of reading a Wikipedia entry, more than of frictionless dialogue.

The interactive mode revolves around providing not just information but insights into Einaudi’s potential viewpoints, thus activating a hypothetical conditional mode, that is, what Einaudi would have thought of a certain issue or problem. By default, the conversation mainly advances with the users asking questions and the chatbot providing answers; however, if the user expresses an opinion, the chatbot may express agreement or disagreement with the user’s viewpoints. When pertinent questions are asked, the chatbot’s replies are usually accompanied by two buttons, the first leading to further insight on the topic, and the second to explore the sources on which its response is based, including bibliographic information of texts by or about Einaudi.

While the first phase was not driven by specific research questions, the second phase started with the preliminary identification of specific themes, issues or problems that the researchers may explore. In fact, the open character of chatbots’ conversational capabilities makes it impossible to sustain a systematic and at the same time comprehensive explorations of all possible themes and responses. To gain meaningful findings, the talkthrough needs therefore to proceed from specific objectives or themes, which we call Talking Points (TPs). In practical terms, each TP corresponds to the use of a range of prompts that test specific aspects of the conversational agent’s functioning.

GPT for cultural heritage

The second TP relates to the cultural heritage dimension of the Einaudi chatbot project. As discussed in the analysis of the context and environment of use, the project moved from an ambitious vision by which the Einaudi Foundation aimed to experiment with the potential as well as the limitations of applying LLMs to cultural heritage, thereby exploring how the technology could in the future complement more traditional pathways for study and research, including archival research. In this TP, we used the talkthrough to question what the chatbot can teach us in this regard.

One of the key technical problems of LLMs is related to the fact that they are unable to provide reference to the sources of data analyzed in the Pre-Training phase. In practical terms, this means that they cannot direct users to the sources from which they gathered information during training (Park and Choi, 2024). The customization developed by the Einaudi Foundation and Reply found a way around this technical limitation by applying a well-known strategy by which data the LLMs’ intrinsic knowledge is complemented with data from external databases (Gao et al., 2023). Since the model is not directly trained on this data, it can be instructed to provide information on the sources from which it has extracted the information. In the Einaudi chatbot, an option is provided at the end of most responses for the user to access original sources, namely writings by Luigi Einaudi (Figure 2). Due to the relatively small corpus of documents employed for the techno-cultural customization of the LLM, the sources provided are relevant but not always specific to the actual responses given by the chatbot. For example, in one interaction, the chatbot commented on the impact of high inflation on electoral results; however, while inflation was mentioned in some of the sources listed by the chatbot, these did not directly address the relationship between inflation and political orientation. Moreover, whenever responses were not relevant to the key topics explored by Einaudi in his work, sources were not included. This was due in most cases to the fact that the information on which such responses were based originated from the model’s intrinsic knowledge. Sources, for instance, were provided as a response to queries including the Cold War, Italian fascism and inflation, while queries on topics such as children, milk, Trump’s election and nuclear energy did not result in the production of a list of sources. In other cases, it appeared to be due to decisions taken during the process of customization. For instance, the system did not include information about his private life because, as the chatbot stated, ‘I am an artificial intelligence programmed to reply as if I was Luigi Einaudi’.

OPEN IN VIEWER

Figure 2.

The interface of the Einaudi chatbot includes an option to view and access original sources, such as Einaudi's writings. Screenshot taken by the authors from https://www.fondazioneeinaudi.it/parla-con-einaudi.

Another significant dimension related to cultural heritage is reliability. A well-known problem of LLMs is their tendency to hallucinate and even spread disinformation (Hicks et al., 2024). In the talkthrough, the Einaudi chatbot proved relatively successful in minimizing this problem, although this was achieved by severely limiting the range of queries with which the chatbot would engage. Some issues, however, emerged in the relationships between the chatbot’s responses and the contents of the indicated sources, and concerns the difficulty to complement the model’s intrinsic knowledge with the use of external sources. Since the corpus of 500,000 words used to achieve the techno-cultural customization of the Einaudi chatbot could not cover all possible conversational themes, when asked about issues that are relevant to its remits (e.g. political and economic topics) but were not directly discussed in Einaudi’s original works, the chatbot activated intrinsic knowledge that may diverge and even contradict the contents of Einaudi’s writings. Given that the chatbot affirms to provide answers based on Einaudi’s views as delineated by his writings, this difficulty highlights a key vulnerability for the application of LLMs for cultural heritage. This is particularly important if one considers how important for cultural heritage institutions is accuracy in the use of sources (Copplestone, 2017). This problem, moreover, can be exacerbated by the impression of authority constructed by the chatbot’s replies and its conversational style. Considering how research in social psychology has shown that appearance of authority and reliability can be achieved through conversational cues (Edwards, 2012), the Einaudi chatbot’s formal, intellectual stance may lead users to overestimate its knowledge and accuracy.