Move cultural heritage knowledge graphs in everyone’s pocket

Based on the analysis described in Section 3 and the overviewed KG aspects and issues, we identified the following challenges that must be faced in designing VA extensions to enable KGQA.

Label retrieval. According to LOD principles [48], every resource must be referred to by a URI. Moreover, KG curators are encouraged to specify human-readable labels to make these URIs understandable to humans. It is crucial to make them callable by VA-based data exploitation tools. To easily configure systems able to query KGs automatically, it is required to exploit a uniform (and standard) property to attach human-readable labels to resources. Most of the KGs attach labels to resources by standards properties, such as rdfs:label, skos:prefLabel or foaf:name. However, some KGs use domain-specific and custom label properties (e.g., EventMedia uses rnews:headline), which makes the label retrieval step even more challenging.

Label coverage. Developers have to carefully check the percentage of resources provided with labels (a.k.a. coverage) to avoid losing a high rate of data by retrieving only URIs attached to human-readable labels.

Label readability. If labels contain codes (e.g., in HU_museum) or are wrongly formatted (e.g., labels are in camel notation, such as hasDate, hasUnit, shipType in NL_maritime), it is hard to recognise the desired resources when pronounced by humans.

Multilingualism. Language support is a desirable property. However, in many cases, labels are defined in just one language. It limits the use and exploitation of available sources.

Label ambiguity. If the same label is attached to several resources, it implies an ambiguous reference to a source of interest. For instance, if Apple is both used for the company and the fruit, it will be up to the VA back-end to solve the pronounced label. While it simplifies the question formulation by the user, it undermines the determinism of the question interpretation. A good trade-off must be detected to maintain the interaction as simple as possible without limiting user control of the desired resources.

Linking approach. To determine the URI corresponding to the pronounced label, developers can rely on i) APIs implemented by the KGs (such as Europeana provides search mechanism), ii) named entity resolution (NER) tools to solve entities (and properties), iii) define a dictionary to maintain a list of URIs for each label of interest, or iv) a combination of them. It affects the complexity, reliability, and size of the back-end. While the dictionary guarantees complete control of the entity and property resolution, it requires developer effort and highly affects the back-end size. As pointed out in the analysis described in Section 3, few CH KGs are provided with APIs. Concerning NER, it is a general solution to solve entity labels, but i) it rarely works on properties, ii) it is hard to configure NER tools to work on KGs different from the one they are developed for, and iii) it strongly affects the reliability of the VA extension under the definition.

SPARQL support. If developers choose to query a SPARQL endpoint or exploit dedicated APIs directly, they have to check SPARQL operators’ support and coverage in defining the mapping between NL and SPARQL queries in the QA tools.

Running time. Requests execution time strongly affects data exploitation tool performance.

Results limit. Results limit posed by KG services must be carefully checked since a low limit can compromise the completeness of the queries and require performing several queries exploiting the OFFSET operator to have a complete reply.

4.2. Principles and methodology

This section describes the proposed approach to design and implement a VA extension to enable KGQA by focusing on Amazon Alexa as a VA provider. It details the introduced concepts related to Alexa skills and the proposed implementation of a KGQA VA extension. It is not a loss of generality since it can be easily adapted to any other VA that enables custom VA extension definition, such as Google Assistant, or in bot implemented by Microsoft Azure Bot Service or Googlebot. We opt for Alexa instead of plausible alternatives as Amazon Alexa holds the provider’s record with the greatest number of sold devices. However, the architecture of the generator leads to easy integration of novel VA providers, such as Google Assistant, that is actually under integration.

Amazon Alexa skills Functionalities in Alexa are called skills. Among the supported types of Alexa skills, we are interested in custom Alexa skills, where we can define the requests the Alexa skill can handle (intents) and the words users say to invoke those requests (utterances) [11]. An Alexa skill developer has to define a set of intents that represent actions that users can do with the resulting VA extension; a collection of sample utterances that specify the words and phrases users can use to invoke the supported intents; an invocation name that identifies and wake-ups the resulting Alexa skill; a cloud-based service that accepts and fulfils these intents. Mapping utterances to intents defines the Alexa skill interaction model. Utterances can contain slots, i.e., variables bound by users when formulating their requests, that can be validated by attaching to each slot a list of valid options during the interaction model definition. The back-end code can be either an AWS Lambda function or a web service. An AWS Lambda (an Amazon Web Services offering) is a service that lets run code in the cloud without managing servers. When the user poses a question, Alexa recognises the activated intent and communicates both the recognised and slot(s) values to the back-end code. Then, the back-end can perform any necessary action to collect results and elaborate a reply [11].

Virtual assistants for question-answering We model each supported SPARQL query template as an intent. The implemented intents (listed in Table 7) are tailored towards SPARQL constructs, and they mainly cover questions related to a single triple enhanced by the refinement of the subject or object class. More in detail, we cover SELECT and ASK queries, class specification, numeric filters, order by to get the superlative and path traversal. Table 7 reports, for each intent, an exemplary NL query that activates the intent, the intent name, an utterance by specifying slots among braces, and the related SPARQL triples. In defining utterances, we separate the supported SPARQL patterns to enable users to assess the query correctness generated out of their input. We also avoid utterance overlapping to ensure, as much as possible, a deterministic intent activation.

When the end-user poses a question, Alexa identifies the activated intent and notifies the back-end by communicating both the activated intent and the slot(s) values. For instance, in the CH use case reported in Fig. 5, users ask for Mona Lisa’s painter. The VA recognises that it corresponds to the getProperty Object intent with utterance what/who is the {property} of {entity}, painter as property slot, and Mona Lisa as entity slot.

OPEN IN VIEWER

Fig. 5.

It is a graphical representation of the virtual assistant extension components where the yellow components are knowledge graph dependent and the VA extension in action in a cultural heritage use case by querying DBpedia.

Consequently, the entity and relation linking phase must be performed. It is worth noting that the performed task is a simplified version of the more general entity and relation linking problem. Entity linking is generally referred to as identifying in a text snippet entities and matching these to the corresponding KG entity. For instance, mapping in the question Who is the wife of the mayor of Rome? the textual evidence of Rome has to be isolated first, and then it can be mapped to the corresponding KG entity. In our case, named entity textual evidence is already detected by VAs, and we have only to map the named entity textual evidence to a KG node (like Rome to the node in the graph representing the city of Rome). To perform this (simplified) linking phase, an alternative is performing a dictionary lookup. In such a case, we store the mapping label URIs in a dictionary by querying KG classes, predicates, and resources URIs and the corresponding labels. The VA extension back-end exploits the dictionary to retrieve the URI(s) corresponding to NL labels. Resolved entities and predicates are used to complete the SPARQL template. We attach to each intent a different SPARQL query template. Consequently, any NL query posed by end-users is matched to the corresponding intent (according to the VA interaction model), and each intent corresponds to a SPARQL query template (according to our approach). Readers can reconstruct the complete SPARQL query corresponding to each intent by proceeding as follows: introducing the SPARQL triple(s) reported in Table 7 with the SELECT operator and appending the optional request of the label attached to the variable of interest. For instance, the triple <e><p>? corresponds to the SPARQL query SELECT DISTINCT ? ?label WHERE{SPARQL triple } OPTIONAL { ? <label>?label. FILTER(LANG(?label)="en")} (supposing that the VA extension language is English). The notation <e> means that the triple is completed by URIs attached to the label e in the dictionary. Once the query has been formulated, it can be posed to the SPARQL endpoint. We opt for running a GET query on the SPARQL endpoint and by asking for results in the JSON format. Once results are returned, the back-end formulates them as an NL reply. We attach to each intent a reply template. The back-end completes it with the resolved entities and retrieved results. The complete reply, i.e., the reply that includes the resolved entities, enables the end-users to inspect how the system interpreted the performed question implicitly. For instance, in the CH use case in Fig. 5, the end-user acknowledges that the painter word has been interpreted as author. It behaves as a step forward in the direction of the explainability of the application back-end logic.

The proposed approach queries KGs in real-time by exploiting up-to-date data and it is entirely KG-independent. Figure 5 makes evident components that must be reconfigured based on the KG of interest and which components can be left unchanged. It is also a general-purpose approach and it can be easily adapted to domain-specific applications (see Section 6). Although, the performance of the implemented approach highly depends on the queried KG. More in detail, the quality of the replies is up to the label coverage; the execution time is up to the endpoint settings; the completeness of the reply depends on the endpoint results limit (if any); the lack of control in accessed URIs is due to the label ambiguity.

As a general process, utterances make no assumption on question interpretation and the application context. The covered SPARQL patterns contain at most three triples. We aim to extend the supported SPARQL patterns by implementing more complex queries. In particular, we are reasoning on iterative queries by consecutive query refinements conversation-based. It enables end-users to iteratively refine their questions, for instance, by applying filters consecutively.

The proposed approach is general enough to be exploited both in querying a single KG and multiple KGs by aggregating query results in the reply formulation step, which means improving the back-end implementation without modifying the general approach. At the moment, the generator can be configured to query a single KG at a time. However, we aim to investigate further how to query multiple KGs.

6.1. Tangible movable category: MMM

MMM [39] is a semantic portal for finding and studying pre-modern manuscripts and their movements, based on linked collections of the Schoenberg Institute for Manuscript Studies, the Bodleian Libraries, and the Institute for Research and History of Texts. In particular, it models physical manuscript objects, the intellectual content of manuscripts, events, places, and people and institutions (referred to as actors) related to manuscripts.

OPEN IN VIEWER

Fig. 7.

MMM use case for the tangible category related to the movable sub-category.

Configuration. We automatically configured the MMM Alexa skill by exploiting the generator configuration component. The returned configuration file is directly used to initialise the generator.

VA extension in action. Fig. 7 reports a simulation of the interaction between humans and the MMM VA extension. We ask for databases aggregated by the MMM portal by posing the How many databases are there? question. Used resources are i) Bibale (which stands for Bib[liothèque médiév]ale), a long-term project of the Codicological Section of the IRHT (The Institute for Research and History of Texts) in Paris; ii) Bodley, i.e., Medieval Manuscripts in Oxford Libraries, and iii) SDBM, i.e., Schoenberg Database of Manuscripts. The user request How many databases are there? match an utterance attached to the getClassInstances intent, which returns the instances of a given class (database in this case). To verify the timeliness of retrieved information, we ask Which database has modified equals to 2020? which corresponds to an utterance matching the getNumericFilterByClass intent that verifies which instance of a given class (database in our use case) has a property (modified in our case) matching a given numerical value (2021 in our case). It replies to the CH community’s need to verify the queried sources and the timeliness of the retrieved information.

The Hungarian Museum [31] provides access to the Museum of Fine Arts Budapest data.

OPEN IN VIEWER

Fig. 8.

Hungarian museum use case for the tangible category related to the immovable sub-category.

Configuration. We manually configured the Hungarian museum Alexa skill by retrieving owl:class, used classes and triples subjects, and the used properties. Labels are rare and are mainly provided in Hungarian, without English translation.

VA extension in action. Figure 8 reports a simulation of the interaction between humans and the Hungarian museum VA extension. By querying What is the creation with the maximum value of participants? we activated the getSuperlative pattern which returns the class instance (creation in our case) corresponding to the maximum (or minimum) value of a given property (had participant in our use case). This scenario simulates the interest of CH lovers in retrieving information about artworks, paintings, sculptures.

The VA extension usually refers to resources by labels. In this case, it returns the creation URL (see the reply in Fig. 8). It makes evident the consequences of lack of labels attached to resources and the difficulties in exploiting them in VA-based applications.

DBTune classical [36] describes concepts and individuals related to the Western Classical Music canon. It includes information about composers, compositions, performers, and influence relationships.

OPEN IN VIEWER

Fig. 9.

DBTune classical use case for the intangible category.

Configuration. We automatically configured the DBTune classical Alexa skill by exploiting the generator configuration component. The returned configuration file is used to initialise the generator after applying basic configuration manipulation, such as identifying which relation can play the role of label predicate (alias5

VA extension in action. Figure 9 reports a simulation of interaction between humans and the DBTune classical VA extension. Who has Beethoven as influenced by? activates the getPropertySubject intent which retrieves the subject of triples where influenced by is the property and Beethoven is the object. This use case addresses the CH community interest in retrieving curiosities about musicians and artists.

Natural Features is part of Scotland’s official statistics [37] that gives access to statistical and geographic data about Scotland from various organisations. In particular, we are interested in aspects concerning geodiversity, ecology, and biodiversity.

OPEN IN VIEWER

Fig. 10.

Natural feature use case for the natural heritage category.

Configuration. We automatically configured the Natural Feature Alexa skill by exploiting the generator configuration component. The returned configuration file is directly used to initialise the generator.

VA extension in action. Figure 10 reports a simulation of interaction between humans and the Natural Feature VA extension. What is the relevance of terrestrial breeding birds? activates getPropertyObject intent which returns the value playing the object role in triples related to terrestrial breeding birds as subject and relevance as predicate. CH lovers and experts joining the user survey on the impact and potentialities of the proposed approach in the CH domain stress that VA extensions might be useful for educational scenarios. In fact, this use case simulates the possibility to deeper domain-specific information for familiarising with terminology or conduct researches.

The UNESCO Thesaurus [42] is a controlled and structured list of terms used in subject analysis and retrieval of documents and publications in education, culture, natural sciences, social and human sciences, communication, and information. Continuously enriched and updated, its multidisciplinary terminology reflects the evolution of UNESCO programs and activities. Like a thesaurus, it mainly provides access to synonyms and related concepts. It also partially behaves like a dictionary by providing term definitions.

Configuration. We manually configured the UNESCO Alexa skill by retrieving (4421) skos:Concept that defines all the thesaurus terms and the used properties. All the concepts are attached to a human-readable label (by skos:prefLabel), while we generate property labels by local names of URIs.

VA extension in action. Figure 11 reports a simulation of the interaction between humans and the UNESCO VA extension. We can ask for the term definitions, e.g., what is intangible cultural heritage? (see Fig. 11). It activates the getDescription intent, i.e., a special case of getPropertyObject where the property is bound to a relation modelling term description. The VA extension retrieves the description (configured as skos:scopeNote) attached to intangible CH, and it returns the term definition. We can also pose ask queries. As an example, Can you verify if intangible cultural heritage as folklore as narrower? activates the getTripleVerification pattern, which model ask queries that verify if the stated triple is modelled in the KG. It replies to the interest of the CH community to clarify and use domain-specific terms properly.

OPEN IN VIEWER

Fig. 11.

UNESCO use case for the thesaurus category.

We demonstrate most of the intents listed in Table 7 by the overviewed use cases. We verify that the proposed approach is general enough to query data concerning different categories of CH, from museums to manuscripts, from music to term definition. Moreover, we also experienced some issues related to aspects pointed out in the CH KG analysis (Section 3) and challenges described in Section 4. In the following, we summarise KG properties that affect VA-based KG exploitation.

Label coverage. To cope with the scarce provision of human-readable labels, they can be generated by local names of URIs, as we performed in UNESCO Thesaurus. This practice can be performed if resources have human-readable URLs. As evidenced in the Hungarian museum use case, the lack of label provision is an obstacle to resource understanding.

Multilingualism. Some KGs, such as Finland datasets, Hungarian museum, Cultura, only provide access to labels in the data provider’s native language without enriching resources with English translations. Lack of multilingualism prevents wider data exploitation.

SPARQL support. A technical detail must be stressed. Before implementing the intents to SPARQL queries mapping, developers must carefully check if the queried endpoint fully supports SPARQL or omits some patterns. For instance, to use alternative predicates, we exploited the VALUES pattern. It is not supported by some of the queried KGs, such as Munnin and CULTURA. It affects the back-end implementation or limits the endpoints that can be exploited by any KG exploitation mean. Moreover, there are endpoints, such as CIDOC-CRM and AAT, that do not support the COUNT aggregator. It affects queries as simple as How many artifacts are hosted in the Uffizi museum.

7.1. Performance of the proposed mechanism

It is relevant to assess the performance of the auto-generated VA extensions as a special case of KGQA over VA compared with systems categorised as traditional KGQA. This evaluation tests the accuracy and the precision of the auto-generated VA extension as an approach to verify to what extent the configuration affects the proposed assessment. It demonstrates that the generation of a VA extension in a single click already returns VA extensions as accurate as systems proposed in the literature evaluated on the same benchmark. Moreover, it also demonstrates that by tuning the generator configuration, end-users can significantly improve the accuracy and precision of the auto-generated VA extension.

This section assesses the usability of an auto-generated Alexa skill according to HETOR10

Participants and setting 5 delegates of the HETOR project joined the usability evaluation of the UNESCO Alexa skill generated by the proposed generator, corresponding to the one described in Section 6. The evaluation took place remotely due to the COVID-19. As the VA extension has not already been published on the Alexa store, we deployed the VA extension on the Alexa developer console and asked participants to interact with the textual interface. We behaved as moderators while asking participants to formulate questions, and we collected their thoughts and reactions.

Protocol The performed protocol follows:

Data collection At the end of the evaluation, the moderator asked for the fulfilment of a final questionnaire to evaluate i) users’ satisfaction based on a Standard Usability Survey (SUS [26]) and ii) their interest in using and proposing the tool by a Behavioural Intentions (BI) survey. The questions of the BI survey are i) “I will use this approach in the future”; ii) “I will recommend others to use the proposed approach.” and users can use a 5-point scale to reply. Moreover, the moderator annotates all the comments and observations raised during the evaluation.

Results The proposed approach achieved a SUS score of 80, close to the higher step, interpreted as a great appreciation of the proposed tool and the propensity to propose it to others. The latter result is verified by the BI survey which achieved a mean score of 4.6.

Besides the tasks explicitly assigned by the moderator, participants started posing queries on their own, asking for the generalisation of mosques and synagogues, the definition of amphitheatre, the generalisation of Catalan or Gothic, generalisation of painting and specialisation of fine arts.

Question templates. Users naturally posed questions according to specific templates as it is the traditional approach used to query Alexa and its VA extensions. However, it requires training to learn the supported templates. Participants inquired the moderator asking for the other supported patterns besides the ones tested in the UNESCO Alexa skill, and they were almost satisfied with the covered templates. In particular, they asked for further details on numerical filters, quantitative queries, and mechanisms to retrieve images. They perceived the query to retrieve object proprieties the easiest and the more natural one. Participants observed that tailoring utterances according to the target user is a crucial aspect. For instance, educational contexts may require simplifying the terminology and adopting a wider way to formulate similar questions. Participants suggested integrating the definition intent with utterances such as What is the meaning of X? We also discussed if a keyword-based search might result in a dirtier but quicker way to retrieve information. Further study in this direction should be performed to verify the expressiveness capability of a keyword-based querying mechanism.

Target age. The proposed mechanism is perceived as a powerful approach above all for young people that are more and more accustomed to query VAs to perform daily tasks. Participants observed that it also seems particularly compliant with very young learners, also in the pre-scholar phase, as vocal commands represent the unique approach they can use as they cannot already write commands. Similarly, this approach might be critical for learners with disabilities that prevent them from typing questions or adopting textual interfaces, which may be too difficult for blind people or people with a limited range of motion.

The role plaid by the data source. Queried data sources play a crucial role in the effectiveness of the resulting VA extension. For instance, the UNESCO Thesaurus is too generic for domain-specific questions, and CH experts also disagree with some of the reported definitions. As an example, they are surprised by the taxonomy proposed by UNESCO for the CH concept, as they expect the well-known taxonomy based on tangible, intangible and natural heritage. As data modelling impacts also the VA extension utterances, it is crucial to evaluate the naturalness of the resulting questions.

Application contexts. The HETOR group really appreciated the proposed approach as a way to provide learners with continuous support to master terminology about CH and become familiar with related concepts. Learners are less and less accustomed to consulting a dictionary to look for the right terminology. The proposed approach allows to familiarise and disambiguate terms and enrich the personal vocabulary. As in the described activities, this proposal has interesting implications for groups of works to retrieve thematic information and images.

Furthermore, the proposed mechanism seems particularly useful in guided tours to guarantee personalised interactions, guided by curiosities avoiding boring prepackaged presentations of artworks and points of interest. VA extensions as virtual guides can overcome the lack of interest in the entire exhibition and too detailed descriptions, lack of customisation in terms of tour duration, interests, and curiosities. It also solves the linguistic gap between visitors and personnel. It enables the possibility to perform tours to the desired speed with the chance to repeat unclear passages without bothering other visitors. If it may be an exciting alternative to audio guides already available in museums, it might be revolutionary for city tours to explore monuments or points of interest spread in a city or minor realities, such as small villages.

This section discusses the perceived impact and utility from the end-user perspective. We proposed an online survey to collect opinions and suggestions. We do not limit ourselves to experts in the CH field but also try to involve CH lovers to take their views into account. Moreover, it is worth noting that we do not limit this survey to experts in the field as we are assessing the perceived impact and utility from the end-users side, meaning that we need to collect opinions by interviewing potential users of the resulting VA extensions.

Participants and setting 86 people joined the online survey administered for one week, from September 15th to September 22nd, 2021. All the participants spontaneously joined the survey in an anonymous form. 73 people are (very) interested in CH by rating their interest at least as 4 out of 5. 24 of them are experts in CH by rating their expertise in CH at least as 4 out of 5. By looking at people who consider themselves experts in the CH, they have limited expertise in Computer Science, stressing that it is crucial to provide the CH community with tools that do not take their competencies in programming, query languages, and computer science for granted.

Data gathering and survey outline The survey has been administered in English and in Italian, and its content is described in Table 10 that reports questions, reply format, and the rationale behind each posed question. The survey is structured in three main sections: i) general information about participants’ expertise and interest in CH, the spread of VAs within the CH community interpreted as people that are either experts or interested in CH, alternative means used to query and explore CH; ii) the perceived utility in terms of application contexts, feeling in adopting the proposed approach instead of traditional CH exploitation means, the perceived impact achieved by spreading CH data by VAs, queries users are interested in to evaluate the intent coverage and to collect ways users naturally pose questions; iii) finally, general suggestions and comments as a free text.

Results This paragraph reports the most common replies and interesting considerations concerning the proposed approach in the field of CH.

Current exploitation means. More than half of the participants that assessed to be interested in CH, i.e., participants that rated their interest at least as 4 out of 5 (56%), query CH data by googling them (in 28 out of 43 cases). They rarely exploit websites dedicated to the CH of interest (in 2 out of 43 cases) or bibliographic sources (in 3 out of 43 cases). It is interesting to notice that in 10 out of 43 cases, CH lovers already exploit VAs to fulfil their curiosities.

The diffusion of VAs within CH lovers and experts is also confirmed by results reported in Table 11 that summarises the most used VA providers and the frequency of their usage. Most of the participants have their favourite VA and usually stick to it without experiencing multiple providers. If a single provider is chosen, Google Assistant appears to be the preferred one. VA usage is still limited to a few days a week, meaning that there are still barriers to the wide exploitation of VAs in this community and it requires overcoming scepticism, perhaps, leveraging on curiosity connected to novelty or demonstrating to the potential users about utility and potentialities.

If compared with traditional text-based interfaces, by asking for activities that can be performed only with VAs, participants recall the potentiality to use VAs with users with disabilities, such as blindness, or situations that impede the usage of a keyword to type questions. It seems to be particularly useful at school during teamwork. By asking for activities that can benefit from the usage of VAs, users underlined the advantage to pose questions rapidly, interactive consultation of sources, simplify lookup operations.

Application contexts. Considering the entire set of replies, independently from participants’ interest and expertise in CH, just in one case, a participant cannot see the potentialities of the proposed approach, while all the other ones selected at least a CH application context that might take advantage of VAs. We first asked participants to choose among a set of options, i.e., in libraries to help look up books, as a virtual guide in museums, and as a learning assistant at school. Figure 12 reports participants’ opinions, who seem to be convinced that our proposal is a promising approach as a virtual guide in museums.

OPEN IN VIEWER

Fig. 12.

Application contexts rated by survey participants.

Furthermore, we also asked users to think about any other application context that can take advantage of VAs. In 39 out of 67 cases, users see the potentiality to adopt the proposed approach as a virtual guide not only in a museum but to guide visitors while wandering in an unknown city, above all while visiting small villages, unconventional destinations, or cities with low population density and high cultural impact, dense of archaeological parks or churches. An interesting consideration has been proposed by more than one participant that assessed that the proposed approach is particularly useful when there is no possibility to type requests, for instance, while driving. Moreover, a user also suggested thinking about the exploitation of VAs in a virtual museum by simulating a real tour also in terms of a tour guide. In 14 out of 67 cases, users state that it might result in a promising individual learning tool at university to learn about terminology and clarify doubts while preparing for exams or scientific contributions, at school to disambiguate terms, at home to deeper knowledge and awareness about CH, for young learners to overcome limits posed by textual interfaces and to leveraging on their curiosity. Moreover, further considerations concern the inclusiveness of the proposed approach able to overcome disabilities related to limited usability of textual interfaces or blindness. Users also proposed VAs as support in superintendence offices, in archives to guide the lookup phase, as support in offices, and (surprisingly!) in hospitals.

Utility and impact. We explicitly asked users to assess the perceived impact of using VAs as a means to spread the culture, interest, and awareness about CH by a 5-Likert scale. Figure 13 graphically represents the perceived impact demonstrating that most of the people think that VAs have the potentiality to wider spread the interest about CH, possibly by leveraging of curiosity of the novelty or providing and immediate access to data of interest.

OPEN IN VIEWER

Fig. 13.

Perceived impact of the capability of VAs to spread the culture of CH.

Analysing topics users are generally interested in addressing by the posed questions, they inquire about point of interest’s curiosities and details, such as “Which is the history of monument x?”, “Which museums are located in x?”, “When x happened?”, “What is the aim of x?”, “Who is the author of x?”, “What is the historical context of x?”, “Where is x?”, “Which is the architectural style of x?”

Participants expected to use the proposed approach to plan a trip. Thus, besides collecting cultural information, users are also interested in collecting practical information about points of interest, such as the accessibility posing questions like “Which is the ticket for visiting x?”, “Does x support an all-inclusive ticket?”,“What is the opening time of x?”, “How do visitors rate place x?”, “What can I visit in city x?”, “Which are the most important artworks hosted in x?”, “Where can I visit x?”, “Are there events in x?”, ‘Which is the most important point of interest in x?”, “Which are the most famous x?”, “Where is the oldest surviving x of the world?”.

Some participants simulated a conversation with a thesaurus by clarifying terms and terminology. Moreover, they were interested in looking for details about the queried sources to assess their reliability or retrieving the list of sources containing information about a given topic. As an alternative, according to the school level and subject, learners might be interested in specific information, such as “Who is restoring x?”, “Who is curator of x?”, “Is x curated by UNESCO?”, “How was called x before date d?”, ‘How tall is x?”. Learners might be interested in the story of the past, such as “Did Romans took baths at the sea?”, “Who was x?”, “What is authored/discovered/invented by x?” and vice versa “What did x author/discover/invent?”, “In which occasion x has been build?”, ‘Why x is famous?”, “Which artist influenced x?”, “Which is the art movement of x?”, “Which are the most important artworks authored in x?”, “What characterise x?”, “How many artworks have been authored by x?”.

Many participants threatened the proposed mechanism as an approach to fulfil general curiosities, such as “Is there any legend behind x?”. An interesting aspect that emerged by the collected replies is that VAs interpreted as vocal assistants can easily perform storytelling and can be queried to tell a random event or curiosity about an artist or a monument, can narrate “How was city x before event y happened?”, “Tell me the story of x”, “Can you describe x?”, “Talk about x.”, “Give me further details about x”, “Tell me curiosities about x.”

The most common questions concern tangible CH, both movable, such as artworks, coins, and documents, and immovable, such as churches, monuments, and castles. However, users are also interested in curiosities related to intangible CH, such as the folklore of local traditions, such as “What are the traditional dances of x?”, music, such as “Which musician has the most albums?”, events, co-occurrences of terms in books and manuscripts, such as “Does x discuss about y?”, “How many times x talk about y?”.

While participants posed questions in most of the cases, some of them used their imaginary VA extension to explore available data. For instance, “Give me all the titles/authors belonging to the topic/category x.”, “Give me artworks related to x”.

Besides textual replies, users are also interested in visualising photos and videos, such as “Show me x” to retrieve examples of presbyters or a building plan, or “Show me video related to x” where x might be an event or a monument.

Looking at the way questions have been formulated, the proposed intents successfully reply to most of them. In most cases, participants formulated complete questions, while they rarely posed commands to collect information. The proposed approach misses the fulfilment of complex queries, which are quite rare in this questionnaire. For instance, currently, it cannot deal with questions like “How long did it take the building of x?” if it implies computing the difference between the foundation and the completion date, composed questions like “Give me artworks painted by the same author of x” requires users to split them into two queries as demonstrated in the performance evaluation.

General comments. As general suggestions, participants noticed that even if our proposal is particularly suitable for the CH field, its usage might be hypothesis in any application context, such as Public Administrations and hospitals. It is crucial to care about queried sources in terms of coverage of topics and reliability. As the interaction model is strictly connected to the target user group, the usability of the resulting VA extension should be carefully evaluated to tune the way questions can be posed, make the interaction as natural as possible and limit the error rate. It might be interesting either to introduce a playful component or to evaluate the combination of VAs and virtual exhibitions. Participants also suggested automatically merging information from multiple sources letting users save time in querying individual sources.

Participants mainly used the comment question to compliment the project, assessing that the project has enormous potential, it guides digital transitions to our country, and it is extremely versatile as it might be applied to any application context. We are in a modern world, and everything is going to be connected with technology. CH should not remain out of this.

This section discusses the perceived impact and utility from the CH data curators’ perspective. We proposed an online survey to collect opinions and suggestions, and we administered it to two different groups of CH experts who are either modelling or are planning to model their data as KGs. This survey collects opinions and comments of potential users of the generator who might decide to propose the resulting VA extensions as data exploitation mean.

Participants and setting 5 people joined the online survey belonging to two different groups of CH experts. While 3 of them are delegates of the HETOR project, the other 2 researchers belong to a research group of Medieval Philosophy at the University of Salerno. The HETOR group mainly models tangible CH concerning local and national CH as tabular data releasing them according to the Open Data directive. They are planning to expose their data as KGs in the future. On the other side, the research group of philosophers is designing an ontology to model their collection of medieval manuscripts by representing the co-occurrence of terms, philosophical concept interpretation, and philosophical movements, both concerning Greek and Latin culture. The two research projects spontaneously joined the survey. They are people interested and experts in CH.

Data gathering and survey outline The survey has been administered in English and in Italian, and its content is described in Table 12 that reports questions, reply format, and the rationale behind each posed question. The survey is structured in three main sections: i) general information about participants’ expertise and interest in CH, the spread of VAs within the CH community interpreted as people that are either experts or interested in CH, alternative means used to query and explore CH; ii) the interest in making their data accessible by VAs; iii) general suggestions and comments as a free text.

Results This paragraph reports the most common replies and considerations related to the exploitation of the proposed approach in the field of CH.

Current exploitation means. The HETOR group performs analysis on their data by using query builders and data visualisation approaches, mainly via SPOD,12

The group of Medieval Philosophy is modelling history of philosophy data and related metadata by ontologies and plans to materialise the related KGs in the next future and to make them accessible to all. Even if planned exploitation tools are still under investigation, they hypothesise exploiting data in data visualisation approaches to guide users in interpreting data. They see potentialities to make them accessible by VAs, reacting with curiosity and enthusiasm but mainly focusing on actual data to improve their accessibility. They are a bit sceptical about making metadata accessible by VAs as they cannot already foresee an application context of interest as only experts are usually interested in metadata, in their opinion.

In both groups, participants stated that their working groups have no competencies in querying languages, such as SPARQL. Thus, providing this community with CH data exploitation tools not requiring technical competencies is crucial.

Application contexts. According to data published by the HETOR group, they are interested in retrieving artworks information, such as location, author, and date. As an example, they desire to pose questions like “How many x are in y?” as a general question to quantify castles in Campania, or museums in Italy, or churches in Naples; “Which is the construction year of x?”, “Where is x?”, “Which are artworks authored by x?”. They also hypothesise to query a VA extension to obtain terms definitions and disambiguation, such as “What is the meaning of x?”. Our proposal is perceived as a promising approach at school to familiarise with terms and concepts also during remote sessions, in museums, or on city tours as virtual guides.

Impact and utility. Participants assess that they would be delighted to query data by pronouncing questions instead of data sheets and query builders. Moreover, they assess that the impact of making CH data accessible by VAs might be very high (grade 5 out of 5). They reacted with enthusiasm to our proposal and are curious about the future applications. They foresee great potentialities given the possibility to spread the interest and the usage to a vast public without constraints on the age and without requiring any technical skills. They only see refrains by people that are still sceptical about the extensive use of technologies, but for sure, it might be useful to engage young CH lovers to deepen their awareness and expertise in CH.

General comments. Participants suggested introducing the possibility of querying multiple data sources at a time, tuning the interaction model according to the target group and the planned application context, and carefully checking the used source in terms of accuracy and reliability. They explicitly stated that they foresee potentialities in this project and that it would be extremely useful in the CH field.