Abstract
This work analyses the usage of different approaches adopted in Wikidata to represent information with weaker logical status (WLS, e.g., uncertain information, competing hypotheses, temporally evolving information). The study examines four main approaches: non-asserted statements, ranked statements, non-existing valued objects, and statements qualified with properties
Introduction
Since 2012, Wikidata [16] has been one of the most outstanding platforms for collecting and sharing Linked Open Data through the web.
Wikidata encompasses a multitude of facts, including some that may be contrasting since they come from different and disagreeing sources. Expecting global consensus on the “true” data would be unrealistic since many facts are disputed or uncertain. Wikidata allows conflicting data to coexist and provides mechanisms to organize this plurality, going beyond the triple-based representation of factual information, for instance, including contextual metadata and constraints over those statements [32,43]. For instance, Wikidata contributors can add time-sensitive information through qualifiers and ranks to represent temporally evolving information (e.g., the number of followers of a YouTube channel that is updated year after year) or multiple coexisting (and possibly competing) claims over the same subject (e.g., maintaining both the old as well as a new theory over some topic). In many such cases, multiple information items are present. Yet, newer or better information does not replace older or less true assertions. However, they coexist next to each other, and one or more mechanisms are used to signal their simultaneous presence and, when appropriate, the currently adopted stance.
We understand these statements as enjoying a somehow weaker logical status than asserted statements: they are neither true nor false, but they are, e.g., true from a specific moment onward but not earlier, or true up to a given moment but not afterwards, or accepted as true by most people but not everybody, etc.
It is a cultural necessity in many (if not all) fields of knowledge to access available data about a complex topic entirely and objectively as they evolve, as different scholars or models interpret them and represent available hypotheses rather than a positive certainty. For instance, Cultural Heritage scholars study attributions, the temporal context of events, the temporal evolution of content, and the contradictions of opinions and assertions so that expressing weak statements, i.e., claims we are not sure about, becomes a necessary tool to increase precise awareness of the currently available data for those who consult or reuse it. Interpretation thus plays a central role in humanities disciplines. Yet, Cultural Heritage knowledge graphs and domain ontologies frequently limit the formalisation of these phenomena or only partially represent them ([14,15], cf. Section 2). Recently, a rekindled interest has been shown in the formalisation of uncertain statements [10,17,37], claiming that interpretation constitutes a focal point in humanities data and metadata. Interestingly, these works prove how different motivations for nuanced statements with varying degrees of truth create a small and consistent number of approaches to express them. We conclude that studying the very idea of weak logical status claims per se, independently of their different justifications, can help shed light on these commonalities and their relative merits and issues. WLS claims are used not only for missing or incomplete information but also for the correct representation of personal opinions or beliefs, for temporally constrained information, for geographically constrained information, etc.
Wikidata supports several patterns to represent situations best expressed with weaker logical status claims. In this paper, we analyse some of these patterns as they are employed in actual collections, both in the humanities and, as a comparison, in hard sciences. A factor that increases complexity is that many of these uses have partially overlapping semantics, i.e., Wikidata contributors can use them for other purposes beyond weaker logical status claims, and this muddles the correct identification and interpretation of the situations we are interested in. We, therefore, want to discuss both the designed use of each approach, its actual usage and success in Wikidata. Finally, we discuss the impact of their ambiguous applications due to the coexistence of multiple uses for the same techniques.
In particular, we analysed four main families of approaches to the weaker logical status of statements, asserted vs. non-asserted statements, ranked statements, unknown objects, and qualified statements. In this paper, we try to answer the following research questions:
RQ1 – How widespread are these approaches in the current state of Wikidata?
RQ2 – How does the cultural domain of the Wikidata topics (and, presumably, of the individuals contributing to the data regarding the Wikidata topics) affect and reflect on the relative success of some WLS types over others?
RQ3 – Does the actual usage of the surveyed approaches match their designed use declared by Wikidata?
In addition, we wonder about how we could improve the clarity and cleanliness of such differentiation.
To perform such analysis, we accessed and downloaded two large sets of topics from Wikidata, one belonging to the Cultural Heritage (visual works of art such as paintings and statues, text documents, and audio-visual entities) and another from astronomy (celestial bodies such as stars and galaxies). Both use multiple fuzzy assertions and hypotheses and, therefore, need assertions with weaker status (e.g., attributions uncertainties or physical locations moving over time for paintings vs. spectral class or radial velocity for stars).
The decision to use a comparative dataset in this study is motivated by the wish to explore the similarities and differences between astronomical and humanities academic practices. Both fields involve studying unique objects, such as stars or books. Yet, the way data is treated differs, with astronomical observations becoming scientific data as soon as they are used as evidence of phenomena [8], while humanities rarely can go beyond learned interpretations.
The data sources also vary, with humanities researchers using historical documents, literature, art, and oral traditions, each having varying levels of reliability and introducing systemic and insurmountable uncertainty. In astronomy, uncertainty is often related to instrumental limitations and observational conditions. Methodologically, astronomy relies on empirical observation, mathematical modelling, and experimental validation. At the same time, humanities research is frequently interpretative and qualitative, and the necessary proof to obtain historical certainty is often unattainable [7]. This difference leads to distinct epistemological foundations, with the humanities acknowledging the subjectivity and cultural bias in interpretations [21], and astronomy seeking to minimise uncertainty through rigorous data collection and adherence to physical principles [11].
Our study’s hypotheses and assumptions include the idea that annotators in Cultural Heritage and astronomy may approach data incompleteness and uncertainty differently, with Cultural Heritage favouring qualitative, context-rich representations of competing hypotheses and astronomy leaning towards more quantitative, data-centric representations. This difference may reflect broader epistemological stances in their respective communities. Additionally, our study assumes that these distinct approaches to handling data incompleteness and uncertainty may impact the ease of integrating data from these fields in interdisciplinary research, with Cultural Heritage data potentially requiring more effort for reconciliation due to its contextual and subjective nature.
Overall our findings show that the amount of weaker logical status statements in Wikidata seems suspiciously low, as only 0,4% of visual artworks report attribution debates, a fairly low figures compared to, e.g., a more reasonable 8,5% coming from the RKD images collection,1
The paper is structured as follows: in Section 2, the state of the art is presented focusing on the representation of WLS claims in RDF (particularly in the context of Cultural Heritage) and how the representation of complex data scenarios in Knowledge Graphs (KGs) is evaluated. In Section 3, we present the approaches provided in Wikidata to encode WLS claims. Section 4 outlines the research objectives, the data acquisition process is briefly described, and the analysis of our Wikidata sample dataset is presented. In Section 5, we present our proposal for improving annotating weaker-logical status knowledge quality. Finally, in Section 6, we summarise our findings and outline our conclusions about the work.
Public Knowledge Graphs such as Wikidata [16], DBpedia [4], Yago [40], and Google Knowledge Graph constitute publicly available collections that can be used for research, either expressing specialist knowledge or general knowledge. In particular, Wikidata is a collaborative public platform built and maintained by a community of contributors.
Weaker logical status statements are natural in many contexts covered by these KGs, but the support for their representations varies considerably. Guidelines, data modelling, and harmonisation (a particularly relevant need for open platforms) can help express them, i.e., concurrent opinions or uncertain claims. In the field of Cultural Heritage studies, the knowledge competition is intriguing. However, some online databases or data models only partially address this issue.
Although domain ontologies represent the domain of cultural heritage hardly ever integrate support for representing interpretations (i.e., hermeneutics) into their models [37], there are some exceptions [15,36].
For instance, CIDOC CRM [15] is a conceptual model developed and maintained by the International Council of Museums (ICOM), widely adopted by many knowledge graphs in the Cultural Heritage domain [12,19]. It offers a formal approach to express weaker logical status claims through instances of classes representing n-ary relations.
CIDOC-CRM [15] adopts n-ary relationships for WLS claims, e.g., via the
CIDOC CRM use of n-ary relation for encoding concurring attributions
Europeana [36] stores approximately 50 million heterogeneous digitised items from museums, libraries, and archives across Europe. Data is collected by content providers (i.e., cultural institutions) using the EDM data model [14] and the use of proxies [29] allows to express conflicting information and track data provenance. However, concurrent statements are not visible on the online pages, and no mechanism is in place to determine which proxy will be made visible when multiple exist.
An interesting instance of an EDM collection is the RKD catalogue, a comprehensive collection of data about Dutch works of art throughout history. By design, RKD allows and gathers contested and discarded attributions of paintings and portraits. Although, at the moment, there is no SPARQL endpoint available for querying the collections, users can browse RKD data through an online catalogue. Interestingly, about 83,600 artwork descriptions in Wikidata have been linked to the RKD dataset via the predicate
Despite the support of representational definitions of weaker logical status claims in EDM, CIDOC-CRM and RDK data models, these weaker forms of information are often poorly reported (reticence) or are expressed in textual annotations rather than being modelled in the data structure (dumping) [6].
The widespread adoption of Wikidata within the Cultural Heritage community has been well-documented [42].5
The list of cultural institutions involved in Wikidata can be found at
Handling WLS in Semantic Web data can be placed within the broader topic of representing and reasoning over data enriched with metadata or contextualised data. The matter has been discussed at length from many different angles. A primary objective is that of reconciliation or integration of multiple data sources. Indeed, effective representation and reasoning about knowledge with heterogeneous viewpoints is one of the objectives for applications concerned with distributed knowledge sources. Yet, semantic web ontologies force a unique, global view of the represented world, in which the axioms are meant to be interpreted as universally true. The same domains are often modelled differently depending on the intended use of an ontology. The problem of reconciliation, therefore, is to bring different world views together to create a single, unified model for representation and reasoning. This may be obtained through formal Interoperability Systems [30] extending the expressive reach of Description Logic, or bridge rules mapping separate contexts determining how the local concepts in the two ontologies map onto each other [9], or extended representation models such as RDFS with Annotations [45]. Different approaches, such as colouring [20] or NDFluents [24], or RDF+ [13], on the other hand, are less interested in obtaining reconciliation and more in representing adequately the semantics of inferences about heterogeneous claims.
To the best of our knowledge, current research has not extensively tackled WLS representation in RDF. However, the representation of complex data scenarios in knowledge bases (and in particular, in Wikidata) has been evaluated according to multiple metrics. For instance, Piscopo and Simperl [38] survey quality metrics from 28 scientific publications on the topic and categorise quality assessments into three dimensions: intrinsic (accuracy, trustworthiness, consistency), context (relevance, completeness and timeliness) and representation (ease of understanding and interoperability). Among quality measures, evaluation of completeness, defined by Faerber et al. [18] as the “presence of all required information in a given dataset”, has been approached through various methods and assessments as comparing data for similar entities [5], measuring entity relatedness [39], evaluating thoroughness of information by determining the completeness of specific attributes of objects [22], assessing low-quality statements thought the analysis of items’ discussion pages, deprecated statements and constraint violations [41], and assessing and comparing data quality across large knowledge bases [1,18]. Additionally, Arnaut et al. [3] surveyed negative knowledge in Wikidata, analysing deleted statements, count predicates, deprecated statements, negated predicates and noValues to measure Wikidata completeness from this point of view.
Overall, among current Cultural Heritage KGs, WLS representation seems to be slightly tackled, showing Wikidata as one of the few platforms providing designed approaches to represent such knowledge. However, little or no evaluation has been conducted specifically on the representation of weaker logical status claims in Wikidata, nor has a comprehensive analysis been carried out to assess the amount of knowledge related to WLS status in Cultural Heritage. In the next section, we detail our proposal to address these shortcomings.
Wikidata represents weaker logical status statements (e.g., for uncertain or debated assertions) using at least three different approaches: ranked statements (Section 3.1), statements with specific qualifiers (Section 3.2) and statements with a non-existing valued object (Section 3.3).
Ranked statements
Ranking of assertions is modelled by the Wikibase data model6
Claims in Wikidata are expressed through statements, a custom reification approach7
Statements do not assert the corresponding claim, but an additional triple must be added to assert the claim’s content. The additional triple (which uses a different prefix) flatly relates the statement’s subject to the statement’s intended object through the statement’s predicate, thus enabling simple query support for asserted facts. The separation between statements and their assertion is selectively provided, allowing easy support for both claims presented as facts (where both the statement and the assertion triple exist) and claims not meant to be considered facts (the statement exists, but no assertion triple is added).
The ranking mechanism is enriched with the representation of asserted and non-asserted statements. Rankings [26] communicate the scientific community’s or Wikidata annotators’ consensus. Disputes are separately hosted in plain text on the corresponding discussion page. Many possible combinations of variously ranked competing statements can be found in the Wikidata collection, with various and debatable interpretations. Ranking is assigned to individual statements using values such as preferred, normal and deprecated).
Note that whether or not a statement is asserted is determined solely by its rank and the absence of higher-ranked statements using the same predicate. The Wikidata engine automatically asserts the statement and it is not the editors’ conscious choice.
The normal ranking is the default ranking for statements. A statement ranked normal can be either asserted or not depending on the existence and intended meaning of competing statements against it. For instance, in Listing 2, “The Scream” by Edvard Munch belongs to the Expressionist period,8
Normal rank
Deprecated statements are meant for claims with a weak logical status and do not represent a correct value in the editors’ view. Deprecated statements are always automatically non-asserted independently of the ranking of the other concurring statements. Wikidata designed use for deprecated ranking is stated to be “used for statements that are known to include errors (i.e. data produced by flawed measurement processes, inaccurate statements) or represent outdated knowledge (i.e. information that was never correct, but was at some point thought to be)”. Additionally, Wikidata negates the use of deprecated ranks for claims which describe “correct historical information, such as previous values of a statement [...]”.9
For instance, Listing 3 expresses the concept that “The Lamentation”,10
Deprecated rank
Preferred statements are meant for claims with a stronger status and representing the currently presumed correct value of a predicate. They are always also asserted. For instance, as shown in Listing 4, a retracted attribution of the painting “Madonna with the Blue Diadem” 11
Even though the first attribution is ranked normal rather than deprecated, we must consider it a superseded claim. This example shows that the nature of normal statements varies depending on whether they coexist or not with competing preferred and/or deprecated claims, and similarly, the presence or absence of assertion triples may vary. The preferred rank designed use is “most current statement”, implying that other concurring statements should represent outdated statements, and “statement that best represents consensus (be it scientific consensus or the Wikidata community consensus)”, implying that other concurring statements should represent concurring discarded statements.12
Preferred and normal ranks
Statements, independently of rank, can be decorated with additional triples annotating contextual information or specifications about the claim itself.13
The complete list of available qualifiers in Wikidata is available at
Following the example from Aljalbout et al. [2], we examined the 150 most frequently used qualifiers in Wikidata and their most commonly used values. The most used qualifiers to use WLS values are
The most frequently used values are: circa, presumably, allegedly, inference, uncertainty, possibly, near, probably, conventional date, disputed.
The most frequently used values are: originally, attribution, hypothesis, often, allegedly, expected, possibly, disputed, rarely, mainly.
For instance, in Listing 5, we see that the painting “Abstract Speed + Sound”21
A qualified statement in Wikidata
Wikidata provides a list of 96 recommended values for nature of statement and 83 recommended values for sourcing circumstances in their respective Property Talk pages. In contrast, no recommended terms are provided for reason for deprecated rank nor reason for preferred rank. However, terms that were used with these properties can be retrieved via a simple SPARQL query,22
List of terms used in Wikidata with reason for deprecated rank
There are three types of basic information structures used to describe entities in Wikibase (called SNAKs, or Some Notation about Knowledge23
Unknown valued statements are claims whose object exists but is not known.26
Unknown-valued statement in Wikidata
Non-existing valued statements28
Non-existing valued statement in Wikidata
Even before checking on the actual usage patterns of these approaches, we can immediately notice the richness of annotations made possible by them, the subtle nuances they afford, and the variety of (potential) sources of ambiguities, overlapping connotations and representation vagueness. In particular, we can summarise three specific problems that are worth further discussion:
Although the separate uses of normal, preferred and deprecated rankings are clear and practical, there are uncertainties when they coexist on the same predicate, especially for the different representations of normal statements when preferred ones are also present or when all three rankings are present.
The sheer number of qualifiers, the differing levels of their respective specificities, and the manifest semantic overlapping of many of them make it hard to guarantee homogeneity and precision in their use. Contextualising qualifiers, be they temporal, provenance or otherwise, does not add to the base information but changes the context within which such information is true. As Patel-Schneider [35] suggests, contextual qualifiers should not be shown to consumers. Still, basic tools (visualisers, contextualisers, reasoners) should be written to take such context into account correctly, and low-level tools should remove facts that are not valid in the selected contexts.
The subtlety in the semantic differences between providing no statement, specifying a
In a way, WLS claims can be seen simply as logical disjunctions of competing claims each of which is separately annotated with context, provenance, confidence, temporal boundaries, etc.: “according to α,
Another way to formally understand WLS claims is to link them to modal statements in modal logic [23], which can be used to understand the coexistence of strong logical status claims, expressed as atomic formulas
e.g.,
Yet, all these reflections are empty and pointless unless we examine how contributors use these approaches to express real WLS claims in their Wikidata contributions. The following section covers this topic.
To generate some analysis about the actual usage of WLS claims and to provide an initial answer to our research questions, we collected three datasets of Wikidata items: one about Cultural Heritage items (visual arts, text documents and audio-visual entities), another about Astronomical objects (galaxies and stars) and one with a selection of random entities reflecting the actual distribution of entities in classes in the whole Wikidata as discussed in Section 1.
The datasets were selected to be approximately comparable in size, and the number of individual statements and under evidence that many types of entities rely on weaker logical status claims when entities undergo re-evaluations due to new pieces of evidence or the recording of different opinions.
Data acquisition
SPARQL query retrieving Wikidata entities to subclasses of work of art (
The first dataset contains Cultural Heritage items (CH), a complete snapshot of the Wikidata records of these cultural assets. All Wikidata entities belonging to the class work of art33
Via
Audio-Visual heritage (CHav): This collection holds information about audio-visual materials that have cultural, historical, or artistic value. They include movies, videos, recordings of music or spoken words, and other audio-visual materials that record a particular event in a specific time or place. The dataset contains 1,251,626 entities and 17,141,394 statements organised in 25,033 JSON files.
Visual heritage (CHv): This collection holds information about visual artefacts with cultural, historical, or artistic value. They include paintings, drawings, sculptures, photographs, decorative arts, etc. The dataset contains 1,078,855 entities and 12,850,825 statements organised in 21,579 JSON files.
Textual heritage (CHt): This collection holds information about written and printed materials with historical or cultural significance. They include books, manuscripts, letters, and other written documents. The dataset contains 625,110 entities and 4,584,444 statements organised in 12,503 JSON files
We also downloaded Wikidata entities of architecture-related classes; they were later discarded due to their fairly lower number as well as for the presence of many statistical ambiguities that could make their evaluation useless (e.g., many entities belonging to these classes should not be considered relevant to Cultural Heritage collections).
The second dataset, chosen to verify our assumptions using a different collection with a similar size, is a collection of astronomical entities organised into two datasets:
Stars (ANs): This collection holds a random selection of 1,199,950 Wikidata entities (of the ~3.3 million existing) belonging to the class Star,35
the ANs dataset was meant to be composed of 24,000 files with 50 entities each, but after running our tests we noticed that a file was corrupt and we chose to discard that contribution.
Galaxies (ANg): This collection holds a random selection of 1,200,000 Wikidata entities (of the ~2 million existing) belonging to the class Galaxy,37
We decided to limit the number of astronomical entities to 1,200,000 to approximately balance them to each other (although the CHt is about half in size with 625,110 entities), as well as the average number of statements for each entity (CHav: 13.7, CHv: 11.9, CHt: 7.3, ANs: 22.9, ANg: 12).
The third dataset is a selection of randomly chosen entities from Wikidata. This dataset was acquired to compare WLS claims in the other datasets with a randomised subset designed to mimic the overall distribution of WLS claims in Wikidata.
Random (R): This dataset comprises 1,159,800 Wikidata entities (starting from a selection of 1.2 million entities from which we removed duplicates) chosen randomly from the most numerous 100 classes to reflect the proportional distribution of entities found in Wikidata.38
In Table 1, we summarise basic information about these collections. All these datasets can be accessed and downloaded from Zenodo39
In the following, we will describe as WLS statements all Wikidata statements showing the use of each approach described in Section 3, regardless of whether they have been used to make weaker logical status claims. Table 1 shows a tabular presentation of our analysis.
Even though critical analysis is a pivotal part of humanities discourses, plainly stated statements with no competing claims are largely the most represented information in the CH dataset: the vast majority of statements here (>99%, in particular 99.74% in CHav, 99.92% in CHv and 99.69% in CHt) are plainly asserted statements with no WLS additions. In contrast, the Astronomical datasets show a reasonably different situation, 83% overall of plainly asserted statements, specifically ANs at 72.58% and ANg at 95%. The overall distribution of the Random (R) dataset showcases a low percentage of WLS claims (1.78%), closer to the CH and the AN datasets. Yet, interestingly, almost the whole percentage is made of non-asserted statements (98.95%) matching a similar distribution in the AN dataset.
Entities, statements and types of WLS statements
Entities, statements and types of WLS statements
When analysing the Random (R) dataset, we notice that the ranking system’s simplicity leads to a clear predominance of deprecated items and, consequently, of non-asserted claims. The other approaches appear to be underutilised in a proportion closer to the AN dataset. Possibly, this is a reflection that, in the CH community, historical uncertainty and the representation of interpretation are more frequent and typical than in other disciplines.
To further explore these data, we can notice that:
Non-asserted statements Of the approaches previously listed (cf. Section 3), non-asserted statements (i.e., variously ranked statements with no corresponding asserted triples) are largely the most frequent approach for representing competing information in both AN and R. The situation is fairly different in the CH collections, non-asserted statements being the most frequently used approach in CHt (81.64%) and CHav (only 86.09%) and almost unused in CHv (3.99%).
Deprecated statements Deprecated claims are visibly a small portion of the overall non-asserted statements, occurring only in 20% of the non-asserted statements of the Cultural Heritage entities, in 30% of the non-asserted statements of Astronomical entities and in the 66% of the non-asserted statements of Random entities. At the same time, about half of the deprecated statements were annotated with the corresponding reason for deprecated rank qualifier (in particular, 45.59% CHt, 25.15% CHv, 64.93% CHav – compare this with basically 0% in both AN datasets and 1.24% in R dataset), proving that scholars in the humanities have a solid interest in annotating provenance of WLS claims on CH data. Yet, only less than 1% of preferred statements have been annotated with the corresponding qualifier reason for preferred rank.
Unknown values Unknown valued statements are not used at all in Astronomical data (absolute 0 in both ANg and ANs out), poorly adopted in the R dataset (0.47%), and sparsely used in the Humanities as well (9.75% in CHav and 10.71% in CHt). Higher is the result for the CHv dataset, with 46.88% of the overall WLS claims using this approach.
Non-existing values Even if they do not represent WLS claims, we examined them in our datasets for contiguity to unknown values. Non-existing values are almost unused in Astronomical data (exactly 4 occurrences in ANs and an absolute 0 in ANg out of more than 7 million WLS claims) and very sparsely used in the Humanities and Random datasets as well: 1.969 statements in CHv, 1.356 statements in CHt and 3.857 statements in R dataset. Fairly higher is the result for the CHav dataset, with 50,611 statements using this approach. This outlier value is probably justified and will be commented on later in this section.
Qualifiers Statements qualified with nature of statement and sourcing circumstances predicates are the least employed out of the surveyed ones, being used in 7.66% of the WLS statements in CHt, in 0.58% of the WLS statements in R and in 4.16% of the CHav statements, present in 0.0008% of the ANs statements and only in one ANg statement. Yet, they are used in 49.13% of the WLS statements of the CHv dataset. This value will be commented later on in this section.
We further surveyed the terms actually used as values for the qualifiers.
We witnessed the use of respectively 200 different values for qualifier nature of statement, 419 for sourcing circumstances and 588 for reason for deprecated rank. These values largely exceed the proposed values specified in the corresponding Wikidata property talk pages (respectively, 194 values for nature of statement and 175 for sourcing circumstances) or property constraints as for the 384 values for reason for deprecated rank). Furthermore, the three sets of actual terms show a considerable overlap of values between them (in our datasets, but also over all of Wikidata), as shown in Fig. 1. This seems to imply that the semantics associated with these values, and indeed the properties themselves, may have been unclear to contributors, who then, in some cases, selected the qualifier in non-predictable ways. Therefore, we decided to group all three sets into a single category (shown as WLS qualified statements in Table 1).
Since the R dataset is not disciplinary, we deemed that the variety of situations occurring across disciplinary boundaries would inevitably pollute any analysis deeper than mere counting, and therefore, in the following sections, we will focus only on the disciplinary datasets.
We further surveyed the terms actually used as values for the qualifiers.
Terms used in qualifiers nature of statement, sourcing circumstances and reason for deprecated rank throughout Wikidata (left) and in the CH datasets (right).
Overall, the three sets contain a variety of terms such as generic contextual information items, e.g., provenance details, as well as domain-specific terms not relevant to our purposes (e.g., show election, declared deserted, or text exceeds character limit), as well as qualifiers we can truly consider suggesting weaker logical statuses (e.g., possibly, disputed, expected, etc.).
Therefore, we ignored the suggested values provided by the Property Talk pages and focused on the actual values found in our datasets. We surveyed the list of terms and selected a subset of 101 terms that seem to concretely refer to WLS claims. This subset of WLS terms appears to be widespread in CH and Random datasets (2,086 occurrences in CHav, 111.641 occurrences in CHv, 1,318 occurrences in CHt and 6,406 occurrences in R), while almost not employed in Astronomical datasets (62 occurrences in ANs and only 1 in ANg).
The distribution of approaches to represent WLS claims in the CH dataset is not homogeneous, as unknown values and WLS-qualified statements are both highly used in the CHv dataset, while non-asserted statements for CHav and CHt. An obvious outlier is the use of one specific qualifier. Indeed, the value circa41
Another outlier seems to be the number of non-existing valued statements, which are present in the CHav dataset with a much higher proportion than elsewhere. In this dataset, non-existing valued statements seem to be heavily employed correctly in specific properties that appear frequently here and not elsewhere, such as
In theory, the approaches to represent WLS claims are
To summarise, it becomes manifest that the prevalence of each approach is quite diverse, even between the datasets of the same domain. Specifically, in CHav the most commonly used approach representing WLS information is non-asserted (86.09%), in CHv it is the WLS Qualified statement (49.13%) followed by unknown value (46.88%), and in CHt it is non-asserted (81.64%). In the Astronomy datasets, non-asserted statements overwhelmingly represent WLS claims, but deprecated statements have a much larger impact on them than in the Cultural Heritage domain.
Top 10 most recurrent properties implied in WLS claims in each disciplinary dataset.
The property analysis provides valuable insights, too, as shown in Fig. 2. We divided the actual usage of WLS approaches by the property where they appear. The x-axis contains, for each dataset, the ten most frequent properties in which WLS statements appear. The y-axis shows in logarithmic scale the number of occurrences of such statements, organised by colour: non-asserted statements (with rank normal), non-asserted statements (with rank deprecated), statements with qualifiers (only WLS-related qualifiers), and non-existing valued statements.
The datasets were analysed by systematically evaluating the properties associated with the surveyed approaches. Each dataset was analysed to identify (1) the most prominent properties of each dataset and (2) the most prominent properties of each dataset with each approach.
Normal ranked, yet non-asserted statements appear in large numbers in CHav for
Qualified statements are largely present in CHv and CHt on properties
Unknown valued statements are primarily used in CHv and CHt datasets and only sparsely in CHav dataset. Their usage seems to be mainly implied in the description of agents in roles in all CH datasets (e.g.,
We can also notice the predominance of non-existing valued statements in CHav (
Besides this, we registered some co-occurrences of the use of unknown and non-existing valued statements with the same properties (e.g.,
To summarise, we list some of the complexities and ambiguities we identified in both the CH and the AN datasets besides their designed use described by Wikidata (cf. Section 3). The list comprehends a more fine-grained distinction of WLS situations.
Evolving situation: The claim is not true at the moment but was correct at some point in the past, and keeping this information is deemed interesting to maintain. For instance, the number of
Evolving knowledge: Because of a new observation or theory, a previous value is considered superseded. This situation is mainly connected to new observations, theories, measurements, guesses and interpretations. For instance, the introduction of a new accepted attribution of a work of art means that the previous one is now deemed as false or at least deprecated, or, in astronomy, the object “15 Orionis”44
Less favoured versions: Similar claims are ranked not because they are either false or true but because one of them is preferred over the others so that they are marked as preferred and asserted while the others are non-asserted. For instance, the
Uncertainty: For instance, the painting “Madame Antoine Arnault”47
Caution: For instance, the “Frontispiece to Christopher Saxton’s Atlas of the Counties of England and Wales State I”49
Imprecision: For instance, the hypothetical entity “IRAS 17163-3907”52
Data entry errors: Data include errors probably introduced during the annotation. For instance, the novel “Invisible Monsters”54
Dumping from pre-existing databases: Some non-existing values may result from an error in the conversion or an empty field of a record after importing an existing database into Wikidata. For instance, the painting “Marshy Landscape”55
The value does not exist: For instance, the first and last entities of a sequence use properties
Model fitting: When the model does not fully support the situation to be described, some arrangements were taken, such as the use of a non-existing value for the property original language of film or TV shows
The value exists but is not known: For instance, the painting “The Welcome Home”58
The previous list shows a series of situations where the same approaches are used for different purposes. All such purposes (except data entry errors) are legitimate. Yet, we fear that users may have trouble differentiating the purpose of each use because the approaches chosen are not sufficiently precise enough to distinguish the specific situation clearly and unambiguously. Rather than suggest forcing all different situations into a single over-encompassing approach, Section 5 lists some increasingly impactful solutions to solve these ambiguities without overly revolutionising the data model.
The datasets presented in the previous section and our analysis of their content allow us to reach some conclusions on the research questions specified in the introduction.
RQ1 – How widespread are these approaches in the current state of Wikidata? – The current state of WLS claims in Wikidata is poor. Even though Wikidata focuses on collecting and referencing the facts claimed elsewhere59
RKD61
See
One may wonder that Dutch and Flemish collections are not representative of the full scale of worldwide types of artworks represented in the CHv dataset. Yet, they provide an interesting starting point for a further comparison. We created a sub-dataset of CHv and further analysed it to improve our understanding of this issue. First of all, it should be noted that, as mentioned, about 83,600 artwork descriptions out of the 267,238 available in RKD have been linked to Wikidata,62
The count of attributions is calculated over the number of claims having the predicate
The number of discarded attributions is calculated over the number of claims having
Comparison between attributions in RKD images collection, CHv dataset and CHv selection of paintings from 17th to 20th century
RQ2 – How does the cultural domain of the Wikidata topics (and, presumably, of the individuals contributing to the data regarding the Wikidata topics) affect and reflect on the relative success of some WLS types over others? – Our data analysis highlighted several peculiarities between the Cultural Heritage and Astronomical datasets. The two families of datasets present many different representational artefacts: while the CH datasets seem to employ, with variable proportions, all the listed approaches, the astronomical datasets employ almost exclusively ranked statements. Additionally, while WLS statements in AN datasets affect a fairly small number of properties, they cover a much wider range of properties in CH, as shown in Fig. 2. These aspects highlight key differences in what the two communities consider weaker logical status: we may hypothesise that deprecations in astronomical data mostly reflect the result of newer and better data. In contrast, the humanities community uses WLS statements for a much larger set of uncertainties due to ignorance, scholarly interpretations and disagreements as hypothesised in Section 1. Thus, it may occur that the specification of the
RQ3 – Does the actual usage of the surveyed approaches match their designed use declared by Wikidata? – Wikidata provides a set of designed uses for WLS claims annotation as described in Section 3. In addition to them, Wikidata contributors have, over time, adopted frequent annotation patterns that are only sometimes aligned with designed uses. Thus, there is much noise and ambiguity in how Wikidata contributors have used approaches provided by Wikidata to represent WLS information in the datasets we studied. This makes it difficult to differentiate and search WLS data. The variety of cases listed at the end of Section 4.2 summarises an incomplete yet vast collection of WLS and non-WLS situations modelled through the same WLS representation approaches. Therefore, it is difficult to search for specific data patterns over the entire dataset and even to interpret individual entities correctly. In particular, such ambiguities can be specifically listed for the surveyed approaches: (1) Ranked statements are used for both representing WLS information as the evolution of opinions in critical debate (evolving knowledge), historical information (evolving situation) and non-related WLS information such as, e.g., less preferred variant. Additionally, despite the different designed uses for preferred and deprecated statements, in practice, they frequently co-occur in the CH dataset for the same properties, showing that annotators arbitrarily choose between these two approaches to represent such information (e.g. discarded attributions are sometimes represented with a non-asserted normal rank and sometimes with a deprecated rank). (2) The selection of terms provided with nature of statement and sourcing circumstance, despite being a very expressive pattern to represent WLS information but also its justification, is not exclusively related to WLS information, so that a subset of terms should be defined for this specific purpose (cf. 101 selected terms in Section 4.2). Additionally, no taxonomy is provided on types of WLS qualifiers. For this reason, automatic extraction of types of uncertainty (such as uncertainty, cautioning, and imprecision as discussed in Section 4.2) cannot be automatically performed. (3) Despite the designed use provided by Wikidata, the two types of missing values statements (noValue and someValue) present a significant co-occurrence within the same properties, indicating an unclear usage similar to the usage of ranked statements.
Furthermore, using the same approaches for WLS and non-WLS-related characterisations makes complex patterns hard to express and identify. For instance, if an artwork AW was supposedly moved from location X to location Y, but we are not certain, both locations X and Y must be represented as WLS, the first because of an evolving situation (AW is not at location X anymore) and the second because of uncertainty since the new location Y is only guessed. Therefore, none of these assertions can be asserted, and none can be ranked as preferred. We need a complete and thorough contextual annotation (e.g., why each claim is discarded), without which disambiguation and full understanding of the state and truth of the relevant predicate is impossible. In Section 5, we suggest a possible pattern to represent such situation (cf. point 5, in particular, normal rank + non-asserted).
Getting down to detailing workable solutions to improve the situation for WLS statements in a project as large and as complex as Wikidata is always running the risk of becoming an exercise in futility. In this section, we respectfully suggest possible actions for WLS statements, starting from very conservative proposals with limited impact to more impacting changes.
We list possible remediation activities for the Wikidata data model and the collection to simplify and disambiguate WLS assertions from the rest. We approach such a complex endeavour with humility and caution, as it may be hard to assess the impact and difficulty of implementing each suggested step from our vantage point.
For this reason, we express our suggestions as an ordered list whose first items are meant as simple cleaning-up activities of little impact and then progress to bolder and more impacting actions that sometimes require not just a modification in the data model but possibly also the systematic update of small, but still numerically relevant, selections of the current datasets.
Require a Require the specification of Create a new and separate Certainty Degree qualifier specifically for WLS statements, separating the reason for the chosen qualification from the certainty or confidence degree of the qualification. Such certainty degree should be scalar and use a limited number of values, avoiding any complexity in distinguishing between terms such as possibly, hypothetical, and dubious. A 5- or 7-item scale would suffice, e.g., non accepted, highly unlikely, unlikely, possible, probable, almost surely, and accepted. Different labels would be perfectly acceptable, even using numerical values instead of labels. Reorganise the values of
Restrict ranking for competing statements to just three (possibly four) different patterns and prevent any other variant:
Preferred + Deprecated: To be used whenever there are several competing statements, and some are chosen to be the best. Accepted statements are set to preferred (and asserted), while the rest are set to deprecated (and not asserted); there are no normal ranks. Both preferred and deprecated statements are fully qualified with Normal rank + asserted: This would be the default situation, to be used when no dispute or disagreement exists and the statement(s) are all equally accepted. All statements are also asserted. Since this is the default, no qualifier is necessary, but it is still possible to specify a Normal rank + non-asserted: To be used when there are several competing statements but none of them stands above the rest as being the most likely. For instance, this would be the case of a work of art not definitely attributed to anyone but for which several competing hypotheses exist. However, none seem more convincing than the others. No statement is asserted, and
A fourth pattern could be allowed for claims for which the only reported value is wrong, but no acceptable alternatives exist. In this case, we could use a deprecated statement for the reported wrong value and a non-existing valued statement with a normal rank to represent the non-existing correct value.
Our work is the first systematic study about the representation of weaker logical status claims (WLS) over Cultural Heritage data in Wikidata. Through WLS claims, uncertain information, competing hypotheses, temporally evolving information, etc., for which a plain and direct assertion is inappropriate, can be expressed. We analysed four patterns used in Wikidata for WLS claims: asserted vs. non-asserted statements, ranked statements, missing values, and qualifiers.
In our analysis, we found several interesting facts. First, the number of statements expressed using a lower logical status is much lower than might have been expected by comparing similar sources. Secondly, the Wikidata data model is far from being too poor to express WLS claims; it offers users an overabundance of approaches, but their applications overlap and are also used for non-WLS applications. Finally, significant differences exist in how datasets from different domains employ these approaches for weaker logical status claims. Domain-specific non-WLS situations can be considered as a justification for much of this variety, and this contributed to the idea that WLS-specific features should be introduced in the Wikidata model to address specifically weaker logical status claims. We proposed a set of increasingly impacting modifications to the data model aiming towards a leaner and more accurate representation of these phenomena, expecting that they can improve data quality and information retrieval, specifically over uncertain, evolving and competing statements.
We are still working toward a complete taxonomy of values for qualifying ranked predicates, as this seems to be, to our eyes, the most rapid and solid way to fully represent both the weaker logical status of a claim and its underlying nature and justification. We plan to publish this taxonomy with a proposal for mapping existing data points to this taxonomy so that no information is lost during conversion.
Resposibility statement
Fabio Vitali and Valentina Pasqual jointly wrote the manuscript’s introduction (Section 1). Valentina Pasqual authored Section 2, covering the state-of-the-art, and Section 3, focused on approaches to representing WLS in Wikidata. Valentina Pasqual collaborated with Alessio di Pasquale on the data analysis section (Section 4). Fabio Vitali is responsible for Section 5, proposing new approaches to represent WLS in Wikidata. All authors contributed to the conclusions section (Section 6). Fabio Vitali and Francesca Tomasi provided critical revisions and feedback throughout the writing process, ensuring the coherence and accuracy of the manuscript. All authors actively participated in the manuscript review, providing intellectual contributions and final approval for the submission.