Creative AI: A new avenue for the Semantic Web?

1.1. From analysis to synthesis

As applications and services are the key to adoption of a given technology by users, what intelligent applications and services might be then facilitated and enabled with Semantic Web and semantic technologies in the next several years?

It comes with no surprise that Semantic Web research evolves influenced by major shift changes in knowledge engineering and AI. Early Semantic Web used mainly deductive reasoning, employing logic-based reasoning services. With growing amounts of data, this has later shifted to an increased interest in applying statistical approaches, i.e., inductive reasoning [11,32]. Both may be classified as analytic tasks, but lately, there is an increasing interest in other types of reasoning. Consider for instance generating justifications and explanations (for explainable AI), which may serve for debugging purposes, and which are closely related to abductive reasoning. Some of recently popular tasks deal with synthesis rather then analysis and aim to generate rather than only analyse artefacts. Domains previously reserved for humans, such as making creative designs and scientific discoveries, are increasingly being addressed by AI [13].

Can thus the Semantic Web, semantic technologies and resources facilitate creative AI, i.e. computationally creative systems that use AI techniques?

1.2. What is creativity?

Creativity, creative reasoning and creative problem solving have been researched in cognitive [6] and computational sciences [15]. Cognitive psychologists aim to understand the human creative process. In her influencial works, Boden [5,7] describes creativity as the ability to come up with ideas or artifacts that are new, surprising, and valuable. The former ones may be concepts, musical compositions, poems but also cooking recipes, or even scientific theories. The latter ones may be paintings, pottery, but also vacuum cleaners, engines, etc. Moreover, many researchers use the term ‘concept’ to refer to a range of things such as abstract ideas in arts, science, and in everyday life [6], and we will also use this term throughout the paper.

1.3. Computational creativity

Can machines be creative? Some time ago it was hardly believable. Ada Lovelace, arguably referred to as the first computer programmer, was reflecting on the Charles Babbage’s mechanical general-purpose computer, the Analytical Engine that it “has no pretensions whatever to originate anything. It can do whatever we know how to order it to perform”. However, with the development of machine learning, it is not needed anymore to explicitly program machines that apparently have begun to reveal creative behaviours [1,13].

The computational creativity research area has emerged concerned with “computational systems which, by taking on particular responsibilities, exhibit behaviours that unbiased observers would deem to be creative” [9]. Computationally creative systems perform various ‘generative acts’ that create exemplars, concepts, or provide an aesthetic evaluation for the generated artefacts. Such systems have already been built in various domains including not only art (fashion, entertainment), but also design and engineering (drugs, devices, processes, e.g. in software composition [37] or program synthesis [30]), and scientific discovery disciplines [27] or even for inventing recipes [48].

1.4. Motivating use cases

In the following, we briefly introduce illustrative use cases for the research in the Semantic Web for computational creativity, while Table 1 breaks them down to: their relation with the Semantic Web topics, (computational) creativity types (described in Section 2) and sample solution methods (a selection of them is elaborated more in Section 3).

Recipe generation The goal is to generate a recipe given the list of desired and excluded ingredients. This goes beyond simply retrieving a recipe based on the specified conditions (ingredient lists), as existing databases may not contain one meeting the conditions. In case of culinary recipes, some ingredients may be desirable, for instance, because they are in the user’s fridge, while others may have to be excluded because they are allergens. The recipe cannot be completely random, it must be plausible also regarding taste and smell. A new recipe may have to be aligned to a particular cuisine or a chef, mimicking his or her style. Techniques that may be used for performing this include data mining/machine learning against Web or Semantic Web recipes, querying remote resources or applying constraints regarding the resulting recipes, and finally blending (mixing) available recipes.

Collaborative scientific discovery Another scenario concerns a creative process of generation of plausible hypotheses from observations. Scientists in their work use to invent hypotheses to explain phenomena, and then design and apply methods to either verify or falsify them. Imagine a computer system not only able to verify hypotheses (which is now increasingly being done with machine learning predictive models), but also to abduce novel hypotheses or to find novel, surprising connections between domains to facilitate scientific discoveries. A seminal example of a system capable of abducing and testing novel hypotheses is Robot Scientist, which originates novel hypotheses in functional genomics and has been shown to make scientific discoveries [27] with use of expressive ontologies to describe the domain of interest and use this formal, machine-readable description for automated reasoning to automate scientific experiments.

Creative design Imagine that as a designer you are able to just tell a computer what you want to design, e.g. that you want to design a table, which uses material X and costs no more than Y, and has weight Z, and tell about the style you like or that is compliant with your company’s aesthetics. In response, the computer produces thousands of new solutions, meeting your requirements and that are also easy to manufacture as they consolidate parts. So called generative design makes it increasingly possible and enables designers and engineers to collaborate with machines to co-create new products, which are not only novel but also more effective in terms of time to produce or impact on the environment. To achieve this, evolutionary computation may be used in computational design, where the generic approach is first to parameterize the topology of an underlying knowledge structure and then use a genetic program to modify it. More challenging example use case than designing a table is video game design as it also must integrate various multimedia data and complex artifacts such as, for instance, a monster or a narrative and must incorporate social and cultural context. This case may involve a team of human and robot designers who exchange creative ideas and solutions using languages shared between humans and machines.

2.1. Exploratory: Generation of new ideas by exploration of a space of concepts

Exploratory creativity refers to search within a pre-defined search space (limited by rules, constraints etc.). It is often modeled as an objective-driven search, using techniques such as constraint satisfaction, evolutionary algorithms, and data mining [36].

Regarding data mining, one may notice that its definition as the nontrivial process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data [20] has commonalities with definitions of computational creativity. Indeed, various techniques of data mining have found their applications in computational creativity, for tasks such as concept creation [47].

Potential for the Semantic Web Ontologies and knowledge graphs may provide conceptualizations for the given domain, including its constraints. As such, they serve to define the search space for generating novel concepts. Use cases such as generating novel recipes also concern procedural knowledge, thus one may pose research questions such as: Are existing ontologies and knowledge graphs sufficient to effectively support creative computing? or What other semantic resources are needed to fuel computationally creative systems?

Regarding methods, concept induction [18,35] and pattern mining [33] have been active areas of research in data mining in the Semantic Web context [32]. Many of these approaches use so-called refinement operators, i.e. functions that ‘traverse’ the search space and generate specializations or generalizations of concepts. For instance, an operator may add a primitive concept as the new conjunct to a complex concept (being an intersection of concepts), replace a primitive concept with its (primitive) subconcept, or add an existential restriction. Those refinements are further evaluated regarding their quality. To assess the quality of generated candidate concepts various measures can be used, not only based on frequency or predictive quality but also such that promote diversity [41] or novelty. The further interesting research question to study would be then: What properties should have refinement operators to support exploratory creativity on the Semantic Web? and What research is needed to define quality measures and evaluation procedures for concept creation with use of Semantic Web technologies that promote novelty?

2.2. Combinational: Novel combinations of familiar ideas

Creativity, understood as unfamiliar combinations of familiar ideas, dates back to the notion of bisociation by Koestler in 1964 [28], who describes creativity as a result of combining distinct frames of reference. The work of Koestler was followed by a subsequent cognitive theory of conceptual blending [19].

2.2.1. Conceptual blending

Conceptual blending is a process of inventing a novel concept (the blend) by combining two familiar input concepts. The framework of conceptual blending proposed by Fauconnier and Turner [19] concerns so-called mental spaces that connect schematic knowledge and frames representing the organization of elements and relations of the familiar knowledge. In the center of the conceptual blending theory, there is a conceptual integration network, which contains such elements as: (i) input spaces, (ii) a generic space (with a structure being an abstraction of commonalities of all the spaces of the system), (iii) a blended space, containing chosen aspects of the structures from the input spaces and its own, created structure, (iv) a partial mapping, connecting chosen aspects of the models in the input mental spaces. This basic framework may be also extended to include additional structure in the blend, that is not copied from the input spaces, via composition (which involves, possibly partial selection of elements), completion, and elaboration.

One of the classical examples of conceptual blending concerns the concepts of house and boat (e.g. [16,19,22]. Figure 1 illustrates one of the possible results, which is a house-boat concept (another example could be a boat-house concept).

OPEN IN VIEWER

Fig. 1.

The houseboat blend (adapted from [16,19,22]).

Various formalisms have been used to represent input spaces, including concept maps, frames, rules and constraints by Pereira [39], Prolog and micro-theories as in the system Divago [40], semantic networks used by Veale and Donoghue [50], description logics by Confalonieri et al., [10], Distributed Ontology Language by Kutz et al. [31], and algebraic specifications by Eppe et al. [16]. Not only concepts may be blended, but also ontologies, as proposed in [31].

The computational challenges associated with conceptual blending are: (i) to compute a generic space (representing what is common to the two input spaces), which can be later specialised to produce meaningful blends with elements from the input concepts and (ii) to ensure that there are no inconsistencies by combining concepts in a too naive and arbitrary way.

Genetic algorithms and neural networks can be used to generate blended concepts, capturing a combination of the inputs.

Potential for the Semantic Web There are multiple options for Semantic Web research in the area of conceptual blending. There have already been proposals to use the Web as a source of background information to generate blends, such as ‘conceptual mash-ups’ proposed by Veale [49]. Can thus these ideas be taken further, and can Linked Data and knowledge graphs be sources of vast amounts of (already structured) knowledge for producing blends? How can we combine or blend existing concepts, semantic frames, and other constructs, and benefit from cross-fertilization? Could we exploit (and how) Ontology Design Patterns [ 4 , 23 ] to represent a generic space?

When input spaces are being combined, another challenge is to compute a generic space automatically, especially for expressive representation languages, and many proposed blending approaches are not capable of it. Can thus automated approaches be developed for computing a generic space automatically, leveraging of reasoning services developed within the Semantic Web area and aimed to compute a most generic concept or of generalisation refinement operators? Even after the generic space is identified, there remains a challenge of a large number of possible combinations to generate blends. Some of them need to be pruned, and, besides using quality measures, can also consistency checking be used and how to prune blends?

2.2.2. Bisociation

The term bisociation was introduced by Koestler [28] to describe the creative act in humor, science and art. It stands for a blend of bi- + association. An association represents a relation between concepts within one domain, and bisociation fuses the information from multiple domains. Elements that are blended are taken from two (previously) unrelated ‘matrices of thought’ (or domains) (see: Fig. 2) to form a new matrix of meaning, applying processes such as abstraction, categorisation, analogies and metaphors, and comparisons.

OPEN IN VIEWER

Fig. 2.

A concept of bisociation, illustrated with a solid line connecting concepts c 1 , c 2 viewed in two different matrices of thought or from two different knowledge bases, versus an association, illustrated with a dashed line, which connects concepts from one matrix of thought or a knowledge base (adapted from [14,28]).

OPEN IN VIEWER

Fig. 3.

A BisoNet with a bridging concept (a concept connecting dense sub-graphs from different domains).

Since bisociative thinking occurs when a problem, idea etc. are viewed in two (or more) ‘matrices of thought’ or domains, to find bisociations it requires to integrate data from different domains. Bisociative Networks (BisoNets) [14] have been proposed as a method to compute Koestler’s bisociation, and to semantically integrate information. BisoNets are based on a k-partite graph structure with nodes representing units of information or concepts and with edges representing their relations. Each partition of a BisoNet contains a certain type of concepts or relations (terms, documents etc.). Kötter et al. [29] discuss three different kinds of bisociations: bridging concepts that connect dense sub-graphs from different domains (see: Fig. 3), bridging graphs that are sub-graphs linking concepts from different domains, and bridging by graph similarity, where bisociations are represented by structurally similar sub-graphs of different domains.

Once a BisoNet is formed, it can be mined for novel, and interesting information, and patterns (bisociations) to support creative discoveries. So-called creative information exploration aims to explore large volumes of heterogeneous information to discover new, surprising and valuable relationships in data that would not be mined by conventional information retrieval and data mining approaches [14]. The discovered links represent non-obvious connections and domain-crossing links, where concepts from various domains are not commonly related. One ‘classic’ example from literature of such non-obvious link regards connecting magnesium and migraine. There had been a body of articles on how migraine can be treated with calcium blockers, and another body of articles (not connected with previous ones) describing how magnesium works as a calcium blocker, but the potential to treat migraine with magnesium had not been realized [26].

Such connections may be discovered by graph mining and analysis techniques.

Potential for the Semantic Web The Semantic Web and Linked Data coincide with the model of BisoNets as heterogeneuos information networks, integrating concepts. How then the modeling choices of Linked Data may impact creative information discovery? Could thus the ideas and methods developed for creative information exploration be used to mine multi-domain Linked Data and vice-versa, i.e. can link discovery approaches developed within Semantic Web research be applied to creative information exploration? Mining potential novel associations between Linked Data [51] have already been explored within Semantic Web research. Can this be taken further? Is research on ontology mapping for bridging domains also relevant here? How to compute which nodes in the Semantic Web would bridge domains in creative ways?

Transformational creativity may be seen as meta-search, i.e. search not only for concepts, but also for rules, that is modifying rules and constraints and the search method. Transformational creativity takes place when the search space itself is also modified. The result are novel concepts in the modified space. This is the most difficult type of creativity to implement in a computational system, as one may argue that if the system is not equipped with some autonomy to change the rules, constraints or even the goals, nor it can be influenced by external information (beyond what a programmer equipped it with) then it only expresses the programmer’s creativity [25]. Therefore, some form of creative autonomy, i.e. when a system not only evaluates creations, but also changes its standards without being given an explicit direction [25], is required for transformational creativity.

But what inputs can a creative system receive to modify its standards? Toivonen [46] points that those may be: i) introspection, and ii) social interaction. As an example for the former take a system able to write songs, which modifies its own goals and operation [45]. It uses constraint programming, where constraints are used declaratively to define a search space of songs. Consequently, a standard constraint solver can then be used to generate songs. A meta-level control component manipulates the constraints at runtime based on self-reflection of the system. Regarding the latter, social interaction, it can be a source of new influences, ideas and feedback, and for developing creative autonomy it can be more plausible if the system is embedded in a broader society of other creators [25].

Potential for the Semantic Web Semantic Web technologies serve well to describe domain and cross-domain knowledge with making explicit constraints existing in a domain. Can we use ontologies and knowledge graphs, which describe an existing domain with its axioms and constraints and, applying a meta-rule for transformational creativity, start dropping constraints and adding new constraints to produce novel artifacts?

One transformatory assumption regarding reasoning in OWL and on the Web versus relational databases is to assume ‘open world’ rather than ’closed world’. Can we also change some other assumptions underlying reasoning on the Web to obtain novel problem settings and surprising and useful results?

The Semantic Web, envisaged as a multi-agent system with all of its technologies provides an opportunity for developing autonomous, creative agents that socially interact to gather new influences and ideas. They need to communicate using common languages and conceptualizations, shared between humans and machines to maintain common conceptual spaces. Due to this setting, the area of transformational creativity provides a big research opportunity to the Semantic Web specifically. How thus we should model and incorporate into a common conceptual space influences from other agents, e.g. other designers and customers, their preferences and aesthetics?