An automatic generation method of cross-modal fuzzy creativity

Abstract

Digital creativity is creative expression derived from cultural creativity and information technology. In order to overcome the problem in the creative generation in the condition of fuzzy and uncertain ideas, an automatic generation method of cross-modal fuzzy creativity (AGMCFC) is proposed. In this subject, fuzzy creative data sets and learning retrieval network are constructed for the sake of extracting original creative data effectively. And the logical correlations between creative objects are acquired dynamically based on the graph neural network. Creative objects and creative styles are generated by using generative adversarial nets technology and style transfer technology, respectively. Then, the projectiles, boundary markers and location words of the creative scene objects are generated by analyzing related attributes of each entity. After adjusting the layout, creative works are automatically generated. A fuzzy creative generating environment is implemented. Experimental results show that the screened number of AGMCFC method is about twice as much as that of manual method, and the accuracy rate of AGMCFC method is improved compared with the manual method. AGMCFC method performs well at creative generation of fuzzy ideas automatically.

Keywords

Generation of fuzzy creativity cross-modal graph neural network creative works

1 Introduction

Fuzzy theory is used to analyze and deal with many problems in the real world. It can be divided into fuzzy mathematics, fuzzy system [4], uncertainty and information, fuzzy decision [22], fuzzy logic and artificial intelligence [1]. Fuzzy theory has been applied in Internet of Vehicles (IoV) [2], ammunitions consumption prediction [11], mechanical and electrical manufacturing [19], expert system [21], data mining [12] and many other fields. The automatic generation of digital creativity is an integrative application of fuzzy system, fuzzy logic and artificial intelligence of fuzzy theory in creativity generation.

Digital creation, as a specific digital simulation, utilizes computers to simulate the human brain to build ideas [17]. It is a new economic form produced by the integration of information technology and cultural creativity. Digital creativity is mainly based on modern digital technologies such as computer graphics, which is different from traditional cultural creativity that treats entity as the vehicle for artistic creation. The overall framework of the digital creative process consists of four layers: motivation, conception, generation and evaluation. These layers are connected closely. Therefore, when the creative idea is fuzzy or uncertain, how to generate digital creative works becomes a complex and difficult problem that fuzzy systems need to be solved. In recent years, many new technologies and methods of digital creativity have emerged, such as Long Short Term Memory (LSTM), Generative Adversarial Networks (GANs) [16], Style Transfer (ST) [25] and so on, which are applied in painting, literature, music, stage and other creative fields. LSTM is used for generating literature works with similar styles, and composing music. GANs is used for generating similar image works and series frames in the movie. ST is used for generating object style. However, all these methods are coarse-grained creative generation or only partial creative generation. These methods do not study the generation process of creative works systematically and completely, therefore, the creative works that generated by these methods are often unable to meet the needs of fuzzy creativity.

Causal reasoning is particularly important for the universal creative process of digital creativity, so the logical correlation between objects should be considered in the process of extracting creative objects and constructing creative works. General machine learning methods (such as deep learning) can only carry out object feature learning, and cannot perform well in logical analysis. However, Graph Neural Network (GNN) defines a class of relational reasoning functions for graphical structure representation as its framework structure, which is suitable for the research foundation of this aspect [20]. GNNs have been explored in many problem areas, including supervised learning tasks, semi-supervised learning tasks, and unsupervised learning tasks. It can be simply divided into three application scenarios [15]: (1) structured scenario, where the data has a clear relationship structure, such as physical system, molecular structure and knowledge graph; (2) unstructured scene, where the relationship structure is not clear, including images, text, etc.; (3) other application scenarios, such as generation model and combinatorial optimization problems. GNN framework structure includes graph neural networks, Modified Probabilistic Neural Network (MPNN) and Non-Linear Neural Network (NLNN), etc. It supports to build complex structures from simple Building Blocks (BB), which is exactly what fuzzy creativity needs to be automatically generated. The automatic generation of fuzzy creativity with the characteristics of self-organization, emergence and self-coordination seems complicated, but strong artificial intelligence technologies with causal reasoning can simulate and support the implementation process, making digital creativity follow rules.

An Automatic Generation Method of Cross-modal Fuzzy Creativity (AGMCFC) based on GNN is proposed. During the proposed method, the GNN is used to calculate the correlation between data labels, the fuzzy logic relations are deduced between various objects, and the position and layout of creative works are given. Then an automatic generator of cross-modal fuzzy creativity is generated.

2 The frame model of the whole universal creativity generation process

Aiming at the creative process of digital creativity, a frame model of the whole universal creativity generation process is proposed (as shown in Fig. 1).

Fig. 1

The framework of the universal creativity generation process.

The specific steps of the framework model are described below.

The cross-modal multi-label fuzzy creative data sets are constructed on the basis of word segmentation and information extraction from texts and images of the key creativity. The image data can be acquired from the Internet or the database, and also can be collected from the latest image samples by wireless sensor network at any time. In order to speed down the energy consumption of the sensors near the sink, α-fraction first strategy for hierarchical model [14] in wireless sensor networks is used. The lifetime of the sensor network is extended, then more image samples are collected.

Based on the cross-modal multi-label fuzzy creative data sets, a cross-modal learning and retrieval network model is built to facilitate the classification and retrieval of image data.

A graph neural network is established to analyze the logical relationship between objects. The correlation between multiple tags is calculated by using GNN and tag semantics, and then the logical relationship between objects in the same mode is deduced.

Creative objects and creative styles are generated. GANs technology is adopted to generate the relevant creative objects if there is no suitable object in the material library. ST technology is adopted to generate the texture of the objects if the texture of the current image does not meet the requirements.

Projective bodies, bound scripts and localizers of creative scene objects are achieved from the related attributes of each entity in the GNN, making the spatial layout of creative entities conform to the logic of realistic placement. In order to ensure the security of the generated creative works, some encryption technologies [13] and [23] can be considered to encrypt them.

An EEG evaluation model based on deep learning [6] is adopted to assess the effectiveness of creativity generation.

3 Key technologies of the automatic generation method of cross-modal fuzzy creativity

Now, the key technologies involved in the implementation process will be analyzed and discussed in detail.

3.1 Construction of cross-modal multi-label fuzzy creative data sets and learning retrieval network

3.1.1 Construction of cross-modal multi-label fuzzy creative data sets

Texts and images of real-life or database are automatically collected and extracted. Due to the instability of data storage and communication transmission, some encryption techniques have been adopted to ensure that they are protected from malicious attacks. In particular, the three-party password-based authenticated key exchange technology can ensure the security in an offline environment [3]. In order to solve the problems of data space heterogeneous and semantic association of the text and image data, it is necessary to construct unified cross-modal multi-label fuzzy creative data sets. The data sets are divided into artistic conception library, object library, style paste gallery and so on.

The data sets are multi-labeled, and it is expressed as Equation (1). $\begin{matrix} {Dataset}_{cross - mode} = \\ ({Dataset}_{image - mode}, {Dataset}_{label - mode}, Map) \end{matrix}$ (1) where Dataset_image-mode is the image data set, Dataset_label-mode is the label data set, Map is the mapping relationship between Dataset_image-mode and Dataset_label-mode, f: Dataset_image-mode $\overset{Map}{\to}$ Dataset_label-mode.

The creative ontology domain consists of three parts: entities, attributes and relationships. Suppose there are M entities, N attributes and T connections in the creative ontology domain. Then, X_entity ={ X₁, X₂, …, X_M }, Y_attribute ={ Y₁, Y₂, …, Y_N }, Z_relationship ={ Z₁, Z₂, …, Z_T }.

For text-image cross-modal data objects, an entity represents an object with attributes such as name, color, emotion, and so on. Many entities constitutes cross-modal multi-label fuzzy creative data sets, as Dataset_cross-mode in Equation (1). Each Entity has its own global properties, as Dataset_label-mode in Equation (1). A relationship is a common property of two entities, as Map in Equation (1), such as above ... , growing in.., located at ... , and so on. A regular function is a function used to map entities or relationships, as Map in Equation (1), which takes one or two parameters and returns an attribute value.

We assume that X_entity is traversed. ∀X_i ∈ X_entity, the target network resource is crawled, then the image data set L and the label set K related to X_i are obtained. Crawler technology is used to automatically crawl labeled image data to expand the current creativity data set according to the entities and attributes of the creative ontology domain (as shown in Fig. 2).

Fig. 2

Construction of multi-label fuzzy creative data sets.

The attributes related to the object are extracted from the labels of the image, and the similarity and correlation of the attributes are calculated. We take the property closest to the entity in the ontology domain as the outer label of the object, and take the original tag of the object as the inner tag. Tagging is essentially to classify the object accordingly. L is added as the ith element in Dataset_image-mode. K is added as the ith element in Dataset_label-mode. In this way, the (Map) _i corresponding to the current element in the Map can be expressed as the mapping relationship between L and K, which is $f_{i} : L_{i} \overset{Map}{\to} K_{i}$ . Now, a hierarchical multi-label graph network structure is constructed as shown in Fig. 3.

Fig. 3

Construction of hierarchical multi-label graph network structure.

Computability analysis: The construction process of cross-modal multi-label fuzzy creative data sets is a series of mapping and storage operations through finite loop and invocation. Four basic operations have been done within each cycle. (1) The current elements of the set X are stored in Dataset_image-mode. (2) The current elements of the set Y are stored in Dataset_label-mode. (3) The current elements of X are mapped to the current elements of Y. (4) The map is stored into Map. The procedure is a loop call to the function. Hence the procedure is computable, and the algorithm complexity is O (4nm). Therefore, the construction process of cross-modal multi-label fuzzy creative data sets is computable based on three aspects of decision, computable function and complexity.

3.1.2 Construction of learning retrieval network

The deep network of each image is introduced to learn feature representation better. The classifier is constructed for each image, where the optimization target is punished by the dependency of label. Firstly, the weights of the deep network are trained in advance by contrast divergence method [5], and the weights of the classifier are randomly initialized. Then, the back propagation method is used to train both the deep network and the classifier. After training, an image can predict the labels of test samples by the output mean of its classifier. A cross-modal learning retrieval network is constructed as shown in Fig. 4.

Fig. 4

Construction of cross-modal learning retrieval network.

Computability analysis: The construction process of graph neural network is actually the construction of multiple entities and their relationships. It is the established by finite steps, and the algorithm complexity is O (N × (N - 1) + N × M). Therefore, the construction process of learning retrieval network is computable based on three aspects of decision, computable function and complexity.

3.2 Calculation of fuzzy logic relationships between creative entities based on GNN

GNN constructs its architecture through logically reason functions of directed graph structures. GNN framework uses simple Building Blocks (BB) to construct complex structures and describes the process of fuzzy logic reasoning. As a Graph, each entity consists of three aspects: nodes, global attributes, and edge relations. A node is represented as v_i, a Global Attributes (GA) is represented as u, and an edge is represented as e_k. A sender node index and a receiver node index are represented as s_k and r_k, respectively. Then, A Graph is defined as a triple, as shown in Equation (2). $G = (V, u, E)$ (2)

In the process of fuzzy creative automatic generation, the construction of cross-modal multi-label fuzzy creative data sets is the premise and basis of logic reasoning in GNN. Appropriate image data is selected to generate digital creative in their logical order.

This is illustrated by a picture of landscape. V = { v_i } _i=1:N^v is represented as a collection of nodes, where v_i is represented as an attribute of the node. If V is a mountain, v_i may be location, color, snow and other attributes. If V is a river or lake, v_i may be location, color, width, wave height and other attributes. u is represented as global attributes, such as altitude, depth, and so on. E = { (e_k, r_k, s_k) } _k=1:N^e is represented as edges, where e_k is represented as an attribute of the edge, r_k is represented as the receiver node index, and s_k is represented as the sender node index. E may be the relationship between different objects, such as spatial relations, the projectiles, boundary markers and location words, and so on. Table 1 shows steps of computation in a full GNN block.

Table 1

The GNN block calculation steps

Algorithm	Steps of computation in a full GNN block
function GraphNetwork (E, V, u)
fork∈ { 1 … N^e } do
$e_{k}^{'} \leftarrow \emptyset^{e} (e_{k}, v_{rk}, v_{sk}, u)$	1.Compute updated edge attributes
end for
fori∈ { 1 … Nⁿ } do
let $E_{i}^{'} = {(e_{k}^{'}, r_{k}, s_{k})}_{r_{k} = i, k = 1 : N^{e}}$
$e_{i}^{'} \leftarrow ρ^{e \to v} (E_{i}^{'})$	2.Aggregate edge attributes per node
$v_{i}^{'} \leftarrow \emptyset^{v} (e_{i}^{'}, v_{i}, u)$	3.Compute updated node attributes
end for
letV′ = { v′ } _i=1:N^v
let $E' = {(e_{k}^{'}, r_{k}, s_{k})}_{k = 1 : N^{e}}$
${\bar{e}}^{'} \leftarrow ρ^{e \to u} (E')$	4.Aggregate edge attributes globally
${\bar{v}}^{'} \leftarrow ρ^{v \to u} (V')$	5.Aggregate node attributes globally
$u' \leftarrow \emptyset^{u} ({\bar{e}}^{'}, {\bar{v}}^{'}, u)$	6.Compute updated global attribute
return (E′, V′, u′)
end function

A GNN block include three update functions ∅ and three aggregation functions ρ, as shown in Equation (3). $\{\begin{matrix} e_{k}^{'} = \emptyset^{e} (e_{k}, v_{r_{k}}, v_{s_{k}}, u) \\ v_{i}^{'} = \emptyset^{v} ({\bar{e}}_{i}^{'}, v_{i}, u) \\ u^{'} = \emptyset^{u} ({\bar{e}}^{'}, {\bar{v}}^{'}, u) \end{matrix}, \{\begin{matrix} {\bar{e}}_{i}^{'} = ρ^{e \to v} (E_{i}^{'}) \\ {\bar{e}}^{'} = ρ^{e \to u} (E^{'}) \\ {\bar{v}}^{'} = ρ^{v \to u} (V^{'}) \end{matrix}$ (3) where $E_{i}^{'} = {(e_{k}^{'}, r_{k}, s_{k})}_{r_{k} = i, k = 1 : N^{e}}, V^{'} = {v_{i}^{'}}_{i = 1 : N^{v}} a n d E^{'} = \cup E_{i}^{'} = {(e_{k}^{'}, r_{k}, s_{k})}_{k = 1 : N^{e}} .$

Then, all edges and nodes related to the graph neural network can be easily traversed and updated according to Equation (3).

In fact, the process of traversing and updating the edges and nodes of the graph neural network is to complete sequential extraction of material objects required for creative works and determination of logical correlations between objects.

Computability analysis: After constructing a graph neural network, it is necessary to calculate the logical correlation between the entities of the network. During this process, nodes, attributes and edges are to be updated. The update process is carried out by finite steps, and the algorithm complexity is $O (C_{N}^{M} + L + M \times (M - 1))$ . Therefore, the calculation of fuzzy logic relationships is computable based on three aspects of decision, computable function and complexity.

3.3 Generation of creative objects and creative styles

At the creative stage, GANs technology is adopted to generate creative objects, and ST technology is adopted to generate creative styles. The stage scene is taken as an example to illustrate its generation process, and its principle can also be extended to other types of creative works.

GANs was first proposed by Ian Goodfellow. It extends existing neural networks for processing data represented in graph domains. Based on the idea of game theory, GANs consists of two networks, a generator and a discriminator [10]. The generator is trained to fool discriminators into producing realistic images, and the discriminator is trained not to be fooled by the generator. First, the generator extracts a noise vector Z through a simple distribution (such as a normal distribution), and upsample this vector to generate the image. In the initial loop, the images look very noisy. Then, the discriminator gets the true and false images and learns to recognize them. Subsequently, the generator receives feedback from the discriminator based on Backpropagation Algorithm, and gradually performs better in the generation of images. Finally, the distribution of the false image is as close to the true image as possible. In other words, fake images are expected to look as real as possible. The main GANs algorithm is shown in Equation (4). $\begin{matrix} {min}_{D} {max}_{G} V (D, G) = E_{X \sim Pdata (X)} [logD (X)] \\ + E_{z \sim Pz (z)} [\log (1 - D (G (z)))] \end{matrix}$ (4)

Computability analysis: In the generation process of creative objects based on GANs, cyclic convolutional neural network is used to generate creative objects in finite loop and iterative computation. We assume that N₁ is the number of neurons in the input layer and N_m is the number of neurons in the output layer, then N₂ to N_m-1 is the number of neurons in each layer of the middle hidden layer. If there are L samples, the algorithm complexity is: O (L × (N₁ × N₂ + N₂ × N₃ + … + N_m-1 × N_m)). Therefore, the generation process of creative objects is computable based on three aspects of decision, computable function and complexity.

After selecting the creative objects or generating the desired creative objects, the desired styles of objects are generated based on ST. The generating principle of ST is to input a white noise image and generate a style image based on it. A content image and a style image are fused based on ST to generate an output image that retains the original content and the style [18]. In other words, the main idea is to give a content object and a style map for fusion generation, and then to introduce a certain style while keeping the content unchanged.

The style-generating framework is divided into two parts, one is the Image Transform Net (ITN) T, and the other is the pre-trained Loss Network (LN) vgg - 16 (as shown in Fig. 5). T takes the content image x as input, and outputs the image y′ after style transfer. Then, content-based image yc (that is x), style-based image ys and the image y′ input the computing characteristics of vgg - 16 [7]. The loss calculation is shown below.

Fig. 5

The network framework diagram of style-generating.

Content loss calculation function $l_{feat}^{φ; j} (y) = \frac{1}{C_{j} H_{j} W_{j}} ∣ ∣ φ_{j} (y^{'}) - φ_{j} (y) ∣ ∣^{2}$ (5) where φ stands for deep convolution network vgg - 16.

Perceived loss calculation function $l_{style}^{G; j} (y) = ∣ ∣ G_{j} (y^{'}) - G_{j} (y) ∣ ∣_{F}^{2}$ (6) where G is the Gram matrix and $G_{j}^{φ} {(x)}_{c^{'}, c} = ∣ ∣ G_{j}^{φ} (y^{'}) - G_{j}^{φ} (y) ∣ ∣$ .

Total loss calculation function ${Loss}_{total} = γ_{1} l_{feat} + γ_{2} l_{style}$ (7) when the specified loss is within the threshold range, the style generation is considered to conform to the quality of the generation.

Computability analysis: In the generation process of creative styles based on ST, the style-generating framework is divided into the Image Transform Net and the pre-trained Loss Network. The calculation functions of ST are composed of finite addition, subtraction, multiplication, division and matrix multiplication. Therefore, the generation process of creative styles is computable based on three aspects of decision, computable function and complexity. To sum up, the generation of creative objects and creative styles is computable.

3.4 Generation of creative works

3.4.1 The placement sequence of scene objects in a creative work

After the construction of the GNN, the order of objects in the creation work has been determined. Relevant attributes between nodes are extracted from the sender to the receiver. However, it may not specify the relationship between entities in the creative point, so the information of this part needs to be supplemented. A generation sequence model is proposed between objects to supplement this missing information. The generation process of virtual creative scenes is the same as the order of scene construction in real life, which follows the principle from bottom to top. The sequence of construction is laying foundations, covering vegetation, placing buildings, placing everyday tools, and finally placing people and animals. Basic placement sequence model is shown in Fig. 6.

Fig. 6

Basic placement sequence model of scene objects.

3.4.2 The composition methods of creative works

In the GNN, after global information is extracted, a basic composition method or a combination of multiple local composition methods is need for the creative works if the layout and composition of creative works is not obtained. The common basic composition methods include pyramid composition method, S shape composition method, diagonal composition method, cross composition method, lateral composition method, full composition method, symmetrical composition method, et al.

Now, an example of “bridges, rivers and houses” is given to explain the creative works generation process. After global information extraction is completed in GNN, if the lack of layout and composition information is found, the corresponding layout and composition information is added to it. Bridges should be above rivers, and the people should be above bridges. These two arrangements follow the pyramid composition method (that is to set large models under small models). Houses should be next to rivers based on both pyramid composition method and lateral composition method. This layout conforms to the logic of real life placement.

3.4.3 The modular script description of fuzzy creative generation process

A graph neural network is constructed based on creative points, and each node and its related attributes are extracted according to the direction of its edges to determine the logical relationship between entities. The modular script description of fuzzy creative generation process is shown in Fig. 7. The function of each model is described below.

Model Searcher: Each entity of the creative node in GNN is traversed. And a < model classification table > is preset. The model files are traversed and compared that conform to relevant attributes imported from the model library of the data sets. After that, the existing models are recorded and the number of the models is counted. At the same time, all recorded model positions are cleared (0, 0, 0). Then, a < model asset table > is generated.

Model Classifier: The model names of the preset < model classification table > are compared with those of the < model asset table > which is generated by the model searcher. Then, a < asset classification table > is generated.

Model Collision Detector: The model collision detector is used to detect whether two models are interspersed or collided in the simulation world of fuzzy creativity generation.

Composition Method Selector: Composition method selector is used to call composition methods according to the actual creative scene. These composition methods include pyramid composition method, S shape composition method and so on.

Creative Scene Arranger: The models selected from the < asset classification table > are arranged according to the construction principle (as shown in Fig. 6). If a model is used, it is recorded in < model arrange table > to count the arranged models.

Fig. 7

Modular script description of fuzzy creative generation process.

When placing objects, the projectiles, boundary markers and location words [8 , 24] of the generated creative scene objects are analyzed by GNN, so as to adjust the appropriate position to complete the composition. If there is no specific boundary mark or location word in the attributes, based on the direction of the edges in GNN, set the boundary mark and location word of the corresponding object to complete the creative scene composition according to the given basic scene object composition order and composition method. The creative scene arranging process without boundary mark of location word in the attributes is shown in Fig. 8.

Fig. 8

The creative scene arranging process without boundary mark of location word in the attributes.

Computability analysis: The generation process of creative works is the extraction and construction of finite steps according to certain rules to construct the space of creative works. Its algorithm complexity depends on the number of detailed steps N of extraction and construction. The algorithm complexity is O (N). Therefore, the generation process of creative works is computable based on three aspects of decision, computable function and complexity.

4 Evaluation of the creative works generation results

A creativity generation environment is designed and implemented. In this environment, cross-modal digital creativity can be generated for text and image data. Besides, the performance of the proposed AGMCFC method is compared with that of the manual method.

4.1 Implementation of fuzzy creative generating environment

Description of experimental data: 15 creative points that meet the needs are extracted from 100 creative points. After word segmentation, 256 related words including entities, attributes and relationships are obtained. There are 66 entity words, 90 attribute words and 100 relationship words. Based on these entity words and attribute words, relevant document data and image data are collected. Meanwhile label data are extracted through the document data. Now, multi-label fuzzy creative data sets are constructed. Examples of multi-label fuzzy creative data sets are shown in Table 2.

Table 2
Examples of cross-modal multi-label fuzzy creative data sets

Serial number Image dataset Entity Attribute Label

1 Desert Dry, boundless... Landscape, desert, dry, boundless, flowing spread...

2 Field Gold, infinity... Landscape, fields, gold, infinity, growth, harvest...

3 Forest Lush, vast ... Landscape, forest, lush, vast, moist...

4 Tree Green, tall... Plants, trees, green, tall, grow, absorb...

5 Flowers Beautiful, fragrance... Plants, flowers, beautiful, fragrant, blooming, attracting...

6 Grass Lush, verdant... Plants, grass, lush, tender green, root, wither...

7 Stone Rugged, hard... Landscape, stone, rugged, hard, scattered, placed...

8 ... ... ... ...

Serial number	Image dataset	Entity	Attribute	Label
1		Desert	Dry, boundless...	Landscape, desert, dry, boundless, flowing spread...
2		Field	Gold, infinity...	Landscape, fields, gold, infinity, growth, harvest...
3		Forest	Lush, vast ...	Landscape, forest, lush, vast, moist...
4		Tree	Green, tall...	Plants, trees, green, tall, grow, absorb...
5		Flowers	Beautiful, fragrance...	Plants, flowers, beautiful, fragrant, blooming, attracting...
6		Grass	Lush, verdant...	Plants, grass, lush, tender green, root, wither...
7		Stone	Rugged, hard...	Landscape, stone, rugged, hard, scattered, placed...
8	...	...	...	...

The fuzzy creative generating environment is designed and implemented. Based on GNN, the logical relationship between creative objects are obtained, then related objects are extracted. The required creative works are constructed based on projective bodies, bound scripts and localizers of the creative point. GANs technology [16] and ST technology [25] are used to achieve object generation and Style generation, respectively. Finally, satisfactory creative works are screened by the EEG evaluation model [6]. The EEG evaluation model is computable. Hence, the whole process of the proposed AGMCFC is computable. An example of fuzzy creative generation process is shown in Fig. 9.

Fig. 9

An example of fuzzy creative generating environment: (a) add a scene; (b) add a sky; (c) add trees; (d) add flowers and plants; (e) add stones.

4.2 The comparison between AGMCFC method and manual method

In order to test the performance of the proposed AGMCFC method, it is compared with manual method at different periods, as shown in Fig. 10.

In Figure 10, the screened number of creative works of the proposed AGMCFC method is significantly larger than that of manual method in unit time. It is about twice as much as that of manual method. The AGMCFC method integrates information technologies with cultural creativity to solve the problem of low efficiency of traditional manual method, therefore, the generation efficiency of AGMCFC method is higher than that of manual algorithm in unit time. AGMCFC method avoids the semantic gap of object extraction by labeling multi-label semantics on each object, so as to achieve relatively accurate extraction. AGMCFC method solves the problem of inaccurate extraction to a large extent. Besides, the accuracy rates of creative generation of AGMCFC method at any time are all above 90% and the accuracy rate of AGMCFC method on average is 9.6% higher than that of manual method. Especially at Time 4 (affected by external interference), the accuracy rate of the proposed AGMCFC method is nearly 20% higher than that of manual method, which demonstrates the feasibility of replacing manual method with AGMCFC method. Hence, the AGMCFC method performs well at creative generation of fuzzy ideas automatically.

Fig. 10

Comparison between AGMCFC method and manual method.

5 Conclusion

An automatic generation method of cross-modal fuzzy creativity is proposed. The logical correlation between objects is calculated by the proposed AGMCFC method based on GNN. The AGMCFC method realizes the automatic generation of creativity under the condition of fuzzy ideas to some extent. It is convenient to extract cross-modal creative objects on the basis of constructing of multi-label fuzzy creative data sets and learning retrieval network. GANs technology and ST technology are used to achieve object generation and style generation, respectively. The spatial layout of creative entities of the AGMCFC method conforms to the logic of realistic placement. Experimental results show that the efficiency and the accuracy of the AGMCFC method are significantly higher than that of the manual method. AGMCFC method can generate satisfactory creative works in the case of fuzzy or uncertain ideas, which demonstrates the feasibility of replacing manual method with AGMCFC method.

Footnotes

Acknowledgments

To the Research Program Foundation of Minjiang University (Grant No. MYK17021), the National Natural Science Foundation of China (Grant No. 61772254), and Beijing Finance Project (Grant No. PXM2019_178214_000004) for their support.

References

Rajeswari

A.M.

and Deisy

, Fuzzy logic based associative classifier for slow learners prediction, Journal of Intelligent and Fuzzy Systems36 (2019), 2691–2704.

Chen

C.-M.

, Xiang

, Liu

and Wang

K.-H.

, A secure authentication protocol for Internet of Vehicles, IEEE ACCESS7 (2019), 12047–12057.

Chen

C.-M.

, Wang

K.-H.

, Yeh

K.-H.

, Xiang

and Wu

T.-Y.

, Attacks and solutions on a three-party password-based authenticated key exchange protocol for wireless communications, Journal of Ambient Intelligence and Humanized Computing10 (2019), 3133–3142.

Hua

, Wang

, Xu

, Li

and Gombay

, Fuzzy system for monitoring energy consumption of wireless sensor network nodes, Journal of Intelligent and Fuzzy Systems35 (2018), 4319–4328.

Geoffrey

E.H.

, Simon

and Yeewhye

, A fast learning algorithm for deep belief nets, Neural Computation18 (2006), 1527–1554.

Zhang

F.Q.

, Mao

Z.J.

, Huang

Y.F.

, Xu

and Ding

G.Y.

, Deep learning models for EEG-based rapid serial visual presentation event classification, Journal of Information Hiding and Multimedia Signal Processing9 (2018), 177–187.

Dahl

G.E.

, Yu

, Deng

, et al., Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition, Audio, Speech, and Language Processing, IEEE Transactions on20 (2012), 30–42.

H.J.

, Research on spatial conceptual model based on natural language processing, Ph.D. Dissertation, Harbin Institute of Technology, 2007.

H.J.

, Li

, Zhao

T.J.

, et al., The research and realization of the layout of objects in 3D Scene based on natural language understanding, Journal of Electronics & Information Technology29 (2007), 1845–1849.

10.

Goodfellow

, Pougetabadie

, Mirza

, et al., Generative adversarial nets, Advances in Neural Information Processing Systems (2014), 2672–2680.

11.

J.H.

, Huang

, Yu

H.M.

and Zhang

M.L.

, Prediction model of ammunition consumption based on fuzzy logic theory, Journal of Ordnance Equipment Engineering40 (2019), 150–153.

12.

J.M.-T.

, Lin

J.C.-W.

and Tamrakar

, High-utility itemset mining with effective pruning strategies, ACM Transactions on Knowledge Discovery from Data (2019). https://doi.org/10.1145/3363571

13.

Pan

J.-S.

, Lee

C.-Y.

, Sghaier

, Zeghid

and Xie

, Novel systolization of subquadratic space complexity multipliers based on toeplitz matrix–vector product approach, IEEE Transactions on Very Large Scale Integration Systems27 (2019), 1614–1622.

14.

Pan

J.-S.

, Kong

, Sung

T.-W.

, Tsai

P.-W.

and Snasel

, Alpha-fraction first strategy for hierarchical wireless sensor networks, Journal of Internet Technology19 (2018), 1717–1726.

15.

Wang

, Overview of graph neural network, Modern Computer23 (2019), 58–62.

16.

, Zhang

, Xue

, et al., Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling, arXiv:1610.07584v2 [cs.CV], 4 Jan 2017.

17.

Lee

K.C.

, DCSE (Digital Creativity Simulation Engine), Digital Creativity Model and Its Relationship with Corporate Performance, Springer International Publishing, 2016.

18.

Gatys

L.A.

, Ecker

A.S.

, Bethge

, Image style transfer using convolutional neural networks, in: Computer Vision and Pattern Recognition, IEEE Computer Society Press, 2016, pp. 2414-2423.

19.

L.Z.

, Ma

and Liao

, Fuzzy control strategy of double motor for screw pile machine, Journal of Mechanical & Electrical Engineering36 (2019), 1083–1088.

20.

Battaglia

P.W.

, Hamrick

J.B.

, Bapst

, et al., Relational inductive biases, deep learning, and graph networks, arXiv:1806.01261v3 [cs.LG], 17 Oct 2018.

21.

Chen

S.M.

, New methodology to fuzzy reasoning for rule-based expert systems, Cybernetics and Systems26 (1995), 237–263.

22.

Lin

T.-L.

, Chuang

C.-H.

, Chen

S.-L.

and et al., An efficient image processing methodology based on fuzzy decision for dental shade matching, Journal of Intelligent and Fuzzy Systems36 (2019), 1133–1142.

23.

T.-Y.

, Chen

C.-M.

, Wang

K.-H.

, Meng

and Wang

E.K.

, A provably secure certificateless public key encryption with keyword search, Journal of the Chinese Institute of Engineers42 (2019), 20–28.

24.

Wang

, Automatic spatial layout of named entity in natural sentence, Master Dissertation, Harbin Institute of Technology, 2014.

25.

Jing

, Yang

, Feng

, et al., Neural style transfer: a review, arXiv:1705.04058v6 [cs.CV], 17 Jun 2018.