Construction of real-time sharing model of energy big data based on end cloud collaboration technology

Abstract

A real-time sharing model of energy big data based on end cloud collaboration technology is built to safely and efficiently share energy big data in all fields. Through the collaboration between the client layer and the cloud platform layer in the end cloud collaboration module, combined with the vertical federation learning algorithm and the homomorphic encryption algorithm, the energy big data knowledge in various fields is extracted and encrypted, and the encrypted knowledge is stored in the cloud platform as shared data. After the blockchain module combines the smart contract identification coding and parsing of such shared ciphertext, the ciphertext key is provided to the data user, and the shared energy big data plaintext is obtained after decryption, so as to realize the real-time security sharing of energy big data. According to the result analysis, the model performs well in data knowledge extraction and encryption, and has a good effect in ensuring the security and reliability of energy big data sharing. At the same time, the identification coding and analysis time of shared data knowledge is relatively short, making energy big data can be shared in real time. These results demonstrate the potential and feasibility of the model in facilitating big data sharing in the energy sector.

Keywords

End cloud collaboration energy big data real-time sharing client cloud platform blockchain

1 Introduction

The big data of energy comes from the energy internet, which is a complex multi network flow system formed by the close coupling of power grid, natural gas network, water supply network, transportation network and other networks, with the energy system as the core and Internet technology as the support [1]. The energy Internet basically realizes real-time monitoring of the operation status of grid nodes, and the collected data is collected into the data center system of the power company through the data acquisition base station or network dedicated line [2]. With the proposal of China’s dual carbon goals, accelerating the transformation of energy structure has brought new opportunities and challenges to the development of energy Internet.Under the deep integration of the energy revolution and the digital revolution, it has become an inevitable trend for big data to help the innovative development of energy Internet [3]. However, to fully tap the value of the big data of the energy Internet, it is necessary not only to tap the operation status data generated in various links such as energy production, transmission and consumption, but also to tap the data from society, economy, climate, human settlements, transportation and other energy systems (such as natural gas network and heating network) that will affect the construction and operation of the energy Internet. Therefore, the problem of data sharing between different data owner entities needs to be solved [4]. The storage and utilization of traditional energy big data is an information island centered on enterprises. The lack of efficient and reliable data sharing mechanism directly hinders the information mining of energy big data. In addition, due to the massive, diversified, streaming and other characteristics of energy Internet big data, there are still large barriers to data sharing among data owner entities [5]. Therefore, in order to solve such problems, it is necessary to study an appropriate data sharing model to realize the effective sharing of energy big data among data owner entities. At present, many scholars have carried out relevant research on data sharing in various fields. For example, the data sharing technology of 5 G UAV based on blockchain, which was studied by Feng et al. [6], is to build a data sharing model by using blockchain combined with attribute encryption technology, enhance the security of shared data, and achieve data authentication and access control based on smart contracts through the authentication mechanism in the model. The technical model can realize data security sharing, but the real-time performance of data sharing is not ideal. The blockchain based V2 G network lightweight data sharing model studied by Hassija, V and others is to create a lightweight protocol through the blockchain distributed ledger technology, and realize network data security sharing in combination with this protocol. Although this model can achieve the security sharing of network lightweight data, its scope of use is too limited, and the real-time nature of data sharing cannot be effectively guaranteed [7].

Cloud computing is a centralized service mode that uses virtualization, on-demand services and other technologies to realize resources (including computing, storage, software, data and other resources). In traditional cloud computing architecture, cloud platforms are usually used as large data centers and computing centers [8]. However, with the arrival of the fifth generation mobile communication (5 G) and the development of the Internet of Things (IoT) technology, facing the challenges of bandwidth consumption, network delay, data privacy protection, etc., the centralized cloud platform only deals with computing tasks that require large computing resources and have low real-time requirements. Based on this, an edge computing architecture of cloud edge collaboration has emerged, that is, end (client) cloud (cloud platform) collaboration technology. This technology has excellent performance in data knowledge mining. It can sink data mining algorithms from the cloud platform to the user end according to the actual needs of users, and achieve data knowledge mining and extraction at the user end, which to a certain extent alleviates bandwidth and real-time problems [9]. However, due to the actual application of this technology, its data is distributed in different management domains, and the organizational structure of cloud edge is different, which brings some challenges to the sharing method of decentralized chain data and trusted computing.

Blockchain is a decentralized, tamper proof, traceable and multi-party jointly maintained distributed database, which can integrate multiple isolated databases that only involve their own business, and store them on multiple nodes jointly maintained by multi-party [10]. Blockchain solves the problem of data trustworthiness by integrating peer-to-peer (P2P) protocol, asymmetric encryption, consensus mechanism, blockchain structure and other technologies. Through the use of blockchain, it is not necessary to rely on any third-party trusted institutions, and multiple parties that do not know each other and do not trust each other can achieve reliable and peer to peer data knowledge sharing and transmission [11].

In conclusion, based on the characteristics and advantages of the end cloud collaboration technology and blockchain technology, this paper designs a real-time sharing model of energy big data, which is composed of the end cloud collaboration module and the blockchain module, to achieve the safe and reliable sharing of energy big data, which is characterized by high security, high reliability and high timeliness. It provides new solutions for knowledge sharing of energy big data. The overall research route of the model is as follows:

The data provider on the client side will mine the energy big data in various fields and form the corresponding resource directory after collection.

After the cloud platform receives the data sharing request from the data user, it sinks the vertical federated learning algorithm to the client. The client uses the algorithm to extract the corresponding initial energy big data knowledge, and encrypts such data knowledge in combination with the homomorphic encryption algorithm. The obtained energy big data knowledge ciphertext is uploaded to the cloud platform memory as shared data.

The blockchain implements identification coding and parsing for the knowledge ciphertext in the cloud platform, and provides the data user with the ciphertext key of the required data knowledge. After the user decrypts, the required shared energy big data is obtained, and the real-time sharing of energy big data is completed.

The experiment verifies the sharing performance of the model from two aspects: the effect of knowledge extraction and encryption of energy big data, and the effect of knowledge identification coding and parsing of shared data.

2 Research on real-time sharing model of energy big data

2.1 Overall architecture of real-time sharing model of energy big data based on end cloud collaboration

With the end cloud collaboration technology as the core, combined with the blockchain technology, build a real-time sharing model of energy big data, realize the extraction of energy big data knowledge based on the end cloud collaboration technology and the safe sharing of energy big data based on the blockchain technology, so as to achieve the goal of safe real-time sharing of energy big data in all fields. The model is mainly composed of the end cloud collaboration module and the blockchain module. The end cloud collaboration module includes two parts: the client layer and the cloud platform layer. The overall architecture of the real-time sharing model of energy big data is shown in Fig. 1.

Fig. 1

Overall architecture of real-time energy big data sharing model.

The model realizes collaboration and interaction among client, cloud platform and blockchain through data upload, data download, metadata transmission, cloud registration, outsourcing encryption and decryption and other interfaces. The functions of the two key modules in the model are as follows:

End cloud collaboration module: through the cloud platform layer in this module sinking the federated learning algorithm to the client layer, the effective mining, collection and knowledge extraction of energy big data in various fields are realized, and the extracted knowledge is stored in each cloud platform of the cloud platform layer respectively, so as to realize the knowledge extraction of energy big data in multiple fields based on end cloud collaboration. The client layer of this module is mainly composed of energy big data providers and users. The energy big data providers upload, update, share and other operations of energy big data in various fields, provide energy big data knowledge sharing ciphertext to the cloud platform layer, develop access strategies, use homomorphic encryption algorithms to calculate the intermediate key ciphertext, generate access metadata and upload it to the blockchain module. The data user obtains access metadata from the blockchain module, that is, after access authorization, the cloud platform layer obtains the energy big data knowledge sharing ciphertext for decryption, and obtains the energy big data knowledge required after decryption.

Blockchain module: the sharing process of energy big data knowledge realizes trusted transactions through the identification code of the blockchain module, and realizes the analysis of knowledge identification code and the automation of authority management during the sharing process through smart contracts, so as to ensure the security and real-time of energy big data knowledge sharing. The smart contract of this module also includes attribute token management contract, outsourcing encryption contract and pre decryption contract. It is mainly responsible for collecting user attribute sets, verifying whether user attributes meet attribute policies, and implementing outsourcing encryption and pre decryption. At the same time, according to the policies formulated by the client, it extracts backup metadata, encodes it into a blockchain global unique identifier, and stores it in the status database, so as to address the shared energy big data knowledge. In blockchain-based energy big data sharing models, a consensus mechanism is usually used to ensure block generation and verification, and to guarantee the consistency and credibility of the data. In the public chain, the common consensus mechanism includes Proof of Work (PoW), which requires participating nodes to compete for block generation rights by solving complex algorithm problems. There are also Proof of Stake (PoS), which determines the probability of obtaining block generation rights based on the number of crypto tokens held. There are also Byzantine consensus algorithms (BFT) based on fault-tolerant Byzantine problems, such as Practical Byzantine Fault Tolerance (PBFT) to achieve Fault Tolerance. Each block usually contains a block header and block body. The block header contains metadata, such as the block’s version number, timestamp, and hash of the previous block. The block body contains a specific transaction or data record, which is usually stored in the block in the form of a transaction or record. Storage can be designed according to the specific application needs, using hash tables, Merkle trees, or other data structures to organize and manage data.

Through the effective integration of the end cloud collaboration module and the blockchain module, the safe and real-time sharing of energy big data in all fields is realized. The fusion mechanism of the two modules is shown in Fig. 2.

Fig. 2

Fusion mechanism of end-cloud collaboration module and blockchain module.

The collaboration mechanism of the end cloud collaboration module and the blockchain module is the basis of their integration. Here, the collaboration mechanism of the two is designed from three levels, namely computing collaboration, storage collaboration and interactive collaboration. Based on this, the client cloud platform interface (end cloud interface), client blockchain interface (end chain interface) and cloud platform blockchain interface (cloud chain interface) are designed respectively to realize the collaborative integration mechanism of the two modules. Among them, the end cloud interface includes ciphertext data upload interface and ciphertext data download interface, the end chain interface includes user registration interface and metadata management interface, and the cloud chain interface includes cloud registration interface, metadata transmission interface, outsourcing encryption and decryption computing interface and integrity metadata transmission interface.

2.2 Energy big data knowledge extraction of the 2-terminal cloud collaboration module

The end cloud collaboration module sinks the federated learning algorithm from the cloud platform layer to the client layer according to the actual needs of data users. It realizes the mining, collection and knowledge extraction of energy big data in various fields on the client layer, extracts valuable knowledge from the energy big data after mining and collection, and achieves the purpose of simplifying shared data and improving the value density of shared data. At the same time, it effectively solves the problems of excessive data transmission load, data privacy and data security in energy big data sharing. The energy big data knowledge extraction framework of the end cloud collaboration module is shown in Fig. 3.

Fig. 3

Energy big data knowledge extraction framework of end-cloud collaboration module.

The whole knowledge extraction process is as follows:

The data provider in the client layer will mine the energy big data sensed and measured by each subject in the energy Internet, and collect the mined data through the edge server node in this layer to establish the resource directory corresponding to the collected data. At the same time, the collected energy big data and the corresponding resource directory will be uploaded to the cloud platform layer together.

After the data user determines the required data source according to the data resource directory on the cloud platform and in combination with its actual needs, it sends the energy data sharing request to the data provider through the blockchain backbone network. After the data user and the data provider reach a data sharing agreement, it puts forward the demand for energy data knowledge extraction to the cloud platform layer, the cloud platform layer sinks the federated learning algorithm to the client layer.

The client layer selects the server nodes that meet the computing power and data requirements, reasonably deploys the federated learning algorithm sunk by the cloud platform layer, and uses the deployed algorithm to process and extract the original energy big data on the client, converts the original energy big data into data knowledge, and the extracted data knowledge is encrypted through the homomorphic encryption algorithm, and uploaded to the cloud platform layer.

After the cloud platform aggregates the data knowledge extracted by each server node of the client, it completes the end cloud collaborative training in combination with steps (2) and (3) to promote the interaction between the client and the cloud platform.

The data user finally obtains the required energy big data knowledge through the blockchain module.

Among them, when the client layer collects energy big data, it mainly collects energy big data in various fields, such as power grids, energy suppliers, power generators, load aggregators, gas departments, heating departments, traffic supervision departments, etc., mined by various data providers [12]. The main collection process is shown in Fig. 4.

Fig. 4

Process diagram of energy big data collection at the client layer.

The energy big data collection process of the client layer mainly includes data source acquisition, peer data transmission, midrange data collection, data distribution and data persistence. Specifically, energy big data collection is a combination of Internet of Things technology and data transmission technology. The energy big data information (including wind power/photovoltaic power station real-time operation data, box transformer/substation transformer real-time operation data, booster station real-time operation data, station active/reactive power control system data, protection and fault information data, energy metering information data, etc.) excavated by the data provider from various fields. It is transmitted to the energy IoT gateway at the middle end of the server according to the specified transmission protocol and transmission path [13]. After the energy IoT gateway prefabricates such data through communication, labeling, pre-processing and other operations, it transmits such data to the distributor through the data IoT Hub in the middle end. The distributor distributes such data to the stream processing process and batch processing process respectively. After processing, it obtains persistent energy big data and completes the energy big data collection process.

Because the data providers in the client layer have different industries, the energy big data mined by each provider is different, and there are obstacles in information exchange between them, which makes it difficult to meet the needs of all data users at the same time [14]. Federated learning technology is a method of implementing shared data mining through distributed machine learning, which has the technical advantage of data privacy protection [15]. Therefore, in order to solve the above problems, this paper selects the longitudinal federal learning algorithm in the federal learning technology as the edge knowledge extraction algorithm, and combines the homomorphic encryption algorithm. On the basis of aligning and encrypting the energy big data knowledge, it establishes a collaborative training mode for the energy data model of each data user, so that each data user can only get the model parameters related to himself. Thus, under the condition of ensuring data security, it can realize knowledge collaborative mining among multiple data sources of energy big data in various fields, and achieve the purpose of collaborative training of multiple data users and knowledge sharing of energy data. The vertical federated learning model is shown in Fig. 5.

Fig. 5

Vertical federation learning model.

The energy big data knowledge extraction process of vertical federated learning algorithm is as follows:

Set B_i represents the i big energy data mined by individual data providers. A indicates the data feature label. C indicates data label information. D indicates the ID of the data sample. Thus, a complete energy big data training dataset can be expressed as B = (A, C, D). The feature of vertical federated learning is the data feature tag of the training dataset I and data label information C the overlap is small, while the overlap of the data sample ID is large. This feature can be expressed as: $A_{i} \neq A_{j}, C_{i} \neq C_{j}, I_{i} = I_{j}, \forall B_{i}, B_{j}, i \neq j$ (1)

The optimization objectives of longitudinal federal learning can be expressed as: $min \sum_{i = 1}^{m} Ef ({u_{j}^{l}}_{l = 1}^{n}, C_{i})$ (2)

Where, m indicates the number of instances in the energy big data training dataset. E represents the master node model parameters of the longitudinal federated learning model. f (·) represents the integration objective function running on the federated master node. $u_{j}^{l}$ represents the l data users, through their sub models j data knowledge obtained from data calculation. n indicates the number of data users participating in the federated learning of this layer. C_i refers to the No i data label information.

The section l sub models run by data consumers are defined by the data j knowledge extracted from the data can be expressed as: $u_{j}^{l} = F_{l} (A_{j}^{l}, E_{l})$ (3)

Where, F_l (·) represents the l Sub model calculation function of data users. $A_{j}^{l}$ represents the l number of data users j feature labels of data. E_lrepresents the l parameters of sub models run by data consumers.

In order to ensure the security of the energy big data knowledge extracted by the vertical federated learning algorithm, we hereby encrypt the extracted data knowledge in combination with the homomorphic encryption algorithm [16]. Homomorphic encryption algorithm mainly includes five parts: initialization algorithm, key generation algorithm, encryption algorithm, decryption algorithm, and master private key decryption algorithm, which are expressed as Ini (k), KeyG, Enc, Dec, KeyDec. At the same time, the encryption algorithm also has the property of double trapdoors, that is, after the plaintext is encrypted by the public key, it can be decrypted not only by the private key, but also by the master key [17]. Each part of the algorithm is as follows:

Initialization algorithm Ini (k): First give the security parameters k indicate modulus G and select two different prime numbers h′ and r′, respectively h = 1 +2h′, r = 1 +2r′, and hr = G. Then select $o \in X_{G^{2}}^{*}$ , where, $X_{G^{2}}^{*}$ means on [1, G²] and G² coprime number, and make o^h′r′ mod G² = kG + 1, k ∈ [1, G - 1]. Finally, get the public parameters hh = (G, k, o), Master Key UK = (h′, r′).

Key generation algorithm KeyG: random selection t ∈ X_{G
²}, get the public key according to the public parameters hk = o^t mod G², private key zk = t.

Encryption algorithm Enc: Clear text of energy big data knowledge extracted by vertical federated learning algorithm u ∈ X_G, with any two clear text messages u₁ and u₂. For example, the addition homomorphism formula for both is: $Enc (u_{1} + u_{2}) = Enc (u_{1}) \otimes Enc (u_{2})$ (4)

Where, Enc (·) indicates encryption operation. ⊗ represents the arithmetic multiplication operation in the encryption domain under the same public key. On this basis, select the random number q ∈ X_{G
²}, use public key encryption to obtain the ciphertext of such energy big data knowledge (V, W), where V and W can be expressed as: ${\begin{matrix} V = o^{q} mod G^{2} \\ W = o^{tq} (uG + 1) mod G^{2} \end{matrix}$ (5)

Decryption algorithm Dec: Input energy big data knowledge ciphertext (V, W) and key zk, output its clear text u is: $u = [- 1 mod G^{2} + \frac{W}{V^{zk}}] / - G$ (6)

Master private key decryption algorithm KeyDec: Using common parameters hh, user’s public key hk, Master Private Key UK and energy big data knowledge ciphertext (V, W), get clear text of such energy big data knowledge u.

2.3 Security sharing of energy big data knowledge of blockchain module

Under the integration mechanism of the end cloud collaboration module and the blockchain module, in order to achieve the safe sharing of energy big data knowledge, it is necessary to uniformly manage the shared energy big data knowledge stored on different clouds, develop a set of identification coding rules, and complete the unique marking, recording and maintenance of shared energy big data knowledge through identification [18]. In addition, in order to address the shared energy big data knowledge, it is necessary to develop a set of corresponding identification resolution methods, complete the identification resolution according to the unified resolution steps, finally locate the storage location of the shared energy big data knowledge in the cloud, and realize the safe sharing of energy big data knowledge [19]. The blockchain based shared energy big data knowledge ID code is a two-level structure, as shown in Fig. 6.

Fig. 6

Coding structure diagram of big data knowledge identification of shared energy based on blockchain.

The identification coding structure diagram mainly includes short identification and long identification. In the identification coding process, short identification is used as the key and long identification is used as the value to generate key value pairs, which are stored in the Fabric status database. The short identification includes the global domain unique identification and the user domain unique identification. The user domain unique identification is used to increase naming autonomy, and the constructed short identification is used to implement the unique identification of energy big data knowledge in the blockchain network. Long ID is a collection of all metadata describing energy big data knowledge. The basic metadata mainly includes the storage location (URL) of energy big data knowledge in the cloud, extended fields, etc. The extended fields can be empty, which is convenient for users to expand [20]. Since the fields in the long ID can be updated, when the physical location of the energy big data knowledge changes, only the storage location in the long ID needs to be updated to address the new physical location through the original short ID.In this way, the identification code has the characteristics that the physical location of the energy big data knowledge can be migrated, and the storage form of the key value pair of the long and short identification provides an effective guarantee for fast addressing.

The identification and resolution process of shared energy big data knowledge is as follows: the identification registrant initiates a registration request, the blockchain automatically codes and generates a globally unique secondary identification according to the identification coding rules, and returns the short identification to the registrant. When an identity registrar sends a request for energy big data knowledge query to the blockchain with this short identity as a parameter, the smart contract in the blockchain implements hierarchical parsing of the secondary identity, and returns a long identity for addressing the cloud platform’s shared energy big data knowledge.

3 Experimental results study

In the experiment, the historical wind turbine monitoring data of a wind farm is selected as the big data of experimental energy, and the actual application effect of this model is tested by sharing such data through this model. The time of such energy big data is from October 11, 2020 to October 31, 2020, the interval between recording is 1 min/time, and the total number of data is 360000. The models and parameters in federated learning can be protected by encryption and stored on the blockchain. The decentralized nature of blockchain ensures the security and immutability of models and parameters. In order to encourage participants to contribute their computing resources and models, incentives on the blockchain can be introduced, such as token rewards or other financial incentives. Select the Hyperledger Fabric blockchain platform based on the needs and objectives of the experiment, and define the number of blockchain nodes to be used in the experiment, including verification nodes and consensus nodes. Define the average time for a transaction to be confirmed on the blockchain to ensure real-time sharing of energy big data requirements. Provide security parameters such as the encryption algorithm used and the corresponding key length. The consensus mechanism used by the selected blockchain platform is PoW. This kind of data is divided into three groups, A, B and C, with 120000 data in each group. Taking some of the data as an example, the big data of the initial experimental energy is presented, as shown in Table 1.

Table 1
Part of initial experimental energy big data

Data number Time Fan number

3 2020-10-11 03 : 15 : 00 A-1

18 2020-10-11 07 : 22 : 00 D-4

34 2020-10-12 01 : 06 : 00 A-5

55 2020-10-11 02 : 17 : 00 B-3

76 2020-10-13 13 : 03 : 00 B-6

89 2020-10-18 10 : 11 : 00 C-7

103 2020-10-20 06 : 48 : 00 C-10

166 2020-10-14 09 : 52 : 00 D-2

281 2020-10-21 00 : 45 : 00 E-1

1016 2020-10-30 21 : 36 : 00 B-8

21061 2020-10-28 18 : 18 : 00 E-11

33657 2020-10-20 15 : 03 : 00 A-6

89567 2020-10-31 12 : 10 : 00 A-10

101680 2020-10-11 20 : 25 : 00 D-3

253681 2020-10-15 07 : 11 : 00 E-2

Data number	Time	Fan number
3	2020-10-11 03 : 15 : 00	A-1
18	2020-10-11 07 : 22 : 00	D-4
34	2020-10-12 01 : 06 : 00	A-5
55	2020-10-11 02 : 17 : 00	B-3
76	2020-10-13 13 : 03 : 00	B-6
89	2020-10-18 10 : 11 : 00	C-7
103	2020-10-20 06 : 48 : 00	C-10
166	2020-10-14 09 : 52 : 00	D-2
281	2020-10-21 00 : 45 : 00	E-1
1016	2020-10-30 21 : 36 : 00	B-8
21061	2020-10-28 18 : 18 : 00	E-11
33657	2020-10-20 15 : 03 : 00	A-6
89567	2020-10-31 12 : 10 : 00	A-10
101680	2020-10-11 20 : 25 : 00	D-3
253681	2020-10-15 07 : 11 : 00	E-2

The big data knowledge of initial experimental energy is extracted through the model in this paper, and the knowledge extraction results are shown in Table 2.

Table 2

Big data knowledge of experimental energy extracted from the model in this paper

Data number	Wind speed of fan/m/s	Active power/kW	Reactive power/kW	Ambient temperature/°C	Gear oil temperature/°C	Gearbox bearing temperature/°C	Generator temperature/°C	Fan direction Angle/°
3	8.5	384.4	9.0	17.6	77.1	77.3	56.0	–7
18	9.8	386.5	9.3	17.6	77.1	77.3	56.0	–7
34	6.3	386.8	9.7	17.5	77.1	77.3	57.0	–2
55	7.7	385.9	8.8	17.7	77.1	77.3	56.0	–5
76	8.4	385.4	8.9	17.6	77.1	77.3	56.0	–2
89	7.3	385.9	9.0	17.5	77.1	77.3	56.0	–3
103	7.7	385.3	8.7	17.7	77.1	77.3	56.0	–6
166	7.7	384.8	8.7	17.6	77.1	77.3	56.0	–7
281	8.5	384.4	9.0	17.8	77.1	77.3	57.0	–7
1016	8.6	385.2	9.1	17.6	77.1	77.3	56.0	–8
21061	8.4	385.7	9.3	17.6	77.1	77.3	56.0	–6
33657	7.8	386.3	8.9	17.7	77.1	77.3	57.0	–8
89567	7.7	384.5	8.7	17.6	77.1	77.3	56.0	–3
101680	9.8	386.1	8.9	17.6	77.1	77.3	56.0	–2
253681	9.6	385.3	8.8	17.5	77.1	77.3	57.0	–4

Table 2 shows that the key knowledge can be extracted from the model in this paper for some of the initial experimental energy big data. It can be seen that the application effect of the knowledge extraction algorithm used in this model is relatively ideal.

In order to comprehensively test the overall effect of extracting energy big data knowledge from this model, normalized mutual information (NMI) index is selected as the evaluation index to evaluate the effect of all experimental energy big data knowledge extracted from this model. The value range of this indicator is [0,1]. The higher the value, the higher the matching degree between the energy big data knowledge extracted from this model and the actual energy big data knowledge, that is, the better the knowledge extraction effect of this model. The statistics of the NMI index values of the overall knowledge extraction results of this model are shown in Fig. 7.

Fig. 7

NMI index value statistics of the overall knowledge extraction results of the model in this paper.

It can be seen from Fig. 7 that with the increase of the amount of extracted energy big data, the NMI index value of the knowledge extraction results of the model in this paper also increases slightly. But on the whole, the NMI index values of the knowledge extraction results of the three groups of energy big data A, B, and C are not different, all within the range of 0.96 0.99. It can be seen that the amount of data has an impact on the NMI index value of the knowledge extraction results of the model in this paper, but the effect is not significant. The overall knowledge extraction effect of this model is good, and the performance is relatively stable, and the practical application is high.

On this basis, continue to use the model in this paper to encrypt the extracted energy big data knowledge, and the encrypted energy big data knowledge ciphertext presentation effect is shown in Table 3.

It can be seen from Table 3 that among the encryption results of some energy big data knowledge in the example, only the wind speed data knowledge of No. 281 is not encrypted, and the other data knowledge has achieved good encryption effect. It can be seen that this model can effectively encrypt the extracted energy big data knowledge, effectively ensuring the security of data knowledge in the sharing process.

Table 3

Ciphertext effect of energy big data knowledge after encryption of the model in this paper

Data number	Wind speed of fan/m/s	Active power/kW	Reactive power/kW	Ambient temperature/°C	Gear oil temperature/°C	Gearbox bearing temperature/°C	Generator temperature/°C	Fan direction Angle/°
3	9825	3360	1241	0136	5211	1589	1036	1598
18	4532	1569	2368	7536	4596	4597	1258	2612
34	0256	2150	0368	2103	6405	9950	3259	2597
55	0205	0026	4458	4651	5994	4520	7782	5632
76	1203	2659	2230	4592	8975	1597	9552	3255
89	2360	2560	2123	8640	2610	5235	1552	2323
103	8598	1250	2203	3680	5687	2525	5462	9569
166	9959	0199	2216	5960	0351	2352	5895	5671
281	8.5	2658	8875	1495	0002	2315	3692	0123
1016	2236	1463	9962	4202	2569	0115	5555	1256
21061	1230	1259	1620	1255	0396	2545	5941	4724
33657	2569	2590	5697	1260	3600	0324	2635	2351
89567	4544	2300	1368	2598	2658	2124	2395	2563
101680	1256	1259	2580	2586	2475	2253	5754	9652
253681	2269	2236	3325	2258	5671	1289	0147	1590

On the basis of the above experiments, the real-time sharing effect of energy big data in this model is finally tested, and the time cost of identification coding and parsing of shared energy big data knowledge in the sharing process is taken as the test indicator. During the inspection process, 8000 pieces of data knowledge were randomly selected from three groups of encrypted energy big data knowledge as shared energy big data knowledge. Through the model in this paper, such shared data knowledge was coded and parsed, the time cost of the coding and parsing process was recorded, and the real-time sharing effect of energy big data in this model was analyzed based on the record results. The recorded identification coding and parsing time cost of the model sharing energy big data knowledge in this paper is shown in Fig. 8.

Fig. 8

Time cost of coding and parsing of shared energy big data knowledge identification of the model in this paper.

It can be seen from Fig. 8 that the time cost of encoding and parsing of the shared energy big data knowledge identifier in this model is positively related to the amount of data. In the process of data knowledge identifier encoding, with the increase of the amount of data, the data knowledge label will also grow, which leads to the increase of the time cost of the overall data knowledge identifier encoding. In the process of data knowledge analysis, due to the growth of data volume, the length of data knowledge to be analyzed has also increased, so the time cost has increased. But on the whole, the time consumption of this model is not high when identifying, coding and parsing the knowledge of shared energy big data. It can realize real-time sharing of energy big data and ensure the timeliness of sharing energy big data.

4 Discussion

This paper combines the end cloud collaboration technology and blockchain technology to build a real-time sharing model for energy big data, and selects the historical wind turbine data of a wind farm as the experimental energy big data to test the sharing effect of the model built. According to the test results, we can know that:

In the process of implementing knowledge extraction for the selected initial experimental energy big data, this model can acquire the key knowledge of each energy big data, and the NMI index values of the overall knowledge extraction results are between 0.96 and 0.99. The knowledge extraction performance is stable and the extraction effect is good, which can lay a solid foundation for the effective sharing of energy big data.

In the process of implementing encryption for the extracted energy big data knowledge, only individual data knowledge has not been effectively encrypted, and other data knowledge can obtain ideal encryption effect. The encryption processing effect is significant, which can effectively guarantee the security of the energy big data knowledge to be shared.

In the process of implementing identification coding and parsing for the ciphertext of big data knowledge about energy to be shared, the amount of data is positively correlated with the time spent, but the average data knowledge has low time spent on identification coding and parsing, which can effectively guarantee the time efficiency and efficiency of energy big data knowledge sharing and realize real-time sharing.

In the encryption results of some energy big data knowledge in this example, only 281 wind speed data knowledge is not encrypted, and other data knowledge has achieved good encryption effect.

5 Conclusion

This paper conducts research on a real-time sharing model of energy big data based on end cloud collaboration technology. In the research, by combining end cloud collaboration technology and blockchain technology, we jointly build a real-time sharing model of energy big data. Among them, end cloud collaboration technology mainly combines vertical federated learning algorithm and homomorphic encryption algorithm, under the collaborative effect of client and cloud platform. Implement knowledge extraction and encryption processing for the initial energy big data in various fields mined by the client, and transfer the processed energy big data ciphertext in various fields to the cloud platform for storage, laying a solid foundation for the subsequent security sharing of energy big data. Blockchain technology mainly uses its attribute token management contract, outsourcing encryption contract, pre decryption contract and other smart contracts to implement identification coding and parsing processing on the cryptogram of shared energy big data stored in the cloud platform, and when data users need to share energy big data, provide the metadata of key cryptogram of shared energy big data knowledge to data users. It is convenient for data users to obtain the cryptogram of shared energy big data knowledge and the energy big data knowledge needed after decryption from the cloud platform. Finally, taking the historical energy big data of a wind farm as an example, the actual application effect of the model in this paper is tested through experiments. The test results show that the model in this paper has a good extraction effect in extracting energy big data knowledge, and at the same time, it can effectively encrypt almost all the extracted data knowledge. When identifying, encoding and parsing the knowledge ciphertext of shared data, the time cost is low, which can provide a reliable guarantee for the safe and real-time sharing of energy big data.

Conflict of interest

The author declares that there is no conflict of interest regarding the publication of this paper.

Data availability

All datasets generated for this study are included within the article.

References

Chan

, Gherardini

P.F.

, Krummel

M.F.

and Fragladakls

G.K.

, A“data sharing trust” model for rapid, collaborative science, Cell 184(3) (2021), 566–570.

and Zhou

, Research and Simulation on Network Acquisition Algorithm of Energy Consumption Monitoring Data, Computer Simulation 39(07) (2022), 297–300+.

Shurrab

, Singh

, Otrok

, Mizouni

, Khadkikar

and Zeineldin

, A Stable Matching Game for V2V Energy Sharing-A User Satisfaction Framework, IEEE Transactions on Intelligent Transportation Systems PP(99) (2021), 1–13.

Perera

A.T.D.

, Wang

, Nik

V.M.

and Scartezzini

, Towards realization of an Energy Internet: Designing distributed energy systems using game-theoretic approach, Applied Energy 283(Feb.1) (2021), 116349.1–116349.12.

, Zhang

, Lv

, Ge

, Yang

and Li

, Big data driven predictive production planning for energy-intensive manufacturing industries, Energy 211(Nov.15 Pt.2) (2020), 118320.1–118320.19.

Feng

, Yu

, Bashir

A.K.

, AI-Otaibi

Y.D.

, Lu

, Chen

et al. Efficient and Secure Data Sharing for 5G Flying Drones: A Blockchain-Enabled Approach, IEEE Network 35(1) (2021), 130–137.

Hassija

, Chamola

, Garg

, Dara

N.G.K.

, Kaddoum

and Jayakody

D.N.

, A Blockchain-Based Framework for Lightweight Data Sharing and Energy Trading in V2G Network, IEEE Transactions on Vehicular Technology 69(6) (2020), 5799–5812.

Liang

, Wu

and Su

, An energy-aware resource deployment algorithm for cloud data centers based on dynamic hybrid machine learning, Knowledge-Based Systems 222(Jun.21) (2021), 107020.1–107020.9.

Ramsay

, A cloud-computing platform for developing and evaluating vocal biomarkers based on home audio recordings: Resources for large-scale data processing and analysis, The Journal of the Acoustical Society of America 148(4) (2020), 2791–2791.

10.

Burns

and Collisson

E.A.

, Blockchain-authenticated sharing of cancer patient genomic and clinical outcomes data, Journal of Clinical Oncology 38(15_suppl) (2020), e19358–e19358.

11.

Shen

, Duan

, Zhu

, Zhang

, Du

and Guizani

, Blockchain-Based Incentives for Secure and Collaborative Data Sharing in Multiple Clouds, 38(6) (2020), 1229–1241.

12.

Gandhi

G.S.

, Vikas

, Ratnam

and Babu

K.S.

, Grid clustering and fuzzy reinforcement-learning based energy-efficient data aggregation scheme for distributed WSN, IET Communications 14(16) (2020), 2840–2848.

13.

Sebastian

, Analysis of the storage capacity and charging and discharging power in energy storage systems based on historical data on the day-ahead energy market in Poland, Energy 213(Dec.15) (2020), 118815.1–118815.9.

14.

Salehi

and Hossain

, Federated Learning in Unreliable and Resource-Constrained Cellular Wireless Networks, 69(8) (2021), 5136–5151.

15.

Jin

, Jiao

, Qian

, Zhang

and Lu

, Budget-Aware Online Control of Edge Federated Learning on Streaming Data With Stochastic Inputs, IEEE Journal on Selected Areas in Communications 39(12) (2021), 3704–3722.

16.

Zhang

, Wei

and Berry

, Faithful Edge Federated Learning: Scalability and Privacy, 39(12) (2021), 3790–3804.

17.

Hadjicostis

C.N.

and Dominguez

A.D.

, Privacy-Preserving Distributed Averaging via Homomorphically Encrypted Ratio Consensus, IEEE Transactions on Automatic Control 65(9) (2020), 3887–3894.

18.

, Ma

, Hu

and Yang

, Blockchain-Based Continuous Auditing for Dynamic Data Sharing in AutonomousVehicular Networks, Computer 54(8) (2021), 33–45.

19.

Yang

, Wen

, Jiang

and Wang

, Blockchain-Based Sharing and Tamper-Proof Framework of Big Data Networking, IEEE Network 34(4) (2020), 62–67.

20.

Cai

, Wu

, Lin

and Cai

, Blockchain-Empowered Big Data Sharing for Internet of Things, International Journal of Web Services Research 18(1) (2021), 58–69.