Optimal defense strategy based on the mean field game model for cyber security

Abstract

With the evolution of research on defense strategies in cyber security, the choice of an optimal strategy has become a key problem in current studies. Focusing on the balance between individual cost and overall network cost, we present an application of mean field game in large-scale defenders in cyber security, where players seek to construct an optimal defense strategy at their minimum cost. The contributions are threefold: first, we propose an individual cost function based on the mean field game in Hilbert space and discuss the overall network cost function, where each player has discrete-time dynamics. Then, the Nash equilibrium of the individual cost function with infinite players is researched. Finally, we establish an optimal condition in which the game equilibrium is the optimal solution to the overall cost function. Numerical examples are provided to illustrate the effectiveness of the presented strategy with an appropriate assumption.

Keywords

Mean field game Hilbert space cyber security

Introduction

With the rapid development of computer networks, the Internet has become an indispensable part of daily life. Meanwhile, the Internet is constantly facing various types of cyber security problems such as malware, virus and worm infection, distributed denial-of-service (DDoS) attacks, data hacking, email bombing, and energy attacks.¹ It is essential to study the defense mechanism technology with respect to cyber security problems to make the network safe.² However, conventional defense mechanisms such as intrusion detection and firewalls are not sufficient to address the current cyber security issues brought by large energy attackers and new techniques.³ New defense methods are required to overcome these problems,^4–6 and the details will be introduced in section “Related works.”

Since an effective defense scheme should include protection strategies for different attack behaviors, it is difficult to derive this type of defense scheme due to limited network resources and cumulative cost. Moreover, most of the existing works^4–9 do not consider the dynamic evolution of attackers, which should be considered in the design of a defense scheme.

In addition to successful defense against attacks, another important aim of a defense system is to minimize its running cost under attacks.¹⁰ In this article, we pay attention to the cyber security problem to minimize the cost of the individual player and the whole network based on the energy state. In this framework, we aim at constructing an optimal defense strategy at the individual cost function, which is also the optimal strategy for the whole network.

In general game models, since we focus on the process of individual decision-making, the optimal defense strategy of the individual player is not the optimal strategy for the overall network. In this article, we try to use mean field game theory by considering a large population of players and try to research the relation between the individual cost and the whole network cost. Mean field game theory was proposed by Lasry and Lions¹¹ and Huang et al.^12,13 as a modeling method for games with a sufficiently large number of players. Each player in the mean field game attempts to minimize its individual energy consumption based on the mean field term, which can be considered as the distribution of nodes with defense strategy.¹⁴ In cyber security, the whole network tries to obtain the maximum defense level at the minimum overall cost. We assume that attackers are randomly distributed depending on the defense strategy. The contributions of this article are as follows:

We formulate the individual cost function and the overall cost function in Hilbert space, which consists of n nodes with different defense mechanisms, where the evolution of defenders depends on the current energy level and the previous moment.

Since the subtle changes among nodes can be negligible if the number of players is sufficiently large, we research the ε-Nash equilibrium of the individual cost function with finite players and prove the existence of the optimal defense strategy of the individual player if $n \to \infty$ .

Finally, the optimal condition between the individual cost function and the overall cost function is given, under which the optimal defense strategy of the individual player is also the optimal defense strategy of the whole network.

This article is organized as follows. We introduce related works in section “Related works.” In section “Mean field game–based cost model in Hilbert space,” we present the individual cost model with the mean field term in Hilbert space and elaborate the overall network cost function based on this mean field game model. Next, we prove the ε-Nash equilibrium of the individual game model in section “ε-Nash equilibrium of the individual cost model.” We discuss the optimal condition using convex optimization theory and simulate the optimal defense strategy in sections “The optimality condition” and “Numerical examples,” respectively. Finally, section “Conclusion and future work” concludes the article.

Related works

Defense schemes have been examined in cyber security,¹⁵ and game theory has been introduced for cyber security problems.^16,17 For example, Khouzani et al.⁴ proposed an optimal control model to evaluate the damage maximization problem inflicted by a malware attack, which minimizes the overall cost of security patches. Huang et al.⁵ translated the targeted attacks into random attacks to study the robustness of interdependent networks. The work of Gao et al.⁶ compared the effectiveness of the current immunization strategies and proposed the best strategy to mitigate virus spread in email networks. However, these studies have not considered the distribution of attacks.

In the study of Dingankar and Brooks,⁷ DDoS attacks were modeled as a non-cooperative game, where the defenders attempted to form an optimal network topology to prevent the attack, while the attackers attempted to deploy zombies in the network. Zargar et al.⁸ classified the current DDoS flooding attacks and introduced a comprehensive DDoS defense mechanism. La et al.⁹ formulated a Bayesian game model to study the optimal defense strategy in the Internet of Things. Moreover, a Markov game model was proposed in the study of Lei et al.¹⁸ to solve the problem of the optimal strategy selection for the moving target defense. In the work of Eisenstadt and Moshaiov,¹⁹ the interaction between attackers and defenders was designed as a zero-sum multi-objective game model in which each player was undecided about its objective preferences. In addition, the popularization of intelligent terminals in edge network has also caused some security problems. In the study of An et al.,²⁰ a proper fog-intrusion detection system was designed in which an optimal intrusion response strategy was studied based on the differential game theory.

Mean field game theory is one of the most practical branches of game theory and has been used to research a class of complex problems with large number of players.²¹ Mean field game model is established by considering various assumptions, for example, players are homogeneous and their behaviors are continuous in time, and the decision-making of each player depends on the mean field term.²² The homogeneity means that the subtle change among players can be negligible if the number of players is sufficiently large. The continuity leads to an approximation of the game model with a large number of players, and the third assumption indicates that the process of decision-making of each player is affected by others through the mean field term.

Mean field game theory has been applied in economics,²³ engineering,²⁴ networks,^25–27 and other fields. Meanwhile, mean field game models for cyber security have been proposed in the works of Wang et al.,²⁸ Miao and Li,²⁹ and Khaliq et al.³⁰ Wang et al.²⁸ considered an attacker and multiple defenders in mobile ad hoc networks. The legitimate nodes of this model may intelligently select actions to decrease their energy consumption and security loss. In the study of Miao and Li,²⁹ we studied the binary interactive behaviors between malicious attackers and defenders and derived the specific solutions for the individual strategy of the active defense behavior and the passive defense behavior, respectively. In the work of Khaliq et al.,³⁰ a mean field game model was designed to solve the problem of the cyber-attack detection technique in ad hoc cognitive radio network. Multiple attackers were considered in the proposed model and each node could detect the attacks without incurring additional overheads.

Nevertheless, none of the above works to achieve optimal defense strategy by considering the whole network cost. Based on the advances in mean field game theory, in this article, we introduce a mean field game model based on the overall network cost for the optimal defense problem with the help of Hilbert space theory, under which this problem can be formulated through the minimum norm problem.

Mean field game-based cost model in Hilbert space

In this section, to analyze the relation between the individual defense mechanism cost and the overall network cost, we propose a mean field game model in Hilbert space with an infinite number of network nodes, which are assumed to be rational. The interactions between attackers and cyber nodes are discussed in Figure 1.

Figure 1.

Interactions between attackers and cyber nodes.

Let n be the number of network nodes with defense mechanisms, such as intrusion detection systems, intrusion prevention systems, firewalls, and distributed responses for DDoS attacks³¹ and m be the number of different defense mechanisms deployed by the individual node. Let H denote the Hilbert space with inner product $〈 x, y 〉$ for $\forall x, y \in H$ , and the norm $‖ u_{i} ‖^{2} = 〈 u_{i}, u_{i} 〉$ for $u_{i} \in H$ .

We use $x_{i} (t)$ to denote the energy consumption level of each player i for $i = 1, 2, \dots, n$ and use $u_{i} (t)$ to denote the defense intensity of player i at time t, where $u_{i} (t)$ is coupled through m components. For example, the first component can be considered as the response time of the defense mechanism, the second component as the probability of monitoring the attacker’s packet, and the third component as the security level of the privacy assets or the information assets.³² We presume $x_{i} (t) \in M_{i} \subset H$ and $u_{i} (t) \in U_{i} \subset H$ , where $M_{i}$ and $U_{i}$ have one component at least. We assume $ω_{i}$ is a random variable which is used to describe the probability distribution of any attacker. Furthermore, we assume that $ζ_{i}$ is the maximum defense level of each node and $ϖ_{i}$ is the maximum attacker level. Hence, the strategy space of each defender can be defined as $[0, ζ_{i}]^{m}$ , meanwhile, the strategy spaces of attackers are $[0, ϖ_{i}]$ .

We define $r (t) = \frac{1}{n} \sum_{i = 1}^{n} u_{i} (t)$ as the security intensity of the whole network, which is called the mean field term. We assume that $f : R^{m} \to R^{m}$ is Fréchet’s differential function and the derivative of $f (r)$ is bounded for any $r \in H$ , then f is Lipschitz continuous function, where there is a Lipschitz constant $k \geq 0$ such that $‖ f (y_{i}) - f (y_{j}) ‖ \leq k ‖ y_{i} - y_{j} ‖$ holds for any $y_{i}, y_{j} \in H (i \neq j)$ .

The cost of deviation from the whole security intensity is written as

‖ u_{i} (t) - f (r (t)) ‖^{2}

(1)

and the cost of energy consumption of the single player is

α_{i} ‖ x_{i} (t) ‖^{2}

(2)

where $α_{i} (i = 1, 2, \dots, n)$ are non-negative real numbers and $\sum_{i = 1}^{n} α_{i} = 1$ .

According to the above analysis, the cost of an individual computer can be formulated as

\begin{matrix} J_{i} (u_{i} (t), x_{i} (t), f (r)) \\ = min_{{u_{i} (t) | t \in [1, T]}} E (\sum_{t = 1}^{T} ({‖ u_{i} (t) - f (r (t)) ‖}^{2} + α_{i} {‖ x_{i} (t) ‖}^{2})) \end{matrix}

(3)

where T is the terminal time of the game.

The current state at time t is related to the state of the previous moment, the current strategy, and attacks. Then, the evolution of the energy consumption level can be expressed by

{\begin{matrix} x_{i} (t) = a_{i} x_{i} (t - 1) + b_{i} u_{i} (t) + ω_{i} (t) \\ x_{i} (t) \in M_{i}, u_{i} (t) \in U_{i}, i = 1, 2, \dots, n \end{matrix}

(4)

where $a_{i}$ and $b_{i}$ are positive parameters.

To simplify equations (3) and (4), we define the state of the energy level at different times as vector $x_{i}$ , that is, $x_{i} = (x_{i}^{(1)}, x_{i}^{(2)}, \dots, x_{i}^{(T)})$ , where $x_{i}^{(1)} = (x_{i, 1}^{(1)}, x_{i, 2}^{(1)}, \dots, x_{i, m}^{(1)})^{'}$ . Similarly, we define $u_{i} = (u_{i}^{(1)}, u_{i}^{(2)}, \dots, u_{i}^{(T)})$ , where $u_{i}^{(1)} = (u_{i, 1}^{(1)}, u_{i, 2}^{(1)}, \dots, u_{i, m}^{(1)})^{'}$ and $ω_{i} = (ω_{i, 1}, ω_{i, 2}, \dots, ω_{i, m})$ . Thus, the cost functions (3) and (4) can be transformed as

\begin{matrix} J_{i} (u_{i}, x_{i}, f (r)) \\ = min_{u_{i}} E ({‖ u_{i} - f (r) ‖}^{2} + α_{i} {‖ x_{i} ‖}^{2}) \\ = min_{u_{i}} E ({‖ u_{i} ‖}^{2} + α_{i} {‖ x_{i} ‖}^{2} + {‖ f (r) ‖}^{2} - 2 u_{i} \cdot f (r)) \end{matrix}

(5)

subjected to

{\begin{matrix} x_{i} = A_{i} x_{i}^{0} + B_{i} u_{i} + C_{i} ω_{i} \\ x_{i} \in M_{i}, u_{i} \in U_{i}, i = 1, 2, \dots, n \end{matrix}

(6)

where $x_{i}^{(0)} = (x_{i, 1}^{(0)}, x_{i, 2}^{(0)}, \dots, x_{i, m}^{(0)})^{'}$ is the initial value of the energy level. $A_{i} = (a_{i}, a_{i}^{2}, \dots, a_{i}^{T})^{'}$ , $B_{i} = b_{i} (\begin{matrix} 1 & 0 & \dots & 0 \\ a_{i} & 1 & \dots & 0 \\ ⋮ & ⋮ & ⋮ & ⋮ \\ a_{i}^{T - 1} & a_{i}^{T - 2} & \dots & 1 \end{matrix})$ , and $C_{i} = (a_{i}^{T - 1}, a_{i}^{T - 2}, \dots, 1)^{'}$ are matrices, where $a_{i} = (a_{i, 1}, a_{i, 2}, \dots, a_{i, m})^{'}$ and $b_{i} = (b_{i, 1}, b_{i, 2}, \dots, b_{i, m})^{'}$ .

In general, the optimal solution to the individual cost function is not the optimal overall network cost. We assume that $g (r)$ is the cost depending on strategies of all defense mechanisms Then, the cost of the whole network can be expressed as

\begin{matrix} G (u_{1}, \dots, u_{n}, x_{1}, \dots, x_{n}, h (r)) \\ = min_{u_{i}} E (\sum_{i = 1}^{n} ({‖ u_{i} ‖}^{2} + α_{i} {‖ x_{i} ‖}^{2}) + g (r)) \end{matrix}

(7)

subjected to

{\begin{matrix} x_{i} = A_{i} x_{i}^{0} + B_{i} u_{i} + C_{i} ω_{i} \\ x_{i} \in M_{i}, u_{i} \in U_{i}, i = 1, 2, \dots, n \end{matrix}

(8)

This problem can be evaluated using the ε-Nash equilibrium, wherein the key problem is under what condition the individual game equilibrium is the optimal solution of the whole network cost function.

ε-Nash equilibrium of the individual cost model

In this section, we analyze the existence of the Nash equilibrium solution of the individual game model (equation (5)) based on the large-sale players. In the following subsections, we first discuss the ε-Nash equilibrium of the game model with finite players. Then, the existence of the optimal defense strategy of the individual player when $n \to \infty$ will be considered.

Definition 1

A set of strategies ${\bar{U}}_{i} = {u_{i}^{*} | i = 1, 2, \dots, n}$ is called an ε-Nash equilibrium with respect to the cost $J_{i}$ for any i, if there exists $ε \geq 0$ such that for each i, the following inequality holds³³

J_{i} (u_{i}^{*}, u_{- i}^{*}) - ε \leq J_{i} (u_{i}, u_{- i}^{*})

(9)

where $u_{i} \in U_{i}$ and $u_{- i} = (u_{1}, \dots, u_{i - 1}, u_{i + 1}, \dots, u_{n})$ .

The inequality in equation (9) holds for a finite number of participants. The ε-Nash equilibrium will degenerate into the general Nash equilibrium as the number of players tends to infinity and ε tends to zero.

Each player in our model is assumed to be rational, and the process of decision-making of each node depends on the mean field term. For the individual cost function (equation (5)), a dynamic evolution $(u_{i}, x_{i})$ and the mean field term will determine the minimum cost function $J_{i}$ . Since $J_{i}$ is difficult to derive because the effect of the mean field term is negligible when the number of players goes to infinity, we can approximate the mean field behavior of the players with a best approximation value $z \in H$ that substitutes function $f (r)$ in objective function (equation (5)). Thus, equation (5) can be written as

J_{i} (z) = min_{{u_{i} | i = 1, \dots, n}} E ({‖ u_{i}^{*} ‖}^{2} + α_{i} {‖ x_{i}^{*} ‖}^{2} + {‖ z ‖}^{2} - 2 u_{i}^{*} \cdot z)

(10)

where $J_{i} (z)$ is the optimal solution of (10) and

{\begin{matrix} x_{i} = A_{i} x_{i}^{0} + B_{i} u_{i} + C_{i} ω_{i} \\ z = f (\frac{1}{n} E \sum_{i = 1}^{n} u_{i}^{*}) \\ u_{i}^{*} \in {\bar{U}}_{i}, i = 1, 2, \dots, n \end{matrix}

(11)

In equation (10), the best approximation term $z \in H$ generated by the individual player converges to the minimum value $J_{i} (z)$ , which includes the optimal solutions selected by each player. Here, we prove that the solutions to the cost function in equations (10) and (11) are the ε-Nash equilibrium solution in equation (5). First, we demonstrate the following assumption and lemmas.

We assume that there exists a constant $K > 0$ such that $E ‖ u_{i} ‖^{2} \leq K$ in the $U_{i}$ for any i. In fact, the compactness is equivalent to the bounded closed subset in finite dimensional space.

Lemma 1

For $r^{*} = \frac{1}{N} \sum_{i = 1}^{N} u_{i}^{*}$ , $r_{- i}^{*} = \frac{1}{N} \sum_{j = 1, j \neq i}^{N} u_{j}^{*} \in U_{i}$ and $u_{i}^{*} \in {\bar{U}}_{i}$ , $| E (f (\frac{1}{N} u_{i} + r_{- i}^{*}) \cdot u_{i}) - E (f (r^{*}) \cdot u_{i}) |$ is convergent for each i.

Proof

For $\forall t \in [0, T]$ , it is well known that $‖ E (u_{i} (t)) ‖ = max_{t} | E (u_{i} (t)) | \leq E (max_{t} | u_{i} (t) |) = E ‖ u_{i} (t) ‖$ . Then, $‖ E (u_{i}) ‖$ is bounded for $u_{i} \in U_{i}$ . In fact, we have $‖ E (u_{i}) ‖ \leq E (‖ u_{i} ‖) \leq \sqrt{k}$ because $E ‖ u_{i} ‖^{2} \leq K$ .

$f (r)$ is a Lipschitz continuous function, thus, there exists $ε_{1} > 0$ such that the following inequality holds

\begin{matrix} | E (f (\frac{1}{n} u_{i} + r_{- i}^{*}) \cdot u_{i}) - E (f (r^{*}) \cdot u_{i}) | \\ = | f (E (\frac{1}{n} u_{i} + \frac{1}{n} \sum_{j = 1, j \neq i}^{n} u_{j}^{*})) \cdot E u_{i} \\ - f (E (\frac{1}{n} u_{i}^{*} + \frac{1}{n} \sum_{j = 1, j \neq i}^{n} u_{j}^{*})) \cdot E u_{i} | \\ \leq \frac{ε_{1}}{n} (‖ E (u_{i}) ‖ + ‖ E (u_{i}^{*}) ‖) \cdot ‖ E (u_{i}) ‖ \end{matrix}

(12)

Thus, $| E (f (\frac{1}{N} u_{i} + r_{- i}^{*}) \cdot u_{i}) - E (f (r^{*}) \cdot u_{i}) |$ is convergent for each i.

End of proof.

Besides, if $n \to + \infty$ , there exists $ε_{n} > 0$ such that the above conclusion holds, that is

\begin{matrix} {lim}_{n \to + \infty} | E (f (\frac{1}{n} u_{n} + r_{- n}^{*}) \cdot u_{n}) - E (f (r^{*}) \cdot u_{n}) | \\ \leq lim_{n \to + \infty} ε_{n} = 0 \end{matrix}

Lemma 2

For $u_{i} \in U_{i}$ , $i = 1, 2, \dots, n$ , $| E (f (r)) - E (f (r_{- i})) |$ is convergent for each i, where $r = \frac{1}{n} \sum_{i = 1}^{n} u_{i} (t) + δ$ .

Proof

Considering $r_{- i} = \frac{1}{n} \sum_{j = 1, j \neq i}^{n} u_{j} + δ$ , we obtain

\begin{matrix} | E (f (r)) - E (f (r_{- i})) | \\ = | E (f (r)) - f (E (r)) + f (E (r)) - f (E (r_{- i})) + f (E (r_{- i})) - E (f (r_{- i})) | \\ \leq | E (f (r)) - f (E (r)) | + | f (E (r)) - f (E (r_{- i})) | + | f (E (r_{- i})) - E (f (r_{- i})) | \end{matrix}

(13)

Hence, we must prove that the inequality in equation (13) holds.

The functional f is a Fréchet’s differential function with respect to r. According to the definition of Fréchet’s differentials, there exists a constant $c_{1} > 0$ such that the following inequality holds

\begin{array}{l} ‖ f (\frac{1}{n} E (u_{i}) + \frac{1}{n} \sum_{j = 1, j \neq i}^{n} E (u_{j})) - f (\frac{1}{n} \sum_{j = 1, j \neq i}^{n} E (u_{j})) - f^{'} (\frac{1}{n} E (u_{i})) (\frac{1}{n} \sum_{j = 1, j \neq i}^{n} E (u_{j})) ‖ \\ \leq \frac{c_{1}}{n} ‖ \sum_{j = 1, j \neq i}^{n} E (u_{j}) ‖ \end{array}

(14)

Equation (14) can be written as

\begin{matrix} ‖ f (\frac{1}{n} E (u_{i}) + \frac{1}{n} \sum_{j = 1, j \neq i}^{n} E (u_{j})) - f (\frac{1}{n} \sum_{j = 1, j \neq i}^{n} E (u_{j})) ‖ \\ - ‖ f' (\frac{1}{n} E (u_{i})) ‖ \cdot ‖ \frac{1}{n} \sum_{j = 1, j \neq i}^{n} E (u_{j}) ‖ \\ \leq \frac{c_{1}}{n} ‖ \sum_{j = 1, j \neq i}^{n} E (u_{j}) ‖ \end{matrix}

(15)

In finite space, there is a constant $c_{2} > 0$ such that $‖ E (\sum_{i = 1}^{n} u_{i}) ‖ \leq c_{2}$ for any $u_{i} \in U_{i}$ . Therefore, the inequality in equation (15) is bounded. In addition, we assume that the derivative of $f (r)$ is bounded with $r \in H$ , so $‖ f' (\frac{1}{n} E (u_{i})) ‖ \leq c_{3}$ with $c_{3} > 0$ .

Consider $0 < ε_{2} = (c_{1} + c_{3}) c_{2} < ε$ such that

‖ f (\frac{1}{n} E (u_{i}) + \frac{1}{n} \sum_{j = 1, j \neq i}^{n} E (u_{j})) - f (\frac{1}{n} \sum_{j = 1, j \neq i}^{n} E (u_{j})) ‖ \leq ε_{2}

(16)

Hence, $| f (E (r)) - f (E (r_{- i})) |$ is convergent.

Next, we prove that the remaining functions of the inequality in equation (13) are convergent. The function $f (r)$ is a continuous function because it is a Fréchet’s differential function with respect to r. According to the continuity of $f (r)$ and the weak convergence theorem,³⁴ we form the random variable $F_{n} = f (r) - f (E (r))$ and analyze its boundedness. Then, we obtain

\begin{matrix} lim_{n \to \infty} F_{n} = lim_{n \to \infty} (f (r) - f (E (r))) \\ = f (lim_{n \to \infty} (r - E (r))) \\ = 0 \end{matrix}

(17)

Therefore, we have $lim_{n \to \infty} ‖ E (f (Er)) - E (f (r)) ‖ = 0$ , where $F_{n}$ satisfies the integrals uniformly bounded on L₁. Hence, there exists $0 < ε_{3} < ε$ such that $| E (f (r)) - f (E (r)) | \leq ‖ E (f (r)) - Ef (E (r)) ‖ \leq ε_{3}$ ; similarly, there exists $0 < ε_{4} < ε$ such that $| f (E (r_{- i})) - Ef (r_{- i}) | \leq ε_{4}$ .

Based on the above analysis, $| E (f (r)) - E (f (r_{- i})) |$ is convergent for each i.

End of proof.

Besides, if $n \to + \infty$ , there exists $ε_{n} > 0$ such that lemma 2 holds, that is, $lim_{n \to + \infty} | E (f (r)) - E (f (r_{- n})) | = 0$ . Building upon lemmas 1 and 2, we will present a proof of existence of the ε-Nash equilibrium solution to equation (10).

Theorem 1

For any $ε > 0$ and n, the following inequality holds

\begin{matrix} E ({‖ u_{i}^{*} ‖}^{2} + α_{i} {‖ x_{i}^{*} ‖}^{2}) + E ‖ z^{*} ‖^{2} \\ - 2 {Eu}_{i}^{*} \cdot E z^{*} - ε \leq E ({‖ u_{i} ‖}^{2} + α_{i} {‖ x ‖}^{2}) + E ‖ z ‖^{2} \\ - 2 E u_{i} \cdot Ez \end{matrix}

(18)

Proof

Similar to definition 1, we must prove that the following inequality holds

\begin{matrix} E ({‖ u_{i}^{*} ‖}^{2} + α_{i} {‖ x_{i}^{*} ‖}^{2}) + E ‖ f (r^{*}) ‖^{2} - 2 E (u_{i}^{*}) \cdot Ef (r^{*}) - ε \\ \leq inf_{u_{i} \in U_{i}, r_{- i}^{*} \in H} E ({‖ u_{i} ‖}^{2} + α_{i} {‖ x_{i} ‖}^{2}) \\ + E ‖ f (\frac{1}{n} u_{i} + r_{- i}^{*}) ‖^{2} - 2 E (u_{i}) \cdot Ef (\frac{1}{n} u_{i} + r_{- i}^{*}) \\ \leq E ({‖ u_{i} ‖}^{2} + α_{i} {‖ x_{i} ‖}^{2}) + ‖ f (\frac{1}{n} u_{i} + r_{- i}^{*}) ‖^{2} \\ - 2 E (u_{i}) \cdot Ef (\frac{1}{n} u_{i} + r_{- i}^{*}) \end{matrix}

(19)

First, we prove that there is an $ε_{5} > 0$ such that the inequality $E (u_{i}^{*}) \cdot Ef (r^{*}) - ε_{5} \leq E (u_{i}) \cdot Ef (\frac{α_{i}}{n} u_{i} + r_{- i}^{*})$ holds.

Because the following inequality holds

\begin{matrix} | E (f (r^{*}) \cdot u_{i}^{*}) - E (f (\frac{α_{i}}{n} u_{i} + r_{- i}^{*}) \cdot u_{i}) | \\ \leq | E (f (r^{*}) \cdot u_{i}^{*}) - f (E r^{*}) \cdot {Eu}_{i}^{*} | \\ + | f (E r^{*}) \cdot {Eu}_{i}^{*} - f (E (\frac{1}{n} u_{i} + r_{- i}^{*})) \cdot {Eu}_{i}^{*} | \\ + | f (E (\frac{1}{n} u_{i} + r_{- i}^{*})) \cdot {Eu}_{i}^{*} - Ef (\frac{1}{n} u_{i} + r_{- i}^{*}) \cdot {Eu}_{i}^{*} | \\ + | Ef (\frac{1}{n} u_{i} + r_{- i}^{*}) \cdot {Eu}_{i}^{*} - Ef (\frac{1}{n} u_{i} + r_{- i}^{*}) \cdot E u_{i} | \\ = I_{1} + I_{2} + I_{3} + I_{4} \end{matrix}

(20)

Based on lemma 1, both $I_{1}$ and $I_{2}$ are convergent. Since $f (r)$ is a Lipschitz continuous function and $‖ E (u_{i}) ‖$ is bounded for any $u_{i} \in U_{i}$ , using lemma 2, $I_{3}$ and $I_{4}$ are also convergent. Hence, the inequality $E (u_{i}^{*}) \cdot E (f (r^{*})) - ε_{5} \leq E (u_{i}) \cdot E (f (\frac{u_{i}}{n} + r_{- i}^{*}))$ holds.

Next, we prove that the following inequality holds

\begin{matrix} E ({‖ f (r^{*}) ‖}^{2} - {‖ f (\frac{1}{n} u_{i} + r_{- i}^{*}) ‖}^{2}) \\ \leq E ‖ f (r^{*}) - f (\frac{1}{n} u_{i} + r_{- i}^{*}) ‖^{2} \\ = E ‖ f (\frac{1}{n} u_{i}^{*} + r_{- i}^{*}) - f (\frac{1}{n} u_{i} + r_{- i}^{*}) ‖^{2} \\ \leq \frac{k}{n} (E {‖ u_{i}^{*} ‖}^{2} + E {‖ u_{i} ‖}^{2}) \end{matrix}

(21)

Since $f (r)$ is a Lipschitz continuous function, there exists a constant k such that $‖ f ‖ \leq k$ , and $‖ E (u_{i}) ‖^{2}$ is bounded for any $u_{i} \in U_{i}$ . Thus, there exists $ε_{6} > 0$ such that the inequality in equation (21) holds.

Since $z^{*} = f (\frac{1}{n} E (\sum_{i = 1}^{n} u_{i}^{*}))$ , for any $ε > 0$ such that the following inequality holds

\begin{matrix} E ({‖ u_{i}^{*} ‖}^{2} + α_{i} {‖ x_{i}^{*} ‖}^{2}) + E ‖ z^{*} ‖^{2} - 2 E (u_{i}^{*}) \cdot E z^{*} - ε \\ \leq E ({‖ u_{i} ‖}^{2} + α_{i} {‖ x ‖}^{2}) + E ‖ z^{*} ‖^{2} - 2 E (u_{i}) \cdot E (z^{*}) \end{matrix}

(22)

End of proof.

Furthermore, if $n \to + \infty$ , there exists $ε_{n} > 0$ such that theorem 1 holds. In the above analysis, each node will follow the optimal defense strategy $u_{i}^{*}$ as deviating from this equilibrium strategy will decrease its individual cost.

The optimality condition

If $n \to \infty$ , the key problem is that under what condition the equilibrium of the individual problem is the overall network optimum if we derive an optimal condition between game models (5) and (7), which will be discussed in this section. Specifically, we elaborate the relation between the overall cost function $g (r)$ and the mean field term $f (r)$ . The overall cost function $g (r)$ can be considered as the coupling term of the individual cost. Since it is difficult to decompose the overall cost function into an individual minimization problem, we translate this problem into the convex optimization problem.

We assumed that $g (r)$ is a convex function and Fréchet’s differential function. Combining the objective functions in equations (5) and (6), we form the Lagrange function as

\begin{matrix} L (u_{1}, u_{2}, \dots, u_{n}, x_{1}, x_{2}, \dots, x_{n}, λ) \\ = E (\sum_{i = 1}^{n} ({‖ u_{i} ‖}^{2} + α_{i} {‖ x_{i} ‖}^{2}) + g (r)) \\ + λ \cdot (\frac{1}{n} E (\sum_{i = 1}^{n} u_{i}) - r) \\ = E (\sum_{i = 1}^{n} ({‖ u_{i} ‖}^{2} + α_{i} {‖ x_{i} ‖}^{2})) \\ + \frac{λ}{n} E (\sum_{i = 1}^{n} u_{i}) + ng (r) - λ r \\ = L_{1} + L_{2} \end{matrix}

(23)

where $λ$ is the Lagrange multiplier.

Let

L_{1} (u_{1}, \dots, u_{n}, λ) = E (\sum_{i = 1}^{n} ({‖ u_{i} ‖}^{2} + α_{i} {‖ x ‖}^{2} + λ \cdot \frac{u_{i}}{n}))

(24)

L_{2} (r, λ) = ng (r) - λ \cdot r

(25)

Thus, the objective function of the overall network cost in equation (7) is equivalent to

\underset{u_{i}, r, λ}{max min} L_{1} + L_{2}

(26)

The optimal solution of equation (26) can be expressed as

L_{1}^{*} = min_{u_{i}} E (\sum_{i = 1}^{n} ({‖ u_{i}^{*} ‖}^{2} + α_{i} {‖ x_{i}^{*} ‖}^{2} + λ \cdot \frac{u_{i}^{*}}{n}))

(27)

r^{*} = \underset{r \in H}{\arg min} L_{2} = ng (r^{*}) - λ \cdot r^{*}

(28)

where $r^{*}$ , $u_{i}^{*}$ , and $L_{1}^{*}$ are the optimal solutions of equations (10) and (26).

The derivative of equation (28) with respect to $r^{*}$ is given in equation (29)

\frac{d}{d r^{*}} (nh (r^{*}) - λ \cdot r^{*}) = 0

(29)

Based on objective function (10), equation (26) indicates that

J_{i} (z) = min_{u_{i}} L_{i}^{*} (u_{i}, λ)

(30)

Since the overall cost function g is Fréchet’s differential function, from equations (10), (27), and (30), we derive the following optimality condition

\frac{d}{d r^{*}} (g (r^{*}) - {‖ f (r^{*}) ‖}^{2} + 2 r^{*} \cdot f (r^{*})) = 0

(31)

According to the above analyses, we provide the following result.

Proposition 1

If the optimal condition $\frac{d}{d r^{*}} (g (r^{*}) - {‖ f (r^{*}) ‖}^{2} + 2 r^{*} \cdot f (r^{*})) = 0$ holds, the ε-Nash equilibrium of the game model $J_{i} = min_{{u_{i} (t), t \in [1, T]}} E (\sum_{t = 1}^{T} ({‖ u_{i} (t) - f (r) ‖}^{2} + α_{i} {‖ x_{i} (t) ‖}^{2}))$ is the optimal solution of the overall cost function

min_{{u_{i} | i = 1, \dots, n}} E (\sum_{i = 1}^{n} ({‖ u_{i} ‖}^{2} + α_{i} {‖ x_{i} ‖}^{2}) + g (r)) .

Numerical examples

In this section, numerical examples are provided to evaluate the proposed scheme. For simplicity, we presume that the random distribution of attackers $ω_{i}$ follows the normal distribution. We define the mean field term as $f (r^{*}) = c r^{*}$ , which satisfies the assumptions. Equation (29) can be written as

\frac{d}{d r^{*}} (h (r^{*}) - (c^{2} - 2 c) {‖ r^{*} ‖}^{2}) = 0

(32)

where c is a real number.

We presume that the iteration number is 50 times greater than time T = 100 s for the simulations. During the game time, each player can provide the energy to reduce the response time of the defense mechanisms or the loss of information assets of nodes. We assume that the coefficient of $x_{i} (t)$ is negative when under attacks. The dynamic evolution of the energy level $x_{i} (t)$ at a fixed time during the iterations is presented in Figure 2. The result shows that the value of $x_{i} (t)$ converges after a few iterations, which means that the change of energy consumption tends to be stable.

Figure 2.

Evolution of energy $x_{i} (t)$ at a fixed time during the iteration.

In addition, we discuss the evolution of the response time and the optimal defense strategy of the node $i = 1$ over time in Figure 3. The result shows that the response time increases with increasing attack intensities, and the defense strategy $u_{i 1} (t)$ can significantly keep the value of the response time within a certain range by incurring slightly greater energy consumption under the attacks $ω_{i}$ , which is assumed to have normal distribution. Similarly, for the security level of the privacy assets or the information assets, the defense strategy $u_{ij} (t)$ can cut down the loss of assets by consuming more energy, so does other security problems. Moreover, in Figure 4, we present the change of the defense strategy of the whole network at a fixed time during the iterations. These results illustrate the convergence property of $f (r)$ .

Figure 3.

Evolution of the response time under attacks over time.

Figure 4.

Evolution of the mean field term $f (r)$ at a fixed time.

In Figure 5, we present the difference between the individual cost function $J_{i}$ and the overall cost function G. The result in Figure 4 shows that the change gradually increases after the initial time and tends to zero by the 25th iteration, which implies that the optimal defense strategy for the individual cost function is also the optimal strategy for the overall cost function.

Figure 5.

Difference between individual cost function $J_{i}$ and overall cost function G.

Using the overall cost function defined in equation (7) and the optimal condition in equation (30), we calculate the overall cost function G over time. Then, we compare the costs of the proposed scheme and the energy-prioritized strategy²⁵ in Figure 6. As shown in Figure 6, it can be seen that the proposed scheme consumes more energy than the energy-prioritized strategy at the beginning because of the level attacks, but then the cost is gradually reduced, which indicates that the node has an optimal strategy with a minimum cost at this time.

Figure 6.

Cost comparison between the proposed scheme and the energy-prioritized strategy.

Conclusion and future work

In this article, we have proposed a security mean field game model in Hilbert space for large-scale defenders in cyber security. In terms of model construction, we have formed an individual cost model and an overall network cost model based on the mean field game in Hilbert space, where the cost function can be expressed by the minimum norm problem. As the number of defense players tends to infinity, ε approaches zero. Thus, we have evaluated the optimal solution using the ε-Nash equilibrium with a finite number of players. We have derived an optimal condition between the mean field term and the overall cost function, where the game equilibrium of the individual cost model corresponds to the minimum cost of the overall network in virtue of a Lagrange function. In this framework, the optimal defense strategy of the individual cost function is also the optimal strategy of the overall cost function.

We have considered that each defense strategy is independent of each other in this article. In future work, we will investigate the interactions among different defense strategies.

Footnotes

Acknowledgements

The authors gratefully acknowledge the anonymous reviewers who read the drafts and made many helpful suggestions.

Handling Editor: Kim-Kwang R Choo

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China under grants 1603116 and 61701020.

ORCID iD

Li Miao

References

Chowdhury

Recent cyber security attacks and their mitigation approaches—an overview. In: International conference on applications and techniques in information security, Cairns, QLD, Australia, 26–28 October 2016, pp.54–65Singapore: Springer.

Wei

Zhang

Liu

et al . Defense strategy of network security based on dynamic classification. KSII T Internet Inf 2015; 9(12): 5116–5134.

Spyridopoulos

Karanikas

Tryfonas

et al . A game theoretic defence framework against DoS/DDoS cyber attacks. Comput Secur 2013; 38: 39–50.

Khouzani

MHR

Sarkar

Altman

Maximum damage malware attack in mobile wireless networks. IEEE/ACM Trans Netw 2012; 20(5): 1347–1360.

Huang

Gao

Buldyrev

et al . Robustness of interdependent networks under targeted attack. Phys Rev E 2011; 83(6): 065101.

Gao

Liu

Zhong

Network immunization and virus propagation in email networks: experimental evaluation and analysis. Knowl Inf Syst 2011; 27(2): 253–279.

Dingankar

Brooks

. Denial of service games. In: Proceedings of the third annual cyber security and information infrastructure research workshop, Oak Ridge, TN, 15–17 May 2007, pp.7–17.

Zargar

Joshi

Tipper

A survey of defense mechanisms against distributed denial of service (DDoS) flooding attacks. IEEE Commun Surv Tutor 2013; 15(4): 2046–2069.

Quek

TQS

Lee

et al . Deceptive attack and defense game in honeypot-enabled networks for the internet of things. IEEE Internet Things 2016; 3: 1025–1035.

10.

Cheng

A differential game model between intrusion detection system and attackers for wireless sensor networks. Wireless Pers Commun 2016; 90(3): 1211–1219.

11.

Lasry

Lions

PL.

Mean field games. Japan J Math 2007; 2(1): 229–260.

12.

Huang

Malhamé

Caines

PE.

Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle. Commun Inf Syst 2006; 6(3): 221–252.

13.

Huang

Caines

Malhame

RP.

Large-population cost-coupled LQG problems with nonuniform agents: Individual-mass behavior and decentralized, ε-Nash equilibria. IEEE Trans Autom Contr 2007; 52(9): 1560–1571.

14.

Bensoussan

Frehse

Yam

Mean field games and mean field type control theory. New York: Springer, 2013.

15.

Zhang

Han

Lai

et al . Survey on cyberspace security. Sci China Inf Sci 2015; 58(11): 1–43.

16.

Liang

Xiao

Game theory for network security. IEEE Commun Surv Tutor 2013; 15(1): 472–486.

17.

Tran

Hong

et al . Game theory for cyber security and privacy. ACM Comp Surv 2017; 50(2): 30.

18.

Lei

Zhang

HQ.

Optimal strategy selection for moving target defense based on Markov game. IEEE Access 2017; 5: 156–169.

19.

Eisenstadt

Moshaiov

Novel solution approach for multi-objective attack-defense cyber games with unknown utilities of the opponent. IEEE Trans Emerg Top Comput Intell 2017; 1(1): 16–26.

20.

Lin

et al . A novel differential game model-based intrusion response strategy in fog computing. Secur Commun Netw 2018; 2018: 1821804.

21.

Guéant

Lasry

Lions

PL.

Mean field games and applications. In: Cousin

Crépey

Guéant

et al . (eds) Paris-Princeton lectures on mathematical finance 2010. Berlin; Heidelberg: Springer, 2011, pp.205–266.

22.

Bensoussan

Chau

MHM

Yam

SCP

. Mean field games with a dominating player. Appl Math Optim 2006; 74: 91–128.

23.

Guéant

Mean field games and applications to economics. PhD Thesis, Université Paris-Dauphine, Paris, 2009.

24.

Djehiche

Tcheukam

Tembine

Mean-field-type games in engineering, 2016, https://arxiv.org/abs/1605.03281

25.

Mériaux

Varma

Lasaulce

Mean field energy games in wireless networks. In: 2012 conference record of the forty sixth Asilomar conference on signals, systems and computers (ASILOMAR), Pacific Grove, CA, 4–7 November 2012, pp.671–675. New York: IEEE.

26.

Tembine

Vilanova

Debbah

Noisy mean field stochastic games with network applications. Report, Ecole Superieure D’electricite, Paris, 2010.

27.

Bauso

Tembine

Basar

Opinion dynamics in social networks through mean-field games. SIAM J Contr Optim 2016; 54(6): 3225–3257.

28.

Wang

Tang

et al . A mean field game theoretic approach for security enhancements in mobile ad hoc networks. IEEE T Wirel Commun 2014; 13(3): 1616–1627.

29.

Miao

Cyber security based on mean field game model of the defender-Attacker strategies. Int J Distrib Sens N. Epub ahead of print 24 October 2017. DOI: 10.1177/1550147717737908.

30.

Khaliq

SBA

Amjad

Abbas

et al . Defence against PUE attacks in ad hoc cognitive radio networks: a mean field game approach. Telecommun Sys 2019; 70: 123–140.

31.

Mirkovic

Reiher

A taxonomy of DDoS attack and DDoS defense mechanisms. ACM SIGCOMM Comp Com 2004; 34(2): 39–53.

32.

Zheng

Tang

. A game theoretic approach for security and quality of service (QoS) co-design in MANETs with cooperative communications. In: MILCOM 2012 military communications conference, Orlando, FL, 29 October–1 November 2012, pp.1–6. New York: IEEE.

33.

Huang

Caines

Malhamé

RP.

Individual and mass behaviour in large population stochastic wireless power control problems: centralized and Nash equilibrium solutions. In: 2003 42nd IEEE conference on decision and control, Maui, HI, 9–12 December 2003, vol. 1, pp.98–103. New York: IEEE.

34.

Giné

Zinn

Central limit theorems and weak laws of large numbers in certain Banach spaces. Z Wahrscheinlichkeit 1983; 62(3): 323–354.