Deep autoencoder-based community detection in complex networks with particle swarm optimization and continuation algorithms

Abstract

Detecting communities is an important multidisciplinary research discipline and is considered vital to understand the structure of complex networks. Deep autoencoders have been successfully proposed to solve the problem of community detection. However, existing models in the literature are trained based on gradient descent optimization with the backpropagation algorithm, which is known to converge to local minima and prove inefficient, especially in big data scenarios. To tackle these drawbacks, this work proposed a novel deep autoencoder with Particle Swarm Optimization (PSO) and continuation algorithms to reveal community structures in complex networks. The PSO and continuation algorithms were utilized to avoid the local minimum and premature convergence, and to reduce overall training execution time. Two objective functions were also employed in the proposed model: minimizing the cost function of the autoencoder, and maximizing the modularity function, which refers to the quality of the detected communities. This work also proposed other methods to work in the absence of continuation, and to enable premature convergence. Extensive empirical experiments on 11 publically-available real-world datasets demonstrated that the proposed method is effective and promising for deriving communities in complex networks, as well as outperforming state-of-the-art deep learning community detection algorithms.

Keywords

Complex networks community detection autoencoder particle swarm optimization continuation method

1 Introduction

In recent years, the research community has made great efforts to attempt to solve the problem of community recognition in complex networks (hereafter referred to as CNs). Community detection is an important multi-disciplinary research area, and is considered vital to understand the structure of networks [1]. It splits a CN member into several communities, whereby nodes within the same community are very similar, and less similar to nodes in other communities. Community detection is applied in many applications, including advertising and recommendation activities, link inference and influence analysis, biological and criminological processes, etc. [2, 3]. In reality, several real networks can be represented as CN, e.g. social networks, citation networks, biochemical networks, computer networks, protein-protein interaction networks, etc. All these types of networks can potentially detect and analyse useful information using a community detection task. [4 –6].

Encouraged by the considerable success achieved with Deep Learning (DL) in many areas, the problem of community recognition was successfully solved with DL models. The similarity between Deep Autoencoder (hereinafter referred to as DAE) and spectral clustering in terms of feature learning and matrix reconstruction inspired the research community to propose several methods for solving the community detection problem with DAE models [7 –9]. These methods are based on the unsupervised DL models (i.e., DAE) [10], which can transform the information from CNs into a low-dimensional space containing useful information that points out the similarity between CN members. The communities are identified by applying clustering algorithms on the obtained low-dimensional space. The following are examples of these models: Sparse Autoencoder (SPAE) [11], Stack Autoencoder (SAE)-based modularity [12], DAE-based modularity [13], and DAE-based nonnegative matrix factorization [14].

DL model training is an NP-heavy optimization problem, typically involving thousands and millions of parameters, which play a key role in increasing the number of local minima [15]. The most popular method for training DL models is the Gradient Descent (GD) with Back-Propagation (BP) algorithm. However, this method has three shortcomings: the slow training process, especially with large amounts of data; sensitivity towards parameter initiation; convergence to local minima [16, 17]. The existing DAE-based community detection methods were built with GD-based BP algorithm. As a result, the limitations of the GD-based BP reduce the effectiveness and efficiency in solving community detection problem. Therefore, it is an interesting subject to find an effective optimization method for the DAE-based community detection in order to overcome the limitations mentioned above.

Metaheuristic algorithms have a good track record in optimizing machine learning models, and are known as efficient methods for solving complex problems and finding optimal or near-optimal solutions in a reasonable time. They can also avoid local minima and increase the likelihood of global convergence [16, 18]. In optimization, however, metaheuristic methods are rarely used in DL models [19]. Therefore, the combination of metaheuristic algorithms for optimizing DL models is an interesting research topic.

Particle Swarm Optimization Algorithm (PSO) is one of the popular global search metaheuristic algorithms, which is a population-based optimization algorithm [20]. In PSO algorithm, the fitness function plays an important role in finding optimal solutions, and it consists of a single objective function; however, it is not effective especially for complex problems such as DAE-based community detection. Thus, it is interesting to establish a fitness function that consists of multi-objective functions, allowing optimization of DL models becomes more effective [21], although the computational complexity will be increased. A continuation method is an iterative method that starts with solving problem in a simple form, and gradually extends to the actual problem. Therefore, it is of great interest to integrate the continuation method with PSO algorithm. The integration will reduce the execution time in optimization and the probability of local minima and premature convergence [22].

In this paper, we present a new DAE model based on PSO and continuation optimization algorithms for community detection in CNs, called (DACDHC). The contributions of this work are summarized as follows: (1) DACDHC method uses PSO algorithm to learn parameters of DAE model. This allows the method to take the advantage of PSO optimization, e.g. avoid local minima and increase the probability of global convergence. Consequently, a useful latent representation of a CN is extracted, which helps in community detection task. (2) Continuation optimization method is combined with PSO algorithm to reduce execution time in optimization, and avoid the probability of local minima and premature convergence. (3) Multi-objective optimization is suggested for DACDHC method since the method includes a DAE model and a clustering algorithm, the two different objective functions used to further improve performance; the first concerns DAE training, i.e. the Mean Square Error (MSE) function, and the second is the modularity Q function, which is related to the quality of communities discovered. In addition, the paper presents the method without the continuation algorithm called DACDH and another method allowing premature convergence called DACDHE.

The paper is organized as follows: section 2 presents an overview of related works; section 3 introduces the preliminaries of autoencoder, PSO and continuation algorithms; section 4 elaborates the proposed method DACDHC in detail. section 5 presents experimental results and analysis of the proposed method; the conclusion is drawn in section 6.

2 Related works

2.1 Community detection with deep learning models

Several studies have proposed using DL models for community detection in the field of CNs. Most of the suggested models were built based on Autoencoder (AE) architecture, which are unsupervised learning models. The main aim of the models is to learn the latent representation of CNs, in which the network structure is preserved and similarity between network members is shown so they can easily be placed into communities. A SAE is proposed for learning the graph embedding [11]. The design was inspired by the similarity between AE model and spectral clustering algorithm. SAE also consists of several encoding and decoding layers. In addition, it uses the loss reconstruction and KL divergence functions. On the other hand, Yang et al. developed a SAE model based on modularity method [12]. The layers of the SAE are trained to create a useful latent representation of the graph. The model is extended to a semi-supervised model by including pairwise constraints. The BP algorithm and greedy layer-wise training process are adopted in the mentioned models.

Recently, Dhilber et al. proposed a SAE-based modularity model [13] as a new way to train the entire model in one training process, in contrast to [12] that adopted a separate training for each layer. In addition, the model is trained to reduce training time by sharing weights across the layers. Similarly, Fanghua et al. integrated the concept of nonnegative matrix factorization into the layer of AE model [14]. The encoder and decoder components are also integrated into the traditional nonnegative matrix factorization to form a uniform loss function. Another type of DL model has been applied in this field of study. For example, a deep sparse filtering model was established to discover communities in CNs [7]. On the other hand, Cao et al. proposed an AE that can integrate network topology and network content [23], whereby Laplace and modularity matrices are fed to the AE model. The experimental results of all the above-mentioned methods demonstrate that these methods have outperformed the traditional methods in the field of community detection. However, most of the existing models encounter the problem of inefficiency and local optimal solutions due to the use of GD with BP algorithm for optimization process.

2.2 Deep learning-based metaheuristic optimization

The success of metaheuristic method in optimization tasks has prompted the research community to solve large and difficult problems using metaheuristic algorithms. Recently, several studies have proposed the use of these algorithms for optimizing DL models to create effective and efficient models. In the field of deep neural networks, the GD is replaced by iterative metaheuristic methods to tune the parameters and overcome limitations in GD with BP. The limitations include the slow training process, sensitivity to parameter initiation and the risk of falling into local minima [16, 18]. In this subsection, we outline the recent and important studies which have used metaheuristic algorithms for training and optimizing DL models.

A PSO-based population algorithm was developed to train neural networks in [24]. The fitness function of particles is the cost function of the model, which is evaluated by the current position and corresponds to the weight matrix. The authors presented a comparative study between PSO-based heuristic method and BP methods. Similarly, Zhang et al. integrated both PSO and BP in a model for neural network training [25]. This integration aims to take advantage of both algorithms. PSO performs a global search at initializing phase of the weights followed by a local search around the global optimum by BP algorithm. On the other hand, ACO algorithm is used to feedforward neural network model [26]. Two optimization algorithms are suggested: to train the model based on the stand-alone ACO optimization algorithm; to use BP and ACO. A bat algorithm was employed to optimize both the structure of artificial neural network and the parameters (i.e., weights) In [27]. The work suggested two modifications to the bat algorithm, which named MBatDNN and MeanBatDNN.

Metaheuristic methods have also been used to optimize DAE model. Parameters optimization (i.e., weights) was achieved using GA-assisted AE model [28]. The proposed method overcomes the problem of tied weights in encoder and decoder models to enhance the accuracy in classifications. On the other hand, PSO algorithm was integrated with AE by incorporating PSO into marginalized Stacked Denoising AE to tune the parameters [29]. PSO can hold the best configuration of the marginalized AE. Despite the good results achieved using these methods, most of them still lack in efficiency or effectiveness, or both.

Optimization by continuation refers to the solving of a problem starting with the easy form and gradually expanding to the actual problem through iterations [30]. The purpose is to reduce optimization time and maintain the performance. A continuation method was proposed for training and optimizing neural network models using metaheuristic algorithms [22]. The experimental results demonstrate that continuation method of the three studied metaheuristic algorithms have improved the efficiency of the training and reduces the execution time by 5–30 % compared to the standard metaheuristics. Similarly, incremental learning is suggested for such task, which means the learning of streaming data that arrive over time [31]. In this method, the models are train based on a new batch of data and an earlier model. This method requires limited memory resources without sacrificing the model accuracy.

2.3 Community detection and PSO algorithm

In this section, we provide an overview of some important works that have been proposed for community detection-based metaheuristic algorithm, solely (i.e., without DL models). Single-objective metaheuristic algorithms, especially PSO, have been suggested for community detection in CNs field. In [32], a method was proposed to solve the problem of community detection using PSO algorithm. The method utilizes network modularity and eigenvectors for optimization process. The method can automatically detect the number of communities and the particles required in a low-dimensional structure. Similarly, the work in [33] proposed the use of PSO to find communities in CNs based on the modularity function. The method aims to maximize the modularity function that meets a fitness function. The method was evaluated in a small network and achieved acceptable results. However, a single objective optimization does not work well and may fall into a local optimum trapping.

Multi-objective optimization algorithms was proposed to overcome the disadvantage in the single objective optimization. For example, a method using multi-objective PSO in [34]. The two objective functions are Kernel k-mean and ratio cut. This work was further extended in [35] through modifications of the two objective functions to solve the problem of low efficiency and large search space. Similarly, an anti-colony multi-objective algorithm was developed for community detection in CNs in [3]. The algorithm optimizes two objective functions: the community score that measures the cluster density; the fitness function of the community that minimizes the external links. Despite their effectiveness, they have only been used for simple and small datasets, in addition to the high computational complexity. These methods too may fall into local optimum solution.

In this work, a DACDHC was developed based on a DAE model to solve the problem of community detection in CNs. PSO and continuation algorithms have effectively and efficiently trained and optimized the model. Two different objective functions were used to further improve the performance; the first concerns AE training, i.e. the Mean Square Error (MSE) function, and the second is the modularity Q function, which is related to the quality of communities discovered. The advantage of PSO optimization includes the ability to avoid local minima and increase the probability of global convergence, while the continuation method reduces execution time and the probability of local minima and premature convergence. In short, the proposed method was designed to benefit from the merits of existing methods to solve the community detection problem.

3 Preliminaries

3.1 Graph concepts

A graph G consists of nodes V and edges E as G (V, E), where the number of nodes and edges is n and m, respectively. A CN is a graph in context theory, and most graph theories can be employed in complex networks. The adjacency matrix A = [a_ij] ∈ R^n×n is the most widely used representation of CNs. The element a_ji denotes the relationship between node i and node j, it can be defined as a_ij = w_ij, if a_ij = 0, which means that if there is no relation between i and j nodes, there is a relationship with the weighting w_ij. The degree of a node is the number of relationships between one node and other nodes in the graph $d (vi) = \sum_{i}^{n} a_{ji}$ , where a high degree indicates the importance of the node in the graph or CN. The Jaccard J similarity measures the similarity between two vectors by dividing the intersection of the vectors by the unions. In complex network, each vector represents the relationship of the node v_i, i = 0, i, 2, …, n to other nodes v_j, j = 0, 1, 2, …, n in a complex network (i.e. the representation of the adjacency matrix). Jaccard can be calculated based on the following equation:

$\begin{matrix} J (v_{i}, v_{j}) = & \frac{(v_{i}, v_{j})_{11}}{(v_{i}, v_{j})_{10} + (v_{i}, v_{j})_{01} + (v_{i}, v_{j})_{11}} \\ (i, j = 1, 2, . . . n) \end{matrix}$ (1)

3.2 Preprocessing

In this phase, each node has a vector of features corresponding to the number of nodes n in the graph, but these features are sparse (i.e. zero, indicating that there is no relationship between the nodes and no common neighbours between them). In addition, the number of features is huge, especially for large networks, and this has a negative effect on the efficiency and effectiveness of the working model. Therefore, a reasonable number of features should be used; it can be achieved by calculating the average of a set of neighbouring features by reducing them to one feature [36]. The number of features that averaged to one is determined by a hyper-parameter called F, to which F belongs (1 > F > n). For example, if we have a simple network with 100 nodes, each node has a vector feature of size 100 according to the Jaccard similarity calculation. When F = 2, this means that two features are reduced to one (i.e., 1, 2=>1, 3, 4=>2,...99, 100=>50. The size of the new features is therefore 50, and in case of F = 4, the new features will be 25, and so on. The new feature size is based on the following: $F_{size} = \frac{n}{F}$ (2)

This method was chosen because it is easy and efficient to create a new version of the data that reflects the original data. In terms of the effect and efficient latent representation in relation to the clustering process, it is derived from the DAE model proposed in this paper.

3.3 Autoencoder and community detection

The autoencoder model is a feedforward neural network that reconstructs the output data so that it is similar to the input data [10]. It has two phases: (1) The first phase is called encoding, where the hidden layer h converts the data x and its features n into a latent representation with feature spaces d, where d is smaller than n. The mapping process is conducted with f1 function, h = f1 (W₁x + b₁). (2) The second phase is called decoding, which attempts to reconstruct the original data from the output of the hidden layer using the f2 function y = f2 (W₂x + b₂). The Sigmoid, Tanh, and Relu activation functions are normally used with the decoding and encoding functions since they are nonlinear mapping functions. The parameters of AE model are referred to as θ = [W₁, b₁, W₂, b₂] and trained by minimizing the loss function: $J_{θ} = \frac{1}{n} \sum_{i = 0}^{n} (x^{i} - y^{i})$ (3) with respect to all samples n. At some point, the loss function must be included in the regularization function to prevent overfitting such as sparsity. AE includes several layers in encoding phase and the same number of layers in decoding phase. The hidden layer is placed between them, which produces the latent representation of the input data (e.g., a CN in this paper). The GD with BP algorithm and greedy layer-wise training process has often been used in training strategy. However, this strategy leads to convergence to local minima in the parameter space and consumes a considerable amount of time, especially with large-scale DL models [17, 37].

Community detection aims to divide the network into dense groups, which typically correspond to entities that are closely related, and may belong to a single community. This is based on the similarity between spectral clustering and unsupervised DL models (i.e. AE model). AE model has proven its ability to solve the community detection problem, which works as follows: AE model receives the CN representation; then, after the model training process, the latent representation with low-dimensional feature spaces of a CN is extracted, showing the similarity between the CN members and ignoring the dissimilarity; finally, a clustering algorithm is applied to the latent representation to assign the nodes to communities [7 , 39].

3.4 PSO and continuation algorithms

Several metaheuristic algorithms were successfully utilized for the deep neural networks training. Metaheuristic algorithms can avoid local minima and increase the likelihood of global convergence [22]. PSO is a population-based optimization algorithm [20]. Since PSO algorithm is a population-based metaheuristic optimization, it has proven to be a good global optimization algorithm for various optimizations. It generates good knowledge that is learned through local experience of each particle. In addition, the algorithm supports constructive collaboration and information sharing between all particles, thereby improving the search performance for a global optimal solution.

PSO uses a stochastic optimization technique and mimics the behaviour of swarms of birds to find the solution for the optimization problem. Many particles are generated stochastically across the search space to construct a swarm. Each particle is generated to represent a solution candidate for the optimization problem, and each particle is defined by a D-dimensional vector containing trainable parameters. Then, an evaluation process based on the fitness function f (X_i) is carried out with a certain number of generations. The particle moves in the search space with a certain velocity. The particle can return to the best previous position based on memory, which is determined by the best previous performance of the individual particle (pbest) and the best previous performance of all particles (gbest) [20].

The current position is expressed as X_i = X_i1, X_i2, X_i3, . . . X_iD). The following equations represent the basic of PSO: $\begin{matrix} V_{id} (t + 1) = {wV}_{id} (t) + c 1 r 1 ({pbest}_{id} (t) - X_{id} (t) \\ + c 2 r 2 ({gbest}_{d} (t) - X_{id} (t)) \end{matrix}$ (4)

$X_{id} (t + 1) = X_{id} (t) + V_{id} (t + 1), i \in [1, p], d \in [1, D]$ (5)

As for updating the best positions: $\begin{matrix} {pbest}_{ib} = X_{i} if (f (X_{i}) < f ({pbest}_{i})), else ({pbest}_{i}), \\ gbest = X_{i} if (f (X_{i}) < f ({gbest}_{i})), else (gbest) \end{matrix}$ (6) where r1 and r2 are random numbers between [0,1]; c1 and c2 are called the cognitive and social parameters, respectively. Based on the expression, the behaviour of the particle is controlled in a way that it either follows its individual experience or the globally best position of the swarms; t and t + 1 denote the generations (iterations), w controls the inertia of the swarm’s movement, p is the number of particles and D is the dimension of trainable parameters. PSO trains deep neural network models and tunes their trainable parameters (i.e. weights and biases) by particles proposing solutions. Each solution represents trainable parameter values of all model layers, which are evaluated by the fitness function. The cost function of the neural network model is used as the fitness function.

Continuation optimization refers to a general approach that is not dealing with the whole problem from the beginning of the optimization, i.e. it starts to solve the problem from simple to difficult, from simple to complex or from small to large. This change occurs progressively through iterations of the optimization [30].

4 The proposed DACDHC method

This subsection presents the details of the proposed method DACDHC, which was used to identify communities with DAE model-based metaheuristic PSO and continuation optimization algorithms. In the DACDHC method, AE model is trained and its parameters are tuned using PSO metaheuristic and continuation optimization algorithms instead of GD-based optimization. PSO proposes several solutions corresponding to the number of particles, each solution involving its own tuning of the model parameters and the assignment of nodes to communities. Figure 1 and Algorithm 1 show the steps and structure of the method (DACDHC). DACDH is a method without continuation algorithm, while DACDHE is similar to DACDH but enables early stopping when premature convergence occurs.

Fig. 1

The steps and structure of proposed methods.

Algorithm 1: Framework of the proposed DACDHC method
Parameters: Particle swarm size P, maximum iterations max_t, cognitive parameters c1 and c2, inertia weight w, hyperparameters kd and F, number of communities C, new dimension of latent representation d
Input: Network representation as a graph G
Output: The latent representation and community labels
1: Step 1)Initialize solutions: create encoder and decoder layers; define activation functions; initialize position particles X ; initialize
2:velocity vectors V ; initialize local positions p best
3: Step 2)Preprocessing phase: obtain similarity matrix of a CN with Eq. (1); prepare the features fed into DAE model with Eq. (2)
4: Step 3) Train DAE model with PSO:
5: Repeat
6: if (Continuation), then do:
7: Run degeneracy method to create a subgraph based on k-core. See subsection 4.1 and Fig. 2
8: Send the new subgraph to DAE.
9: Update k according Eq. (7)
10: if (k==0), then Continuation = False
11: otherwise , send the full graph (G) to DAE
12: Compute the output from hidden layer (latent representation); compute the output from decoding layer (reconstruction data)
13: Assign nodes to communities: select head nodes; calculate the distance between each node and head nodes; assign the node to the
14: nearest community
15: Calculate the fitness value using the two objective functions: minimize MSE (Eq. (3)) and maximize Q modularity (Eq. (8))
16: Update the local solutions ( pbest ) and global solution ( gbest ) according Eq. (6)
17: Calculate the velocity of each particle according Eq. (4)
18: Update the position of each particle according Eq. (5)
19: if (DACDHE), then early_stop = True, otherwise early_stop = False. See subsection 4.3
20: t = t + 1
21: Until(t > max_t or early_stop)
22: Step 5) Clustering process: Feed the new representation of CN node into a k-mean algorithm to divide the CN nodes into communities
23: Step 6) Returning output: return the latent representation and community labels

The proposed methods start with the preprocessing phase. The phase receives the graph nodes n and edges e, then finds the similarity representation of the graph based on the Jaccard similarity using Equation (1).

The features are also reduced to a certain number of features by Equation (2). At this stage the data is ready to be sent to DAE model for the training process. The data is then subjected to the DACDHC, DACDH or DACDHE method. The methods are explained in detail in the following subsections.

4.1 DACDHC

In DACDHC method, the output of preprocessing step is sent to the continuation algorithm based on the degeneration or k-core method. Then, the initialization of the particles produces first solutions. The initial solutions are sent to the continuation algorithm based on the degeneration or k-core method. The continuation algorithm sends the CN data and initial solutions to DAE model for a training process in several phases. The early phases send small size sub-CNs with nodes that have a high degree (d), and gradually, the subgraph size is increased based on the degree until the entire CN is reached over iterations. The evaluation of the solution is performed on a single batch because the second fitness function (i.e. the modularity function) requires all CN or sub-CN nodes that are in the training process. Therefore, the continuation starts sending the important nodes to the model and the CN size is gradually increased over the iterations.

K-core refers to the maximum subgraph generated, whereby all vertices have a degree of at least k. Increasing a subgraph based on node degree makes it possible to create a new subgraph with new nodes based on k degrees (i.e. a k-degenerated graph is an undirected graph in which each subgraph has a vertex with a degree of at least k). In this work, the degree was based on the original graph, not on the degree of the generated subgraph. The basic continuation process includes the following steps: k degrees are initialized to guarantee that around 5 % of the total size of the graph is generated; next, the change from the previous k-degenerate phase to the next phase is determined, which allows the addition of another subgraph with new nodes alongside the nodes in the current subgraph, where the change is based on the kd value. The subgraphs are generated as follows: $\begin{matrix} \begin{matrix} g_{k_{1}}^{\land} \subset g_{k_{2}}^{\land} . . . \subset g_{k_{z}}^{\land}, g_{k_{z}}^{\land} \equiv G, k_{1} = max_{d} - (1 * kd), \\ k_{z} = max_{d} - z * kd = 0, z = \frac{{max}_{d}}{kd} \end{matrix} \end{matrix},$ (7) where max_d is the maximum degree in graph G; kd a hyper-parameter involving an integer number greater than 1, which indicates the difference in max node degree between the current generation of subgraph and the next generation (Note: If the node degree is small, the number of nodes added to the subgraph is large until the full graph is reached with degree = 0); z is an integer that gives $k = \max_{d} - - (z \times kd) = 0$ , which refers to the newly generated subgraph involving all nodes with a degree from 0 to max_d. In this case, all graph nodes are added to the training process, whereby $\overset{\land}{g}$ is a subset of training graph fed to the DAE model, with the model parameters trained using PSO based on Equations (4 –6). The training patterns is performed as follows: ${\overset{\land}{X}}_{max d - kd} \subset {\overset{\land}{X}}_{max d - 2 kd} . . . \subset {\overset{\land}{X}}_{max d - zd}, {\overset{\land}{X}}_{max d - zd} \equiv X$

Consequently, the local solutions for all particles are expressed as follows:

${\overset{\land}{pbest}}_{1} \subset {\overset{\land}{pbest}}_{2} . . . \subset {\overset{\land}{pbest}}_{3}, {\overset{\land}{pbest}}_{max d - zd} \equiv pbest$

On the other hand, the global solutions are expressed as follows: ${\overset{\land}{gbest}}_{1} \subset {\overset{\land}{gbest}}_{2} . . . \subset {\overset{\land}{gbest}}_{3}, {\overset{\land}{gbest}}_{max d - zd} \equiv gbest$ where the fitness function that consists of MSE and modularity Q functions is gradually improved as the problem is expanded, i.e. the subgraph is expanded based on the degeneration or K-core method until the real and complete graph is established $f (θ, \overset{\land}{X_{1}}) ⩽ f (θ, \overset{\land}{X_{2}}) ⩽ . . . f (θ, {\overset{\land}{X}}_{max d - zd})$

For clarifications, this is an example of how the process continuation works, which details are shown in Fig. 2 (a-e). In Fig. 2 (a), the original graph, where red nodes have the highest degree, followed by green, orange, and pink nodes. The graph has 20 nodes, the maximum degree is max_d = 9 and the difference between the phases is kd = 3, which indicate the increasing graph size from the current generation to the next generation. The beginning allows for creation of at least 5 % of the original graph by adding nodes with degree of max_d or max_d - kd. The subgraph is progressively extended in phases over iterations up to z + 1 = 4 phases, where k = 9, 6, 3, 0, as shown in Fig. 2 (b), (c), (d) and (e). In Fig. 2 (b), a subgraph that has nodes with a degree greater than or equal to 9 is selected, and (c) a subgraph with nodes having a degree of at least 6, until the final phase, in which the entire graph nodes with degree k = max _ d -- (z × kd) = 0, is added. The continuation optimization process feeds the subgraphs into DAE model and PSO optimization, progressively by iterations.

Fig. 2

A simple graph to illustrate the continuation-based k-degenerate method.

In the k generation of subgraphs, all operations of the PSO and AE are performed for all generations. In addition, fitness function is calculated as MSE-AE and Q-modularity functions in each phase and generation. The MSE-AE is calculated based on Equation (3) while the Q- modularity is calculated based on (8). The Q-modularity function requires the assignment of nodes to clusters. Consequently, the trainable parameters and fitness values are updated iteratively.

After the continuation process selects the subgraph (i.e. sub-CN), the initial solutions and the selected subgraph are sent to the PSO algorithm to train AE model, which extracts the latent representation for the nodes in the subgraph; the latent representation is then sent to the clustering process. Next, based on the extracted latent representation, the nodes are assigned to the appropriate communities. This phase begins with the selection of the head nodes having the highest degree, where the number of head nodes corresponds to the number of communities, identified as cluster centres. Then, the distance between each node and the head nodes is calculated using Euclidean distance, and the nodes are assigned to the nearest head node to form the communities. The assignment of nodes and model parameters are sent to the evaluation phase.

In evaluation phase, the fitness function includes two objective functions: the first is the minimization of MSE using Equation (3); the second is the maximization of the modularity Q using Equation (8), but generally added to MSE as minimization (1-Q), to be employed for the same goal of community detection and MSE. The fitness function starts with receiving AE parameters and calculates the MSE based on Equation (3). In the first few iterations of the optimization process, the fitness function continues to calculate only the MSE. After the MSE value has improved and reached a certain value known as the MSE improvement value (MSEI), which is less than one, the second objective function is added and used alongside the MSE, which is a Q-modularity function based on the following equation: $Q = \frac{1}{2 M} \sum_{i = 1}^{N} \sum_{j = 1}^{N} (A_{ij} - \frac{d_{i} d_{j}}{2 M}) δ (ci, cj),$ (8) where d_i represents the degree of the vertex, i, N is the number of nodes in the subgraph, and m is the total number of edges in the subgraph. A_ij is the element in the adjacent matrix, and d_ij =1 if vertex, i, and j belong to the same community c, otherwise d_ij =0.

Based on the fitness function, the globally best solution is selected from all proposed solutions represented by the particles; the locally best solution is also selected in the same manner. At this point, the first iteration of the optimization is already completed, moving to the next iteration, which starts with creation of a new solution based on the selected local and global solutions using Equations (4) and (5). The new solutions are close to the best local and global solutions. The solutions refer to AE parameters, q, including weights, W and biases, b, which lead to the assignment of CN nodes to communities. All operations mentioned above are then repeated until the maximum iterations are reached. Finally, the latent representation of CN nodes is obtained from the established communities. The steps and structure of DACDHE are shown in Fig. 1 and Algorithm 1. Additional operations in the method are shown in blue alongside the existing ones in black.

4.1 DACDH

This method follows the same steps as in the DACDHC method, except that the continuation method is excluded. And the model deals with the whole CN from the beginning of the training to the end. The method starts with the PSO to initialize the particles, which are the solutions for tuning parameters of AE model (i.e. weights and biases) and the assignment of nodes to communities. Then, the latent representation is extracted and the nodes are assigned to the communities. This is followed by the evaluation phase, during which the two objective functions MSE and Q are calculated. Based on the evaluation, PSO updates the local and global solutions and create new solutions that are close to the best global and local solutions using Equations (4) and (5). All of the above operations are repeated until the convergence is reached by the end of iterations. The steps are listed down in black in Fig. 1 and in Algorithm 1, where continuation parameter is False.

4.2 DACDHE

This method is similar to DACDH, which starts with feeding the entire graph to the model. However, it allows for premature convergence when the model gets stuck at a certain point of optimization, which is when the discrimination threshold is E = 0.1e-10. If the threshold is reached with 5 consecutive iterations, the method aborts processing to save computational costs. The additional steps in the method are shown in red alongside the existing steps in black in Fig. 1 and in Algorithm 1, where the early stop parameter is True.

5 Experiments

In this section, extensive experiments carried out to evaluate the proposed methods with respect to their efficiency and effectiveness are described. First, the experimental setting, datasets, evaluation metrics and comparison models are presented followed by the four available models in this area. Finally, the experimental results and meaningful analyses of the selected results are described in detail.

5.1 Experimental setting

The proposed method was evaluated on the benchmark dataset using a single machine that has Core i7 processor, 8 GB RAM and NVIDIA GeForce (GT 840) with 4 GB memory. The Python programing language and Pyswarm package [41] were utilized.

The suggested methods were evaluated and analysed on 11th common real CNs. The datasets were selected because they are commonly used by the researchers working in the area of community detection. Furthermore, the diversity of these datasets in terms of content, size and sources, allows an accurate evaluation of the proposed methods. Six datasets of small networks contain the truth labels (i.e., each node was assigned to real community) namely Karate, School6, School7, Dolphns, Football, and Pol-books. Another five datasets ranged from medium to large networks, namely Political-blogs, Citeser, Cora, Facebook 1, and Facebook 2. The details of these networks are listed in Table 1.

Table 1
Dataset information

Dataset Nodes Edges Communities Refs.

Karate 34 78 2 [42]

dolphins 62 159 2 [43]

School 6 68 220 6 [44]

school 7 68 220 7

Football 115 613 12 [45]

Pol-books 105 441 3 [46]

Pol-blogs 1,490 16,718 2 [47]

Cora 2,708 5,429 7 [48]

Citeser 3312 4732 6

Facebook 1 4039 81800 12 [49]

Facebook 2 22,470 171,002 4 [50]

Dataset	Nodes	Edges	Communities	Refs.
Karate	34	78	2	[42]
dolphins	62	159	2	[43]
School 6	68	220	6	[44]
school 7	68	220	7
Football	115	613	12	[45]
Pol-books	105	441	3	[46]
Pol-blogs	1,490	16,718	2	[47]
Cora	2,708	5,429	7	[48]
Citeser	3312	4732	6
Facebook 1	4039	81800	12	[49]
Facebook 2	22,470	171,002	4	[50]

The preprocessing phase included a hyperparameter F, which averages several adjacent features to a single feature, where it is defined depending on the size of the dataset as follows: F = 2 for Karate Football and Pol-books networks, and F = 20, 30, 35, 40, and 200 for Political-blogs, Cora, Citeser, Facebook 1, and Facebook 2 networks, respectively. The F value was set based on the best value obtained from the experiments.

DAE was set up as follows: Sigmoid and Relu-wise elements were used for the proposed method, where Relu was used for the hidden layer and Sigmoid for the encoding and decoding layers. DL architecture includes 3 encoding layers as well as decoding layers. PSO was set up as follows: Particles p = 20, c1 = 0.5, c2 = 0.3, and inertia weight w = 0.9. The settings for the continuation method were determined as described above. The hyperparameter MSEI was set to 0.9. The number of iterations for the optimization was set to 100.

5.2 Evaluation metrics

Two accurate metrics are presented here to validate the effectiveness of community detection methods: the first is Modularity Q, and the second is Normalized Mutual Information (NMI) [51]. The modularity Q metric was applied to evaluate the quality of community detection on real CNs since the truth label was unknown in most real CNs; it is defined by Equation (8). As for NMI, it was used for CNs with the known basic truth; it is defined as follows:

$\begin{matrix} NMI (A, B) \\ = \frac{- 2 \sum_{i = 1}^{C_{A}} \sum_{j = 1}^{C_{B}} N_{ij} log (N_{ij} n / N_{i} N_{j})}{\sum_{i = 1}^{C_{A}} C_{i} log (N_{i} / n) + \sum_{j = 1}^{C_{B}} C_{j} log (N_{j} / n)}, \end{matrix}$ (9) where A is the real labels, B is the expected labels; N_ij is the number of nodes in clusters i and j of A and B; N_i and N_j is the number of nodes in cluster i and j, respectively; n is the number of nodes. The maximum value of NMI refers to the high similarity between real and expected clusters.

On the other hand, in regard to the method efficiency evaluation, the time consumption and the speedup metric were adopted [52]. Speed up of Method A over Method B can generally be defined as follows: $speed - up (A over B) = \frac{Method A}{Method B} (time)$ (10)

5.3 Comparing models

Four recently available DAE-based methods for community detection are presented in this section. These methods were used to validate DACDHC, DACDH, DACDHE methods as they have been used for GD-based BP optimization, and procedures are available online.

SPAE [11]. It is one of the earliest methods proposed with DL for community detection.

SAE based Modularity (SAEM) [12]. It maximizes modularity with DL for community detection.

DAE based Nonnegative Matrix Factorization (DANMF) [14]. It combines DAE and nonnegative matrix factorization principles for community detection.

DAE based Modularity (DAEM) [13]. It uses the sharing of weights across the layers to reduce the memory and time complexity of the method. The implementation of these methods is available online at GitHub’s pages namely SPAE 1 , SAEM 2 , DANMF 3 and DAEM 4 .

5.4 Results and discussions

5.4.1 Effectiveness evaluation

In this section we evaluate the effectiveness of the proposed methods (DACDHC, DACDH, and DACDHE). The suggested methods were applied to six different real datasets, which are presented in Table 1. The mean value of the NMI and Q modularity metrics over ten runs were reported. The performance of the four state-of-the-art algorithms mentioned above was also evaluated using the same datasets. Tables 2 and 3 show the experimental results obtained using the state-of-art and the proposed methods to solve the problem of community detection with DAE model.

Table 2
Performance evaluation based on NMI

Dataset SPAE SAEM DANMF DAEM DACDH DACDHE DACDHC

Karate 1.0 1.0 1.0 0.84 1 1.0 1.0

Dolphins 0.89 0.88 0.89 0.83 0.95 0.75 1.0

School 6 0.73 0.88 0.75 0.67 0.90 0.76 0.90

School 7 0.76 0.90 0.78 0.61 0.90 0.78 0.88

Football 0.92 0.92 0.87 0.90 0.96 0.92 0.96

Polbooks 0.54 0.581 0.56 0.60 0.68 0.54 0.68

Polblogs 0.15 0.51 0.48 0.52 0.55 0.44 0.56

Dataset	SPAE	SAEM	DANMF	DAEM	DACDH	DACDHE	DACDHC
Karate	1.0	1.0	1.0	0.84	1	1.0	1.0
Dolphins	0.89	0.88	0.89	0.83	0.95	0.75	1.0
School 6	0.73	0.88	0.75	0.67	0.90	0.76	0.90
School 7	0.76	0.90	0.78	0.61	0.90	0.78	0.88
Football	0.92	0.92	0.87	0.90	0.96	0.92	0.96
Polbooks	0.54	0.581	0.56	0.60	0.68	0.54	0.68
Polblogs	0.15	0.51	0.48	0.52	0.55	0.44	0.56

Table 3

Performance evaluation based on Q

Dataset	SPAE	SAEM	DANMF	DAEM	DACDH	DACDHE	DACDHC
Karate	0.37	0.37	0.37	0.36	0.37	0.37	0.38
Dolphins	0.38	0.37	0.38	0.37	0.38	0.38	0.40
School 6	0.53	0.50	0.57	0.55	0.59	0.54	0.60
School 7	0.565	0.44	0.54	0.4	0.58	0.54	059
Football	0.598	0.58	0.56	0.59	0.60	0.58	0.60
Polbooks	0.49	0.47	0.50	0.49	0.51	0.49	0.52
Polblogs	0.01	0.38	0.41	0.42	0.42	0.40	0.41

Based on Table 2, the results demonstrate that the DACDH method provides comparable and competitive results compared to other methods. In the Karate network, we found that the extracted communities matched the ground-truth communities and reported the NMI value of 1.0. For most of the remaining datasets, we found that DACDH achieved a higher NMI value than the state-of-the art methods (i.e., SPAE, SAEM, DANMF, and DAEM). The results also demonstrate the reduced performance using DCCDHE method as compared to that of the state-of-the-art methods, as the former focuses on reducing the running time. Table 2 shows that the best NMI was reported from using DACDHC on most datasets, with a value of 1.0, 1.0, 0.90, 0.88, 0.96, 0.68, and 0.56 for Karate, School6, School7, Dolphins, Football, Polbooks, and Polblogs networks, respectively. DACDHC outperformed other methods by exploiting the advantages in DL and PSO algorithm optimization, which allows for an escape from the local optimal solutions. Furthermore, the continuation method helps in avoiding local minima and premature convergence by preventing the procedures from getting stuck at a particular optimization point through progressive addition of new data to the model. As a result, the performance improved significantly, as shown in Table 2.

The quality of community detection was taken into account in this work, which was evaluated using the modularity Q metric. The evaluation demonstrates that there is a denser relationship within communities than the relationship between communities. Table 3 compares the result of modularity Q obtained using the suggested methods applied to the seven real CNs. The method DACDH performed slightly better than other methods (i.e., SPAE, SAEM, DANMF, and DAEM) when applied to most of the CNs. The gain is due to the optimization-based PSO, which can overcome the local optimization; the traditional models are often carried out using gradient optimization by BP algorithms (Note: GD optimization has been adopted in all of the state-of-the-art methods). Overall, DACDHC outperformed other methods proposed in this study when applied to most of the datasets. The outperformance can also be interpreted using NMI values in the same manner as discussed above. On the other hand, the weakest performance was observed in the use of DCCDHE due to the early stop; the method improves efficiency by allowing termination of optimization process when the model is stuck at one point.

The performance of DACDHC was also evaluated in medium and large networks namely Cora, Citeser, Facebook and Facebook 2, solely based on with Q-metric due to the unknown ground truth of some datasets such as Facebok as shown in Table 4. The results in Table 4 show that DACDHC records the best Q metric values for most datasets compared to other proposed methods. There is also a significant drop in performance observed in the use of DCCDHE method. The performance of DACDH was slightly lower than DACDHC. In general, the results obtained from the performance analysis in medium and large networks do not differ that of small networks (i.e., Karate, Dolphins, School6, School7, Football, Polbooks and Polblogs).

Table 4

Performance evaluation based on Q with large networks

Dataset	DACDH	DACDHE	DACDHC
Cora	0.36	0.29	0.37
Citeser	0.49	0.43	0.50
Facebook 1	0.76	0.71	0.79
Facebook 2	0.50	0.42	0.50

The results presented in Tables 2 –4 show the outperformance of the proposed methods compared to the four state-of-the-art methods. This is due to the newly proposed fitness function that combines two objective functions (i.e. MSE and Q). The cost function of AE (MSE) is designed to extract a latent representation for each node of a CN with low-dimensional feature space and to preserve the original structure of the CN representation. Modularity function (Q) directs the extracted latent representation towards the task of community detection as the function binds each node to the appropriate cluster and performs a fast evaluation based on the Q. In other words, the new representation of CN nodes is similar for nodes with the same structure. As a result, the clustering algorithm assigns similar nodes to the same community, enabling the separation of different communities.

5.4.2 Efficiency evaluation

A) Running time: this paper also takes the efficiency of the proposed methods into account. This subsection presents the running time rendered by the proposed methods. As discussed in the previous section, the DCCDHC method is identified as the best in terms of effectiveness, followed by DACDH and DCCDHE; worst performance was rendered by DCCDHE as the method focuses solely on efficiency that it compromises the accuracy.

The efficiency is determined by the running time required to operate the method. Figure 3(a-e) shows the running time rendered by DACDH, DACDHC and DACDHE when applied to the 11th real CNs presented in Table 1. In Fig. 3(a), it is clearly observed that the running time of DACDHC is longer than DACDHE but shorter than DACDH when applied to Karate and Dolphins dataset, signifying the highest efficiency rendered by DACDHE and the lowest by DACDH. Similar results are observed in Fig. 3(b-e) for all of the remaining real CNs, namely School6, School7, Football, Polbooks and Polblos, Cora, Citeser, Facebook, and Facebook 2. It is important to note that the results also demonstrate that the difference in the running time of the proposed methods becomes larger as the CN grows larger.; Fig. 3(a) presents the CNs with the smallest size (i.e., 34 and 62 nodes) while Fig. 2(e) presents the CNs with the largest size (i.e., 4039 and 22470 nodes). In general, the efficiency of DACDHC and DACDHE is improved as the size of the CN grows in comparison to DACDH.

Fig. 3

Running time of the proposed methods on different 11th real complex networks.

DACDH is the original design that combines the PSO heuristic algorithm and DAE model to find communities in CNs. From the results in Fig. 3(a-e), it is clear that DACDH requires the longest running time when applied to all real CNs because the method deals with the entire CN from the start of the training until the end. Furthermore, the modularity fitness function requires the assignment of all nodes to all clusters prior to evaluation in each iteration of the optimization. The method also has an optimization threshold, which at some point can be stuck in premature convergence; even after several iterations, only small improvement in the optimization can be achieved. On the other hand, DACDHE method allows for a stop in optimization process at a certain threshold value to avoid wasting time on optimization that does not make sense. Therefore, the results indicate that DACDHE renders the highest efficiency as compared to other suggested methods. Notably, DACDHE sacrifices performance, as observed in the performance evaluation presented in Tables 2, 3, and 4. DACDH does not allow for an escape in the event of being stuck in premature convergence regardless of the number of iterations.

On the other hand, the results demonstrate that the use of DACDHC has not only improved the performance, but also has improved the efficiency in comparison to the original version of the method (i.e., DACDH). The improvement is due to the continuation-based degeneration method, whereby the graph or CN of different sizes (i.e. subgraphs based on degree of node) from small up to the original size of the CN are progressively fed to the model. As a result, the model runs fast during the early iterations on the small subgraphs because the evaluation phase (i.e., the calculation of the Euclidean distance between nodes and centres, the assignment of nodes to clusters, the calculation of MSE and Q objective functions) is performed on small data. The method also benefits from the early optimization performed on small subgraphs during the early iterations. Thus, the performance and efficiency of the are improved.

The number of communities is also considered in evaluating the efficiency of the proposed methods. Figure 4 illustrates the running time of the proposed methods when applied to CNs with communities of different sizes. It is clearly shown that with the increasing number of communities, the running time increases. This is due to the objective function of modularity, which requires the nodes to be assigned to the nearest cluster centre and the Euclidean distance to be calculated, which leads to an increase in overall process optimization.

Fig. 4

Running time of the proposed methods on CNs have communities with different sizes.

The largest fraction of time for optimization is spent on fitness function as it consists of two objective functions, namely MSE and Q; Q assigns each node to a suitable cluster and evaluates the detected clusters according to Equation (8). The functions are also influenced by the size of CN and the number of iterations required for the optimization. Thus, according to Fig. 3(a-e), DACDHC outperforms DACDH as the size of CN gradually increases throughout the optimization process in the former. In this case, the DACDHC runs fast during most of optimization iterations, especially during the early iterations. On the other hand, DACDHE surpasses DACDH too as it takes advantage of the premature convergence, which allows for a stop in the optimization process.

B) Speedup gain: the speedup gain of DACDHC and DACDHE methods as compared to DACDH, the original version of the proposed method, was calculated. Firstly, the speedup gain of DACDHC over DACDH was calculated, followed by calculation of the speedup gain of DACDHE over DACDH. Figure 5 demonstrates the greater performance of DACDHC than DACDH with the following speedup: 1.3X speedup reported in small networks, namely Karate, Dolphins, School6, School7, Football, and Polbooks; 1.8X speedup in medium networks, namely Polbooks, Cora, and Cteser; around 2X and 3X speedup in large networks, namely Facebook, and Facebook 2, respectively. Figure 5 illustrates the greater performance of DACDHC than DACDH with the following speedup: 4X speedup reported in small networks, namely Karate, Dolphins, School6, School7, Football, and Polbooks; about 6X speedup in medium networks, namely Polblooks, Cora, and Cteser; about 6X and 10X speedup in large networks, namely Facebook and Facebook 2, respectively.

Fig. 5

Running time of the proposed methods on CNs have communities with different sizes.

The results demonstrate that the speedup of DACDHE is greater than DACDHC due to the reason discussed above, which is related to premature convergence and its focus in improving the efficiency by compromising effectiveness. DACDHC uses a continuation strategy to improve both the effectiveness and the efficiency of the original method, DACDH. It is also found that the speedup of DACDHC is enhanced with the increase in network size; the continuation strategy allows the model to start the iterations on small network by including the important nodes with high degree and progressively proceed to a lower degree until the original size is reached. Via DACDHE method, it is also found that the speedup increases as the size of the network increases; however, the increment is unstable as it depends on the occurrence of premature convergence.

In summary, we find that the integration of metaheuristic algorithm, exemplified by PSO, to optimize DAE-based community detection method has demonstrated promising performance as compared to the traditional optimization that relies on GD optimization using BP algorithm. The integration overcomes the limitations in traditional optimization, which include the local optimization and the high computational cost of BP. Furthermore, we find that the integration of continuation method to metaheuristic PSO algorithm plays an important role in improving both efficiency and performance as demonstrated by the results obtained from the use of DACDHC method. The integration prevents the process from getting stuck at a particular optimization point by progressively adding new data to the model, in the same way that it improves efficiency. We also find that enabling premature convergence leads to significant performance losses despite the contribution to the efficiency gains, as observed in the results obtained from the application of DACDHE method.

6 Conclusion

In this paper, DACDHC is presented as a new method for DAE-based community detection using PSO and continuation optimization algorithms. Via this method, PSO trains DAE model to extract the useful latent representation and use it for community detection. PSO was used to overcome the limitations in GD optimization with BP algorithms, which includes the probability of local minima and the inefficiency of BP algorithm. Furthermore, a continuation strategy was employed to reduce the time required for the optimization and avoid the local minima and premature convergence. A continuation algorithm feeds the CN data progressively through iterations based on the k-core method. The proposed method employs two objective functions that include minimizing MSE and maximizing Q modularity function, which refers to the quality of the communities detected. In addition, the paper presents on DACDH method, which operates similarly to DACDHC; the continuation strategy is excluded in DACDH and it deals with the entire CN from the start of processing. On the other hand, the DACDHE method allows for premature convergence to save the unreasonable computational costs of the optimization. Extensive experiments were performed on the 11th real-world CNs to evaluate the performance of the proposed methods. Based on results obtained, we find that the proposed method demonstrated the best performance in comparison to other methods. Furthermore, we find that the integration of the continuation strategy into the PSO algorithm can be considered efficient and beneficial in improving both efficiency and performance of DAE-based community detection in CNs. We also find that enabling premature convergence improves efficiency gains, but compromises the performance. Our future work will focus on combining DAE model with other metaheuristic methods for community detection. Parallel computation should be investigated to further improve the model efficiency in future.

Footnotes

Acknowledgments

This work was supported by the Fundamental Research Grant Scheme (FRGS) from the Ministry of Higher Education and Multimedia University, Malaysia (Project ID: FRGS/1/2018/ICT02/ MMU/02/1).

https/github.com/zepx/graphencoder

yangliang.github.io/code/DC.zip

https/github.com/smartyfh/DANMF

https//github.com/dilberdilu/community-detection-DL

References

Javed

M.A.

, Younis

M.S.

, Latif

, Qadir

and Baig

, Community detection in networks: A multidisciplinary review, Journal of Network and Computer Applications 108 (2018), 87–111. doi: https://doi.org/10.1016/j.jnca.2018.02.011

Khatoon

and Banu

W.A.

, A survey on community detection methods in social networks, International Journal of Education and Management Engineering 5(1) (2015), 8. doi: 10.5815/ijeme.2015.01.02

Sani

N.S.

, Manthouri

and Farivar

, A multi-objective ant colony optimization algorithm for community detection in complex networks, Journal of Ambient Intelligence and Humanized Computing 11(1) (2020), 5–21. doi: https://doi.org/10.1007/s12652-018-1159-7

Newman

M.E.J.

and Girvan

, Finding and evaluating community structure in networks, Physical Review E 69(2) (2004), doi: 10.1103/physreve.69.026113

Dong

, Wang

and Liu

, Spectral based hypothesis testing for community detection in complex networks, Information Sciences 512 (2020), 1360–1371. doi: https://doi.org/10.1016/j.ins.2019.10.056

Al-Ayyoub

, Al-andoli

, Jararweh

, Smadi

and Gupta

, Improving fuzzy C-mean-based community detection in social networks using dynamic parallelism, Computers & Electrical Engineering 74 (2019), 533–546. doi: https://doi.org/10.1016/j.compeleceng.2018.01.003

Xie

, Gong

, Wang

and Yu

, Community discovery in networks with deep sparse filtering, Pattern Recognition 81 (2018), 50–59. doi: 10.1016/j.patcog.2018.03.026

Jin

, Ge

, Li

, Lu

, He

and Fogelman-Soulie

, Using Deep Learning for Community Discovery in Social Networks, in 2017 IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI) 2017 IEEE, 160–167. doi: https://doi.org/10.1109/ICTAI.2017.00035

Al-Andoli

, Cheah

W.P.

and Tan

S.C.

, Deep learning-based community detection in complex networks with network partitioning and reduction of trainable parameters, Journal of Ambient Intelligence and Humanized Computing (2020), 1–19. doi: https://doi.org/10.1007/s12652-020-02389-x

10.

Pinaya

W.H.L.

, Vieira

, Garcia-Dias

and Mechelli

, Autoencoders, in Machine Learning: Elsevier, (2020), 193–208.

11.

Tian

, Gao

, Cui

, Chen

and Liu

T.-Y.

, Learning deep representations for graph clustering, in Twenty-Eighth AAAI Conference on Artificial Intelligence, (2014).

12.

Yang

, Cao

, He

, Wang

and Zhang

, Modularity Based Community Detection with Deep Learning, in IJCAI 16 (2016), 2252–2258.

13.

Dhilber

and Bhavani

S.D.

, Community Detection in Social Networks Using Deep Learning, in International Conference on Distributed Computing and Internet Technology 2020 Springer, 241–250. doi: https://doi.org/10.1007/978-3-030-36987-3_15

14.

, Chen

and Zheng

, Deep autoencoder-like nonnegative matrix factorization for community detection, in Proceedings of the 27th ACM International Conference on Information and Knowledge Management 2018 ACM, 1393–1402. doi: https://doi.org/10.1145/3269206.3271697

15.

Dauphin

Y.N.

, Pascanu

, Gulcehre

, Cho

, Ganguli

and Bengio

, Identifying and attacking the saddle point problem in high-dimensional non-convex optimization, in Advances in neural information processing systems (2014), 2933–2941.

16.

Fong

, Deb

and Yang

X.-s.

, How metaheuristic algorithms contribute to deep learning in the hype of big data analytics, in Progress in Intelligent Computing Techniques: Theory, Practice, and Applications: Springer, (2018), 3–25.

17.

Šíma

, Back-propagation is not efficient, Neural Networks 9(6) (1996), 1017–1023.

18.

J.-h.

, Xu

W.-h.

and Chen

Y.-t.

, Novel back propagation optimization by cuckoo search algorithm, The Scientific World Journal (2014), 2014.

19.

J.-B.

, Evolutionary manifold regularized stacked denoising autoencoders for gearbox fault diagnosis, Knowledge-Based Systems 178 (2019), 111–122. doi: https://doi.org/10.1016/j.knosys.2019.04.022

20.

Shi

and Eberhart

R.C.

, Empirical study of particle swarm optimization, in Proceedings of the 1999 congress on evolutionary computation-CEC99 (Cat. No. 99TH8406) 3 (1999), IEEE, 1945–1950, doi: https://doi.org/10.1109/CEC.1999.785511

21.

Chiandussi

, Codegone

, Ferrero

and Varesio

F.E.

, Comparison of multi-objective optimization methodologies for engineering applications, Computers & Mathematics with Applications, 63(5) (2012), 912–942.

22.

Rojas-Delgado

, Trujillo-Rasúa

and Bello

, A continuation approach for training Artificial Neural Networks with metaheuristics, Pattern Recognition Letters 125 (2019), 373–380.

23.

Cao

, Jin

and Dang

, Autoencoder Based Community Detection with Adaptive Integration of Network Topology and Node Contents, in International Conference on Knowledge Science Engineering and Management 2018 Springer, 184–196. doi: https://doi.org/10.1007/978-3-319-99247-1_16

24.

Gudise

V.G.

and Venayagamoorthy

G.K.

, Comparison of particle swarm optimization and backpropagation as training algorithms for neural networks, in Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS’03 (Cat. No. 03EX706) 2003 IEEE, 110–117. doi: https://doi.org/10.1109/SIS.2003.1202255

25.

Zhang

J.-R.

, Zhang

, Lok

T.-M.

and Lyu

M.R.

, A hybrid particle swarm optimization–back-propagation algorithm for feedforward neural network training, Applied Mathematics and Computation 185(2) (2007), 1026–1037. doi: https://doi.org/10.1016/j.amc.2006.07.025

26.

Mavrovouniotis

and Yang

, Training neural networks with ant colony optimization algorithms for pattern classification, Soft Computing 19(6) (2015), 1511–1522. doi: https://doi.org/10.1007/s00500-014-1334-5

27.

Jaddi

N.S.

, Abdullah

and Hamdan

A.R.

, Optimization of neural network model using modified bat-inspired algorithm, Applied Soft Computing 37 (2015), 71–86. doi: https://doi.org/10.1016/j.asoc.2015.08.002

28.

David

O.E.

and Greental

, Genetic algorithms for evolving deep neural networks, in Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation 2014 ACM, 1451–1452. doi: https://doi.org/10.1145/2598394.2602287

29.

Sui

, Bennamoun

and Togneri

, Deep feature learning for dummies: A simple auto-encoder training method using Particle Swarm Optimisation, Pattern Recognition Letters 94 75–80 (2017). doi: https://doi.org/10.1016/j.patrec.2017.03.021

30.

Mobahi

and Fisher

J.W.

III , A theoretical analysis of optimization by Gaussian continuation, in Twenty-Ninth AAAI Conference on Artificial Intelligence (2015).

31.

Gepperth

, Hammer

, Incremental learning algorithms and applications, European Symposium on Artificial Neural Networks (ESANN), (2016), Bruges, Belgium. ffhal-01418129.

32.

Shi

, Liu

and Liang

, PSO-based community detection in complex networks, in 2009 Second International Symposium on Knowledge Acquisition and Modeling 3 (2009), IEEE, 114–119. doi: https://doi.org/10.1109/KAM.2009.195

33.

Cai

, Shi

, Zhu

, Qiao

and Hu

, An algorithm Q-PSO for community detection in complex networks, in IEEE, 2017 16th International Symposium on Distributed Computing and Applications to Business, Engineering and Science (DCABES) 2017, 76–79. doi: https://doi.org/10.1109/DCABES.2017.23

34.

Gong

, Cai

, Chen

and Ma

, Complex network clustering by multiobjective discrete particle swarm optimization based on decomposition, IEEE Transactions on Evolutionary Computation 18(1) (2013), 82–97. doi: https://doi.org/10.1109/TEVC.2013.2260862

35.

Liu

, Du

, Jiang

and Zeng

, Multiobjective Particle Swarm Optimization Based on Network Embedding for Complex Network Community Detection, IEEE Transactions on Computational Social Systems 7(2) (2020), 437–449. doi: https://doi.org/10.1109/TCSS.2020.2964027

36.

SpolaúR

, Cherman

E.A.

, Monard

M.C.

and Lee

H.D.

, A comparison of multi-label feature selection methods using the problem transformation approach, Electronic Notes in Theoretical Computer Science 292 (2013), 135–151.

37.

Bhatia

and Rani

, DFuzzy: a deep learning-based fuzzy clustering model for large graphs, Knowledge and Information Systems 57(1) (2018), 159–181. doi: 10.1007/s10115-018-1156-3

38.

Sperlí

, A deep learning based community detection approach, in Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing 2019 ACM, 1107–1110.

39.

Wang

, Zhang

and Chai

, Deep Auto-encoded Clustering Algorithm for Community Detection in Complex Networks, Chinese Journal of Electronics 28(3) (2019), 489–496.

40.

Batagelj

and Zaversnik

, An O (m) algorithm for cores decomposition of networks, arXiv preprint cs/0310049, (2003).

41.

Miranda

, PySwarms: a research toolkit for particle swarm optimization in Python, Journal of Open Source Software 3(21) (2018), 433–https://doi.org/10.21105/joss.00433

42.

Zachary

W.W.

, An information flow model for conflict and fission in small groups, Journal of Anthropological Research 33(4) (1977), 452–473. doi: https://doi.org/10.1086/jar.33.4.3629752

43.

Lusseau

, Schneider

, Boisseau

O.J.

, Haase

, Slooten

and Dawson

S.M.

, The bottlenose dolphin community of Doubtful Sound features a large proportion of long-lasting associations, Behavioral Ecology and Sociobiology 54(4) (2003), 396–405. doi: https://doi.org/10.1007/s00265-003-0651-y

44.

Xie

, Kelley

and Szymanski

B.K.

, Overlapping community detection in networks: The state-of-the-art and comparative study, Acm Computing Surveys (csur) 45(4) (2013), 1–35. doi: https://doi.org/10.1145/2501654.2501657

45.

Girvan

and Newman

M.E.

, Community structure in social and biological networks, Proceedings of the National Academy of Sciences 99(12) (2002), 7821–7826. doi: https://doi.org/10.1073/pnas.122653799

46.

Newman

M.E.

, Modularity and community structure in networks, Proceedings of the national academy of sciences 103(23) (2006), 8577–8582. doi: https://doi.org/10.1073/pnas.0601602103

47.

Adamic

L.A.

and Glance

, The political blogosphere and the 2004 US election: divided they blog, in Proceedings of the 3rd international workshop on Link discovery, (2005), 36–43. doi: https://doi.org/10.1145/1134271.1134277. [Online]. Available: https://dl.acm.org/doi/abs/10.1145/1134271.1134277

48.

Yang

, Jin

, Chi

and Zhu

, Combining link and content for community detection: a discriminative approach, in Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, (2009), 927–936. doi: https://doi.org/10.1145/1557019.1557120

49.

Leskovec

and Mcauley

J.J.

, Learning to discover social circles in ego networks, in Advances in neural information processing systems, (2012), 539–547.

50.

Rozemberczki

, Allen

and Sarkar

, Multi-scale Attributed Node Embedding, arXiv preprint arXiv:1909. 13021, (2019).

51.

Danon

, Diaz-Guilera

, Duch

and Arenas

, Comparing community structure identification, Journal of Statistical Mechanics: Theory and Experiment 2005(09) (2005), P09008.

52.

and Gorton

, Performance evaluation for parallel systems: A survey. Citeseer, (1997).