Hyper-Laplacian regularized multi-view subspace clustering with jointing representation learning and weighted tensor nuclear norm constraint

Abstract

In recent years, tensor-Singular Value Decomposition (t-SVD) based tensor nuclear norm has achieved remarkable progress in multi-view subspace clustering. However, most existing clustering methods still have the following shortcomings: (a) It has no meaning in practical applications for singular values to be treated equally. (b) They often ignore that data samples in the real world usually exist in multiple nonlinear subspaces. In order to solve the above shortcomings, we propose a hyper-Laplacian regularized multi-view subspace clustering model that joints representation learning and weighted tensor nuclear norm constraint, namely JWHMSC. Specifically, in the JWHMSC model, firstly, in order to capture the global structure between different views, the subspace representation matrices of all views are stacked into a low-rank constrained tensor. Secondly, hyper-Laplace graph regularization is adopted to preserve the local geometric structure embedded in the high-dimensional ambient space. Thirdly, considering the prior information of singular values, the weighted tensor nuclear norm (WTNN) based on t-SVD is introduced to treat singular values differently, which makes the JWHMSC more accurately obtain the sample distribution of classification information. Finally, representation learning, WTNN constraint and hyper-Laplacian graph regularization constraint are integrated into a framework to obtain the overall optimal solution of the algorithm. Compared with the state-of-the-art method, the experimental results on eight benchmark datasets show the good performance of the proposed method JWHMSC in multi-view clustering.

Keywords

Multi-view subspace clustering representation learning hyper-Laplacian regularization weighted tensor nuclear norm

1 Introduction

In the era of big data, the speed of data generation and collection is increasing rapidly, and multi-view data is becoming more and more common in practical applications. Compared to traditional data, which describes objects from a single perspective, multi-view data is semantically richer and more useful, but more complex [1]. For instance, the same news can be reported by articles in different news centers. A document can be translated into multiple languages, such as French, German, Japanese, etc. A web page can be described by pictures, text or links. Due to the consistency and complementarity of multi-view data, the description of data samples can be enriched and the learning effect can be improved. Therefore, the analysis and learning of multi-view data have attracted more and more attention.

Clustering is a widely used unsupervised learning method, which uses a certain similarity measure to group a given dataset, so that the similarity of samples in the same clusters is as large as possible, and the similarity of samples in different clusters is as small as possible. Multi-view clustering as an important method of multi-view learning, which aims to make full use of the consistency and complementarity between multiple views of data to obtain more effective and accurate learning results than using only a single-view of data. The existing multi-view clustering methods can be roughly divided into five categories: collaborative training clustering methods [2 –4], graph clustering methods [5 –8], subspace clustering methods [9 –12], multi-kernel learning clustering methods [13, 14], and deep learning clustering methods [15, 16]. This paper mainly studies methods based on subspace clustering.

The purpose of subspace clustering is to learn a robust coefficient matrix or similarity matrix commonly used for spectral clustering. Sparse subspace clustering (SSC) [17] and low-rank subspace clustering (LRR) [18] are two classic subspace clustering methods based on spectral clustering. The former adds a sparsity constraint on the coefficient matrix represented by the sample linearly, while the latter is a low-rank constraint on the coefficient matrix. SSC lacks a global constraint on the coefficient matrix. Although LRR has a low-rank global constraint, the coefficient matrix obtained is often dense, which is not conducive to subspace division. On the basis of LRR and SSC, a large number of multi-view clustering algorithms have been proposed [19 –28]. In order to recover a shared similarity matrix for the final clustering task from multiple views, Xia et al. [19] proposed a robust multi-view spectral clustering (RMSC) method based on low rank and sparse decomposition. Considering that the original features may contain noise, Zhang et al. [20] tried to find latent representation and developed a latent multi-view subspace clustering (LMSC) method. Chen et al. [21] designed a multi-view clustering in latent embedding space (MCLES) which firstly mapped each view data into a common representation matrix, and then learned a global similarity matrix for spectral clustering. By relaxing the constraint of the global similarity matrix, Chen et al. [22] presented a relaxed multi-view clustering in latent embedding space (R-MCLES) wihch avoided the optimization problem of quadratic programming. Hu et al. [23] learned a sparse consensus similarity matrix from multiple views and introduced a multi-view spectral clustering (S-MVSC) method based on sparse graph learning. Considering the different contributions of multiple views, Nie et al. [24] proposed a self-weighted multi-view clustering with multiple graphs (SwMC) method. In order not to introduce additional parameters, Nie et al. [25] developed a parameter-free auto-weighted multiple graph learning (AMGL) method. Subsequently, Nie et al. [26] put forward a multi-view clustering via adaptively weighted procrustes (AWP) method. In order to avoid post-processing and reduce high computational cost, Hu et al. [27] came up with a multi-view spectral clustering via integrating nonnegative embedding and spectral embedding (NESE) method. For an overview of multi-view learning, please refer to [29].

Recently, by stacking the similarity matrices of multiple views into a 3-order tensor, the tensor-based multi-view spectral clustering methods have received widespread attention [30 –35]. To be specific, the high-order correlations between multiple views can be captured by using the low-rank of the tensor. For instance, Zhang et al. [30] first proposed a low-rank tensor constrained multi-view subspace clustering (LT-MSC) method, which regards the similarity matrices constructed by different views through self-representations learning as the lateral slices of tensor. However, the tensor nuclear norm is not a tight convex relaxation of ℓ₁-norm. In order to deal with this problem, Lu et al. [36] developed a tensor nuclear norm based on tensor-singular value decomposition (t-SVD). Subsequently, Xie et al. [31] presented an unifying multi-view self-representations for clustering by tensor multi-rank minimization (t-SVD-MSC), which effectively captures the high-order information embedded in the multi-view data. However, they ignore the local geometric structure embedded in the high-dimensional ambient space. To solve this problem, Xie et al. [32] introduced a hyper-Laplacian regularized multilinear multiview self-representations for clustering (HLR-M²VS). Combining the idea of robust principal component analysis, Wu et al. [33] used a tensor nuclear norm based on t-SVD to preserve the low-rank property of the essential tensor, a new method of essential tensor learning for multi-view spectral clustering (ETLMSC) is proposed. In order to deal with different types of errors, Wang et al. [34] developed a multi-view clustering error robust low-rank tensor approximation (ELRTA) method. Taking into account the limitations of step-by-step learning transition probability matrix, Chen et al. [35] adopted a one-step strategy to directly learn the transition probability matrix under the framework of robust principal component analysis.

Although these tensor-based spectral clustering methods have achieved good performance in multi-view clustering, they still have some limitations. First, they ignore that different singular values have significant difference, which limits their ability and flexibility in dealing with many practical problems. Second, they ignore that the data samples in the real world often exist in multiple nonlinear subspaces, which cannot effectively captures the local geometry structure of the data. In order to overcome the above problems, we proposes a hyper-Laplacian regularized multi-view subspace clustering (JWHMSC) model that joints representation learning and weighted tensor nuclear norm constraint for multi-view clustering. The Fig. 1 shows the framework of the proposed JWHMSC. The main contributions of our proposed model are as follows.

The subspace representation matrices of all views are stacked into a low-rank constraint tensor, which not only captures the global structure of all views, but also exploits the complementary information between different views.

hyper-Laplace graph regularization is adopted to preserve the local geometric structure embedded in the high-dimensional ambient space.

WTNN based on t-SVD is introduced to treat singular values differently, which can make better use of the prior information of singular values. Therefore, the proposed JWHMSC can more accurately reveal the category information of the sample data distribution.

Representation learning, WTNN constraint, and hyper-Laplacian graph regularization constraint are integrated into a framework to obtain the overall optimal solution of the algorithm.

An effective algorithm based on the alternating direction method of multipliers (ADMM) is proposed to solve the JWHMSC model. Experiments on eight benchmark datasets validate our proposed method is better than many state-of-the-art clustering approaches.

Fig. 1

Flowchart of JWHMSC method. (a) Multi-view Data. (b) Subspace Representations. (c) First, the subspace representations ${Z^{(v)}}_{v = 1}^{V}$ of different views are stacked into a tensor Z , and then the tensor Z is rotated to obtain the tensor $\tilde{Z}$ . (d) Jointly optimize the tensor $\tilde{Z}$ by using the t-SVD based WTNN constraint and the local geometry structure constraint of the hyper-Laplacian graph regularization.

It should be pointed out that the most similar method to our proposed JWHMSC is HLR-M²VS [32]. In fact, the model of HLR-M²VS can be regarded as a special case of our proposed model JWHMSC. When the weights of WTNN are all set to 1, the proposed model of JWHMSC degenerates to the model of HLR-M²VS.

The rest of this paper is structured as follows. Section 2 briefly reviews single-view subspace clustering and multi-view subspace clustering. Section 3 introduces the proposed JWHMSC method which provides an optimization solution process, convergence and complexity analysis. The experimental results and analysis are introduced in detail in Section 4. In Section 5, we present the conclusion and future work.

2 Related work

In this part, we briefly review the single-view subspace clustering method, and then introduce the multi-view self-representations subspace clustering method.

2.1 Single-view subspace clustering

Among the single-view subspace clustering methods, the self-representation subspace clustering model is the most typical and important. Self-representations subspace clustering is based on the assumption that any data point in the space can be obtained by a linear combination of other sample points. Commonly used single-view subspace clustering methods can be expressed in the following form

$\begin{matrix} min_{Z, E} Ψ (Z) + λ φ (E) \\ s . t . X = XZ + E, diag (Z) = 0, \end{matrix}$ (1) where $X \in ℝ^{d \times n}$ is the data matrix, n is the number of samples, each column is a sample, and d is the feature dimension. Ψ ( Z ) is the regularization that applies the desired property to the representation matrix $Z \in ℝ^{n \times n}$ , φ ( E ) is used to eliminate noise E , and λ is used to balance the two terms in the formula. In [17 , 37], the main difference between their works lies in the choice of Ψ. For example, the ℓ₁-norm constraint on Z was imposed to capture the local structure of the data in SSC [17]. While the low-rank constraint on Z was imposed to capture the global structure of the data in LLR [18]. The F-norm was adopted to Z and E in the model of least squares regression (LSR) [37]. Although these models have achieved better performance in single-view subspace learning, they may not be able to process the real data and are only applicable to single-view data.

2.2 Multi-view subspace clustering

Since multi-view data can provide more category information, many single-view subspace clustering methods are extended to multi-view subspace clustering [30 –32]. The multi-view subspace clustering model based on self-representation learning has the following general formula

$\begin{matrix} min_{Z^{(v)}, E^{(v)}} \sum_{v = 1}^{V} Ψ (Z^{(v)}) + λ φ (E) \\ s . t . X^{(v)} = X^{(v)} Z^{(v)} + E^{(v)}, diag (Z^{(v)}) = 0, \\ E = [E^{(1)}; \dots; E^{(V)}], (v = 1, \dots, V), \end{matrix}$ (2) where X ^(v), $Z^{(v)} \in ℝ^{n \times n}$ and E ^(v) represent the data matrix, subspace representation matrix and reconstruction error matrix of the v-th view, respectively. V is the number of views. Like Equation (1), the studies in [30 , 39] use different regularizers to obtain different objective functions. For instance, the works in [38, 39] the nuclear norm and ℓ₁-norm are utilized to preserve consistency and diversity. Zhang et al. [30] adopted the nuclear norm of the tensor-unfolding matrix and ℓ_2,1-norm to explore the high-order correlations between different views. However, the aforementioned nuclear norm is not a tight convex relaxation of ℓ₁-norm. In order to deal with this problem, Xie et al. [31] presented a new tensor nuclear norm based on tensor-singular value decomposition (t-SVD), which achieves satisfactory results. Zhang et al. [36] proved that the tensor nuclear norm based on t-SVD is an effective convex relaxation of ℓ₁-norm. Most of these methods perform low-rank constraints on tensors composed of similar matrices to obtain the global structure between different views, but the local geometric information between samples is ignored. Although the local geometric structure of the data sample is considered in the model HLR-M²VS proposed by xie et al. [32], the singular value is treated equally and lacks practical significance. Therefore, it may not be able to accurately reveal the category information of the sample.

3 The proposed algorithm

In this section, some notations and peliminaries will be provided, then we introduce the JWHMSC model, and give the solution process of the model. Finally, the convergence and complexity of the model are analyzed.

3.1 Notations and peliminaries

For a 3-order tensor $A \in ℝ^{n_{1} \times n_{2} \times n_{3}}$ , $\bar{A} = fft (A, [], 3)$ denotes the fast Fourier transformation (FFT) of a tensor A along the 3rd dimension. In the same way, $A = ifft (\bar{A}, [], 3)$ . Next, we give some important definitions.

Definition 1. [41]. For a 3-order tensor $A \in ℝ^{n_{1} \times n_{2} \times n_{3}}$ , we denote $\bar{A}$ as a block diagonal matrix with each block on diagonal as the frontal slice ${\bar{A}}^{(i)}$ of A . $\bar{A}$ has the following form $\bar{A} = bdiag (\bar{A}) = [\begin{matrix} {\bar{A}}^{(1)} \\ {\bar{A}}^{(2)} \\ ⋱ \\ {\bar{A}}^{(n_{3})} \end{matrix}] .$ (3)

Definition 2. [41]. (t-SVD Based Tensor Nuclear Norm). For a 3-order tensor $A \in ℝ^{n_{1} \times n_{2} \times n_{3}}$ , the t-SVD based tensor nuclear norm is defined as $\begin{matrix} ∥ A ∥_{★} = \sum_{i = 1}^{n_{3}} ∥ {\bar{A}}^{(i)} ∥_{*} \\ = \sum_{i = 1}^{n_{3}} \sum_{j = 1}^{\min (n_{1}, n_{2})} σ_{j} ({\bar{A}}^{(i)}), \end{matrix}$ (4) where $σ_{j} ({\bar{A}}^{(i)})$ denotes the j-th largest singular value of ${\bar{A}}^{(i)}$ .

Most existing t-SVD based tensor low-rank approximation methods [10 , 33] mainly solve the following tensor nuclear norm minimization problem

$\arg min_{X} \frac{1}{2} ∥ X - A ∥_{F}^{2} + α ∥ X ∥_{★} .$ (5)

The optimal solution of Equation (5) can be solved by solving n₃ independent optimization problems. We use b to represent the number of singular values of ${\bar{A}}^{(i)}$ , the i-th (i = 1, 2, ⋯ , n₃) optimization problem is as follows

$\begin{matrix} \arg min_{{\bar{X}}^{(i)}} \frac{1}{2} ∥ {\bar{X}}^{(i)} - {\bar{A}}^{(i)} ∥_{F}^{2} \\ + \sum_{j = 1}^{b} α * σ_{j} ({\bar{X}}^{(i)}) . \end{matrix}$ (6)

The most low-rank matrix ${\bar{X}}^{(i) *}$ in Equation (6) can well be recovered by soft-thresholding of singular values of ${\bar{A}}^{(i)}$ with the same parameter α. In practical applications, this is unreasonable, because different singular values may have different importance and they should be treated differently. As shown in literature [9], the weighted tensor nuclear norm minimization (WTNNM) method is proposed as follows

$\arg min_{X} \frac{1}{2} ∥ X - A ∥_{F}^{2} + α ∥ X ∥_{ω, ★},$ (7) where ∥ · ∥ _{ω
,★} represents the weighted tensor nuclear norm, which is defined

$\begin{matrix} ∥ X ∥_{ω, ★} = \sum_{i = 1}^{n_{3}} ∥ {\bar{X}}^{(i)} ∥_{ω, ★} \\ = \sum_{i = 1}^{n_{3}} \sum_{j = 1}^{\min (n_{1}, n_{2})} ω_{j} * σ_{j} ({\bar{X}}^{(i)}), \end{matrix}$ (8) where $σ_{j} ({\bar{X}}^{(i)})$ represents the j-th largest sinular value of ${\bar{X}}^{(i)}$ , ω_j represents the j element of vector ω .

3.2 Objective function

In multi-view self-representation learning, the weighted tensor nuclear norm is used to perform low-rank constraints on the tensor composed of representation coefficients, which can not only obtain the complementary information between different views, but also describe the global structure of the sample data. At the same time, the hpyer-Laplace [40] constraint is adopted to constrain the self-representation coefficients in order to maintain the local geometric structure of the sample data. Thereform, the objective function of our proposed JWHMSC model is shown below

$\begin{matrix} min_{Z^{(v)}, E^{(v)}, S} ∥ E ∥_{2, 1} + λ_{1} \sum_{v = 1}^{V} tr (Z^{(v)} L_{h}^{(v)} {Z^{(v)}}^{T}) \\ + λ_{2} ∥ S ∥_{ω, ★} + \frac{λ_{3}}{2} ∥ S - Z ∥_{F}^{2} \\ s . t . X^{(v)} = X^{(v)} Z^{(v)} + E^{(v)}, v = 1, 2, \dots, V, \\ Z = Φ (Z^{(1)}, Z^{(2)}, \dots, Z^{(V)}), \\ E = [E^{(1)}; E^{(2)}; \dots; E^{(V)}], \end{matrix}$ (9) where Φ (·) is an operator that merges different subspace representation matrix Z ^(v) to a 3-order tensor Z , and then rotates its dimensionality to n × V × n. $L_{h}^{(v)}$ represents the view-specfic hyper-Laplacian matrix which is build on the subspace representation matrix Z ^(v). The hyper-Laplacian regularized term, i.e., $\sum_{v = 1}^{V} tr (Z^{(v)} L_{h}^{(v)} {Z^{(v)}}^{T})$ is constructed based on the assumption that if two data samples are close in the intrinsic geometry of the sample space, their mappings are also close to each other in self-representations sample space. ∥ · ∥ _{ω
,★} denotes the weighted tensor nuclear norm. The item E = [ E ⁽¹⁾ ; ⋯ ; E ^(V)] is constructed by vertically concatenating together along the column of errors corresponding to each view.

3.3 Optimization

We use the Alternating Direction Method of Multipliers (ADMM) [42] to optimize the proposed model, and the constructed augmented Lagrangian function is as follows

$\begin{matrix} L ({Z^{(v)}}_{v = 1}^{V}, {E^{(v)}}_{v = 1}^{V}, S) = ∥ E ∥_{2, 1} \\ + λ_{1} \sum_{v = 1}^{V} tr (Z^{(v)} L_{h}^{(v)} {Z^{(v)}}^{T}) + λ_{2} ∥ S ∥_{ω, ★} \\ + \frac{λ_{3}}{2} ∥ S - Z ∥_{F}^{2} + \sum_{v = 1}^{V} < Y^{(v)}, X^{(v)} \\ - X^{(v)} Z^{(v)} - E^{(v)} > + \frac{μ}{2} \sum_{v = 1}^{V} ∥ X^{(v)} \\ - X^{(v)} Z^{(v)} - E^{(v)} ∥_{F}^{2}, \end{matrix}$ (10) where Y ^(v) is the Lagrangian multiplier of the v-th view, μ is the penalty parameter, and <· , · > represents the inner product of any two matrices.

Fixing E and S , we have $Φ_{(v)}^{- 1} (S) = S^{(v)}, Φ_{(v)}^{- 1} (Z) = Z^{(v)}$ , where Φ (·) ^-1 represents the inverse function of Φ (·), and its subscript (v) represents the extraction of the v-th frontal slice. The solutions of ${Z^{(v)}}_{v = 1}^{V}$ are as follows

$\begin{matrix} min_{Z^{(v)}} λ_{1} tr (Z^{(v)} L_{h}^{(v)} {Z^{(v)}}^{T}) + \frac{λ_{3}}{2} ∥ S^{(v)} - Z^{(v)} ∥_{F}^{2} \\ + < Y^{(v)}, X^{(v)} - X^{(v)} Z^{(v)} - E^{(v)} > + \frac{μ}{2} \\ ∥ X^{(v)} - X^{(v)} Z^{(v)} - E^{(v)} ∥_{F}^{2} . \end{matrix}$ (11)

Equation (11) is a smooth convex problem. Therefore, we take the derivative of the variable Z ^(v) and set the derivative to zero. According to the properties $\frac{\partial X^{T} AX}{\partial X} = (A + A^{T}) X$ and $\frac{\partial β^{T} X}{\partial X} = β$ , the solution process is as follows

$\begin{matrix} (λ_{3} I + μ {X^{(v)}}^{T} X^{(v)}) Z^{(v)} + Z^{(v)} (2 λ_{1} L_{h}^{(v)}) \\ = λ_{3} S^{(v)} + Q^{(v)}, \end{matrix}$ (12) where Q ^(v) = X ^(v)^T Y ^(v) + μ X ^(v)^T X ^(v) - μ X ^(v)^T E ^(v). In literature [43], the above equation is a Sylvester equation, and the classical Bartels-Stewart algorithm can solve it effectively [43]. In our method, we use the lyap function of the Matlab Control Toolbox to solve the problem in Equation (12).

Fixing ${Z^{(v)}}_{v = 1}^{V}$ and S , solving for E , Equation (10) becomes the following form

$\begin{matrix} E^{*} = \arg min_{E} \frac{1}{μ} ∥ E ∥_{2, 1} + \frac{1}{2} ∥ E - (X^{(v)} \\ - X^{(v)} Z^{(v)} + \frac{1}{μ} Y^{(v)}) ∥_{F}^{2} . \end{matrix}$ (13)

Lemma 4.1 in [18] can solve the problem in Equation (13), the introduction of the lemma is as follows.

Lemma 1. [18]. For any given matrix F and parameters δ, β > 0, the optimal solution to the following problem $min_{D} δ ∥ D ∥_{2, 1} + \frac{β}{2} ∥ D - F ∥_{F}^{2}$ (14) is given by $D_{:, j} = {\begin{matrix} \frac{∥ F_{:, j} ∥_{2} - \frac{δ}{β}}{∥ F_{:, j} ∥_{2}} F_{:, j}, & if ∥ F_{:, j} ∥_{2} > \frac{δ}{β} \\ 0, & otherwise . \end{matrix}$ (15)

Here D _:,j and F _:,j represent the j-th column of matrix D and F respectively. Therefore, the optimal solution of Equation (13) is as follows $E_{:, i}^{*} = {\begin{matrix} \frac{∥ H_{:, i} ∥_{2} - \frac{1}{μ}}{∥ H_{:, i} ∥_{2}} H_{:, i}, & if ∥ H_{:, i} ∥_{2} > \frac{1}{μ} \\ 0, & otherwise . \end{matrix}$ (16) where H is constructed by vertically concatenating the matrix $X^{(v)} - X^{(v)} Z^{(v)} + \frac{Y^{(v)}}{μ}$ along the columns. H _:,i represents the i-th column of matrix H = [ H ⁽¹⁾ ; ⋯ ; H ^(V)].

Fixing E and ${Z^{(v)}}_{v = 1}^{V}$ , solving for S , Equation (10) becomes the following form

$\begin{matrix} S^{*} = \arg min_{S} \frac{λ_{2}}{λ_{3}} ∥ S ∥_{ω, ★} \\ + \frac{1}{2} ∥ S - Z ∥_{F}^{2} . \end{matrix}$ (17)

To solve Equation (17), we show the following theorem.

Theorem 1. [9]. Given a 3-order tensor $A \in ℝ^{n_{1} \times n_{2} \times n_{3}}$ , and the t-SVD of A is A = U * J * V ^T. For the weighted tensor nuclear norm minimization problem $X^{*} = \arg min_{X} τ ∥ X ∥_{ω, ★} + \frac{1}{2} ∥ X - A ∥_{F}^{2}$ (18) the optimal solution is $\begin{matrix} X^{*} = {prox}_{τ ∥ \cdot ∥_{ω, ★}} (A) \\ = U * ifft (Γ_{τ ω} (\bar{J})) * V^{T}, \end{matrix}$ (19) where $Γ_{τ ω} (\bar{J})$ is a tensor stacked by $Γ_{τ ω} ({\bar{J}}^{(1)}), Γ_{τ ω} ({\bar{J}}^{(2)}), \dots, Γ_{τ ω} ({\bar{J}}^{(n_{3})})$ . $Γ_{τ ω} ({\bar{J}}^{(j)}) (j = 1, 2, \dots, n_{3})$ is the generalized softthresholding operator with weight vector ω , defined as $\begin{matrix} Γ_{τ ω} ({\bar{J}}^{(j)}) = diag (γ_{1}, γ_{2}, \dots, γ_{\min (n_{1}, n_{2})}), \\ γ_{i} = \max ({\bar{J}}_{ii}^{(j)} - τ ω_{i}, 0), i = 1, 2, \dots, \min (n_{1}, n_{2}) . \end{matrix}$ (20)

From theorem 1, the solution of Equation (17) is as follows

$\begin{matrix} S^{*} = {prox}_{\frac{λ_{2}}{λ_{3}} ∥ \cdot ∥_{ω, ★}} (Z) \\ = U * ifft (Γ_{\frac{λ_{2}}{λ_{3}} ω} (\bar{J})) * V^{T}, \end{matrix}$ (21) where Z = U * J * V ^T is the t-SVD of Z , $\bar{J} = fft (J, [], 3)$ .

In addition, we update the Lagrange multipliers ${Y^{(v)}}_{v = 1}^{V}$ as shown below

$Y^{(v)} = Y^{(v)} + μ (X^{(v)} - X^{(v)} Z^{(v)} - E^{(v)}) .$ (22)

The whole solution process of Equation (9) is shown in Algorithm 1, in which the stopping criterion is given as follows

$\sum_{v = 1}^{V} ∥ X^{(v)} - X^{(v)} Z^{(v)} - E^{(v)} ∥_{\infty} < ε .$ (23)

Here ε is a predefined tolerance, we set its value to 10^-7. Once the convergence condition in Equation (93) is satisfied, we construct the final similarity matrix by unfolding the tensor representation S into a set of view-specific matrices (i.e., ${S^{(v)}}_{v = 1}^{V}$ ). Finally, we apply the spectral clustering algorithm to obtain the final clustering results [44].

Algorithm 1 JWHMSC
1: Input: Multi-view data: X ⁽¹⁾, ⋯ , X ^(V), parameters: λ₁, λ₂, λ₃ > 0.
2: Parameter Setup:μ = 10^-5, ρ = 2, μ_max = 10¹⁰, ε = 10^-7 and the number of k-nearest neighbor k_nn = 5.
3: Initialize: E ^(v) = S ^(v) = Y ^(v) = 0, ∀ v.
4: repeat
5: Construct hyper-Laplacian matrices ${L_{h}^{(v)}}_{v = 1}^{V}$ from ${Z^{(v)}}_{v = 1}^{V}$ .
6: // for Z
7: Update ${Z^{(v)}}_{v = 1}^{V}$ by solving Equation (12).
8: // for E
9: Update E according to Equation (16).
10: Update S by using Equation (21).
11: // for Y
12: Update Y ^(v) by using Equation (22).
13: Update μ by using μ = min (ρμ, μ_max).
14: until the stopping criterion in Equation (23) is met.
15: Obtain similarity matrix $A = \frac{1}{V} \sum_{v = 1}^{V} \| S^{(v)} \| + \| {S^{(v)}}^{T} \|$ .
16: Apply spectral clustering on similarity matrix A .
17: Output: Clustering result C .

3.4 Convergence and complexity analysis

In literature [45], the strong convergence of the ADMM method with two blocks is proved. However, as the number of blocks increases, the convergence property of the ADMM method is difficult to prove. In our model, these blocks $L_{h}^{(v)}$ , Z ^(v), E ^(v) and S are included. In theory, it is difficult to prove the global convergence of JWHMSC. However, we can use Equation (9) to prove that JWHMSC converges to the local optimal solution of the objective function. The main reason is that when fixing other variables and solving one of them, there is closed-form solution and each update of a single variable is convex. In Section 4.6, Fig. 7 shows the convergence curves of our proposed method on all datasets. We can easily find that our proposed JWHMSC method converges very quickly.

For the computation complexity, as shown in Algorithm 1, our model is mainly composed of five stages, which are introduced separately. Define n to represent the number of data samples, V is the number of views. The first process is to solve hyper-Laplacian matrix $L_{h}^{(v)}$ , it costs O (Vn²log (n)) for all the views. The second process is to solve Z ^(v), and the complexity is O (Vn³) for all views. In the third step, the complexity of updating the variable E is O (Vn²) in each iteration. In the fourth step, update the variable S costs O (Vnlog (n)). The last process is to update the Lagrangian multiplier Y ^(v), it costs O (V) for all the views. Therefore, the total computational complexity of our proposed method is O (t (Vn²log (n) + Vn³ + Vn² + Vnlog (n) + V)) + O (n³). In which t is the number of iterations and O (n³) is the cost of spectral clustering.

4 Experiments

In this section, the public standard datasets used in the experiment are first introduced, and then the comparison algorithms and experimental results and analysis are displayed. Finally we represent the parameter setting, convergence analysis, running time, weighted values analysis and the visualization results of some clustering algorithms. All experiments are conducted in Matlab 2019a on the computer with Intel(R) Core(TM) i5-4210U processors, 1.70 GHz CPU and 12 GB of memory.

4.1 Datasets

In our experiments, eight benchmark datasets are used to validate our method. They are one-handred plant species leaves dataset 1 (100leaves), 3 source dataset 2 (3source), BBC dataset 3 (BBC), Handwritten digit dataset 4 (Hdigit), Newsgroups dataset 5 (NGs), BBC-Sport dataset 6 , Caltech101-20 dataset [23], Calte ch101-07 dataset [23] respectively. Their detailed information can be found in literatures [6] and [23]. Table 2 briefly describes the characteristics of these datasets. Some sample images of these datasets are shown in Figs. 2–5.

Table 1
Statistics of different datasets

Datasets Instances Views Clusters Objective View1 View2 View3 View4 View5 View6

100leaves 1600 3 100 Plant 64 64 64 - - -

3source 169 3 6 Text 3560 3631 3068 - - -

BBC 685 4 5 Text 4659 4633 4665 4684 - -

Hdigit 2000 2 10 Digit 784 256 - - - -

NGs 500 3 5 Text 2000 2000 2000 - - -

BBC-Sport 544 2 5 Text 3183 3203 - - - -

Caltech101-20 2386 6 20 Object 48 40 254 1984 512 928

Caltech101-07 1474 6 7 Object 48 40 254 1984 512 928

Datasets	Instances	Views	Clusters	Objective	View1	View2	View3	View4	View5	View6
100leaves	1600	3	100	Plant	64	64	64	-	-	-
3source	169	3	6	Text	3560	3631	3068	-	-	-
BBC	685	4	5	Text	4659	4633	4665	4684	-	-
Hdigit	2000	2	10	Digit	784	256	-	-	-	-
NGs	500	3	5	Text	2000	2000	2000	-	-	-
BBC-Sport	544	2	5	Text	3183	3203	-	-	-	-
Caltech101-20	2386	6	20	Object	48	40	254	1984	512	928
Caltech101-07	1474	6	7	Object	48	40	254	1984	512	928

Table 2

Clustering results of ACC (%) on the tested multi-view datasets

Datasets	SPC_best	S-MVSC	SwMC	AMGL	AWP	GMC	MCGL	NESE	HLR-M²VS	JWHMSC
100leaves	65.3	89.2	57.1	77.7	80.0	82.4	74.4	89.4	88.7	92.4
3source	61.5	71.8	63.3	69.1	65.7	69.2	72.8	59.2	62.8	75.3
BBC	82.9	89.1	47.1	74.2	88.8	69.3	69.1	87.2	89.8	99.4
Hdigit	68.1	99.5	59.6	89.4	71.8	99.4	98.8	98.7	98.1	100.0
NGs	85.4	97.0	93.6	93.2	72.6	98.2	96.6	97.4	99.0	99.2
BBC-Sport	95.4	72.0	72.8	65.8	72.2	73.9	72.6	98.3	99.6	99.8
Caltech101-20	51.4	59.4	44.1	53.1	50.7	45.6	51.8	56.3	59.8	61.2
Caltech101-07	69.0	65.9	71.5	67.0	60.1	69.2	73.7	67.4	67.0	69.3

Fig. 2

The sample image of the 100leaves dataset.

Fig. 3

The sample image of the Hdigit dataset.

Fig. 4

The sample image of the Caltech101-07 dataset.

Fig. 5

The sample image of the BBC-Sport dataset.

4.2 Compared methods

We compare JWHMSC with nine state-of-the-art clustering algorithms, which are SPC_best [44], S-MVSC [23], SwMC [24], AMGL [25], AWP [26], GMC [6], MCGL [46], NESE [27] and HLR-M²VS [32] respectively.

4.3 Evaluation metrics

In our experiments, three evaluation metrics are used to evaluate all algorithms, which are ACC (Accuracy), NMI (Normalized Mutual Information) and ARI (Adjusted Rand Index) respectively. These evaluation metrics have been extensively used in previous studies [6 , 32], and their detailed information can be found in [31]. For each evaluation metric, the larger the value, the better the result.

4.4 Parameter setting

In our proposed JWHMSC method, four parameters are involved, which are λ₁, λ₂, λ₃ and ω_i respectively. Their value ranges are [0.01,0.4], [0.1,1], [0.5,5] and (0,100] respectively. Specifically, λ₁, λ₂, λ₃ and ω are set to 0.07, 0.1, 1 and [5;10;100] respectively on 100leaves dataset. λ₁, λ₂, λ₃ and ω are set to 0.1, 0.1, 1 and [0.5;1;5] respectively on 3source dataset. λ₁, λ₂, λ₃ and ω are set to 0.1, 0.1, 1 and [5;10;100;0.1] respectively on BBC dataset. λ₁, λ₂, λ₃ and ω are set to 0.04, 0.9, 1 and [10;5] respectively on Hdigit dataset. λ₁, λ₂, λ₃ and ω are set to 0.1, 0.1, 1 and [0.5;1;5] respectively on NGs dataset. λ₁, λ₂, λ₃ and ω are set to 0.1, 0.1, 1 and [1;0.1] respectively on BBC-Sport dataset. λ₁, λ₂, λ₃ and ω are set to 0.19, 0.9, 1 and [0.5;1;5;10;0.1;100] respectively on Caltech101-20 dataset. λ₁, λ₂, λ₃ and ω are set to 0.19, 0.9, 1 and [0.5;1;5;10;0.1;100] respectively on Caltech101-07 dataset. Since we also use hypergraph, the construction of the hyperedges involves k-nearest neighbor. Therefore, we study the performance of JWHMSC with various parameter k ranging in {2, 3, 4, 5, 6, 7, 8, 9, 10}, as shown in Fig. 6. It is easy to see that the JWHMSC method we proposed is not sensitive to the value of k on the Caltech101-07, BBC-Sport, BBC, Hdigit and NGs datasets. For convenience, in our experiment, the value of k is simply set to 5. For AMGL, AWP, GMC and NESE, they have no additional parameters. For S-MVSC, SwMC, MCGL and HLR-M²VS, we carefully adjust their parameters to optimize their performance.

4.5 Performance evaluation

Tables 2–4 shows the clustering results of all algorithms on eight datasets. The best clustering results are shown in bold. We report the average of 20 runs of all algorithms. The standard deviations of all algorithms on all datasets are less than 0.1. Due to page limitations, we do not display them. From Tables 2–4, we draw the following observations.

In most cases, the performance of JWHMSC is better than all the compared algorithms, especially on the 100leaves, BBC, Hdigit, NGs and BBC-Sport datasets. Compared with the second-best method, take ACC as an example, our method obtains significant improvement around 3.0%, 2.5%, 9.6%, 0.5%, 1.4% on the 100leaves, 3source, BBC, Hdigit and Caltech101-20 datasets respectively. This may be because our method preserves the local geometric structure embedded in the high-dimensional ambient space through the hyper-Laplacian constraint, and considers the prior information of singular values through the t-SVD based WTNN constraint, which effectively utilizes the complementary information between different views.

Compared with HLR-M²VS, on the 3source dataset, our method indicates a significant increase of 12.5%, 3.9% and 13.6% w.r.t. ACC, NMI and ARI respectively. On the BBC dataset, our method shows 9.6%, 23% and 22.8% of relative improvement w.r.t. ACC, NMI and ARI respectively. This may be because our method introduces a weighted tensor nuclear norm and treats each singular value differently, that is, considering the prior knowledge of singular values. Numerical results show that the clustering performance can be improved by applying different weights to each singular value.

T-SVD based tensor low-rank methods, including HLR-M²VS and our method, have significantly improved the clustering results on BBC, NGs, BBC-Sport, and Caltech101-20 datasets compared with other algorithms. This may be because the tensor low-rank methods consider the high-order correlations embedded in the multi-view data. In addition, the complementary information among different views can be explored more efficiently and thoroughly by the t-SVD based tensor low-rank methods.

Overall, the clustering results of the single-view standard spectral clustering algorithm are inferior to the multi-view clustering algorithms. It shows that by fully fusing multi-view data, the separability of multi-view data can be improved. SwMC algorithm achieves lower performance than the best spectral clustering method in all single-view data. It is probably because the heterogeneous features underlying different views leads to the poor similarity matrices of SwMC.

Table 3
Clustering results of NMI (%) on the tested multi-view datasets

Datasets SPC_best S-MVSC SwMC AMGL AWP GMC MCGL NESE HLR-M²VS JWHMSC

100leaves 82.1 95.2 76.0 89.5 90.5 90.3 85.4 94.2 96.0 97.7

3source 59.4 66.7 44.3 55.3 57.4 54.8 56.2 62.0 59.1 63.0

BBC 60.0 73.1 21.2 53.7 73.8 48.5 46.7 71.8 74.8 97.8

Hdigit 70.6 98.7 65.2 93.4 77.3 98.5 97.2 97.0 96.1 99.9

NGs 65.1 90.7 85.2 90.7 61.8 93.9 89.8 91.5 96.5 97.2

BBC-Sport 85.1 68.9 67.6 58.8 66.0 70.5 70.8 93.9 98.5 99.2

Caltech101-20 51.5 59.4 31.0 53.8 54.3 38.5 43.0 55.1 67.9 69.1

Caltech101-07 52.9 54.2 54.8 52.4 50.8 60.6 54.2 57.1 67.2 68.4

Datasets	SPC_best	S-MVSC	SwMC	AMGL	AWP	GMC	MCGL	NESE	HLR-M²VS	JWHMSC
100leaves	82.1	95.2	76.0	89.5	90.5	90.3	85.4	94.2	96.0	97.7
3source	59.4	66.7	44.3	55.3	57.4	54.8	56.2	62.0	59.1	63.0
BBC	60.0	73.1	21.2	53.7	73.8	48.5	46.7	71.8	74.8	97.8
Hdigit	70.6	98.7	65.2	93.4	77.3	98.5	97.2	97.0	96.1	99.9
NGs	65.1	90.7	85.2	90.7	61.8	93.9	89.8	91.5	96.5	97.2
BBC-Sport	85.1	68.9	67.6	58.8	66.0	70.5	70.8	93.9	98.5	99.2
Caltech101-20	51.5	59.4	31.0	53.8	54.3	38.5	43.0	55.1	67.9	69.1
Caltech101-07	52.9	54.2	54.8	52.4	50.8	60.6	54.2	57.1	67.2	68.4

Table 4

Clustering results of ARI (%) on the tested multi-view datasets

Datasets	SPC_best	S-MVSC	SwMC	AMGL	AWP	GMC	MCGL	NESE	HLR-M²VS	JWHMSC
100leaves	53.2	85.8	33.1	53.7	73.7	49.7	39.0	84.6	86.8	91.7
3source	47.3	54.2	30.5	45.2	44.2	44.3	48.4	48.8	47.4	61.0
BBC	64.0	76.9	13.7	53.6	75.8	47.9	46.0	71.4	76.0	98.8
Hdigit	58.6	98.9	50.7	88.6	66.4	98.7	97.5	97.1	96.0	99.9
NGs	66.4	92.6	86.7	90.2	56.7	95.5	91.7	93.6	97.5	98.0
BBC-Sport	88.2	60.1	57.5	47.6	58.2	60.1	60.4	95.7	99.0	99.5
Caltech101-20	39.2	51.3	11.5	30.1	38.7	12.8	16.1	44.2	46.4	48.0
Caltech101-07	47.3	51.3	46.6	41.4	45.7	59.4	48.0	54.3	57.4	59.7

Table 5

Average running time (in seconds) on all datasets

Datasets	S-MVSC	SwMC	AMGL	AWP	GMC	MCGL	NESE	HLR-M²VS	JWHMSC
100leaves	2.62	109.45	16.09	2.98	7.78	83.62	11.01	204.94	310.50
3source	0.10	0.69	0.20	0.15	0.47	1.76	0.34	6.15	6.41
BBC	0.79	59.49	2.50	0.95	82.85	32.39	2.61	67.86	98.17
Hdigit	3.17	88.13	67.37	2.11	12.50	242.65	5.20	237.17	314.77
NGs	0.23	35.26	1.82	0.33	1.29	7.64	1.17	19.09	20.60
BBC-Sport	0.33	20.41	1.84	0.34	15.68	9.59	1.27	21.38	23.94
Caltech101-20	6.34	527.08	46.26	9.92	34.11	640.81	36.27	1173.34	1583.40
Caltech101-07	2.57	504.02	18.18	3.69	19.02	352.75	14.93	339.13	463.17

4.6 Convergence analysis

Figure 7 shows the empirical convergence analysis of our method on eight benchmark datasets. The vertical axis represents the error defined as $\sum_{v = 1}^{V} ∥ X^{(v)} - X^{(v)} Z^{(v)} - E^{(v)} ∥_{\infty}$ . After 17 iterations, the error yields a stable value, which means that our JWHMSC algorithm can converge within a few iterations.

Fig. 6

Performance of JWHMSC with various parameter k ranging in {2, 3, 4, 5, 6, 7, 8, 9, 10}.

Fig. 7

Convergence analysis on all datasets.

4.7 Running time

Table 5 reports the average running time of our method and all compared methods on eight benchmark datasets. It is easy to see that the multi-view graph clustering methods run faster than the multi-view subspace clustering methods, which include HLR-M²VS and JWHMSC. The graph based method S-MVSC runs the fastest, the main reason is that the S-MVSC method can directly obtain the closed-form solutions without iteration in the process of solving variables. For the multi-view subspace methods, we also find that in most cases, our proposed method is more effective than the competitors, which is also verified in Tables 2–4.

4.8 Weighted values analysis

Figures 8 and 9 show ACC and NMI of our JWHMSC method versus weighted coefficient ω on the BBC and 3source datasets, respectively. It can be seen that the clustering performance of our method is in a state of large fluctuation with the change of different weighted values. When other variables are fixed, our JWHMSC algorithm gets the best performance with weighted coefficient ω = [5 ; 10 ; 100 ; 0.1] on the BBC dataset, and ω = [0.5 ; 1 ;5] on the 3source dataset, respectively. This illustrates the importance of prior knowledge of singular values.

Fig. 8

Analysis of the weighted coefficient ω on the BBC dataset (λ₁ = 0.1, λ₂ = 0.1, λ₃ = 1).

Fig. 9

Analysis of the weighted coefficient ω on the 3source dataset (λ₁ = 0.1, λ₂ = 0.1, λ₃ = 1).

Fig. 10

Visualization of the embedding results on Hdigit dataset.

4.9 Visualization

As shown in Fig. 10(a)–(k), which are obtained by performing t-distributed stochastic neighbor embedding (t-SNE) algorithm on each view, SPC, S-MVSC, SwMC, AMGL, AWP, MCGL, NESE, GMC, HLR-M²VS and JWHMSC. We can see that compared with Fig. 10(a)–(k), our proposed JWHMSC method can effectively group similar samples into the same clusters.

5 Conclusion

In this paper, we put forward a novel hyper-Laplacian regularized multi-view subspace clustering method by jointing representation learning and weighted tensor nuclear norm constraint, which is referred as JWHMSC. To be specific, in our proposed model, the subspace representation matrices of all views are stacked into a low-rank constraint tensor to capture the global structure of all views and exploit the complementary information between different views. In addition, the hyper-Laplace graph regularization is adopted to preserve the local geometric structure embedded in the high-dimensional ambient space. Considering that different singular values have different importance, a WTNN based on t-SVD is introduced to treat singular values differently, which can make better use of the prior information of singular values. Experiments on eight benchmark datasets with different applications validate our proposed method is superior to many state-of-the-art clustering approaches. In future work, we are interested in combining our method with representation learning of deep learning to consider large-scale datasets.

Footnotes

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (No.61866033), the Introduction of Talent Research Project of Northwest Minzu University (No.xbmuyjrc201904), the Gansu Provincial First-class Discipline Program of Northwest Minzu University (No.11080305), the Leading Talent of National Ethnic Affairs Commission (NEAC), the Young Talent of NEAC, and the Innovative Research Team of NEAC (2018) 98.

References

, Lin

, Vasilakos

A.V.

, Wang

, An overview of recent multi-view clustering, Neurocomputing 402 (2020), 148–161.

Kumar

, DaumÃl’.

, A co-training approach for multi-viewspectral clustering, Proceedings of the 28th internationalconference on machine learning (ICML-11) (2011), 393–400.

Kumar

, Rai

, Daume.

, Co-regularized multi-view spectral clustering, Advances in Neural Information Processing Systems 24 (2011), 1413–1421.

, Zhang

, Xia

, Gao

, Low-rank tensor constrained co-regularized multi-view spectral clustering, Neural Networks 132 (2020), 245–252.

Wang

, Yang

, Liu

, Fujita

, A study of graph-based system for multi-view clustering, Knowledge-Based Systems 163 (2019), 1009–1019.

Wang

, Yang

, Liu

, GMC: Graph-based multi-view clustering, IEEE Transactions on Knowledge and Data Engineering 32(6) (2019), 1116–1129.

, Tang

, Liu

, Zheng

, Yue

, Zhang

, Zhu

, Consensus Graph Learning for Multi-view Clustering IEEE Transactions on Multimedia, {PP(99) (2021), 1–1.

Gao

, Wan

, Liang

, Wang

, Liu

, Shao

, Multi-view projected clustering with graph learning, Neural Networks 126 (2020), 335–346.

Gao

, Xia

, Wan

, Xie

, Zhang

, Tensor-SVD based graph learning for multi-view subspace clustering, Proceedings of the AAAI Conference on Artificial Intelligence 34(4) (2020), 3930–3937.

10.

Zhang

G.Y.

, Zhou

Y.R.

, Wang

C.D.

, Huang

, He

X.Y.

, Joint representation learning for multi-view subspace clustering, Expert Systems with Applications {166 (2021), 113913.

11.

Xie

, Liu

, Pan.

, Centralized joint sparse representation for multi-view subspace clustering,&, Fuzzy Systems 39(1) (2020), 1213–1226.

12.

Wang

, Han

, Zhang

, Tao

, Cai

, Exsavi: Excavating both sample-wise and view-wise relationships to boost multi-view subspace clustering, Neurocomputing 418 (2020), 66–78.

13.

Liu

, Wang

, Zhu

, Li

, Zhu

, Liu...

, Yin

, Absent multiple kernel learning algorithms, IEEE Transactions on Pattern Analysis and Machine Intelligence 42(6) (2019), 1303–1316.

14.

Weng

, Zhou

, Chen

, Peng

, Cai

, Enhancing multi-view clustering through common subspace integration by considering both global similarities and local structures, Neurocomputing 378 (2020), 375–386.

15.

Zheng

, Zhu

, Ma

, Li

, Tian

, Multi-view subspace clustering networks with local and global graph information, Neurocomputing 449 (2021), 15–23.

16.

Huang

, Zhou

J.T.

, Zhu

, Zhang

, Lv

, Peng

, Deep Spectral Representation Learning From Multi-View Data, IEEE Transactions on Image Processing 30 (2021), 5352–5362.

17.

Elhamifar

, Vidal.

, Sparse subspace clustering: Algorithm, theory, and applications, IEEE Transactions on Pattern Analysis and Machine Intelligence 35(11) (2013), 2765–2781.

18.

Liu

, Lin

, Yan

, Sun

, Yu

, Ma

, Robust recovery of subspace structures by low-rank representation, IEEE Transactions on Pattern Analysis and Machine Intelligence 35(1) (2012), 171–184.

19.

Xia

, Pan

, Du

, Yin

, Robust multi-view spectral clustering via low-rank and sparse decomposition, Proceedings of the AAAI Conference on Artificial Intelligence 28(1) (2014), 2149–2155.

20.

Zhang

, Hu

, Fu

, Zhu

, Cao

, Latent multi-viewsubspace clustering, Proceedings of the IEEE Conference onComputer Vision and Pattern Recognition (2017), 4279–4287.

21.

Chen

M.S.

, Huang

, Wang

C.D.

, Huang

, Multi-view clustering in latent embedding space, Proceedings of the AAAI Conference on Artificial Intelligence 34(04) (2020), 3513–3520.

22.

Chen

M.S.

, Huang

, Wang

C.D.

, Huang

, Lai

J.H.

, Relaxed multi-view clustering in latent embedding space, Information Fusion 68 (2021), 8–21.

23.

, Nie

, Chang

, Hao

, Wang

, Li

, Multi-view spectral clustering via sparse graph learning, Neurocomputing 384 (2020), 1–10.

24.

Nie

, Li

, Self-weighted Multiview Clustering with Multiple Graphs, IJCAI (2017), 2564–2570.

25.

Nie

, Li

, Parameter-free auto-weighted multiple graph learning: a framework for multiview clustering and semi-supervised classification, IJCAI (2020), 1881–1887.

26.

Nie

, Tian

, Li.

, Multiview clustering via adaptively weighted procrustes,&, Data Mining (2020), 2022–2030.

27.

, Nie

, Wang

, Li

, Multi-view spectral clustering via integrating nonnegative embedding and spectral embedding, Information Fusion 55 (2020), 251–259.

28.

Zhao

, Zong

, Zhang

, Liu

, Yu

, Incomplete multi-view spectral clustering,&, Fuzzy Systems 38(3) (2020), 2991–3001.

29.

Yang

, Wang

, Multi-view clustering: A survey, Big Data Mining and Analytics 1(2) (2018), 83–107.

30.

Zhang

, Fu

, Liu

, Cao

, Low-rank tensor constrained multiview subspace clustering, Proceedings of the IEEE International Conference on Computer Vision (2015), 1582–1590.

31.

Xie

, Tao

, Zhang

, Liu

, Zhang

, Qu

, On unifying multi-view self-representations for clustering by tensor multi-rank minimization, International Journal of Computer Vision 126(11) (2018), 1157–1179.

32.

Xie

, Zhang

, Qu

, Dai

, Tao

, Hyper-Laplacian regularized multilinear multiview self-representations for clustering and semisupervised learning, IEEE Transactions on Cybernetics 50(2) (2018), 572–586.

33.

, Lin

, Zha

, Essential tensor learning for multi-view spectral clustering, IEEE Transactions on Image Processing 28(12) (2019), 5910–5922.

34.

Wang

, Chen

, Jin

, Cen

, Li

, Zhang

, Error-robust low-rank tensor approximation for multi-view clustering, Knowledge-Based Systems 215 (2021), 106745.

35.

Chen

, Xiao

, Hua

, Zhou

, Adaptive transitionprobability matrix learning for multiview spectral clustering, IEEE Transactions on Neural Networks and Learning Systems (2021), 1–15.

36.

Zhang

, Ely

, Aeron

, Hao

, Kilmer

, Novel methods formultilinear data completion and de-noising based on tensor-SVD, Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (2014), 3842–3849.

37.

, Lu, H. Min, Z.Q. Zhao, L. Zhu, D.S. Huang and S. Yan, Robustand efficient subspace segmentation via least squares regression, Springer, Heidelberg, European conference on computer vision (2012), 347–360 Berlin.

38.

Brbić

, Kopriva

, Multi-view low-rank sparse subspace clustering, Pattern Recognition 73 (2018), 247–258.

39.

Zhu

, Lu

, Zhou

, Structured general and specific multi-view subspace clustering, Pattern Recognition 93 (2019), 392–403.

40.

Zhou

, Huang

, SchÃűlkopf

, Learning with hypergraphs: Clustering, classification, and embedding, Advances in Neural Information Processing Systems 19 (2006), 1601–1608.

41.

Kilmer

M.E.

, Martin

C.D.

, Factorization strategies for third-order tensors, Linear Algebra and its Applications 435(3) (2011), 641–658.

42.

Boyd

, Parikh

, Chu

, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends in Machine Learning 3(1) (2011), 1–122.

43.

Bartels

R.H.

, Stewart

G.W.

, Solution of the matrix equation AX+ XB= C [F4][J], Communications of the ACM 15(9) (1972), 820–826.

44.

A.Y.

, Jordan

M.I.

, Weiss

, On spectral clustering: Analysis and an algorithm, Advances in Neural Information Processing Systems (2002), 849–856.

45.

Lin

, Chen

, Ma

, The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices, arXiv preprint arXiv:, (2010).

46.

Zhan

, Zhang

, Guan

, Wang

, Graph learning for multiview clustering, IEEE Transactions on Cybernetics 48(10) (2017), 2887–2895.