Deep multiple non-negative matrix factorization for multi-view clustering

Abstract

Multi-view clustering aims to group similar samples into the same clusters and dissimilar samples into different clusters by integrating heterogeneous information from multi-view data. Non-negative matrix factorization (NMF) has been widely applied to multi-view clustering owing to its interpretability. However, most NMF-based algorithms only factorize multi-view data based on the shallow structure, neglecting complex hierarchical and heterogeneous information in multi-view data. In this paper, we propose a deep multiple non-negative matrix factorization (DMNMF) framework based on AutoEncoder for multi-view clustering. DMNMF consists of multiple Encoder Components and Decoder Components with deep structures. Each pair of Encoder Component and Decoder Component are used to hierarchically factorize the input data from a view for capturing the hierarchical information, and all Encoder and Decoder Components are integrated into an abstract level to learn a common low-dimensional representation for combining the heterogeneous information across multi-view data. Furthermore, graph regularizers are also introduced to preserve the local geometric information of each view. To optimize the proposed framework, an iterative updating scheme is developed. Besides, the corresponding algorithm called MVC-DMNMF is also proposed and implemented. Extensive experiments on six benchmark datasets have been conducted, and the experimental results demonstrate the superior performance of our proposed MVC-DMNMF for multi-view clustering compared to other baseline algorithms.

Keywords

Deep non-negative matrix factorization multi-view clustering deep AutoEncoder graph regularization constraints

1. Introduction

Multi-view data collected from different views or multiple data sources are prevalent in many real-life applications [1]. Different views generally contain different features, and they complement each other to construct more comprehensive information for samples. Therefore, it is more reasonable to integrate the features of multiple views together to enhance the performance of learning tasks (such as clustering and classification) rather than only depending on a single view. Multi-view learning has been explored in different fields, such as social multimedia [18], computer vision [4, 25], and bioinformatics informatics [20].

As one of the primary research directions of multi-view learning, multi-view clustering attracts more and more attention because it is capable of analyzing a large amount of unlabeled multi-view data. Multi-view clustering is devoted to grouping samples into the different clusters by exploring heterogeneous information in multi-view data. However, multi-view clustering is generally faced with the following two challenges. On the one hand, data of different views are often heterogeneous, such as the Fourier feature and texture feature describing an image. The heterogeneity requires multi-view clustering algorithms not only to learn consensus information but also to capture complementary information amongst different views; On the other hand, there may be a large amount of hierarchical information between multi-view data and the obtained low-dimensional representation. Taking face datasets as an example, face recognition depends not solely on the appearance difference of face but also on some other attributes such as facial posture and facial expression. The traditional one level clustering algorithms transform the complex hierarchical information into a single layer mapping so that they cannot obtain the desired clustering results.

In order to overcome the problem of data heterogeneity, various multi-view clustering algorithms [12, 14, 21, 24, 25, 30, 31] have also been developed in recent years. Amongst them, multi-view clustering algorithms based on NMF [13, 14, 25, 30, 31] have been proposed owing to its interpretability and representation ability. Given a non-negative data matrix $X\in\Re_{+}^{m\times n}$ for $n$ samples and $m$ features, NMF-based clustering methods attempt to seek two low-rank non-negative matrices $U\in\Re_{+}^{m\times d}$ and $V\in\Re_{+}^{d\times n}$ so that their product can adequately approximate the input data matrix, i.e. $X\approx UV$ ( $U/V$ is called the basis/representation matrix), where each column of the representation matrix $V$ is the low-dimensional representation of an input data in the low-dimensional space, and the basis matrix $U$ is the mapping between the original data and the obtained low-dimensional representation. Some multi-view clustering algorithms [25, 31] apply NMF to each view and then force the representation matrix learned from each view to be consensus with a global representation matrix, others learn a fixed common latent subspace via a shared low-dimensional representation matrix [15]. However, these algorithms use the shallow matrix factorization framework to learn multiple single layer mappings between multi-view data and the low-dimensional representation directly, thus ignoring the complex hierarchical information. For the purpose of capturing the hierarchical information, several deep matrix factorization algorithms have been proposed [22, 28], which use deep factorization structures to further factorize the basis matrix $U$ , such as $X\approx U_{1}U_{2}U_{3}\ldots U_{p}V_{p}$ . Although these deep matrix factorization algorithms indeed ameliorate the clustering performance, most of them are only developed for clustering single view data and only consist of the Decoder Component. Sun et al. [19] pointed out that the Decoder-only Component faces two main difficulties. One is that it needs a high-cost auxiliary algorithm to find the corresponding low-dimensional representation of the sample. The other is that Decoder-only Component cannot directly project the input data into the consensus representation.

Recently, deep AutoEncoder [8] has received widespread attention due to its wide application, such as network embedding [7] and pattern recognition [6, 17]. By combining Encoder and Decoder Components, deep AutoEncoder can reduce the gap between the lower-level abstractions and higher-level abstractions of the input data, so that the learned abstract representation can be seen as a more compact representation of the input data. Considering the superior structure of deep AutoEncoder as well as the shortcomings of the existing deep MVC algorithms, it is reasonable to design a multi-view clustering framework based on Encoder and Decoder Components.

Based on the discussion above, in this paper, we propose a new deep multi-view clustering framework based on AutoEncoder [8], named deep multiple non-negative matrix factorization (DMNMF), as shown in Fig. 1. It factorizes multi-view data ( $\{X^{k}\}_{k=1}^{K}$ ) iteratively to extract the abstract level representation $V_{P}$ , employs graph regularizers to capture the local geometrical information and makes the obtained consensus representation matrix smooth through the orthogonal constraint of the base matrices. More specifically, DMNMF framework is composed of multiple Encoder Components and Decoder Components with deep structure. Each pair of Encoder Component and Decoder Component are used to hierarchically factorize the input data from each view to capture the hierarchical information. Specifically, each Encoder Component transforms the input data into the low-dimensional representation, each Decoder Component reconstructs the input data from the low-dimensional representation, and the structure of these two Components is symmetric. Meanwhile, these Encoder and Decoder Components are integrated into an abstract level to learn a common low-dimensional representation, aiming to capture the heterogeneous information across multi-view data.

In addition, the local geometric information within each view is also an essential factor affecting clustering performance. If data samples are similar in the original data space, they are also similar in the new low-dimensional feature space [3]. To preserve the local geometric information of each view, graph regularizers [3] are introduced into our DMNMF framework. Specifically, we encode the geometric information of each view by constructing the corresponding nearest neighbor graph. If data samples are connected in the graph, they are similar in low-dimensional feature space. Furthermore, the symmetry between Encoder Components and Decoder Components naturally imposes the orthogonal constraint on the basis matrix, which further makes the obtained consensus representation matrix smooth.

Based on the proposed DMNMF framework, we propose and implement a multi-view clustering algorithm, and carry it on six real-world datasets to evaluate the clustering performance by comparing with other baseline algorithms.

Figure 1.

The framework of our proposed DMNMF. $X^{k}$ represents the $k$ -th view samples. $U_{i}^{k}$ and $V_{i}^{k}(i<p)$ represents the basic matrix and the representation matrix of the $i$ -th layer factorization at the $k$ -th view, respectively. $V_{i}^{k}$ is also used as the input of the $(i{+}1)$ -th layer NMF. Each blackish green dashed ellipse denotes an Encoder Component in each view, and each purple dashed ellipse denotes a Decoder Component in each view.

The main contributions can be summarized in the following four aspects:

•

A DMNMF framework is proposed to cluster multi-view data. DMNMF framework can integrate heterogeneous information across multi-view data through the common low-dimensional representation and capture hierarchical information contained in multi-view data by using AutoEncoders with deep structures.

•

Graph regularizers are introduced to the DMNMF framework. This enables the learned low-dimensional representation to preserve more comprehensive local geometric information. Meanwhile, the symmetry between Encoder Components and Decoder Components makes the learned representation much smoother.

•

A novel multi-view clustering algorithm is proposed and implemented based on the proposed DMNMF framework. As far as we know, this is the first algorithm to introduce Encoder Components and Decoder Components with the deep structure for multi-view clustering.

•

Extensive experiments have been carried out on six benchmark datasets. The results demonstrate that the proposed algorithm is superior to other multi-view clustering algorithms in terms of the accuracy (ACC) and the normalized mutual information (NMI) metrics.

The rest of the paper is organized as follows. Section 2 offers a brief overview of related work. The details of DMNMF are presented in Section 3. Section 4 provides extensive experiments and results, and in Section 5, conclusions are given.

2. Related work

2.1 Multi-view clustering

In recent years, various multi-view clustering algorithms have been investigated from different aspects and they are roughly classified into six categories. The first category of algorithm is graph-based clustering algorithms [24], which constructed the affinity graph of each view, fused them into a unified graph and used some graph techniques for clustering. The second category of algorithm is spectral clustering-based algorithms [12], which looked for the best graph adjacency matrix first and then used spectral clustering to obtain the cluster result. The third category of algorithm is subspace clustering-based algorithms [9], which projected multi-view data into a common coefficient matrix, and then use spectral clustering or k-means clustering. The fourth category of algorithm is NMF-based algorithms [14, 15, 23, 25, 29, 30, 31], which applies NMF to each view and then fused representation matrices learned from different views to be a common consensus matrix. The fifth category of algorithm is ensemble learning-based algorithms [21], which clustered each view into a Basic Partitions (BPs) and integrated multiple BPs into a consensus one. The sixth category of algorithm is canonical correlation analysis-based algorithms [5], which projected multi-view data into multiple low-dimensional representations and maximized the correlated information among them. The above six categories of algorithms are not strictly exclusive, and some algorithms may be compatible with multiple characteristics at the same time.

Amongst the above six categories algorithms, NMF-based multi-view algorithms have attracted more and more attention due to its powerful interpretability. Liu et al. [14] developed a joint multi-view clustering algorithm based on NMF (MultiNMF). MultiNMF pushed the learned information of each view towards a common consensus representation matrix rather than fixing it. To reveal the geometric structure hidden in multi-view data space, a graph regularized multi-view NMF algorithm [25] was developed by considering the local geometric information of data space of each view. Moreover, a multi-manifold regularized NMF (MMNMF) algorithm [31] was also proposed to cluster multi-view data, which incorporates consensus manifold and consensus coefficient matrix with multi-manifold regularization to respect the local geometric information of the multi-view data space. Wang et al. [23] incorporated the concept factorization to handle the data containing negative. However, these algorithms only preserved the local geometric information of data space, neglecting the local geometric information of feature space. To preserve the local geometric information of the data space and the feature space simultaneously, Luo et al. [15] developed a dual-regularized multi-view NMF algorithm that jointly factorized data matrices of different views into the basis matrices and a common representation matrix (DMvNMF).

It is worth noting that ensemble clustering applies the idea of ensemble learning in supervised learning to multi-view clustering to obtain reasonable clustering performance. Therefore, ensemble clustering combined multiple clustering results obtained independently from single-view data as the final results, ignoring the connection between multiple views. On the contrary, MVC algorithms gradually integrates the information in multi-view data during the process learning.

2.2 Multi-view clustering via deep matrix factorization

Most traditional multi-view clustering algorithms are intrinsically shallow frameworks and cannot analyze intricate natural data well. Self-Organizing Mapping (SOM) [10] is a neural network-based clustering algorithm. It is essentially a two-layer neural network that includes an input layer and an output layer (competition layer), so it cannot capture complex hierarchical information. Meanwhile, SOM is single-view clustering algorithm, failing to capture the connection between views. In recent years, deep learning has achieved great success in many fields due to its powerful feature learning capabilities. Driven by studies in the field of deep learning, many algorithms on deep matrix factorization have also been proposed [22, 28]. Ye et al. [28] proposed a deep NMF algorithm based on AutoEncoder, called DaNMF, for community detection. This algorithm integrates an Encoder Component and a Decoder Component with deep structures, and these two components are mutually constrained and guided in the factorization process. However, this algorithm is also only applicable to the single-view data field, ignoring heterogeneous information in multi-view data. In [30], Zhao et al. proposed a deep multi-view clustering, called DMF-MVC, and tried to capture different levels of attribute information through different layers. Xu et al. [26] developed a semi-supervised deep multi-view factorization algorithm called Deep Multi-View Concept Learning (DMCL), which hierarchically performs non-negative factorization of the data. DMCL not only takes into account part of the label information of the data but also explicitly models the consistent and complementary information among multi-view data. However, these two algorithms are only considered as Decoder Components and ignore the importance of Encoder Components, so that they cannot inherit the ability of representation learning of AutoEncoder. Besides, DMCL requires partial label information, which is not suitable for clustering without label information.

In this paper, compared with traditional MVC algorithms, we capture the hierarchical information of multi-view data in a deep factorization manner. Different from the existing deep MVC algorithms which are only seen as the Decoder Components, our proposed framework jointly learns a more compact representation by integrating Encoder and Decoder Components in a unified framework without known label information, where these two Components are mutually constrained and enhanced in a way of sharing weight to achieve a better optimal solution.

3. Deep multiple NMF for multi-view clustering

In this section, we first give some necessary notations, and then introduce DMNMF framework in detail, and discuss the corresponding iterative update optimization scheme.

3.1 Notations and problem statement

Let $X=\{X^{k}\in\Re_{+}^{m^{k}\times n}\}_{k=1}^{K}$ denotes multi-view data, where $K$ denotes the number of views, $n$ is the number of samples, $X^{k}$ denotes the $k$ -th view of data, and $m^{k}$ is the feature dimensionality in the $k$ -th view. $X_{:j}^{k}$ denotes the $j$ -th sample of the $k$ -th view. Let $J$ be the number of clusters. Throughout this paper, denotes the Frobenius norm of a matrix and $tr(\cdot)$ denotes the trace of a matrix.

Let $\{A^{k}\}_{k=1}^{K}$ be the affinity graphs transformed from $\{X^{k}\}_{k=1}^{K}$ by using the method of [11]. In this study, we jointly and hierarchically factorize each of the affinity graphs $\{A^{k}\}_{k=1}^{K}$ to obtain the final representation. The purpose of doing this lies in that $\{A^{k}\}_{k=1}^{K}$ can describe the relationship between data samples embedded in the non-linear manifold structure.

Definition 1. Multi-view Clustering with hierarchical and heterogeneous information (MVC-HH). MVC-HH aims to group similar samples into the same cluster and dissimilar samples into different clusters by integrating the hierarchical and heterogeneous information in multi-view data and searches for consistent and complementary clusters across different views.

3.2 DMNMF framework

The proposed DMNMF framework is exhibited in Fig. 1. DMNMF framework is composed of multiple Encoder Components and Decoder Components with deep structure. Each pair of Encoder Component and Decoder Component are used to hierarchically factorize the input data from each view to capture the hierarchical information, and all Encoder and Decoder Components are integrated into an abstract level to learn a common low-dimensional representation, aiming to capture the heterogeneous information across multi-view data. In each pair of Decoder Components and Encoder Components, Decoder Component and Encoder Component are symmetrical and share the same basis matrix. At the abstract level, the common low-dimensional representation can be shared and updated simultaneously by Encoder and Decoder Components of different views.

3.2.1 Decoder Components

In DMNMF framework, each of affinity graphs $\{A^{k}\}_{k=1}^{K}$ is decomposed into $p+1$ non-negative matrices in the following form:

$\displaystyle A^{k}\approx U_{1}^{k}U_{2}^{k}\ldots U_{p}^{k}V_{p}$ (1)

Where $V_{p}\in\Re_{+}^{d\times n},U_{1}^{k}\in\Re_{+}^{r_{0}\times r_{1}},\ldots,U_{% i}^{k}\in\Re_{+}^{r_{i-1}\times r_{i}},\ldots,U_{p}^{k}\in\Re_{+}^{r_{p-1}% \times r_{p}}(1\leqslant i\leqslant p)$ , $d$ represents the dimensionality of the final obtained representation matrix and we set $n=r_{0}\geqslant r_{1}\ldots\geqslant r_{p-1}\geqslant r_{p}{=}d$ .

The Eq. (1) takes a hierarchy of $p$ layers of hidden representations of the input data into consideration, which can also be given by the following factorizations:

$\displaystyle V_{p-1}^{k}\approx U_{p}^{k}V_{p}$ $\displaystyle\ldots$ $\displaystyle V_{2}^{k}\approx U_{3}^{k}\cdot\cdot\cdot U_{p}^{k}V_{p}$ (2) $\displaystyle V_{1}^{k}\approx U_{2}^{k}\cdot\cdot\cdot U_{p}^{k}V_{p}$ $\displaystyle A^{k}\approx U_{1}^{k}\cdot\cdot\cdot U_{p}^{k}V_{p}$

By hierarchy factorizations, the samples from the same cluster are forced to be closer layer by layer in the low-dimensional representation space. To learn the basis matrices $\{U_{i}^{k}\}$ , the objective function of Decoder Components is given as follows:

$\displaystyle\min O_{D}=\sum_{k=1}^{K}{\left\|{A^{k}-U_{1}^{k}U_{2}^{k}\ldots U% _{p}^{k}V_{p}}\right\|_{F}^{2}}$ (3) $\displaystyle s.t.\,\,V_{p}\geqslant 0,\{U_{i}^{k}\}\geqslant 0,\forall i=1,2,% \ldots,p$

3.2.2 Encoder Components

Encoder Components in DMNMF framework try to project affinity graphs into the low-dimensional representation space with the help of the basics matrices $\{U_{i}^{k}\}^{T}$ , i.e., $V_{p}\approx\sum_{k=1}^{K}{\left\|{U_{p}^{kT}\ldots U_{2}^{kT}U_{1}^{kT}A^{k}}% \right\|_{F}^{2}}$ . The objective function of the Encoder Components is given as follows:

$\displaystyle\min O_{E}=\sum_{k=1}^{K}{\left\|{V_{p}-U_{p}^{kT}\ldots U_{2}^{% kT}U_{1}^{kT}A^{k}}\right\|_{F}^{2}}$

(4) $\displaystyle s.t.\,\,V_{p}\geqslant 0,\{U_{i}^{k}\}\geqslant 0,\forall i=1,2,% \ldots,p$
3.2.3 Graph regularization

In order to preserve the local geometric information of each view, graph regularizers are introduced to our proposed framework, as follows:

$\displaystyle\lambda O_{\textit{reg}}=\lambda\sum_{k=1}^{K}{tr(V_{p}L^{k}V_{p}% ^{T})}$ (5)

where $\lambda$ is the weight of graph regularizers. $L^{k}$ is the Laplacian matrix of the affinity graph $A^{k}$ . $L^{k}=D^{k}-A^{k}$ ( $D^{k}$ is calculated by $D_{ii}^{k}=\sum_{j}{A_{ij}^{k}})$ .

Equation (5) integrates the local geometric information across different views to make the final common representation matrix more consensus. In the proposed DMNMF framework, $A^{k}\approx U_{1}^{k}U_{2}^{k}\ldots U_{p}^{k}V_{p}$ and $V_{p}\approx U_{p}^{kT}\ldots U_{2}^{kT}U_{1}^{kT}A^{k}$ . Thus we have $V_{p}\approx(U_{1}^{k}U_{2}^{k}\ldots U_{pk})^{T}(U_{1}^{k}U_{2}^{k}\ldots U_{% p}^{k})V_{p}$ , imposing an implicit orthogonal constraint to $\{U_{i}^{k}\}$ , so that rows of $\{U_{i}^{k}\}$ keeps sparse, further making the common representation matrix smooth.

3.2.4 The loss function of DMNMF

Based on the above discussion, by combining Encoder Components, Decoder Components, and the graph regularizers, the unified loss function of DMNMF is given by following Eq. (3.2.4):

$\displaystyle\mathop{\min}\limits_{V_{p},\{U_{i}^{k}\}}O=O_{D}+O_{E}+\lambda O% _{\textit{reg}}$ $\displaystyle=\sum_{k=1}^{K}\left(\left\|{A^{k}-U_{1}^{k}U_{2}^{k}\ldots U_{p}% ^{k}V_{p}}\right\|_{F}^{2}+\left\|{V_{p}-U_{p}^{kT}\ldots U_{2}^{kT}U_{1}^{kT}% A^{k}}\right\|_{F}^{2}+\lambda tr(V_{p}L^{k}V_{p}^{T})\right)$

(6) $\displaystyle=\overbrace{\mathop{\sum}\limits_{k=1}^{K}{\left\|{A^{k}-U_{1}^{k% }U_{2}^{k}\ldots U_{p}^{k}V_{p}}\right\|_{F}^{2}}}^{O_{D}}+\overbrace{\mathop{% \sum}\limits_{k=1}^{K}{\left\|{V_{p}-U_{p}^{kT}\ldots U_{2}^{kT}U_{1}^{kT}A^{k% }}\right\|_{F}^{2}}}^{O_{E}}+\overbrace{\lambda\mathop{\sum}\limits_{k=1}^{K}{% tr(V_{p}L^{k}V_{p}^{T})}}^{\lambda O_{\textit{reg}}}$ $\displaystyle s.t.\,\,V_{p}\geqslant 0,\{U_{i}^{k}\}\geqslant 0,\forall i=1,2,% \ldots,p$
3.3 Updating rules

Equation (3.2.4) is not convex in both $\{U_{i}^{k}\}$ and $V_{p}$ or $\{V_{i}^{k}\}_{i=1}^{p-1}$ . Hence, we develop the alternating multiplicative updating optimization scheme to solve the optimization problem of DMNMF framework, where each time we optimize one group of variables while keeping the other groups fixed. To improve the speed of matrix factorization, we use unsupervised pre-training to initialize the algorithm parameters, similar to pre-straining in deep learning [8]. Specifically, for the $k$ -th view, we first factorize $A^{k}$ as $A^{k}=U_{1}^{k}V_{1}^{k}$ using standard NMF. Then, we treat the learned $V_{1}^{k}$ as the input matrix for 2-th layer and continue to factorize it iteratively until the final layer. For $p$ -th layer, we choose to initialize $V_{p}$ randomly.

3.3.1 Updating the basis matrices $\{U_{i}^{k}\}$

By keeping $V_{p}$ or $\{V_{i}^{k}\}_{i=1}^{p-1}$ fixed, minimization of Eq. (3.2.4) is rewritten as:

$\displaystyle\mathop{\min}\limits_{U_{i}^{k}}O(U_{i}^{k})=\left\|{A^{k}-\Psi_{% i-1}^{k}U_{i}^{k}\Phi_{i+1}^{k}V_{p}}\right\|_{F}^{2}+\left\|{V_{p}-\Phi_{i+1}% ^{kT}U_{i}^{kT}\Psi_{i-1}^{kT}A^{k}}\right\|_{F}^{2}$

(7) $\displaystyle s.t.\,\,U_{i}^{k}\geqslant 0$

where $\Psi_{i-1}^{k}=U_{1}^{k}U_{2}^{k}\ldots U_{{i-1}}^{k}$ and $\Phi_{i+1}^{k}=U_{i+1}^{k}\ldots U_{p}^{k}$ . When $i=1$ , we set $\Psi_{0}^{k}=I$ ( $I$ is the identity matrix). Similarity, when $i=p$ , $\Phi_{p+1}^{k}=I$ .

To optimize Eq. (3.3.1), let $\Theta_{i}^{k}$ be the Lagrangian multiplier to control the non-negative constraint on $U_{i}^{k}$ , deriving the following Lagrangian objective function:

$\displaystyle\mathop{\min}\limits_{U_{i}^{k},\Theta_{i}^{k}}O(U_{i}^{k},\Theta% _{i}^{k})=\left\|{A^{k}-\Psi_{i-1}^{k}U_{i}^{k}\Phi_{i+1}^{k}V_{p}}\right\|_{F% }^{2}+\left\|{V_{p}-\Phi_{i+1}^{kT}U_{i}^{kT}\Psi_{i-1}^{kT}A^{k}}\right\|_{F}% ^{2}-tr(\Theta_{i}^{k}U_{i}^{kT})$ (8) $\displaystyle s.t.\,\,U_{i}^{k}\geqslant 0$

The partial derivatives of $O(U_{i}^{k},\Theta_{i}^{k})$ with respect to $U_{i}^{k}$ are:

$\displaystyle\frac{\partial O(U_{i}^{k},\Theta_{i}^{k})}{\partial U_{i}^{k}}=-% 4\Psi_{i-1}^{kT}A^{k}V_{p}^{kT}\Phi_{i+1}^{kT}+2\Pi_{i}^{k}$ (9)

where $\Pi_{i}^{k}=\Psi_{i-1}^{kT}\Psi_{i-1}^{k}U_{i}\Phi_{i+1}^{k}V_{p}V_{p}^{T}\Phi% _{i+1}^{kT}+\Psi_{i-1}^{kT}A^{k}A^{kT}\Psi_{i-1}^{k}U_{i}\Phi_{i+1}^{k}\Phi_{i% +1}^{kT}$ . Using Karush-Kuhn-Tucker conditions [2] $\Theta_{i}^{k}U_{i}^{kT}=0$ , we get the following equations for $U_{i}^{k}$ :

$\displaystyle({-4}\Psi_{i-1}^{k}A^{k}V_{p}^{kT}\Phi_{i+1}^{kT}+2\Pi_{i}^{k})% \odot U_{i}^{k}=0$ (10)

Where $\odot$ denotes the element-wise product.

Based on Eq. (10), we can get the update rule for $U_{i}^{k}$ shown in Eq. (11):

$\displaystyle U_{i}^{k}\leftarrow U_{i}^{k}\odot\frac{2\Psi_{i-1}^{kT}A^{k}V_{% p}^{kT}\Phi_{i+1}^{kT}}{\Pi_{i}^{k}}$ (11)
3.3.2 Updating the consensus representation $V_{p}$

$V_{p}$ represents the final learned low-dimensional representation of multi-view data samples. When $\{U_{i}^{k}\}$ are fixed and $i=p$ , minimization of Eq. (3.2.4) is reduced to:

$\displaystyle\mathop{\min}\limits_{V_{p}}O(V_{p})=\sum\limits_{k=1}^{K}\left(% \left\|{A^{k}-\Psi_{p}^{k}V_{p}}\right\|_{F}^{2}+\left\|{V_{p}-\Psi_{p}^{kT}A^% {k}}\right\|_{F}^{2}+\lambda tr(V_{p}L^{k}V_{p}^{T})\right)$

(12) $\displaystyle s.t.\,\,V_{p}\geqslant 0$

The steps of optimizing $V_{p}$ are similar to those of optimizing $\{U_{i}^{k}\}$ . So we derive the following updating rule for $V_{p}$ , as follows:

$\displaystyle V_{p}\leftarrow V_{p}\odot\frac{\sum\limits_{k=1}^{K}{(2\Psi_{p}% ^{kT}A^{k}+\lambda V_{p}A)}}{\sum\limits_{k=1}^{K}{(\Psi_{p}^{kT}\Psi_{p}^{k}V% _{p}+V_{p}+\lambda V_{p}D^{k})}}$ (13)
3.3.3 Updating the feature matrix $\{V_{i}^{k}\}_{i=1}^{p-1}$

$V_{i}^{k}$ denotes the factorization result of the $(i-1)$ -th layer of the $k$ -th view. When $\{U_{i}^{k}\}$ are fixed and $i<p$ , minimization of Eq. (3.2.4) is rewritten to a convex optimization problem:

$\displaystyle\mathop{\text{min}}\limits_{V_{i}}O(V_{i})=\sum_{k=1}^{K}\left(% \left\|A^{k}-\Psi_{i}^{k}V_{i}^{k}\right\|_{F}^{2}+\left\|V_{i}^{k}-\Psi_{i}^{% kT}A^{k}\right\|_{F}^{2}\right)$

(14) $\displaystyle s.t.\,\,V_{i}^{k}\geqslant 0$

The steps of optimizing $\{V_{i}^{k}\}_{i=1}^{p-1}$ are similar to those of optimizing $\{U_{i}^{k}\}$ . So the finally updating rule for $V_{i}^{k}$ is formulated as follows:

$\displaystyle V_{i}^{k}\leftarrow V_{i}^{k}\odot\frac{2\Psi_{i}^{kT}A^{k}}{% \Psi_{i}^{kT}\Psi_{i}^{k}V_{i}^{k}+V_{i}^{k}}$ (15)

Hereto, all the updating rules have been derived. Based on the above analysis, we propose a multi-view clustering algorithm based on DMNMF framework, which is summarized in Algorithm 1. After obtaining the final common representation $V_{p}$ , the standard clustering algorithm (i.e., $K$ -means) is utilized to cluster $V_{p}$ to identify the corresponding cluster labels.

Algorithm 1 MVC-DMNMF

Input: Multi-view data $\{X^{k}\}_{k=1}^{K}$ ; the regularization parameter, $\lambda$ ; the size of each layer, layers, max iterator iter.

Output: The cluster labels of $X$

1. Construct the affinity graphs $\{A^{k}\}_{k=1}^{K}$ .

// Pre-training phase:

2. Pre-training $\{U_{i}^{k}\}$ , $\{V_{i}^{k}\}_{i=1}^{p-1}$ , $V_{p}$ of each layer of each view;

// Fine-tuning phase:

3. While convergence is not reached Do

4. For $i=$ 1 to $p$ Do // Starting Update Layers

5. For $k=$ 1 to $K$ Do // Starting Update Each View

6. IF $i$ lower $p$

7. Update $U_{i}^{k}$ according to Eq. (11);

8. Update $V_{i}^{k}$ according to Eq. (15);

9. End

10. IF $i$ equals $p$

11. Update $V_{p}$ according to Eq. (3.3.2);

12. End

13. End

14. End

15. End

16. Clustering the final representation matrix $V_{p}$ using the standard clustering algorithm

17. Return the cluster labels of $X$

3.4 Computational complexity analysis

Algorithm 1 MVC-DMNMF
Input: Multi-view data $\{X^{k}\}_{k=1}^{K}$ ; the regularization parameter, $\lambda$ ; the size of each layer, layers, max iterator iter.
Output: The cluster labels of $X$
1. Construct the affinity graphs $\{A^{k}\}_{k=1}^{K}$ .
// Pre-training phase:
2. Pre-training $\{U_{i}^{k}\}$ , $\{V_{i}^{k}\}_{i=1}^{p-1}$ , $V_{p}$ of each layer of each view;
// Fine-tuning phase:
3. While convergence is not reached Do
4. For $i=$ 1 to $p$ Do // Starting Update Layers
5. For $k=$ 1 to $K$ Do // Starting Update Each View
6. IF $i$ lower $p$
7. Update $U_{i}^{k}$ according to Eq. (11);
8. Update $V_{i}^{k}$ according to Eq. (15);
9. End
10. IF $i$ equals $p$
11. Update $V_{p}$ according to Eq. (3.3.2);
12. End
13. End
14. End
15. End
16. Clustering the final representation matrix $V_{p}$ using the standard clustering algorithm
17. Return the cluster labels of $X$

Algorithm 1 consists of three steps, i.e., constructing affinity graphs step, the pre-training step, and the fine-tuning step. For simplify, we set the input feature dimensionalities for all views to $m$ . In constructing affinity graphs step, the running time for constructing affinity graphs $\{A^{k}\}_{k=1}^{K}$ is $O(Kn^{2}m)$ . In the pre-training step, the standard NMF algorithm is used to train each layer of each view. The computational complexity for the pre-training step is of order $O(\textit{Kpt}_{p}(n^{2}r+nr))$ , where $t_{p}$ is the number of iterations to complete convergence and $r$ is the largest layer size in all layers. The computational complexity for the fine-tuning step is of order $O(\textit{Kpt}_{f}(n^{2}r+nr^{2}+r^{3}))$ , where $t_{f}$ is the number of iterations in the fine-tuning step. In normal circumstances, $r<n$ , thus the complexity is $O(\textit{Kpt}_{f}(n^{2}r+nr^{2}))$ . All in all, the overall time complexity is $O(Kp(t_{f}+t_{p})(n^{2}r+nr^{2})+Kn^{2}m)$ .

4. Experiments and results

In this section, we implement the proposed MVC-DMNMF algorithm by using Matlab language. In order to demonstrate the effectiveness of the proposed algorithm, a series of experiments are conducted on six benchmark datasets, including the comparison of performance between our algorithm and baseline algorithms, the visualization of low-dimensional representation, the investigation of the parameter sensitivity and the convergence of our algorithm. All experiments are run on the platform of Ubuntu Linux 16.04 with Intel(R) Core(TM) i7-7820X CPU at 3.60GHz and 64GB memory size.

4.1 Datasets

Six benchmark datasets include 3Sources,1

¹
wurlhttp://mlg.ucd.ie/datasets/3sources.html.

BBC,2

http://mlg.ucd.ie/datasets/segment.html.

BBCSport

{}^{2}

, NGs,3

http://lig-membres.imag.fr/grimal/data.html.

Wikipedia,4

⁴

http://www.svcl.ucsd.edu/projects/crossmodal/.

and Digits.5

⁵

http://archive.ics.uci.edu/ml/datasets/Multiple+Features.

The essential statistics of them are displayed in Table 1, where # instance, # view, and # cluster represent the number of samples, views, and clusters, respectively. Moreover, the details information of these datasets are described as follows:

Table 1

Statistics of the six datasets

Dataset	#Instance	#View	#Cluster
3Sources	169	3	6
BBC	685	4	5
BBCSport	544	2	5
NGs	500	3	5
Wikipedia	693	2	10
Digits	2000	2	10

3Sources. It is collected from three popular online news sources: BBC, Reuters, and Guardian. From February to April 2009, there were 948 news articles covering 416 different news stories. Of these reports, 169 were reported from three sources. Each story was manually annotated with one of the six topical labels: business, entertainment, health, politics, sport and technology.

BBC dataset (BBC). It consists of 685 documents collected from the BBC news website. Each document was split into four segments and was manually annotated with one of five topical labels.

BBCSport dataset (BBCSport). It consists of 544 documents collected from the BBCSport website. Each document was split into two segments and was manually annotated with one of the five topical labels.

Newsgroup dataset (NGs). It is a subset of the 20 Newsgroup datasets. It consists of newsgroup documents. Each raw document was pre-processed with three different feature extraction methods (giving three views), and was annotated with one of five topical labels.

Wikipedia. It consists of 693 articles collected from Wikipedia’s featured articles collection. We have split them into sections based on section headings, and assign each image to the section in which it was placed by the authors. Then this dataset was pruned to keep only sections that contained a single image and at least 70 words.

UCI Handwritten Digit dataset (Digits). This handwritten digits (0–9) data is from the UCI repository. The dataset consists of 2000 examples, including two views, the first is the 76 Fourier coefficients, and the second view is the 240 pixel averages in 2 $\times$ 3 windows.

4.2 Evaluation metric

The clustering results are evaluated by comparing the obtained cluster labels with the original labels provided by the datasets. Two standard metrics, the accuracy (ACC) and the normalized mutual information metric (NMI) [27], are selected to measure the effectiveness of the proposed algorithm. ACC is used to compute the percentage of agreement between the true labels and the clustering labels, which is defined as:

$\displaystyle\text{ACC}=\frac{\sum_{i=1}^{n}1\left\{{\overline{C_{i}}=C_{i}}% \right\}}{n}$ (16)

where $n$ is the total number of samples; $C_{i}$ and $\overline{C_{i}}$ is the true label and the clustering label of $i$ -th sample, respectively. $1\{x\}$ is the indicator equation, when the result is assigned to be 1 if the predicted result is the same as the true result and 0 otherwise.

The normalized mutual information is employed to measure the similarity of two clusters, which is defined as:

$\displaystyle\text{NMI}=\frac{\sum_{j,\ell}{\frac{n_{j\ell}}{n}\log\frac{nn_{j% \ell}}{n_{j}\hat{n}_{\ell}}}}{\sqrt{\left({\sum_{j}{n_{j}\log\frac{n_{j}}{n}}}% \right)\left({\sum_{\ell}{\hat{n}_{\ell}\log\frac{\hat{n}_{\ell}}{n}}}\right)}}$ (17)

Where $n_{j}$ is the number of data contained in cluster $C_{j}(1\leqslant j\leqslant J)$ , $\hat{n}_{\ell}$ is the number of data belonging to the class ${\cal L}_{\ell}(1\leqslant\ell\leqslant J)$ , and $n_{j\ell}$ denotes the number of data that are in the intersection between cluster $C_{j}$ and ${\cal L}_{\ell}$ .

For these two metrics (ACC and NMI), the larger value indicates better clustering performance.

4.3 Baseline algorithms

We use the following four baseline algorithms:

DaNMF:6

⁶
https://github.com/smartyfh/DANMF.

A single-view clustering method [28]. It dealt with each view respectively, and the best result is reported as the result of DaNMF, denoted as DaNMF-BST. MVCC:7

⁷

https://github.com/cswanghao/Clustering.

A multi-view clustering algorithm based on concept matrix factorization (CF) [23]. It incorporated graph regularizers into CF. In the comparison experiment below, we use the original multi-view data

\{X^{k}\}_{k=1}^{K}

and the affinity graphs

\{A^{k}\}_{k=1}^{K}

as inputs to MVCC algorithm, respectively, and the best results are retained for the final experimental evaluation. GBS-KO:8

⁸

https://github.com/cswanghao/gbs.

A graph-based multi-view clustering algorithm, which can automatically detect the contribution weight of each view and fuse multi-view information into a unified sharing space [24]. It first constructs the affinity graph and then uses it as input for subsequent steps to complete the clustering task. DMF-MVC:9

⁹

https://github.com/hdzhao/DMF_MVC.

A deep Semi-NMF-based multi-view clustering algorithm [30]. It can obtain a common representation from diverse views. Similar to MVCC, we also use the original multi-view data

\{X^{k}\}_{k=1}^{K}

and the affinity graphs

\{A^{k}\}_{k=1}^{K}

as inputs to DMF-MVC algorithm, respectively, and the best results were retained for final experimental evaluation.

The codes of all baseline algorithms are downloaded from the GitHub home page of authors. We adjust the parameters of all baseline algorithms by greedy search to obtain their best clustering performance.

4.4 Clustering performance

Table 2 gives the layer size configuration and the values of the regularization parameter $\lambda$ of MVC-DMNMF. In this experiments, we first investigate the clustering performance of MVC-DMNMF with a single hidden layer. The number of the nodes in the input layer is equal to that of samples in each dataset, for example, “169-6-169” in the first row, “169” indicates the number of nodes in the input layer and output layer, and “6” indicates the number of nodes in the hidden layer. After obtaining the final low-dimensional representation, we choose $K$ -means to get the clustering results. Without loss of generality, for all algorithms, we run each algorithm 20 times and report the mean and the standard deviation of clustering results in Tables 3 and 4. For all datasets, the best clustering results are highlighted in bold.

Table 2
The layer size configuration of MVC-DMNMF and the values of the regularization parameter $\lambda$

Dataset	#Layers configurations	# $\lambda$
3Sources	169-6-169	1
BBC	685-5-685	1
BBCSport	544-5-544	0.1
NGs	500-5-500	0.1
Wikipedia	693-10-693	0.001
Digits	2000-10-2000	0.001

Table 3

ACCs (mean (standard deviation)) of algorithms on six datasets

Method	ACC (%)
	BBC	BBCSport	NGs	3Sources	Wikipedia	Digits
DaNMF-BST	75.87 (5.62)	82.83 (4.68)	59.29 (4.46)	66.01 (4.85)	54.03 (3.69)	78.73 (5.99)
MVCC	84.67 (6.74)	86.10 (3.62)	89.12 (2.81)	76.09 (1.55)	59.97 (0.86)	94.55 (1.61)
GBS-KO	69.34 (0)	80.7 (0)	98.2 (0)	69.23 (0)	44.73 (0)	83.00 (0)
DMF-MVC	52.83 (0)	68.38 (0)	48.82 (0)	54.67 (0.31)	56.80 (0)	73.88 (1.0)
DMNMF	88.32 (0)	92.46 (0)	97.59 (0)	79.85 (0)	60.35 (1.30)	93.65 (0)

Table 4

NMIs (mean (standard deviation)) of algorithms on six datasets

Method	NMI (%)
	BBC	BBCSport	NGs	3Sources	Wikipedia	Digits
DaNMF-BST	55.10 (4.37)	65.98 (3.35)	45.28 (2.25)	57.95 (3.32)	53.53 (1.74)	79.41 (2.86)
MVCC	67.97 (4.93)	76.55 (1.51)	76.60 (2.62)	69.79 (1.64)	51.49 (1.35)	89.98 (1.66)
GBS-KO	56.27 (0)	76.00 (0)	93.92 (0)	62.16 (0)	41.50 (0)	87.65 (0)
DMF-MVC	31.24 (0)	51.04 (0)	20.85 (0)	38.10 (0.81)	54.39 (0)	78.69 (0.32)
DMNMF	74.91 (0)	80.14 (0)	92.21 (0)	74.6 (0)	56.10 (1.94)	87.03 (0)

From Tables 3 and 4, we derive the following observations:

•

ACCs and NMIs of MVCC and MVC-DMNMF are higher than those of DaNMF-BST on all datasets. ACCs and NMIs of GBS-KO are higher than those of DaNMF-BST except for BBC, BBCSport, and Wikipedia datasets. Among four multi-view clustering algorithms, only DMF-MVC has poor performance (its ACCs and NMIs are only higher than those of DaNMF-BST on Wikipedia dataset). The above experimental comparison shows that in most cases, the clustering algorithm that combines multi-view information can obtain excellent clustering performance than only using single-view information.

•

ACCs and NMIs of MVC-DMNMF are higher than those of GBS-KO on all datasets except for NGs dataset. This shows that our algorithm is superior to GBS-KO in the subsequent processing steps of the affinity graph, which proves the effectiveness of our proposed algorithm to a certain extent.

•

Compared with MF-based algorithms MVCC and DMF-MVC, MVC-DMNMF performs better or comparable in most datasets, which proves the importance of introducing the structure of Encoder Component and the necessity of considering matrix factorization of Encoder Component and Decoder Component simultaneously to obtain an effective low-dimensional representation.

Furthermore, in order to verify whether MVC-DMNMF can obtain lower encoder error and reconstruction error, we calculated the encoder error and reconstruction error of DMF-MVC and MVC-DMNMF according to $\sum_{k=1}^{K}{\left\|{V_{p}-U^{k}A^{k}}\right\|_{F}^{2}}$ and $\sum_{k=1}^{K}{\left\|{A^{k}-U^{k}V_{p}}\right\|_{F}^{2}}$ , respectively. The calculated errors are shown in Table 5, and the lowest error is highlighted in bold. We can observe from Table 5 that encoder error of DMF-MVC is larger than its decoder error, and decoder error of MVC-DMNMF is larger than encoder error except for 3Sources dataset, but the sum error of “Encoder+Decoder” of MVC-DMNMF is smaller than that of DMF-MVC on all datasets except for BBC dataset. This indicates that in clustering multi-view data, multiple AutoEncoder structures can obtain a better low-dimensional representation, which verifies the necessity of Encoder Components.

Table 5

Error Comparison on six datasets

Datasets	Method	Encoder	Decoder	Encoder+Decoder
Digits	DMF-MVC	808.33	1.33	809.67
	DMNMF	1.43	7.03	8.45
Wikipedia	DMF-MVC	454.52	0.22	454.75
	DMNMF	2.10	13.27	15.37
BBC	DMF-MVC	116.68	1.33	118.01
	DMNMF	13.87	142.62	156.49
BBCSport	DMF-MVC	92.84	1.22	94.05
	DMNMF	6.64	62.94	69.58
3Sources	DMF-MVC	53.18	8.42	61.60
	DMNMF	44.78	6.70	51.48
NGs	DMF-MVC	102.94	1.05	103.99
	DMNMF	11.05	88.54	99.59

Figure 2.

2D visualization of the representations learned in different layers of DMNMF on BBCSport dataset.

Figure 3.

2D visualization of consensus representations on BBCSport dataset obtained from four compared algorithms: DaNMF, DMF-MVC, MVCC, and GBS-KO.

4.5 Visualization

To show whether MVC-DMNMF can capture complex hierarchical information, we visualize the learned hidden representation (i.e., the feature matrix $\{V_{i}^{k}\}_{i=1}^{p-1}$ and the common representation matrix $V_{p}$ ) by utilizing the standard t-SNE [16] tool. The visualization results on BBCSport10

¹⁰
http://mlg.ucd.ie/datasets/segment.html.

dataset are shown in Fig. 2, where points belonging to the same cluster share the same color. In Fig. 2, (a), (b), and (c) belong to the first view, and (e), (f), and (g) belong to the second view. In (a), (b), (d) and (e) of Fig. 2, it is difficult to distinguish which cluster each sample point belongs to. As the layers go deeper, points belonging to the same cluster gather closer and closer to each other, as shown from (a) to (c) and (e) to (f) in Fig. 2. Moreover, it can be seen from (c) and (f) of Fig. 2 that the outlines of different clusters are becoming clearer and clearer. This demonstrates that the proposed algorithm is capable of uncovering the hierarchical information of the input data in a deep factorization way.

Further, in order to verify that our proposed algorithm can capture more heterogeneous information, the corresponding visualization results on BBCSport dataset are shown in Fig. 3, where the common representations obtained by our proposed algorithm and the other four algorithms are visualized respectively: (a) DaNMF-BST, (b) DMF-MVC, (c) MVCC, (d) GBS-KO and (e) DMNMF. In (a), (b) and (d) of Fig. 3, there are many overlaps between sample points of different colors, and the sample points of the same color appear very scattered, which is difficult to identify which cluster the samples belong to. Although both (c) and (e) show a more compact representation, there are fewer overlaps between sample points of different colors in (e) and there is a clearer clustering outline between the different clusters. In addition, compared with (c) and (f) of Fig. 2 that only utilize single view information, (e) of Fig. 3 has a clearer outline and fewer sample points overlapped with the help of multi-view information. This indicates that MVC-DMNMF can capture more heterogeneous information and preserve the latent geometric structure embedded in multi-view data space.

4.6 Parameter sensitivity

In our proposed MVC-DMNMF, there are three parameters: the graph regularizers constraints parameter $\lambda$ , the number of hidden layers, and the number of nodes in each hidden layer. Next, we will evaluate the effects of three parameters for MVC-DMNMF.

Figure 4 shows the effect of the parameter $\lambda$ on clustering performance, which is turned in the range of {10 ${}^{-3}$ , 10 ${}^{-2}$ , 10 ${}^{-1}$ , 1, 10 ${}^{1}$ , 10 ${}^{2}$ , 10 ${}^{3}$ }. It can be observed from Fig. 4 that the clustering results of MVC-DMNMF on most datasets go down when $\lambda>1$ . For 3Sources and BBC datasets, MVC-DMNMF achieves the best performance when $\lambda$ is 1. For BBCSport and NGs datasets, MVC-DMNMF achieves the best performance when $\lambda$ is 10 ${}^{-1}$ and for Wikipedia and Digits datasets MVC-DMNMF has the best performance when $\lambda$ is 10 ${}^{-3}$ . The results show that in order to obtain an excellent cluster performance, it is crucial to select the appropriate parameters, and graph regularization of data space indeed helps to learn a better common low-dimensional representation.

Figure 4.

Clustering results evaluated by ACC and NMI with the regularized parameter $\lambda$ .

Figure 5 shows the effect of the number of hidden layers on clustering performance, which is turned from 1 to 21. Note that since Encoder Components and Decoder Components are symmetrical, the number of hidden layers of the overall algorithm is increased by two layers when Decoder Components add a hidden layer. The number of the $i$ -th hidden layer nodes of Decoder Components is $i\ast h$ , $h=\lfloor{\textit{sqrt}(n)}\rfloor+1$ ( $\lfloor\cdot\rfloor$ rounds the element to the nearest integer less than or equal to that element. and sqrt is the square root of the element). As we can see from Fig. 5, as the number of hidden layer nodes increases, the clustering results of DMNMF remain in a stable range.

Figure 6 shows how clustering performance changes with the number of hidden layer nodes increasing. This experiment sets up a three hidden layer structure of DMNMF. The number of the first layer nodes of Encoder Components and Decoder Components are $j\ast h$ , $1\leqslant j\leqslant 10$ . As we can see from Fig. 6, as the number of hidden layer nodes increases, the clustering results of MVC-DMNMF also remain in a stable range.

Overall, MVC-DMNMF is less affected by the number of hidden layers and the number of nodes in each hidden layer. This demonstrates the superiority of our proposed algorithm, which can achieve promising performance for multi-view clustering, though, without the very deep layer structures.

Figure 5.

ACCs and NMIs of DMNMF on all datasets with the number of hidden layers.

Figure 6.

ACCs and NMIs of DMNMF on all datasets with the number of hidden layer nodes.

Figure 7.

Convergence curve of algorithms DMNMF.

4.7 Convergence analysis

Figure 5 shows the convergence curve of the proposed algorithm. In each sub-figure, the x-axis and the y-axis denote the iteration number and the corresponding objective function value, respectively. As can be observed from Fig. 7, the objective function value decreases sharply with 50 iterations and then becomes steadily afterward except Digits dataset. For Digits dataset, the convergence of the algorithm is relatively slower compared with other datasets because the number of samples is large. Nevertheless, after 300 iterations, the algorithm converges. This indicates that MVC-DMNMF converges efficiently.

5. Conclusions

Heterogeneity and hierarchicy are two challenges of multi-view clustering. This study has investigated how to consider both heterogeneity and hierarchicy simultaneously in the process of clustering. To this end, we propose the DMNMF framework composed of multiple Encoder Components and Decoder Components with deep structures, where the hierarchical information is captured by the deep structures, and heterogeneous information across multi-view data are combined by an abstract level. We also develop and implement MVC-DMNMF based on DMNMF framework, and carry out a series of experiments on six real-world datasets. The experimental results demonstrate that DMNMF is capable of uncovering the hierarchical information of input data in a deep factorization way and can capture heterogeneous information and preserve the latent geometric structure embedded in multi-view data space. Therefore, MVC-DMNMF can improve clustering performance than single-view, Decoder-only Component, and shallow structure clustering algorithms.

In this paper, the parameter of graph regularizers is set by users, which is a difficult task. So how to set the parameter of graph regularizers automatically is our next work.

Footnotes

Acknowledgments

This work was supported by the National Natural Science Foundation of China (61762090, 61262069, 61472346, and 61662086), The Natural Science Foundation of Yunnan Province (2016FA026, 2015FB114), the Project of Innovative Research Team of Yunnan Province (2018HC019), and Program for Innovation Research Team (in Science and Technology) in University of Yunnan Province (IRTSTYN), the Education Department Foundation of Yunnan Province (2019J0005) and Yunnan University’s Research Innovation Fund for Graduate Students.

References

Blum

and Mitchell

, Combining labeled and unlabeled data with co-training, in: Proceedings of the Eleventh Annual Conference on Computational Learning Theory, Citeseer, 1998, pp. 92–100.

Boyd

and Vandenberghe

, Convex optimization, Cambridge university press, 2004.

Cai

Han

and Huang

T.S.

, Graph regularized nonnegative matrix factorization for data representation, IEEE Transactions on Pattern Analysis and Machine Intelligence 33(8) (2011), 1548–1560.

Chao

and Sun

, Multi-kernel maximum entropy discrimination for multi-view learning, Intelligent Data Analysis 20(3) (2016), 481–493.

Chaudhuri

Kakade

S.M.

Livescu

and Sridharan

, Multi-view clustering via canonical correlation analysis, in: Proceedings of the 26th Annual International Conference on Machine Learning, ACM, 2009, pp. 129–136.

Diasse

and Li

, Multi-view deep unsupervised transfer leaning via joint auto-encoder coupled with dictionary learning, Intelligent Data Analysis 23(3) (2019), 555–571.

Gao

and Huang

, Deep Attributed Network Embedding, in: Proceedings of the 28th International Joint Conference on Artificial Intelligence, 2018, pp. 3364–3370.

Hinton

G.E.

Osindero

and Teh

Y.-W.

, A fast learning algorithm for deep belief nets, Neural Computation 18(7) (2006), 1527–1554.

Huang

Cao

Dai

Chao

and Shi

, Self-weighted multi-view subspace clustering with low rank tensor constraint, Procedia CIRP 83 (2019), 665–669.

10.

Kohonen

Oja

Simula

Visa

and Kangas

, Engineering applications of the self-organizing map, Proceedings of the IEEE 84(10), 1358–1384.

11.

Kuang

Ding

and Park

, Symmetric nonnegative matrix factorization for graph clustering, in: Proceedings of the 2012 SIAM International Conference on Data Mining, SIAM, 2012, pp. 106–117.

12.

Kumar

Rai

and Daume

, Co-regularized multi-view spectral clustering, in: Advances in Neural Information Processing Systems, 2011, pp. 1413–1421.

13.

Lee

D.D.

and Seung

H.S.

, Learning the parts of objects by non-negative matrix factorization, Nature 401(6755) (1999), 788.

14.

Liu

Wang

Gao

and Han

, Multi-view clustering via joint nonnegative matrix factorization, in: Proceedings of the 2013 SIAM International Conference on Data Mining, SIAM, 2013, pp. 252–260.

15.

Luo

Peng

Guan

and Fan

, Dual regularized multi-view non-negative matrix factorization for clustering, Neurocomputing 294 (2018), 1–11.

16.

Maaten

L.v.d.

and Hinton

, Visualizing data using t-SNE, Journal of Machine Learning Research 9(Nov) (2008), 2579–2605.

17.

Park

Hoshi

and Kemp

C.C.

, A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder, IEEE Robotics & Automation Letters 3(3), 1544–1551.

18.

J.-H.

Chang

W.-Y.

and Tseng

V.S.

, Effective social content-based collaborative filtering for music recommendation, Intelligent Data Analysis 21(S1) (2017), S195–S216.

19.

Sun

B.-J.

Shen

Gao

Ouyang

and Cheng

, A non-negative symmetric encoder-decoder approach for community detection, in: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, ACM, 2017, pp. 597–606.

20.

Sun

and Kranzler

H.R.

, Multi-view biclustering for genotype-phenotype association studies of complex diseases, in: 2013 IEEE International Conference on Bioinformatics and Biomedicine, IEEE, 2013, pp. 316–321.

21.

Tao

Liu

Ding

and Fu

, From ensemble clustering to multi-view clustering, in: Proceedings of the 27th International Joint Conference on Artificial Intelligence, 2017, pp. 2843–2849.

22.

Trigeorgis

Bousmalis

Zafeiriou

and Schuller

B.W.

, A deep matrix factorization method for learning attribute representations, IEEE Transactions on Pattern Analysis and Machine Intelligence 39(3) (2017), 417–429.

23.

Wang

Yang

and Li

, Multi-view clustering via concept factorization with local manifold regularization, in: 2016 IEEE 16th International Conference on Data Mining (ICDM), IEEE, 2016, pp. 1245–1250.

24.

Wang

Yang

Liu

and Fujita

, A study of graph-based system for multi-view clustering, Knowledge-Based Systems 163 (2019), 1009–1019.

25.

Wang

Kong

and Zhang

, Feature extraction via multi-view non-negative matrix factorization with local graph regularization, in: Image Processing (ICIP), 2015 IEEE International Conference on, IEEE, 2015, pp. 3500–3504.

26.

Guan

Zhao

Niu

Wang

and Wang

, Deep Multi-View Concept Learning, in: the 28th International Joint Conference on Artificial Intelligence, 2018, pp. 2898–2904.

27.

Liu

and Gong

, Document clustering based on non-negative matrix factorization, in: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval, ACM, 2003, pp. 267–273.

28.

Chen

and Zheng

, Deep Autoencoder-like Nonnegative Matrix Factorization for Community Detection, in: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, ACM, 2018, pp. 1393–1402.

29.

Zhang

Gao

Zhao

Huo

Yin

Liu

and Zheng

, Multi-view clustering based on graph-regularized nonnegative matrix factorization for object recognition, Information Sciences 432 (2018), 463–478.

30.

Zhao

Ding

and Fu

, Multi-view clustering via deep matrix factorization, in: Thirty-First AAAI Conference on Artificial Intelligence, AAAI press, 2017, pp. 2921–2927.

31.

Zong

Zhang

Zhao

and Zhao

, Multi-view clustering via multi-manifold regularized non-negative matrix factorization, Neural Networks 88 (2017), 74–89.

Deep multiple non-negative matrix factorization for multi-view clustering

Abstract

Keywords

1. Introduction

2.1 Multi-view clustering

2.2 Multi-view clustering via deep matrix factorization

3. Deep multiple NMF for multi-view clustering

3.1 Notations and problem statement

3.2 DMNMF framework

3.2.1 Decoder Components

(4) s . t . V p ⩾ 0 , { U i k } ⩾ 0 , ∀ i = 1 , 2 , … , p 3.2.3 Graph regularization

3.3.1 Updating the basis matrices { U i k }

4. Experiments and results

4.1 Datasets

1 wurlhttp://mlg.ucd.ie/datasets/3sources.html.

6 https://github.com/smartyfh/DANMF.

Table 2 The layer size configuration of MVC-DMNMF and the values of the regularization parameter λ

10 http://mlg.ucd.ie/datasets/segment.html.

5. Conclusions

Footnotes

Acknowledgments

References

(4) $\displaystyle s.t.\,\,V_{p}\geqslant 0,\{U_{i}^{k}\}\geqslant 0,\forall i=1,2,% \ldots,p$
3.2.3 Graph regularization

3.3.1 Updating the basis matrices $\{U_{i}^{k}\}$

¹
wurlhttp://mlg.ucd.ie/datasets/3sources.html.

⁶
https://github.com/smartyfh/DANMF.

Table 2
The layer size configuration of MVC-DMNMF and the values of the regularization parameter $\lambda$

¹⁰
http://mlg.ucd.ie/datasets/segment.html.