Maximum entropy discrimination (MED) is a general framework for discriminative estimation which integrates the principles of maximum entropy and maximum margin. In this paper, we propose a novel approach named multi-kernel MED (MKMED) for multi-view learning (MVL), which takes advantage of the complementary principle for MVL. Multiple kernels encode the similarities in different views. We obtain a kernel matrix by multiple kernel combination to make use of the complementary information in different views. Based on the kernel matrix obtained by multiple kernel combination, we can proceed MVL within the MED framework. The experimental results on multiple datasets demonstrate the effectiveness of the proposed MKMED. MKMED outperforms the single-view MEDs and a competing MVL mothod named SVM-2K, and is competitive with the state-of-the-art multi-view MED (MVMED) and even sometimes exceeds it.
BlumA. and MitchellT., Combining labeled and unlabeled data with co-training, in: Proceedings of the 11th Annual Conference on Computational Learning Theory (1998), 92-100.
2.
HardoonD.R., SzedmakS. and Shawe-TaylorJ., Canonical correlation analysis: An overview with application to learning methods, Neural Computation16 (2004), 2639-2664.
3.
McFeeB. and LanckrietG., Learning multi-modal similarity, Journal of Machine Learning Research12 (2011), 491-523.
4.
XieP. and XingE.P., Multi-modal distance metric learning, in: Proceedings of the 23th International Joint Conference on Artificial Intelligence (2013), 1806-1812.
5.
NigamK. and GhaniR., Analyzing the effectiveness and applicability of co-training, in: Proceedings of the 9th International Conference on Information and Knowledge Management (2000), 86-93.
6.
SunS., A survey of multi-view machine learning, Neural Computing and Applications23 (2013), 2031-2038.
7.
XuC., TaoD. and XuC., A survey on multi-view learning, CoRR abs/1304.5634, 2013.
8.
SindhwaniV., NiyogiP. and BelkinM., A co-regularization approach to semi-supervised learning with multiple views, in: Proceedings of the 22th International Conference on Machine Learning Workshop on Learning with Multiple Views (2005), 74-79.
9.
WangW. and ZhouZ.H., A new analysis of co-training, in: Proceedings of the 27th International Conference on Machine Learning (2010), 1135-1142.
10.
YuS., KrishnapuramB., RosalesR. and RaoR.B., Bayesian co-training, Journal of Machine Learning Research12 (2011), 2649-2680.
11.
ChenN., ZhuJ., SunF. and XingE.P., Large-margin predictive latent subspace learning for multiview data analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence34 (2012), 2365-2378.
12.
HotellingH., Relations between two sets of variants, Biometrika28 (1936), 321-377.
13.
WhiteM., ZhangX., SchuurmansD. and YuY., Convex multi-view subspace learning, Advances in Neural Information Processing Systems25 (2012), 1682-1690.
14.
LewisD.P., JebaraT. and NobleW.S., Support vector machine learning from heterogeneous data: An empirical analysis using protein sequence and structure, Bioinformatics22 (2006), 2753-2760.
15.
JaakkolaT., MeilaM. and JebaraT., Maximum entropy discrimination, Advances in Neural Information Processing Systems12 (1999), 470-476.
16.
JebaraT. and JaakkolaT., Feature selection and dualities in maximum entropy discrimination, in: Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence (2000), 291-300.
17.
JebaraT., Multi-task feature and kernel selection for SVMs, in: Proceedings of the 21st International Conference on Machine Learning (2004), 55-62.
18.
JebaraT., Multitask sparsity via maximum entropy discrimination, Journal of Machine Learning Research12 (2011), 75-110.
19.
LongP.M. and WuX., Mistake bounds for maximum entropy discrimination, Advances in Neural Information Processing Systems17 (2004), 833-840.
20.
ZhuJ. and XingE.P., Maximum entropy discrimination Markov networks, Journal of Machine Learning Research10 (2009), 2531-2569.
21.
ZhuJ., XingE.P. and ZhangB., Laplace maximum margin Markov networks, in: Proceedings of the 25th International Conference on Machine Learning (2008), 1256-1263.
22.
SunS. and ChaoG., Multi-view maximum entropy discrimination, in: Proceedings of the 23th International Joint Conference on Artificial Intelligence (2013), 1706-1712.
23.
LanckrietG.R.G., CristianiniN., BartlettP., GhaouiL.E. and JordanM.I., Learing the kernel matrix with semidefinite programming, Journal of Machine Learning Research5 (2004), 27-72.
24.
MaoQ., TangI.W., GaoS. and WangL., Generalized multiple kernel learning with data-dependent priors, IEEE Transactions on Neural Networks and Learning Systems (2014). DOI: 10.1109/TNNLS.2014.2334137.
25.
SonnenburgS., RätschG. and SchäferC., A general and efficient multiple kernel learning algorithm, Advances in Neural Information Processing Systems19 (2006), 1273-1280.
26.
SubrahmanyaN. and ShinY.C., Sparse multiple kernel learning for signal processing applications, IEEE Transactions on Pattern Analysis and Machine Intelligence32 (2010), 788-798.
27.
FarquharJ., HardoonD., MengH., Shawe-TaylorJ. and SzedmakS., Two view learning: SVM-2K, theory and practice, Advances in Neural Information Processing Systems18 (2005), 355-362.
28.
LiJ. and SunS., Nonlinear combination of multiple kernels for support vector machines, in: Proceedings of the 20th International Conference on Pattern Recognition (2010), 2889-2892.
29.
DasguptaS., LittmanM.L. and McAllesterD., Pac generalization bounds for co-training, Advances in Neural Information Processing Systems25 (2012), 375-382.
