Abstract
Proximal support vector machine via generalized eigenvalue (GEPSVM) is a recently proposed binary classification technique which aims to seek two nonparallel planes so that each one is closest to one of the two datasets while furthest away from the other. In this paper, we proposed a novel method called Manifold Regularized Proximal Support Vector Machine via Generalized Eigenvalue (MRGEPSVM), which incorporates local geometry information within each class into GEPSVM by regularization technique. Each plane is required to fit each dataset as close as possible and preserve the intrinsic geometric structure of each class via manifold regularization. MRGEPSVM is also extended to the nonlinear case by kernel trick. The effectiveness of the method is demonstrated by tests on some examples as well as on a number of public data sets. These examples show the advantages of the proposed approach in both computation speed and test set correctness.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
C. J. C. Burges, A Tutorial on Support Vector Machines for Pattern Recognition, Data Mining and Knowledge Discovery, 2(2) (1998) 1–43.
R. Tempo, G. Calafiore, and F. Dabbene, Randomized Algorithms for Analysis and Control of Uncertain Systems: With Applications, (Springer-Verlag, London, 2012), pp. 123–134.
O. L. Mangasarian, and E. W. Wild, Multisurface Proximal Support Vector Machine Classification via Generalized Eigenvalues, Pattern Analysis and Machine Intelligence, 28(1) (2006) 69–74.
R. Khemchandani, and S. Chandra, Twin Support Vector Machines for Pattern Classification, Pattern Analysis and Machine Intelligence, 29(5) (2007) 905–910.
Y. Zhang, Z. Dong, S. Wang, et al., Preclinical Diagnosis of Magnetic Resonance (MR) Brain Images via Discrete Wavelet Packet Transform with Tsallis Entropy and Generalized Eigenvalue Proximal Support Vector Machine (GEPSVM), Entropy, 17(4) (2015) 1795–1813.
F. Dufrenois, and J. C. Noyer, Generalized eigenvalue proximal support vector machines for outlier description, in 2015 International Joint Conference on Neural Networks (IJCNN), (Killarney, Ireland, 2015), pp. 1–9.
W. J. Chen, Y. H. Shao, D. K. Xu, et al., Manifold proximal support vector machine for semi-supervised classification, Applied intelligence, 40(4) (2014) 623–638.
J. Wang, Geometric Structure of High-Dimensional Data and Dimensionality Reduction, (Springer-Verlag, Berlin, 2012), pp. 203–220.
Z. Zhang, T. W. S. Chow, and M. Zhao, M-Isomap: Orthogonal Constrained Marginal Isomap for Nonlinear Dimensionality Reduction, IEEE transactions on cybernetics, 43(1) (2013) 180–191.
X. He, P. Niyogi, Locality Preserving Projections, in Advances in Neural Information Processing Systems, Vol. 16 (MIT, 2004), pp. 153–160.
H. T. Chen, H. W. Chang, and T. L. Liu, Local Discriminant Embedding and Its Variants, in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), Vol. 2 (IEEE Computer Society, Washington DC, 2005), pp. 846–853.
C. Chen, L. Zhang, J. Bu, et al., Constrained Laplacian Eigenmap for dimensionality reduction, Neurocomputing, 73(4) (2010) 951–958.
S. Sun, Multi-view Laplacian Support Vector Machines, in Advanced Data Mining and Applications, (Springer-Verlag, Berlin, 2011), pp. 209–222.
C. M. Bishop, Pattern Recognition and Machine Learning, (Springer-Verlag, New York, 2006).
M. Belkin, P. Niyogi, Laplacian Eigenmaps for Dimensionality Reduction and Data Representation, Neural Computation, 15(6) (2003) 1373–1396.
A. N. Tikhonov, and V. Y. Arsenin, Solutions of illposed problems, (VH Winston and Sons, Washington DC, 1977).
X. Yang, S. Chen, B. Chen, et al., Proximal support vector machine using local information, Neurocomputing, 73(1) (2009) 357–365.
Y. H. Shao, C. H. Zhang, X. B. Wang, et al., Improvements on Twin Support Vector Machines, IEEE Transactions on Neural Networks, 22(6) (2011) 962–968.
D. Zhang, F. Wang, C. Zhang, et al., Multi-View Local Learning. in Proceedings of the 23rd AAAI Conference on Artificial Intelligence, Vol. 2 (AAAI, 2008), pp. 752–757.
J. Cheng, Q. Liu, H. Lu, et al., Supervised kernel locality preserving projections for face recognition, Neurocomputing, 67 (2005) 443–449.
Z. Zheng, X. Huang, Z. Chen, et al., Regression analysis of locality preserving projections via sparse penalty, Information Sciences, 303 (2015) 1–14.
M. Wu, and B. Schölkopf, A Local Learning Approach for Clustering, in Advances in Neural Information Processing Systems. Vol. 19 (MIT, 2006), pp. 1529–1536.
M. Wu, K. Yu, S. Yu, et al., Local Learning Projections, in Proceedings of the 24th Annual International Conference on Machine Learning (ICML 2007), (ACM, Corvallis, 2007), pp. 1039–1046.
T. Joachims, Training linear SVMs in linear time, in Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, (ACM, New York, 2006), pp. 217–226.
S. Shalev-Shwartz, Y. Singer, N. Srebro, et al., Pegasos: primal estimated sub-gradient solver for SVM, Mathematical Programming, 127(1) (2011) 3–30.
D. Cai, X. He, and J. Han, Semi-supervised Discriminant Analysis, in 2007 IEEE 11th International Conference on Computer Vision, (IEEE, Rio de Janeiro, 2007), pp. 1–7.
M. Lichman, UCI Machine Learning Repository, (University of California, Irvine, 2013), http://archive.ics.uci.edu/ml.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
About this article
Cite this article
Liang, J., Zhang, Fy., Xiong, Xx. et al. Manifold Regularized Proximal Support Vector Machine via Generalized Eigenvalue. Int J Comput Intell Syst 9, 1041–1054 (2016). https://doi.org/10.1080/18756891.2016.1256570
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1080/18756891.2016.1256570