Unsupervised, Supervised and Semi-supervised Dimensionality Reduction by Low-Rank Regression Analysis

TANG Kewei; ZHANG Jun; ZHANG Changsheng; WANG Lijun; ZHAI Yun; JIANG Wei

doi:10.1049/cje.2021.05.002

Volume 30 Issue 4

Jul. 2021

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2021 > 30(4): 603-610

TANG Kewei, ZHANG Jun, ZHANG Changsheng, et al., “Unsupervised, Supervised and Semi-supervised Dimensionality Reduction by Low-Rank Regression Analysis,” Chinese Journal of Electronics, vol. 30, no. 4, pp. 603-610, 2021, doi: 10.1049/cje.2021.05.002

Citation:

TANG Kewei, ZHANG Jun, ZHANG Changsheng, et al., “Unsupervised, Supervised and Semi-supervised Dimensionality Reduction by Low-Rank Regression Analysis,” Chinese Journal of Electronics, vol. 30, no. 4, pp. 603-610, 2021, doi: 10.1049/cje.2021.05.002

Citation:

PDF( 1531 KB)

Unsupervised, Supervised and Semi-supervised Dimensionality Reduction by Low-Rank Regression Analysis

doi: 10.1049/cje.2021.05.002

1. School of Mathematics, Liaoning Normal University, Dalian 116029, China;
2. College of Computer Science and Artificial Intelligence, Wenzhou University, Wenzhou 325035, China;
3. Research Center for Information Science Theory and Methodology, Institute of Scientific and Technical Information of China, Beijing 100038, China;
4. E-Government Research Center, Chinese Academy of Governance, Beijing 100089, China

Funds:

This work is supported by the National Natural Science Foundation of China (No.62076115, No.61702243, No.61771229), Program of Star of Dalian Youth Science and Technology (No.2019RQ033), and the LiaoNing Revitalization Talents Program (No.XLYC1907169).

Received Date: 2019-01-28
Available Online: 2021-07-19
Publish Date: 2021-07-05

Abstract

Abstract

Techniques for dimensionality reduction have attracted much attention in computer vision and pattern recognition. However, for the supervised or unsupervised case, the methods combining regression analysis and spectral graph analysis do not consider the global structure of the subspace; For semi-supervised case, how to use the unlabeled samples more effectively is still an open problem. In this paper, we propose the methods by Low-rank regression analysis (LRRA) to deal with these problems. For supervised or unsupervised dimensionality reduction, combining spectral graph analysis and LRRA can make a global constraint on the subspace. For semi-supervised dimensionality reduction, the proposed method incorporating LRRA can exploit the unlabeled samples more effectively. The experimental results show the effectiveness of our methods.
- Dimensionality reduction,
- Low-rank,
- Supervised,
- Unsupervised,
- Semi-supervised

FullText(HTML)

References(38)

References

X. Li, G. Jia, J. Li, et al. "A face hallucination algorithm via an LLE coefficients prior model", Chinese Journal of Electronics, Vol.27, No.6, pp.1234-1240, 2018.

Z. Wang, J. Zhen, Y. Li, et al., "Multi-feature multimodal biometric recognition based on quaternion locality preserving projection", Chinese Journal of Electronics, Vol.28, No.4, pp.789-796, 2019.

W. Min, J. Liu and S. Zhang, "Sparse weighted canonical correlation analysis ", Chinese Journal of Electronics, Vol.27, No.3, pp.459-466, 2018.

X. Wang, Y. Kong and Y. Cheng, "Dimensionality reduction for hyperspectral data based on sample-dependent repulsion graph regularized auto-encoder", Chinese Journal of Electronics, Vol.26, No.6, pp.1233-1238, 2017.

M. Turk and A. Pentland, "Face recognition using eigenfaces", Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Lahaina, Maui, Hawaii, USA, pp.586-591, 1991.

P.N. Belhumeur, J.P. Hespanha and D.J. Kriegman, "Eigenfaces vs. fisherfaces:Recognition using class specific linear projection", IEEE Transactions on Patten Analysis and Machine Intelligence, Vol.19, No.7, pp.711-720, 1997.

X. He, S. Yan, Y. Hu, et al., "Face recognition using laplacianfaces", IEEE Transactions on Patten Analysis and Machine Intelligence, Vol.27, No.3, pp.328-340, 2005.

H. Li, T. Jiang and K. Zhang, "Efficient and robust feature extraction by maximum margin criterion", Proc. of the Neural Information Processing Systems, Vancouver, British Columbia, Canada, pp.97-104, 2003.

S. Yan, D. Xu, B. Zhang, et al., "Graph embedding and extensions:A general framework for dimensionality reduction", IEEE Transactions on Patten Analysis and Machine Intelligence, Vol.29, No.1, pp.40-51, 2007.

R. Liu, Z. Lin, Z. Su, et al., "Feature extraction by learning lorentzian metric tensor and its extensions", Pattern Recognition, Vol.43, No.10, pp.3298-3306, 2010.

T. Zhang, D. Tao and J. Yang, "Discriminative locality alignment", Proc. of the European Conference on Computer Vision, Marseille, France, pp.725-738, 2008.

D. Cai, X. He and J. Han, "Srda, An efficient algorithm for large-scale discriminant analysis", IEEE Transactions on Knowledge and Data Engineering, Vol.20, No.1, pp.1-12, 2008.

D. Cai, X. He and J. Han, "Sparse projections over graph", Proc. of the Twenty-Third AAAI Conference on Artificial Intelligence, Chicago, Illinois, USA, pp.610-615, 2008.

Y. Deng, Y. Li, Y. Qian, et al., "Visual words assignment via information-theoretic manifold embedding", IEEE Transactions on Cybernetics, Vol.44, No.10, pp.1924-1937, 2014.

M. Sugiyama, T. Ide, S. Nakajima, et al., "Semi-supervised local Fisher discriminant analysis for dimensionality reduction", Machine Learning, Vol.78, No.1-2, pp.35-61, 2010.

Y. Song, F. Nie, C. Zhang, et al., "A unified framework for semi-supervised dimensionality reduction", Pattern Recognition, Vol.41, No.9, pp.2789-2799, 2008.

X. Yang, H. Fu, H. Zha, et al., "Semi-supervised nonlinear dimensionality reduction", Proc. of International Conference on Machine Learning, Pittsburgh, Pennsylvania, USA, pp.25-29, 2006.

D. Zhang, Z. Zhou and S. Chen, "Semi-supervised dimensionality reduction", Proc. of the SIAM International Conference on Data Mining, Minneapolis, MN, USA, pp.629-634, 2007.

D. Cai, X. He and J. Han, "Semi-supervised discriminant analysis", Proc. of IEEE 11th International Conference on Computer Vision, Rio de Janeiro, Brazil, pp.1-7, 2007.

W. Yang, S. Zhang and W. Liang, "A graph based subspace semi-supervised learning framework for dimensionality reduction", Proc. of the European Conference on Computer Vision, Marseille, France, pp.664-677, 2008.

G. Liu, Z. Lin, S. Yan, et al., "Robust recovery of subspace structures by low-rank representation", IEEE Transactions on Patten Analysis and Machine Intelligence, Vol.35, No.1, pp.171-184, 2013.

R. Liu, Z. Lin, F. De la Torre, et al., "Fixed-rank representation for unsupervised visual learning", Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Providence, Rhode Island, USA, pp.598-605, 2012.

K. Tang, R. Liu, Z. Su, et al., "Structure-constrained low-rank representation", IEEE Transactions on Neural Networks and Learning Systems, Vol.25, No.12, pp.2167-2179, 2014.

K. Tang, J. Zhang, Z. Su, et al., "Bayesian low-rank and sparse nonlinear representation for manifold clustering", Neural Processing Letters, Vol.44, No.3, pp.719-733, 2016.

K. Tang, D. Dunson, Z. Su, et al., "Subspace segmentation by dense block and sparse representation", Neural Networks, Vol.75, pp.66-76, 2016.

K. Tang, Z. Su, W. Jiang, et al., "Superpixels for large dataset subspace clustering", Neural Computing and Applications, Vol.31, pp.8727-8736, 2019.

K. Tang, Z. Su, W. Jiang, et al., "Robust subspace learning-based low-rank representation for manifold clustering", Neural Computing and Applications, Vol.31, pp.7921-7933, 2019.

K. Tang, Z. Su, Y. Liu, et al., "Subspace segmentation with a large number of subspaces using infinity norm minimization", Pattern Recognition, Vol.89, pp.45-54, 2019.

V. Chandrasekaran, S. Sanghavi, P. A. Parrilo, et al., "Rank-sparsity incoherence for matrix decomposition", SIAM Journal on Optimization, Vol.21, No.2, pp.572-596, 2011.

E.J. Candés and Y. Plan, "Tight oracle inequalities for low-rank matrix recovery from a minimal number of noisy random measurements", IEEE Transactions on Information Theory, Vol.57, No.4, pp.2342-2359, 2011.

E.J. Candés, X. Li, Y. Ma, et al., "Robust principal component analysis?", Journal of the ACM, Vol.58, No.3, pp.1-11, 2011.

Z. Lin, M. Chen and Y. Ma, "The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices", available at https://arxiv.org/abs/1009.5055, 2019-1-20.

J.F. Cai, E.J. Candés and Z. Shen, "A singular value thresholding algorithm for matrix completion", SIAM Journal on Optimization, Vol.20, No.4, pp.1956-1982, 2010.

P.J. Phillips, P.J. Flynn, W.T. Scruggs, et al., "Overview of the face recognition grand challenge", Proc. of IEEE Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, pp.947-954, 2005.

T. Sim, S. Baker and M. Bsat, "The cmu pose, illumination, and expression database", IEEE Transactions on Patten Analysis and Machine Intelligence, Vol.25, No.12, pp.1615-1618, 2003.

K.C. Lee, J. Ho and D.J. Kriegman, "Acquiring linear subspaces for face recognition under variable lighting", IEEE Transactions on Patten Analysis and Machine Intelligence, Vol.27, No.5, pp.684-698, 2005.

J. Ho, M.H. Yang, J. Lim, et al., "Clustering appearances of objects under varying illumination conditions", Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Madison, WI, USA, pp.11-18, 2003.

B.S. Manjunath and W.Y. Ma, "Texture features for browsing and retrieval of image data", IEEE Transactions on Patten Analysis and Machine Intelligence, Vol.18, No.8, pp.837-842, 1996.

Relative Articles

Supplements(0)

Cited By

Proportional views