The Kernel Dynamics of Convolutional Neural Networks in Manifolds

WU Wei; JING Xiaoyuan; DU Wencai

doi:10.1049/cje.2020.10.004

Volume 29 Issue 6

Dec. 2020

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2020 > 29(6): 1185-1192

WU Wei, JING Xiaoyuan, DU Wencai. The Kernel Dynamics of Convolutional Neural Networks in Manifolds[J]. Chinese Journal of Electronics, 2020, 29(6): 1185-1192. doi: 10.1049/cje.2020.10.004

Citation:

WU Wei, JING Xiaoyuan, DU Wencai. The Kernel Dynamics of Convolutional Neural Networks in Manifolds[J]. Chinese Journal of Electronics, 2020, 29(6): 1185-1192. doi: 10.1049/cje.2020.10.004

WU Wei, JING Xiaoyuan, DU Wencai. The Kernel Dynamics of Convolutional Neural Networks in Manifolds[J]. Chinese Journal of Electronics, 2020, 29(6): 1185-1192. doi: 10.1049/cje.2020.10.004

Citation:

WU Wei, JING Xiaoyuan, DU Wencai. The Kernel Dynamics of Convolutional Neural Networks in Manifolds[J]. Chinese Journal of Electronics, 2020, 29(6): 1185-1192. doi: 10.1049/cje.2020.10.004

PDF( 1046 KB)

The Kernel Dynamics of Convolutional Neural Networks in Manifolds

doi: 10.1049/cje.2020.10.004

1.
School of Computer Science, Wuhan University, Wuhan 430072, China;
2.
Institute of Deep-sea Science and Engineering, Chinese Academy of Sciences, Sanya 572000, China;
3.
Institute of Data Science, City University of Macau, Macau 999078, China

Funds: This work was supported by the National Natural Science Foundation of China (No.61933013, No.U1736211), the Strategic Priority Research Program of the Chinese Academy of Sciences (No.XDA22030301), the Foundation of Macau (No.MF1809, No.MF1713).

More Information

Corresponding author: JING Xiaoyuan (corresponding author) received the Ph.D. degree in Pattern Recognition and Intelligent System from the Nanjing University of Science and Technology, in 1998. He was a Professor with the Department of Computer, Shenzhen Research Student School, Harbin Institute of Technology, 2005. Now he is a Professor with the School of Computer Science, Wuhan University, China. His research interests include pattern recognition, machine learning, image processing, artificial intelligence, and software engineering. He has published more than 100 papers, such as TIP, TIFS, TSE, TCB, CVPR, AAAI, IJCAI, ICSE and PR. (Email:jingxy_2000@126.com)
Received Date: 2019-07-23
Publish Date: 2020-12-25

Abstract

Abstract

We propose a novel expression from manifolds to define Convolutional neural network (CNN). The layered structure is proceeded by integration in limited space continuously, with weights adjusted including value and direction in neural manifolds. Status transfer functions are proposed to simulate the kernel dynamics as a control matrix. We theoretically analyze the stability and controllability of kernel-based CNNs, and verify our findings by numerical experiments.
- Convolutional neural networks,
- Kernelbased convolution,
- Learning dynamics,
- Neural manifolds

FullText(HTML)

References(23)

References

F. Chollet. "Xception:Deep learning with depthwise separable convolutions", IEEE Conference on Computer Vision & Pattern Recognition, pp.1-8, 2016.

C. Szegedy, V. Vanhoucke, S. Ioffe, et al. "Rethinking the inception architecture for computer vision", IEEE Conference on Computer Vision & Pattern Recognition, pp.2818-2826, 2016.

J. Yosinski, J. Clune, A. Nguyen, et al. "Understanding neural networks through deep visualization", 31st International Conference on Machine Learning, pp. 1-15, 2015.

Z. Lian, X. Jing, et al. "DropConnect regularization method with sparsity constraint for neural networks", Chinese Journal of Electronics, Vol.25, No.1, pp.152-158, 2016.

Y. Zha, M. Wu, T. Ku, et al. "Deep Tracking Algorithm Research Based on Location-Sensitive Model", Acta Electronica Sinica, Vol.47, No.10, pp.2076-2082, 2019.

W. Guo, H. Wei, J. Zhao, et al. "Numerical analysis near singularities in rbf networks", The Journal of Machine Learning Research, Vol.19, No.1, pp.1-39, 2018.

F. Cousseau, T. Ozeki, and S. Amari. "Dynamics of learning in multilayer perceptrons near singularities", IEEE Transactions on Neural Networks, Vol.19, No.8, pp.1313-1328, 2008.

S. Amari, H. Park, and T. Ozeki. "Singularities affect dynamics of learning in neuromanifolds", Neural Computation, Vol.18, No.5, pp.1007-1065, 2006.

R. Vidal, J. Bruna, R. Giryes, and S. Soatto. "Mathematics of deep learning", in arXiv:1712.04741, pp.1-10, 2017.

C. Pratik, O. Adam, O. Stanley, et al."Deep relaxation:partial differential equations for optimizing deep neural networks", Research in the Mathematical Sciences, Vol.5, No.3, pp.30, 2018.

L. Fan, W. Zuo, Y. Wang, X. Wang. "Research on Recommender System Model Based on Differential Privacy and Time Series", Acta Electronica Sinica, Vol.45, No.9, pp.2057-2064, 2017.

X. Zheng, X. Sun, J. Lu, Z. Xian, L. Li. "Action Recognition Based on Deep Learning and Artificial Intelligence Planning", Acta Electronica Sinica, Vol.47, No.8, pp.1661-1668, 2019.

A.M. Saxe, J.L. McClelland, and S. Ganguli. "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks", in arXiv:1312.6120, pp.1-22, 2013.

T. H. Lee, H. M. Trinh, and J. H. Park. "Stability analysis of neural networks with time-varying delay by constructing novel lyapunov functionals", IEEE Transactions on Neural Networks & Learning Systems, Vol.29, No.9, pp.4238-4247, 2017.

Y. Zhang, J. Zheng, et al. "A Text Sentiment Classification Modeling Method Based on Coordinated CNN-LSTM-Attention Model", Chinese Journal of Electronics, Vol.28, No.1, pp.124-130, 2019.

H. Sun, Z. Zhang, L. Peng, and X. Duan. "An elementary Introduction to Information Geometry", Science Press, Beijing, China, 2016.

J. Park and I. W. Sandberg. "Universal approximation using radial-basis-function networks", Neural Computation, Vol.3, No.2, pp.246-257, 1991.

S.J. Pan and Q. Yang. "A survey on transfer learning", IEEE Transactions on Knowledge & Data Engineering, Vol.22, No.10, pp.1345-1359, 2010.

X. Jing, X. Zhu, F. Wu, et al. "Super-resolution person re-identification with semi-coupled low-rank discriminant dictionary learning", IEEE Transactions on Image Processing, Vol.26, No.3, pp.1363-1378, 2017.

S. Amari. "Natural gradient works efficiently in learning", Neural Computation, Vol.10, pp.251-276, 1998.

H. Wei, K. Zhang, F. Cousseau, and S. Amari. "Dynamics of learning near singularities in layered networks", Neural Computation, Vol.20, No.3, pp.813-843, 2014.

S. Sabour, N. Frosst, and Geoffrey E. Hinton. "Dynamic routing between capsules", 31st Conference on Neural Information Processing Systems, pp.1-11, 2017.

B. Li. "Parametric definition and production of directional convolution kernel", Chinese Journal of Computers, Vol.11, pp.701-704, 1988.

Relative Articles

Supplements(0)

Cited By

Proportional views