Volume 29 Issue 6
Dec.  2020
Turn off MathJax
Article Contents
WU Wei, JING Xiaoyuan, DU Wencai. The Kernel Dynamics of Convolutional Neural Networks in Manifolds[J]. Chinese Journal of Electronics, 2020, 29(6): 1185-1192. doi: 10.1049/cje.2020.10.004
Citation: WU Wei, JING Xiaoyuan, DU Wencai. The Kernel Dynamics of Convolutional Neural Networks in Manifolds[J]. Chinese Journal of Electronics, 2020, 29(6): 1185-1192. doi: 10.1049/cje.2020.10.004

The Kernel Dynamics of Convolutional Neural Networks in Manifolds

doi: 10.1049/cje.2020.10.004
Funds:  This work was supported by the National Natural Science Foundation of China (No.61933013, No.U1736211), the Strategic Priority Research Program of the Chinese Academy of Sciences (No.XDA22030301), the Foundation of Macau (No.MF1809, No.MF1713).
More Information
  • Corresponding author: JING Xiaoyuan (corresponding author) received the Ph.D. degree in Pattern Recognition and Intelligent System from the Nanjing University of Science and Technology, in 1998. He was a Professor with the Department of Computer, Shenzhen Research Student School, Harbin Institute of Technology, 2005. Now he is a Professor with the School of Computer Science, Wuhan University, China. His research interests include pattern recognition, machine learning, image processing, artificial intelligence, and software engineering. He has published more than 100 papers, such as TIP, TIFS, TSE, TCB, CVPR, AAAI, IJCAI, ICSE and PR. (Email:jingxy_2000@126.com)
  • Received Date: 2019-07-23
  • Publish Date: 2020-12-25
  • We propose a novel expression from manifolds to define Convolutional neural network (CNN). The layered structure is proceeded by integration in limited space continuously, with weights adjusted including value and direction in neural manifolds. Status transfer functions are proposed to simulate the kernel dynamics as a control matrix. We theoretically analyze the stability and controllability of kernel-based CNNs, and verify our findings by numerical experiments.
  • loading
  • F. Chollet. "Xception:Deep learning with depthwise separable convolutions", IEEE Conference on Computer Vision & Pattern Recognition, pp.1-8, 2016.
    C. Szegedy, V. Vanhoucke, S. Ioffe, et al. "Rethinking the inception architecture for computer vision", IEEE Conference on Computer Vision & Pattern Recognition, pp.2818-2826, 2016.
    J. Yosinski, J. Clune, A. Nguyen, et al. "Understanding neural networks through deep visualization", 31st International Conference on Machine Learning, pp. 1-15, 2015.
    Z. Lian, X. Jing, et al. "DropConnect regularization method with sparsity constraint for neural networks", Chinese Journal of Electronics, Vol.25, No.1, pp.152-158, 2016.
    Y. Zha, M. Wu, T. Ku, et al. "Deep Tracking Algorithm Research Based on Location-Sensitive Model", Acta Electronica Sinica, Vol.47, No.10, pp.2076-2082, 2019.
    W. Guo, H. Wei, J. Zhao, et al. "Numerical analysis near singularities in rbf networks", The Journal of Machine Learning Research, Vol.19, No.1, pp.1-39, 2018.
    F. Cousseau, T. Ozeki, and S. Amari. "Dynamics of learning in multilayer perceptrons near singularities", IEEE Transactions on Neural Networks, Vol.19, No.8, pp.1313-1328, 2008.
    S. Amari, H. Park, and T. Ozeki. "Singularities affect dynamics of learning in neuromanifolds", Neural Computation, Vol.18, No.5, pp.1007-1065, 2006.
    R. Vidal, J. Bruna, R. Giryes, and S. Soatto. "Mathematics of deep learning", in arXiv:1712.04741, pp.1-10, 2017.
    C. Pratik, O. Adam, O. Stanley, et al."Deep relaxation:partial differential equations for optimizing deep neural networks", Research in the Mathematical Sciences, Vol.5, No.3, pp.30, 2018.
    L. Fan, W. Zuo, Y. Wang, X. Wang. "Research on Recommender System Model Based on Differential Privacy and Time Series", Acta Electronica Sinica, Vol.45, No.9, pp.2057-2064, 2017.
    X. Zheng, X. Sun, J. Lu, Z. Xian, L. Li. "Action Recognition Based on Deep Learning and Artificial Intelligence Planning", Acta Electronica Sinica, Vol.47, No.8, pp.1661-1668, 2019.
    A.M. Saxe, J.L. McClelland, and S. Ganguli. "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks", in arXiv:1312.6120, pp.1-22, 2013.
    T. H. Lee, H. M. Trinh, and J. H. Park. "Stability analysis of neural networks with time-varying delay by constructing novel lyapunov functionals", IEEE Transactions on Neural Networks & Learning Systems, Vol.29, No.9, pp.4238-4247, 2017.
    Y. Zhang, J. Zheng, et al. "A Text Sentiment Classification Modeling Method Based on Coordinated CNN-LSTM-Attention Model", Chinese Journal of Electronics, Vol.28, No.1, pp.124-130, 2019.
    H. Sun, Z. Zhang, L. Peng, and X. Duan. "An elementary Introduction to Information Geometry", Science Press, Beijing, China, 2016.
    J. Park and I. W. Sandberg. "Universal approximation using radial-basis-function networks", Neural Computation, Vol.3, No.2, pp.246-257, 1991.
    S.J. Pan and Q. Yang. "A survey on transfer learning", IEEE Transactions on Knowledge & Data Engineering, Vol.22, No.10, pp.1345-1359, 2010.
    X. Jing, X. Zhu, F. Wu, et al. "Super-resolution person re-identification with semi-coupled low-rank discriminant dictionary learning", IEEE Transactions on Image Processing, Vol.26, No.3, pp.1363-1378, 2017.
    S. Amari. "Natural gradient works efficiently in learning", Neural Computation, Vol.10, pp.251-276, 1998.
    H. Wei, K. Zhang, F. Cousseau, and S. Amari. "Dynamics of learning near singularities in layered networks", Neural Computation, Vol.20, No.3, pp.813-843, 2014.
    S. Sabour, N. Frosst, and Geoffrey E. Hinton. "Dynamic routing between capsules", 31st Conference on Neural Information Processing Systems, pp.1-11, 2017.
    B. Li. "Parametric definition and production of directional convolution kernel", Chinese Journal of Computers, Vol.11, pp.701-704, 1988.
  • 加载中


    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Article Metrics

    Article views (159) PDF downloads(56) Cited by()
    Proportional views


    DownLoad:  Full-Size Img  PowerPoint