MENG Deyu, SUN Lina. Some New Trends of Deep Learning Research[J]. Chinese Journal of Electronics, 2019, 28(6): 1087-1091. doi: 10.1049/cje.2019.07.011
Citation: MENG Deyu, SUN Lina. Some New Trends of Deep Learning Research[J]. Chinese Journal of Electronics, 2019, 28(6): 1087-1091. doi: 10.1049/cje.2019.07.011

Some New Trends of Deep Learning Research

doi: 10.1049/cje.2019.07.011
Funds:  This work is supported by the National Natural Science Foundation of China (No.61661166011, No.11690011, No.61603292, No.61721002, No.U1811461).
  • Received Date: 2019-08-21
  • Rev Recd Date: 2019-08-27
  • Publish Date: 2019-11-10
  • Deep learning has been attracting increasing attention in the recent decade throughout science and engineering due to its wide range of successful applications. In real problems, however, most implementation stages for applying deep learning still require inevitable manual interventions, which naturally conducts difficulty in its availability to general users with less expertise and also deviates from the intelligence of humans. It is thus a challenging while critical issue to enhance the level of automation across all elements of the entire deep learning framework, like input amelioration, model designing and learning, and output adjustment. This paper tries to list several representative issues of this research topic, and briefly describe their recent research progress and some related works proposed along this research line. Some specific challenging problems have also been presented.
  • loading
  • K.M. He, X.Y. Zhang, S.Q. Ren, et al., "Deep residual learning for image recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, USA, pp. 770-778, 2016.
    Y. Sun, Y. Chen, X. Wang, et al., "Deep learning face representation by joint identification-verification", Advances in Neural Information Processing Systems, pp.1988-1996, 2014.
    D. Silver, A. Huang, C.J. Maddison, et al., "Mastering the game of go with deep neural networks and tree search", Nature, Vol.529, No.7587, pp.484-489, 2016.
    H. Sun and Y. Pang, "GlanceNets-efficient convolutional neural networks with adaptive hard example mining", Science China Information Sciences, Vol.61, No.10, pp.109101:1-109101:3, 2018.
    Y.Q. Jia, E. Shelhamer, J. Donahue, et al., "Caffe:Convolutional architecture for fast feature embedding", Proceedings of the 22nd ACM International Conference on Multimedia, pp.675-678, 2014.
    M. Abadi, A. Agarwal, P. Barham, et al., "Tensorflow:Largescale machine learning on heterogeneous distributed systems", arXiv preprint, arXiv:1603.04467, 2016.
    N. Ketkar, "Introduction to pytorch", Deep Learning with Python, pp.195-208, 2017.
    T.Q. Chen, M. Li, Y.T. Li, et al., "Mxnet:A flexible and efficient machine learning library for heterogeneous distributed systems", arXiv preprint, arXiv:1512.01274, 2015.
    J. Deng, W. Dong, R. Socher, et al., "Imagenet:A largescale hierarchical image database", 2009 IEEE Conference on Computer vision and Pattern Recognition, Miami Beach, FL, USA, pp.248-255, 2009.
    "The PASCAL Visual Object Classes Homepage", http://host.robots.ox.ac.uk/pascal/VOC/.html,2019-1-1.
    "The COCO Common Objects Web Sit", http://mscoco.org/.html,2019-9-10.
    "Labeled faces in the wild web sit", http://vis-www.cs.umass.edu/lfw/.html,2019-9-17.
    "FCVID:Fudan-Columbia video dataset homepage", http://bigvid.fudan.edu.cn/FCVID/.html,2018-2-18.
    B. Neyshabur, S. Bhojanapalli, D. McAllester, et al. "Exploring generalization in deep learning", Advances in Neural Information Processing Systems, Long Beach, California, USA, pp. 5947-5956, 2017.
    D. Arpit, S. Jastrzebski, N. Ballas, et al. "A closer look at memorization in deep networks", Proceedings of the 34 the International Conference on MachineLearning, pp.233-242, 2017.
    R. Novak, Y. Bahri, D.A. Abolafia, et al., "Sensitivity and generalization in neural networks:An empirical study", International Conference on Learning Representations, Vancouver, Canada, 2018.
    Y. Wang, P. Lin and Y. Hong, "Distributed regression estimation with incomplete data in multi-agent networks", Science China Information Sciences, Vol.61, No.9, pp.168-181, 2018.
    T.Y. Lin, P. Goyal, R. Girshick, et al., "Focal loss for dense object detection", Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, pp. 2980-2988, 2017.
    L. Jiang, D.Y. Meng, T. Mitamura, et al., "Easy samples first:Self-paced reranking for zero-example multimedia search", Proceedings of the 22nd ACM International Conference on Multimedia, pp.547-556, 2014.
    S. Burr, "Active learning literature survey", Report, 2009.
    Q. Xie, Q. Zhao, D.Y. Meng, et al., "Kronecker-basisrepresentation based tensor sparsity and its applications to tensor recovery", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.40, No.8, pp.1888-1902, 2017.
    J. Shu, Z.B. Xu and D.Y. Meng, "Small sample learning in big data era", arXiv preprint, arXiv:1808.04572, 2018.
    X.Y. Dong, L. Zheng, F. Ma, et al., "Few-example object detection with model communication", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.41, No.7, PP.1641-1654, 2018.
    D. Hassabis, D. Kumaran, C. Summerfield, et al., "Neuroscience-inspired artificial intelligence", Neuron,Vol. 95, No.2, pp.245-258, 2017.
    E.J. Bjerrum, "Smiles enumeration as data augmentation for neural network modeling of molecules", arXiv preprint, arXiv:1703.07076, 2017.
    I.J. Goodfellow, J. Pouget-Abadie, M. Mirza, et al., "Generative adversarial nets", International Conference on Neural Information Processing Systems, pp.2672-2680, 2014.
    D. Kingma and M. Welling, "Auto-encoding variational bayes", arXiv preprint,arXiv:1312.6114, 2017.
    T. Elsken, J.H. Metzen and F. Hutter, "Neural architecture search:A survey", Journal of Machine Learning Research Vol.20, No.55, pp.1-21, 2019.
    B. Zoph and Q.V. Le, "Neural architecture search with reinforcement learning", arXiv preprint, arXiv:1611.01578, 2016.
    B. Zoph, V. Vasudevan, J. Shlens, et al., "Learning transferable architectures for scalable image recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Utah, USA, pp.8697-8710, 2018.
    K. Gregor and L.C. Yann, "Learning fast approximations of sparse coding", Proceedings of the 27th International Conference on International Conference on Machine Learning, Haifa, Israel, pp.399-406, 2010.
    Q. Xie, M. Zhou, Q. Zhao, et al., "Multispectral and hyperspectral image fusion by ms/hs fusion net", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, pp.1585-1594, 2019.
    Z.Ch. Long, Y.P. Lu, X.Zh. Ma, et al., "PDE-net:Learning PDEs from data", arXiv preprint, arXiv:1710.09668, 2017.
    Q.X. Li, L. Chen, C. Tai, et al., "Maximum principle based algorithms for deep learning", The Journal of Machine Learning Research, Vol.18, No.1, pp.5998-6026, 2017.
    D.Y. Meng, Q. Zhao and L. Jiang, "A theoretical understanding of self-paced learning", Information Science, Vol.414, pp.319-328, 2017.
    M. Andrychowicz, M. Denil, S. Gomez, et al., "Learning to learn by gradient descent by gradient descent", Advances in Neural Information Processing Systems, pp. 3981-3989, 2016.
    K. Li and J. Malik, "Learning to optimize", arXiv preprint, arXiv:1606.01885, 2016.
    L.J. Wu, F. Tian, Y.C. Xia, et al., "Learning to teach with dynamic loss functions", Advances in Neural Information Processing Systems, pp.6466-6477, 2018.
    Y. Fan, F. Tian, T. Qin, et al., "Learning to teach", arXiv preprint, arXiv:1805.03643, 2018.
    M.Y. Ren, W.Y. Zeng, B. Yang, et al., "Learning to reweight examples for robust deep learning", arXiv preprint, arXiv:1803.09050, 2018.
    J. Shu, Q. Xie, L.X. Yi, et al., "Meta-weight-net:Learning an explicit mapping for sample weighting", arXiv preprint, arXiv:1902.07379, 2019.
    M. Pratama and D.H. Wang, "Deep stacked stochastic configuration networks for lifelong learning of non-stationary data streams", Information Sciences, Vol.495, pp.150-174, 2019.
    M. Xiao, F. Zhang, Y. Li, et al. "Robust sparse representation based face recognition in an adaptive weighted spatial pyramid structure", Science China Information Sciences, Vol.61, No.1, PP.012101:1-012101:13, 2018.
    J.C. Shi, Y. Yu, Q. Da, et al., "Virtual-taobao:Virtualizing real-world online retail environment for reinforcement learning", Proceedings of the AAAI Conference on Artificial Intelligence, pp.4902-4909, 2019.
    A. Nagabandi, I. Clavera, S. Liu, et al., "Learning to adapt in dynamic, real-world environments through metareinforcement learning", arXiv preprint, arXiv:1803.11347, 2018.
    H.W. Yong, D.Y. Meng, W.M Zuo, et al., "Robust online matrix factorization for dynamic background subtraction", IEEE Transactions on Pattern Analysis and Machine Intelligence,Vol.40, No.7, pp.1726-1740, 2017.
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Article Metrics

    Article views (316) PDF downloads(3291) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return