FSCIL-EACA: Few-Shot Class-Incremental Learning Network Based on Embedding Augmentation and Classifier Adaptation for Image Classification

ZHANG Ruru; E Haihong; SONG Meina

doi:10.23919/cje.2022.00.396

Volume 33 Issue 1

Jan. 2024

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2024 > 33(1): 139-152

Ruru ZHANG, Haihong E, Meina SONG, “FSCIL-EACA: Few-Shot Class-Incremental Learning Network Based on Embedding Augmentation and Classifier Adaptation for Image Classification,” Chinese Journal of Electronics, vol. 33, no. 1, pp. 139–152, 2024 doi: 10.23919/cje.2022.00.396

Citation:

Ruru ZHANG, Haihong E, Meina SONG, “FSCIL-EACA: Few-Shot Class-Incremental Learning Network Based on Embedding Augmentation and Classifier Adaptation for Image Classification,” Chinese Journal of Electronics, vol. 33, no. 1, pp. 139–152, 2024 doi: 10.23919/cje.2022.00.396

Citation:

Ruru ZHANG, Haihong E, Meina SONG, “FSCIL-EACA: Few-Shot Class-Incremental Learning Network Based on Embedding Augmentation and Classifier Adaptation for Image Classification,” Chinese Journal of Electronics, vol. 33, no. 1, pp. 139–152, 2024 doi: 10.23919/cje.2022.00.396

PDF( 3494 KB)

FSCIL-EACA: Few-Shot Class-Incremental Learning Network Based on Embedding Augmentation and Classifier Adaptation for Image Classification

doi: 10.23919/cje.2022.00.396

ZHANG Ruru^{1, 2
,},
E Haihong^{1, 2
,},
SONG Meina^{1, 2
,
,}

1.
School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China
2.
Education Department Information Network Engineering Research Center, Beijing University of Posts and Telecommunications, Beijing 100876, China

More Information

Author Bio:
Ruru ZHANG was born in 1992. She is studying at Beijing University of Posts and Telecommunications for a doctorate in computer science and technology. Her interests include artificial intelligence and medical image processing. (Email: zrr@bupt.edu.cn)

Haihong E was born in 1982. She graduated from Beijing University of Posts and Telecommunications with a doctor’s degree in computer science and technology. At present, she is a Professor at Beijing University of Posts and Telecommunications. She has been engaged in research and teaching in the fields of big data, artificial intelligence, and cloud-native services for a long time. (Email: ehaihong@bupt.edu.cn)

Meina SONG was born in 1974. She graduated from Beijing University of Posts and Telecommunications with a doctor’s degree in computer science and technology. She is a Professor at Beijing University of Posts and Telecommunications and director of the Information Network Engineering Research Center of the Ministry of Education. His main research direction is artificial intelligence and its application in the fields of finance and medicine. (Email: mnsong@bupt.edu.cn)
Corresponding author: Email: mnsong@bupt.edu.cn
Received Date: 2022-11-19
Accepted Date: 2023-06-16

Available Online: 2023-06-26

Publish Date: 2024-01-05

Abstract

Abstract

The ability to learn incrementally is critical to the long-term operation of AI systems. Benefiting from the power of few-shot class-incremental learning (FSCIL), deep learning models can continuously recognize new classes with only a few samples. The difficulty is that limited instances of new classes will lead to overfitting and exacerbate the catastrophic forgetting of the old classes. Most previous works alleviate the above problems by imposing strong constraints on the model structure or parameters, but ignoring embedding network transferability and classifier adaptation (CA), failing to guarantee the efficient utilization of visual features and establishing relationships between old and new classes. In this paper, we propose a simple and novel approach from two perspectives: embedding bias and classifier bias. The method learns an embedding augmented (EA) network with cross-class transfer and class-specific discriminative abilities based on self-supervised learning and modulated attention to alleviate embedding bias. Based on the adaptive incremental classifier learning scheme to realize incremental learning capability, guiding the adaptive update of prototypes and feature embeddings to alleviate classifier bias. We conduct extensive experiments on two popular natural image datasets and two medical datasets. The experiments show that our method is significantly better than the baseline and achieves state-of-the-art results.
- Few-shot class-incremental learning,
- Embedding augmentation,
- Classifier adaptation,
- Image classification

FullText(HTML)

References(54)

References

[1]	D. Pal, V. Bundele, R. Sharma, et al., “Few-shot open-set recognition of hyperspectral images with outlier calibration network,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA, pp. 2091–2100, 2022.
[2]	S. Fang, X. B. Pan, S. M. Xiang, et al., “Meta-MSNet: Meta-learning based multi-source data fusion for traffic ﬂow prediction,” IEEE Signal Processing Letters, vol. 28, pp. 6–10, 2021. doi: 10.1109/LSP.2020.3037527
[3]	Q. Wu, S. T. Miao, Z. L. Chai, et al., “Fine-grained image classification with global information and adaptive compensation loss,” IEEE Signal Processing Letters, vol. 29, pp. 36–40, 2022. doi: 10.1109/LSP.2021.3123453
[4]	D. W. Zhou, H. J. Ye, and D. C. Zhan, “Learning placeholders for open-set recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, pp. 4399–4408, 2021.
[5]	D. W. Zhou, Y. Yang, and D. C. Zhan, “Learning to classify with incremental new class,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 6, pp. 2429–2443, 2022. doi: 10.1109/TNNLS.2021.3104882
[6]	S. A. Rebuff, A. Kolesnikov, G. Sperl, et al., “iCaRL: Incremental classifier and representation learning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 5533–5542, 2017.
[7]	F. M. Castro, M. J. Marín-Jiménez, N. Guil, et al., “End-to-end incremental learning,” in Proceedings of the 15th European Conference on Computer Vision, Munich, Germany, pp. 241–257, 2018.
[8]	S. H. Hou, X. Y. Pan, C. C. Loy, et al., “Lifelong learning via progressive distillation and retrospection,” in Proceedings of the 15th European Conference on Computer Vision, Munich, Germany, pp. 452–467, 2018.
[9]	D. Isele and A. Cosgun, “Selective experience replay for lifelong learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA, pp. 3302–3309, 2018.
[10]	Y. Xiang, Y. Fu, P. Ji, et al., “Incremental learning using conditional adversarial networks,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, South Korea, pp. 6618–6627, 2019.
[11]	S. H. Hou, X. Y. Pan, C. C. Loy, et al., “Learning a unified classifier incrementally via rebalancing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, pp. 831–839, 2019.
[12]	F. Zhu, X. Y. Zhang, C. Wang, et al., “Prototype augmentation and self-supervision for incremental learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, pp. 5867–5876, 2021.
[13]	Z. C. Pan, X. H. Yu, M. H. Zhang, et al., “SSFE-Net: Self-supervised feature enhancement for ultra-fine-grained few-shot class incremental learning,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA, pp. 6264–6273, 2023.
[14]	F. Zhu, Z. Cheng, X. Y. Zhang, et al., “Class-incremental learning via dual augmentation,” in Proceedings of the 35th Conference on Neural Information Processing Systems, Online, pp. 14306–14318, 2021.
[15]	X. Y. Tao, X. P. Hong, X. Y. Chang, et al., “Few-shot class-incremental learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 12180–12189, 2020.
[16]	A. Kukleva, H. Kuehne, and B. Schiele, “Generalized and incremental few-shot learning by explicit learning and calibration without forgetting,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, pp. 9000–9009, 2021.
[17]	P. Mazumder, P. Singh, and P. Rai, “Few-shot lifelong learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, virtually, pp. 2337–2345, 2021.
[18]	K. L. Chen and C. G. Lee, “Incremental few-shot learning via vector quantization in deep embedded space,” in Proceedings of the 9th International Conference on Learning Representations, Virtual Event, Austria, 2021.
[19]	C. Zhang, N. Song, G. S. Lin, et al., “Few-shot incremental learning with continually evolved classifiers,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, pp. 12450–12459, 2021.
[20]	G. Y. Shi, J. X. Chen, W. L. Zhang, et al., “Overcoming catastrophic forgetting in incremental few-shot learning by finding ﬂat minima,” in Proceedings of the 35th Conference on Neural Information Processing Systems, Virtual, pp. 6747–6761, 2021.
[21]	H. Shin, J. K. Lee, J. Kim, et al., “Continual learning with deep generative replay,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, pp. 2994–3003, 2017.
[22]	M. Y. Zhai, L. Chen, F. Tung, et al., “Lifelong GAN: Continual learning for conditional image generation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, South Korea, pp. 2759–2768, 2019.
[23]	J. Kirkpatrick, R. Pascanu, N. Rabinowitz, et al., “Overcoming catastrophic forgetting in neural networks,” Proceedings of the National Academy of Sciences of the United States of America, vol. 114, no. 13, pp. 3521–3526, 2017. doi: 10.1073/pnas.1611835114
[24]	R. Aljundi, F. Babiloni, M. Elhoseiny, et al., “Memory aware synapses: Learning what (not) to forget,” in Proceedings of the 15th European Conference on Computer Vision, Munich, Germany, pp. 144–161, 2018.
[25]	F. Zenke, B. Poole, and S. Ganguli, “Continual learning through synaptic intelligence,” in Proceedings of the 34th International Conference on Machine Learning, Sydney, NSW, Australia, pp. 3987–3995, 2017.
[26]	F. Sung, Y. X. Yang, L. Zhang, et al., “Learning to compare: Relation network for few-shot learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 1199–1208, 2018.
[27]	W. B. Li, L. Wang, J. L. Xu, et al., “Revisiting local descriptor based image-to-class measure for few-shot learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, pp. 7253–7260, 2019.
[28]	W. B. Li, J. L. Xu, J. Huo, et al., “Distribution consistency based covariance metric networks for few-shot learning,” in Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, Honolulu, HI, USA, article no. 1060, 2019.
[29]	H. J. Ye, H. X. Hu, D. C. Zhan, et al., “Few-shot learning via embedding adaptation with set-to-set functions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 8805–8814, 2020.
[30]	J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, pp. 4080–4090, 2017.
[31]	O. Vinyals, C. Blundell, T. Lillicrap, et al., “Matching networks for one shot learning,” in Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain, pp. 3637–3645, 2016.
[32]	C. Zhang, Y. J. Cai, G. S. Lin, et al., “DeepEMD: Few-shot image classification with differentiable earth mover’s distance and structured classifiers,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 12200–12210, 2020.
[33]	C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in Proceedings of the 34th International Conference on Machine Learning, Sydney, NSW, Australia, pp. 1126–1135, 2017.
[34]	S. K. Liu, A. J. Davison, and E. Johns, “Self-supervised generalisation with meta auxiliary learning,” in Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, BC, Canada, article no.150, 2019.
[35]	A. Chavan, R. Tiwari, U. Bamba, et al., “Dynamic kernel selection for improved generalization and memory efficiency in meta-learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, pp. 9851–9860, 2022.
[36]	Y. Y. Zhao, L. N. Wang, Y. D. Tian, et al., “Few-shot neural architecture search,” in Proceedings of the 38th International Conference on Machine Learning, Virtual Event, pp. 12707–12718, 2021.
[37]	Y. J. Du, X. T. Zhen, L. Shao, et al., “Hierarchical variational memory for few-shot learning across domains,” in Proceedings of the Tenth International Conference on Learning Representations, Virtual Event, 2022.
[38]	H. B. Zhao, Y. J. Fu, M. T. Kang, et al., “MgSvF: Multi-grained slow vs. fast framework for few-shot class-incremental learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence, in press, 2021.
[39]	D. W. Zhou, F. Y. Wang, H. J. Ye, et al., “Forward compatible few-shot class-incremental learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, pp. 9036–9046, 2022.
[40]	H. Liu, L. Gu, Z. X. Chi, et al., “Few-shot class-incremental learning via entropy-regularized data-free replay,” in Proceedings of the 17th European Conference on Computer Vision, Tel Aviv, Israel, pp. 146–162, 2022.
[41]	Z. X. Chi, L. Gu, H. Liu, et al., “MetaFSCIL: A meta-learning approach for few-shot class incremental learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, pp. 14146–14155, 2022.
[42]	X. L. Xu, S. S. Niu, Z. Wang, et al., “Multi-feature space similarity supplement for few-shot class incremental learning,” Knowledge-Based Systems, vol. 265, article no. 110394, 2023. doi: 10.1016/J.KNOSYS.2023.110394
[43]	S. Gidaris and N. Komodakis, “Dynamic few-shot visual learning without forgetting,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 4367–4375, 2018.
[44]	K. H. Tang, J. Q. Huang, and H. W. Zhang, “Long-tailed classification by keeping the good and removing the bad momentum causal effect,” in Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada, article no.128, 2020.
[45]	L. L. Jing and Y. L. Tian, “Self-supervised visual feature learning with deep neural networks: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 11, pp. 4037–4058, 2021. doi: 10.1109/TPAMI.2020.2992393
[46]	Y. Z. Yang and Z. Xu, “Rethinking the value of labels for improving class-imbalanced learning,” in Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada, article no.1618, 2020.
[47]	Z. W. Liu, Z. Q. Miao, X. H. Zhan, et al., “Large-scale long-tailed recognition in an open world,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, pp. 2532–2541, 2019.
[48]	X. L. Wang, R. Girshick, A. Gupta, et al., “Non-local neural networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 7794–7803, 2018.
[49]	A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention is all you need,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, pp. 6000–6010, 2017.
[50]	D. J. Chen, H. Y. Hsieh, and T. L. Liu, “Adaptive image transformer for one-shot object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, pp. 12242–12251, 2021.
[51]	H. Y. Zhang, M. Cissé, Y. N. Dauphin, et al., “Mixup: Beyond empirical risk minimization,” in Proceedings of the 6th International Conference on Learning Representations, Vancouver, BC, Canada, 2018.
[52]	H. Borgli, V. Thambawita, P. H. Smedsrud, et al., “HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy,” Scientific Data, vol. 7, no. 1, article no. 283, 2020. doi: 10.1038/s41597-020-00622-y
[53]	N. C. F. Codella, D. Gutman, M. E. Celebi, et al., “Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC),” in Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington, DC, USA, pp. 168–172, 2018.
[54]	H. F. Wang, Z. F. Wang, M. N. Du, et al., “Score-CAM: Score-weighted visual explanations for convolutional neural networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA, pp. 111–119, 2020.