FGM-SPCL: Open-Set Recognition Network for Medical Images based on Fine-Grained data Mixture and Spatial Position Constraint Loss

ZHANG Ruru; E Haihong; YUAN Lifei; WANG Yanhui; WANG Lifei; SONG Meina

doi:10.23919/cje.2023.00.081

Article Contents

Article Navigation > Chinese Journal of Electronics > 2022 > Accepted Manuscript

Ruru ZHANG, Haihong E, Lifei YUAN, et al., “FGM-SPCL: Open-Set Recognition Network for Medical Images based on Fine-Grained data Mixture and Spatial Position Constraint Loss,” Chinese Journal of Electronics, vol. x, no. x, article no. , xxxx doi: 10.23919/cje.2023.00.081

Citation:

Ruru ZHANG, Haihong E, Lifei YUAN, et al., “FGM-SPCL: Open-Set Recognition Network for Medical Images based on Fine-Grained data Mixture and Spatial Position Constraint Loss,” Chinese Journal of Electronics, vol. x, no. x, article no. , xxxx doi: 10.23919/cje.2023.00.081

Citation:

PDF( 3209 KB)

FGM-SPCL: Open-Set Recognition Network for Medical Images based on Fine-Grained data Mixture and Spatial Position Constraint Loss

doi: 10.23919/cje.2023.00.081

ZHANG Ruru^{1, 2
,},
E Haihong^{1, 2
,},
YUAN Lifei^3
,,
WANG Yanhui^3
,,
WANG Lifei^3
,,
SONG Meina^{1, 2
,
,}

1.
School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, 100876, China
2.
Education Department Information Network Engineering Research Center, Beijing University of Posts and Telecommunications, Beijing, 100876, China
3.
Hebei Eye Hospital, Hebei, 054001, China

More Information

Author Bio:
Ruru ZHANG was born in 1992. At present, she is studying at Beijing University of Posts and Telecommunications for a doctorate in computer science and technology. Her interests include artificial intelligence and medical image processing. (Email: zrr@bupt.edu.cn)

Haihong E was born in 1982. She graduated from Beijing University of Posts and Telecommunications with a doctor’s degree in computer science and technology. At present, she is a professor at Beijing University of Posts and Telecommunications. She has been engaged in research and teaching in the fields of big data, artificial intelligence and cloud native services for a long time. (Email: ehaihong@bupt.edu.cn)

Lifei YUAN was born in 1986. He graduated from Tianjin Medical University with a master’s degree in ophthalmology. His main research direction is fundus disease. (Email: yuanlifei2341@163.com)

Yanhui WANG was born in 1981. She graduated from Tianjin Medical University with a master’s degree in ophthalmology. Her main research direction is fundus disease. (Email: yanhuiwang1981@163.com)

Lifei WANG (co-corresponding authorr) was born in August 1976. She graduated from Sun Yat-Sen University majoring in Ophthalmology with a doctorate degree. She is currently Secretary of the Party Committee of Hebei Provincial Eye Hospital. Her main research direction is fundus disease. (Email: wlfhb@126.com)

Meina SONG (corresponding author) was born in 1974. She graduated from Beijing University of Posts and Telecommunications with a doctor’s degree in computer science and technology. At present, she is a professor of Beijing University of Posts and Telecommunications and director of the Information Network Engineering Research Center of the Ministry of Education. His main research direction is artificial intelligence and its application in the fields of finance and medicine. (Email: mnsong@bupt.edu.cn)
Corresponding author: Email: mnsong@bupt.edu.cn
Received Date: 2022-03-22
Accepted Date: 2022-03-22

Available Online: 2022-03-22

Abstract

Abstract

The current intelligent auxiliary diagnosis models all follow the closed-set recognition setting. After the model is deployed online, the input data is often not completely controlled. Diagnosing an untrained disease as a known category would lead to serious medical malpractice. Therefore, realizing the open set recognition is significant to the safe operation of the intelligent auxiliary diagnosis model. Currently, most open-set recognition models are studied for natural images, and it is very challenging to obtain clear and concise decision boundaries be-tween known and unknown classes when applied to fine-grained medical images. Therefore, we propose an Open-Set Recognition Network for Medical Images based on Fine-Grained data Mixture and Spatial Position Constraint Loss (FGM-SPCL) in this work. First, considering the fine graininess of medical images and the diversity of unknown samples, we propose a fine-grained data Mixture (FGM) method to simulate unknown data by performing a mixing operation on known data to expand the coverage of unknown data difficulty levels. Secondly, in order to obtain a concise and clear decision boundary, we propose a Spatial Position Constraint Loss (SPCL) to control the position distribution of prototypes and samples in the feature space and maximize the distance between known classes and unknown classes. Finally, we validate on a private ophthalmic OCT dataset, and extensive experiments and analyses demonstrate that FGM-SPCL outperforms state-of-the-art models.
- Few-Shot Class-Incremental Learning,
- Embedding Augmentation,
- Classifier Adaptation,
- Image Classiﬁcation

FullText(HTML)

References(28)

References

[1]	L. Schiff, B. Migliori, Y. Chen, et al., “Integrating deep learning and unbiased automated high-content screening to identify complex disease signatures in human fibroblasts,” Nature Communications, vol. 13, no. 1, article no. 1590, 2022. doi: 10.1038/s41467-022-28423-4
[2]	G. Y. Wang, X. H. Liu, J. Shen, et al., “A deep-learning pipeline for the diagnosis and discrimination of viral, non-viral and COVID-19 pneumonia from chest X-ray images,” Nature Biomedical Engineering, vol. 5, no. 6, pp. 509–521, 2021. doi: 10.1038/s41551-021-00704-1
[3]	K. Zhang, X. H. Liu, J. Shen, et al., “Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of COVID-19 pneumonia using computed tomography,” Cell, vol. 181, no. 6, pp. 1423–1433.e11, 2020. doi: 10.1016/j.cell.2020.04.045
[4]	Z. H. Xia, P. H. Wang, G. G. Dong, et al., “Spatial location constraint prototype loss for open set recognition,” Computer Vision and Image Understanding, vol. 229, article no. 103651, 2023. doi: 10.1016/J.CVIU.2023.103651
[5]	S. Kong and D. Ramanan, “OpenGAN: Open-set recognition via open data generation,” in Proceedings of 2021 IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, pp. 793–802, 2021.
[6]	W. Moon, J. Park, H. S. Seong, et al., “Difficulty-aware simulator for open set recognition,” in Proceedings of the 17th European Conference on Computer Vision, Tel Aviv, Israel, pp. 365–381, 2022.
[7]	D. W. Zhou, H. J. Ye, and D. C. Zhan, “Learning placeholders for open-set recognition,” in Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, pp. 4399–4408, 2021.
[8]	G. Y. Chen, P. X. Peng, X. Q. Wang, et al., “Adversarial reciprocal points learning for open set recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 11, pp. 8065–8081, 2021. doi: 10.1109/TPAMI.2021.3106743
[9]	Z. Y. Ge, S. Demyanov, Z. Chen, et al., “Generative OpenMax for multi-class open set classification,” in Proceedings of British Machine Vision Conference 2017, London, UK, 2017.
[10]	W. J. Scheirer, A. de Rezende Rocha, A. Sapkota, et al., “Toward open set recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 7, pp. 1757–1772, 2013. doi: 10.1109/TPAMI.2012.256
[11]	A. Bendale and T. Boult, “Towards open world recognition,” in Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, pp. 1893–1902, 2015.
[12]	P. R. M. Júnior, R. M. De Souza, R. de O. Werneck, et al., “Nearest neighbors distance ratio open-set classifier,” Machine Learning, vol. 106, no. 3, pp. 359–386, 2017. doi: 10.1007/s10994-016-5610-8
[13]	H. Zhang and V. M. Patel, “Sparse representation-based open set recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 8, pp. 1690–1696, 2017. doi: 10.1109/TPAMI.2016.2613924
[14]	X. Sun, Z. N. Yang, C. Zhang, et al., “Conditional Gaussian distribution learning for open set recognition,” in Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 13477–13486, 2020.
[15]	H. J. Zhang, A. Li, J. Guo, et al., “Hybrid models for open set recognition,” in Proceedings of the 16th European Conference on Computer Vision, Glasgow, UK, pp. 102–117, 2020.
[16]	R. Yoshihashi, W. Shao, R. Kawakami, et al., “Classification-reconstruction learning for open-set recognition,” in Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, pp. 4011–4020, 2019.
[17]	P. Oza and V. M. Patel, “C2AE: Class conditioned auto-encoder for open-set recognition,” in Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, pp. 2302–2311, 2019.
[18]	H. M. Yang, X. Y. Zhang, F. Yin, et al., “Convolutional prototype network for open set recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 5, pp. 2358–2370, 2022. doi: 10.1109/TPAMI.2020.3045079
[19]	Y. Shu, Y. M. Shi, Y. W. Wang, et al., “P-ODN: Prototype-based open deep network for open set recognition,” Scientific Reports, vol. 10, no. 1, article no. 7146, 2020. doi: 10.1038/s41598-020-63649-6
[20]	C. G. Lyu, Y. X. Chen, Z. J. Chen, et al., “Automatic epilepsy detection based on generalized convolutional prototype learning,” Measurement, vol. 184, article no. 109954, 2021. doi: 10.1016/J.MEASUREMENT.2021.109954
[21]	J. Lu, Y. L. Xu, H. Li, et al., “PMAL: Open set recognition via robust prototype mining,” in Proceedings of the 36th AAAI Conference on Artificial Intelligence, virtual event, pp. 1872–1880, 2022.
[22]	G. Y. Chen, L. M. Qiao, Y. M. Shi, et al., “Learning open set network with discriminative reciprocal points,” in Proceedings of the 16th European Conference on Computer Vision, Glasgow, UK, pp. 507–522, 2020.
[23]	D. Mehta, Y. Gal, A. Bowling, et al., “Out-of-distribution detection for long-tailed and fine-grained skin lesion images,” in Proceedings of the 25th International Conference on Medical Image Computing and Computer Assisted Intervention, Singapore, Singapore, pp. 732–742, 2022.
[24]	K. M. He, X. Y. Zhang, S. Q. Ren, et al., “Deep residual learning for image recognition,” in Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 770–778, 2016.
[25]	H. Borgli, V. Thambawita, P. H. Smedsrud, et al., “HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy,” Scientific Data, vol. 7, no. 1, article no. 283, 2020. doi: 10.1038/s41597-020-00622-y
[26]	A. Krizhevsky, et al., Technical Report, 2009. (查阅网上资料, 未找到本条文献信息, 请确认)
[27]	L. Neal, M. Olson, X. Fern, et al., “Open set learning with counterfactual images,” in Proceedings of the 15th European Conference on Computer Vision, Munich, Germany, pp. 620–635, 2018.
[28]	S. Vaze, K. Han, A. Vedaldi, et al., “Open-set recognition: A good closed-set classifier is all you need,” in Proceedings of the 10th International Conference on Learning Representations, Virtual Event, 2022.