Dilated Convolutional Pixels Affinity Network for Weakly Supervised Semantic Segmentation

ZHANG Zhe; WANG Bilin; YU Zhezhou; LI Zhiyuan

doi:10.1049/cje.2021.08.007

Volume 30 Issue 6

Nov. 2021

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2021 > 30(6): 1120-1130

ZHANG Zhe, WANG Bilin, YU Zhezhou, et al., “Dilated Convolutional Pixels Affinity Network for Weakly Supervised Semantic Segmentation,” Chinese Journal of Electronics, vol. 30, no. 6, pp. 1120-1130, 2021, doi: 10.1049/cje.2021.08.007

Citation:

ZHANG Zhe, WANG Bilin, YU Zhezhou, et al., “Dilated Convolutional Pixels Affinity Network for Weakly Supervised Semantic Segmentation,” Chinese Journal of Electronics, vol. 30, no. 6, pp. 1120-1130, 2021, doi: 10.1049/cje.2021.08.007

Citation:

PDF( 3117 KB)

Dilated Convolutional Pixels Affinity Network for Weakly Supervised Semantic Segmentation

doi: 10.1049/cje.2021.08.007

ZHANG Zhe^1,2,
WANG Bilin^1,2,
YU Zhezhou^{1,2
,
,},
LI Zhiyuan¹

1. College of Computer Science and Technology, Jilin University, Changchun 130012, China;
2. Key Laboratory for Symbol Computation and Knowledge Engineering of National Education Ministry, Changchun 130012, China

Funds:

This work is supported by the Development Project of Jilin Province of China (No.20200801033GH, No.2020122328JC), Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University (No.20180520017JH), and the Fundamental Research Funds for the Central Universities, JLU.

Received Date: 2021-03-27
Rev Recd Date: 2021-06-01

Available Online: 2021-09-23

Publish Date: 2021-11-05

Abstract

Abstract

This paper studies semantic segmentation primarily under image-level weak-supervision. Most stateof-the-art technologies have recently used deep classification networks to create small and sparse discriminatory seed regions of each interest target as pseudo-labels for training segmentation networks, which achieve inferior performance compared with the fully supervised setting. We propose a Dilated convolutional pixels affinity network (DCPAN) to localize and expand the seed regions of objects to bridge this gap. Although introduced dilated convolutional units enable capture of additional location information of objects, it falsely highlighted true negative regions as dilated rate enlarge. To address this problem, we properly integrate dilated convolutional units with different dilated rates and self-attention mechanisms to obtain pixel affinity measure matrix for promoting classification network to generate high-quality object seed regions as pseudo-labels; thus, the performance of the segmentation network is boosted. Furthermore, although our approach seems simple, our method obtains a competitive performance, and experiments show that the performance of DCPAN outperforms other state-ofart approaches in weakly-supervised settings, which only use image-level labels on the Pascal VOC 2012 dataset.
- Weakly supervised,
- Semantic segmentation,
- Convolutional neural networks,
- Dilated convolution,
- Self-attention mechanism

FullText(HTML)

References(45)

References

J. Long, E. Shelhamer and T. Darrell, "Fully convolutional networks for semantic segmentation", Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, pp.3431-3440, 2015.

L. C. Chen, G. Papandreou, I. Kokkinos, et al., "Semantic image segmentation with deep convolutional nets and fully connected CRFs", Proc. of International Conference on Learning Representations (ICLR), San Diego, California, USA, pp.1-14, 2015.

J. Dai, K. He and J. Sun, "BoxSup:Exploiting bounding boxes to supervise convolutional networks for semantic segmentation", Proc. of IEEE International Conference on Computer Vision (ICCV), Santiago, Chlie, pp.1635-1643, 2015.

D. Lin, J. Dai, J. Jia, et al., "ScribbleSup:Scribble-supervised convolutional networks for semantic segmentation", Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp.3159-3167, 2016.

A. Bearman, O. Russakovsky, V. Ferrari, et al., "What's the point:Semantic segmentation with point supervision", Proc. of European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, pp.549-565, 2016.

P. O. Pinheiro and R. Collobert, "From image-level to pixellevel labeling with convolutional networks", Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, pp.1713-1721, 2015.

X. Qi, Z. Liu, J. Shi, et al., "Augmented feedback in semantic segmentation under image level supervision", Proc. of European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, pp.90-105, 2016.

Y. Wei, X. Liang, Y. Chen, et al., "STC:A simple to complex framework for weakly-supervised semantic segmentation", IEEE Trans. Pattern Anal. Mach. Intell., Vol.39, No.11, pp.2314-2320, Nov. 2017.

Y. Wei, J. Feng, X. Liang, et al., "Object region mining with adversarial erasing:A simple classification to semantic segmentation approach", Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp.1568-1576, 2017.

J. Lee, E. Kim, S. Lee, et al., "FickleNet:Weakly and semi-supervised semantic image segmentation using stochastic inference", Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp.5267-5276, 2019.

Y. Wei, H. Xiao, H. Shi, et al., "Revisiting dilated convolution:A simple approach for weakly-and semi-supervised semantic segmentation", Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp.7268-7277, 2018.

M. Cordts, M. Omran, S. Ramos, et al., "The cityscapes dataset for semantic urban scene understanding", Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp.3213-3223, 2016.

P. Papadopoulos, A. D. F. Clarke, F. Keller, et al., "Training object class detectors from eye tracking data", Proc. of European Conference on Computer Vision (ECCV), Zurich, Switzerland, pp.361-376, 2014.

B. Zhou, A. Khosla, A. Lapedriza, et al., "Learning deep features for discriminative localization", Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, pp.2921-2929, 2016.

A. Kolesnikov and C. H. Lampert, "Seed, expand and constrain:Three principles for weakly-supervised image segmentation", Proc. of European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, pp.695-711, 2016.

Z. Huang, X. Wang, J. Wang, et al., "Weakly-supervised semantic segmentation network with deep seeded region growing", Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp.7014-7023, 2018.

J. Ahn and S. Kwak, "Learning pixel-level semantic affifinity with image-level supervision for weakly supervised semantic segmentation", Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp.4981-4990, 2018.

A. Chaudhry, P. K. Dokania and P. Torr, "Discovering classspecifific pixels for weakly-supervised semantic segmentation", Proc. of British Machine Vision Conference, London, UK, pp.20.1-20.13, 2017.

X. Wang, S. You, X. Li, et al., "Weakly-supervised semantic segmentation by iteratively mining common object features", Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp.1354-1362, 2018.

L. -C. Chen, G. Papandreou, I. Kokkinos, et al., "DeepLab:Semantic image segmentation with deep convolutional nets atrous convolution and fully connected CRFs", IEEE Trans. Pattern Anal. Mach. Intell., Vol.40, No.4, pp.834-848, 2018.

L. -C. Chen, Y. Zhu, G. Papandreou, et al., "Encoderdecoder with atrous separable convolution for semantic image segmentation", Proc. of European Conference on Computer Vision (ECCV), Munich, Germany, pp.801-818, 2018.

J. Fu, J. Liu, H. Tian, et al., "Dual attention network for scene segmentation", Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp.3141-3149, 2019.

F. Lyu, L. Li, S. S. Victor, et al., "Multi-label image classification via coarse-to-fine attention", Chinese Journal of Electronics, Vol.28, No.6, pp.1118-1126, 2019.

K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition", Proc. of International Conference on Learning Representations, San Diego, California, USA, pp.1-14, 2015.

K. He, X. Zhang, S. Ren, et al., "Deep residual learning for image recognition", Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp.770-778, 2016.

M. Everingham, L. Van Gool, C. K. I. Williams, et al., "The Pascal visual object classes (VOC) challenge", International Journal of Computer Vision, Vol.88, No.2, pp.303-338, 2010.

O. Ronneberger, P. Fischer, and T. Brox, "U-Net:Convolutional networks for biomedical image segmentation", Proc. of International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), Munich, Germany, pp.234-241, 2015.

Deyu. M and Lina. Sun, "Some new trends of deep learning research", Chinese Journal of Electronics, Vol.28, No.6, pp.1087-1091, 2019.

G. Papandreou, L. -C. Chen, K. P. Murphy, et al., "Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation", Proc. of IEEE International Conference on Computer Vision (ICCV), Santiago, Chlie, pp.1742-1750, 2015.

J. Pont-Tuset, P. Arbeláez, J. T. Barron, et al., "Multiscale combinatorial grouping for image segmentation and object proposal generation", IEEE Trans. Pattern Anal. Mach. Intell., Vol.39, No.1, pp.128-140, Jan. 2017.

J. Fan, Z. Zhang, T. Tan, et al., "CIAN:Cross-image affinity net for weakly supervised semantic segmentation", Proc. of AAAI Conference on Artificial Intelligence, New York, NY, USA pp.10762-10769, 2020.

Y. Wang, J. Zhang, M. Kan, et al., "Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation", Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp.12272-12281, 2020.

J. Deng, W. Dong, R. Socher, et al., "ImageNet:A large-scale hierarchical image database", Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, FL, USA, pp.248-255, 2009.

S. Yang, Y. Kim, Y. Kim, et al., "Combinational class activation maps for weakly supervised object localization", Proc. of IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, CO, USA, pp.2930-2938, 2020.

P. Krähenbühl and V. Koltun, "Efficient inference in fully connected CRFS with Gaussian edge potentials", Proc. of Conference and Workshop on Neural Information Processing Systems (NIPS), Granada, Spain, pp.109-117, 2011.

B. Hariharan, P. Arbelaez, L. Bourdev, et al., "Semantic contours from inverse detectors", Proc. of IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain, pp.991-998, 2011.

N. Ketkar, "Introduction to pytorch", Deep Learning with Python, eBook, https://link.springer.com/book/10.1007/978-1-4842-2766-4#toc,pp.195-208,2017.

Y. Jia, E. Shelhamer, J. Donahue, et al., "Caffe:Convolutional architecture for fast feature embedding, " Proceedings of the 22nd ACM International Conference on Multimedia, pp.675-678, 2014.

R. Fan, Q. Hou, M. -M. Cheng, et al., "Associating inter-image salient instances for weakly supervised semantic segmentation", Proc. of European Conference on Computer Vision (ECCV), Munich, Germany, pp.367-383, 2018.

S. Hong, J. Oh, H. Lee, et al., "Learning transferrable knowledge for semantic segmentation with deep convolutional neural network", Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, pp.3204-3212, 2016.

S. Hong, D. Yeo, S. Kwak, et al., "Weakly supervised semantic segmentation using Web-crawled videos", Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp.7322-7330, 2017.

P. -T. Jiang, Q. Hou, Y. Cao, et al., "Integral object mining via online attention accumulation", Proc. of IEEE International Conference on Computer Vision (ICCV), Seoul, Korea (South), pp.2070-2079, 2019.

T. Zhang, G. Lin, J. Cai, et al., "Decoupled spatial neural attention for weakly supervised semantic segmentation", IEEE Transactions on Multimedia, Vol.21, No.11, pp.2930-2941, 2019.

W. Shimoda, and K. Yanai, "Self-supervised difference detection for weakly-supervised semantic segmentation", Proc. of IEEE International Conference on Computer Vision (ICCV), Seoul, Korea (South), pp.5207-5216, 2019.

Z. Wu, C. Shen and A. V. D. Hengel, "Wider or deeper:Revisiting the resnet model for visual recognition", Pattern Recognition, Vol.90, No.C, pp.119-133, 2019.

Relative Articles

Supplements(0)

Cited By

Proportional views