Confidential Image Super-resolution with Privacy Protection

HAN Yiran; LIU Jianwei; DENG Xin; JING Junpeng; ZHANG Yanting

doi:10.23919/cje.2023.00.034

Article Contents

Article Navigation > Chinese Journal of Electronics > 2024 > Accepted Manuscript

Yiran HAN, Jianwei LIU, Xin DENG, et al., “Confidential Image Super-resolution with Privacy Protection,” Chinese Journal of Electronics, vol. x, no. x, pp. 1–11, xxxx doi: 10.23919/cje.2023.00.034

Citation:

Yiran HAN, Jianwei LIU, Xin DENG, et al., “Confidential Image Super-resolution with Privacy Protection,” Chinese Journal of Electronics, vol. x, no. x, pp. 1–11, xxxx doi: 10.23919/cje.2023.00.034

Citation:

PDF( 10178 KB)

Confidential Image Super-resolution with Privacy Protection

doi: 10.23919/cje.2023.00.034

HAN Yiran^{1, 2
,},
LIU Jianwei^{1, 2
,},
DENG Xin^{1
,
,},
JING Junpeng^1
,,
ZHANG Yanting^1
,

1.
School of Cyber Science and Technology, Beihang University, Beijing 100191, China
2.
Beihang University Hangzhou Innovation Institute, Hangzhou 310051, China

More Information

Author Bio:
Yiran HAN received the B.E. degree in Software college of Northeastern University, China, in 2019. She currently is studing for a doctor’s degree in School of Cyber Science and Technology, Beihang University, Beijing, China. Her research direction focuses on privacy protection, artificial intelligence security, information security, computer network security, and cryptographic protocol design. (Email: hanyiran@buaa.edu.cn)

Jianwei LIU (Senior Member, IEEE) received the Ph.D. in communication engineering from Xidian University, China in 1998, and his B.S. and M.S. degrees in electronic engineering from Shandong University, China in 1985 and 1988. He is currently a professor and dean of School of Cyber Science and Technology, Beihang University. His current research interests include cryptographic protocol design, security on wireless and mobile network, computer network security, and Cryptography. He has published 6 books and more than 200 papers in his research fields. He is a senior member of the Chinese Institute of Electronics, and director of the Chinese Association for Cryptologic Research, and member of Teaching steering committee of information security of MOE, China. He has been awarded the first prize of technological invention of China. (Email: liujianwei@buaa.edu.cn)

Xin DENG (Member, IEEE) received the master’s degree in electrical information engineering from Beihang University, Beijing, China, in 2016, and the PhD degree in electrical and electronic engineering from Imperial College London, London, U.K., in March 2020. She is currently an associate professor with the Department of Cyber Science and Technology, Beihang University, Beijing, China. Her research interests include sparse coding with applications in image and video processing, machine learning, and multimodal signal processing. (Email: cindydeng@buaa.edu.cn)

Junpeng JING received the BS degree from Beihang University (BUAA), Beijing, China, in July 2020. He is currently working toward the MS degree with the School of Cyber Science and Technology, Beihang University, Beijing, China.He was awarded three academic scholarships from Beihang University, including a national scholarship. His research interests include image hiding and reversible image conversion. (Email: junpengjing@buaa.edu.cn)

Yanting ZHANG received Ph.D. degree in Communication and Information System, Beihang University, 2023. She is currently an engineer in China Electronics Standardization Institute. Her research interests include artificial intelligence security, cryptography, information security and privacy. (Email: zhangyanting@cesi.cn)
Corresponding author: Email: cindydeng@buaa.edu.cn
Available Online: 2024-03-20

Abstract

Abstract

Nowadays, people are getting used to upload images to a third-party for post-processing, such as image denoising and super-resolution. However, this may easily lead to the disclosure of the privacy in the confidential images. One possible solution is to encrypt the image before sending it to the third party, however, the encrypted image can be easily detected by a malicious attacker in the transmission channel. In this paper, we propose a confidential image super-resolution method namely HSR-Net, which firstly hide the secret image and then super-resolve it in the hidden domain. The HSR-Net is composed of three important modules: image hiding module (IHM), image super-resolution module (ISM), and image revealing module (IRM). The IHM aims to encode secret image and hide it into a cover image to generate the stego image. The stego image looks similar to the cover image but contains the information of the secret image. Then, the third party uses the ISM to perform image super-resolution on the stego image. After that, the user can reveal the super-resolved secret image from the stego image. Our HSR-Net has two advantages. Firstly, it ensures that the third party cannot not directly operate on the secret image to protect the user’s privacy. In addition, due to the similarity between the stego image and cover image, we can reduce the attacker’s suspicion to further improve the image security. The experimental results on various datasets, including DIV2K dataset and Flickr2K dataset. The PSNR of IHM is 38.81dB, the PSNR of ISM is 28.91 dB, and the PSNR of IRM is 23.51 dB, which verify that our HSR-Net is able to achieve image super-resolution and protect user’s privacy simultaneouly.
- Confidential image,
- Image hiding,
- Super-resolution,
- Privacy protection model

FullText(HTML)

References(41)

References

[1]	N. F. Johnson and S. Jajodia, “Exploring steganography: Seeing the unseen,” Computer, vol. 31, no. 2, pp. 26–34, 1998. doi: 10.1109/MC.1998.4655281
[2]	A. D. Ker, “A general framework for structural steganalysis of LSB replacement,” in Proceedings of the 7th International Workshop on Information Hiding, Barcelona, Spain, pp. 296–311, 2005.
[3]	J. Fridrich, M. Goljan, and R. Du, “Reliable detection of LSB steganography in color and grayscale images,” in Proceedings of the 2001 Workshop on Multimedia and Security: New Challenges, Ottawa, Canada, pp. 27–30, 2001.
[4]	C. T. Hsu and J. L. Wu, “Hidden digital watermarks in images,” IEEE Transactions on Image Processing, vol. 8, no. 1, pp. 58–68, 1999. doi: 10.1109/83.736686
[5]	J. J. K. O. Ruanaidh, W. J. Dowling, and F. M. Boland, “Phase watermarking of digital images,” in Proceedings of the 3rd IEEE International Conference on Image Processing, Lausanne, Switzerland, pp. 239–242, 2008.
[6]	M. Barni, F. Bartolini, and A. Piva, “Improved wavelet-based watermarking through pixel-wise masking,” IEEE Transactions on Image Processing, vol. 10, no. 5, pp. 783–791, 2001. doi: 10.1109/83.918570
[7]	K. Curran and K. Bailey, “An evaluation of steganography methods,” International Journal of Digital Evidence, in press, 2003. (查阅网上资料, 未找到本条文献信息, 请确认). K. Curran and K. Bailey, “An evaluation of steganography methods,” International Journal of Digital Evidence, in press, 2003. (查阅网上资料, 未找到本条文献信息, 请确认).
[8]	A. Westfeld and A. Pfitzmann, “Attacks on steganographic systems: Breaking the Steganographic utilities EzStego, Jsteg, Steganos, and S-Tools-and some lessons learned,” in Proceedings of the Third International Workshop on Information Hiding, Dresden, Germany, pp. 61–76, 2000.
[9]	N. Provos and P. Honeyman, “Hide and seek: An introduction to steganography,” IEEE Security & Privacy, vol. 1, no. 3, pp. 32–44, 2003. doi: 10.1109/MSECP.2003.1203220
[10]	S. Baluja, “Hiding images in plain sight: Deep steganography,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, pp. 2066–2076, 2017.
[11]	G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” Science, vol. 313, no. 5786, pp. 504–507, 2006. doi: 10.1126/science.1127647
[12]	P. Wu, Y. Yang, and X. Q. Li, “StegNet: Mega image steganography capacity with deep convolutional network,” Future Internet, vol. 10, no. 6, article no. 54, 2018. doi: 10.3390/fi10060054
[13]	J. R. Zhu, R. Kaplan, J. Johnson, et al., “HiDDeN: Hiding data with deep networks,” in Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany, pp. 682–697, 2018.
[14]	X. Y. Weng, Y. Z. Li, L. Chi, et al., “High-capacity convolutional video steganography with temporal residual modeling,” in Proceedings of the 2019 on International Conference on Multimedia Retrieval, Ottawa, Canada, pp. 87–95, 2019.
[15]	D. X. Dai, Y. J. Wang, Y. H. Chen, et al., “Is image super-resolution helpful for other vision tasks?” in Proceedings of the 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Placid, NY, USA, pp. 1–9, 2016.
[16]	S. Thurnhofer and S. K. Mitra, “Edge-enhanced image zooming,” Optical Engineering, vol. 35, no. 7, pp. 1862–1870, 1996. doi: 10.1117/1.600619
[17]	F. Fekri, R. M. Mersereau, and R. W. Schafer, “A generalized interpolative VQ method for jointly optimal quantization and interpolation of images,” in Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, Seattle, WA, USA, pp. 2657–2660, 1998.
[18]	W. T. Freeman and E. C. Pasztor, “Learning low-level vision,” in Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece, pp. 1182–1189, 1999.
[19]	H. Chang, D. Y. Yeung, and Y. M. Xiong, “Super-resolution through neighbor embedding,” in Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, 2004.
[20]	J. C. Yang, J. Wright, T. Huang, et al., “Image super-resolution as sparse representation of raw image patches,” in Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA, pp. 1–8, 2008.
[21]	R. Timofte, V. De, and L. Van Gool, “Anchored neighborhood regression for fast example-based super-resolution,” in Proceedings of the IEEE International Conference on Computer Vision, Sydney, NSW, Australia, pp. 1920–1927, 2013.
[22]	R. Timofte, V. De Smet, and L. Van Gool, “A+: Adjusted anchored neighborhood regression for fast super-resolution,” in Proceedings of the 12th Asian Conference on Computer Vision, Singapore, Singapore, pp. 111–126, 2015.
[23]	C. Dong, C. C. Loy, K. M. He, et al., “Learning a deep convolutional network for image super-resolution,” in Proceedings of the 13th European Conference on Computer Vision, Zurich, Switzerland, pp. 184–199, 2014.
[24]	J. Kim, J. K. Lee, and K. M. Lee, “Accurate image super-resolution using very deep convolutional networks,” in Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 1646–1654, 2016.
[25]	C. Dong, C. C. Loy, and X. O. Tang, “Accelerating the super-resolution convolutional neural network,” in Proceedings of the 14th European Conference on Computer Vision, Amsterdam, The Netherlands, pp. 391–407, 2016.
[26]	K. M. He, X. Y. Zhang, S. Q. Ren, et al., “Deep residual learning for image recognition,” in Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 770–778, 2016.
[27]	B. Lim, S. Son, H. Kim, et al., “Enhanced deep residual networks for single image super-resolution,” in Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, pp. 1132–1140, 2017.
[28]	C. Ledig, L. Theis, F. Huszar, et al., “Photo-realistic single image super-resolution using a generative adversarial network,” in Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 105–114, 2017.
[29]	Y. Tai, J. Yang, X. M. Liu, et al., “MemNet: A persistent memory network for image restoration,” in Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy, pp. 4549–4557, 2017.
[30]	Y. L. Zhang, K. P. Li, K. Li, et al., “Image super-resolution using very deep residual channel attention networks,” in Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany, pp. 294–310, 2018.
[31]	J. S. Choi and M. Kim, “A deep convolutional neural network with selection units for super-resolution,” in Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, pp. 1150–1156, 2017.
[32]	G. Huang, Z. Liu, L. Van Der Maaten, et al., “Densely connected convolutional networks,” in Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 2261–2269, 2016.
[33]	Y. L. Zhang, Y. P. Tian, Y. Kong, et al., “Residual dense network for image restoration,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 7, pp. 2480–2495, 2021. doi: 10.1109/TPAMI.2020.2968521
[34]	M. Haris, G. Shakhnarovich, and N. Ukita, “Deep back-projection networks for super-resolution,” in Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 1664–1673, 2018.
[35]	I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, et al., “Generative adversarial nets,” in Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, Canada, pp. 2672–2680, 2014.
[36]	T. Y. Lin, M. Maire, S. Belongie, et al., “Microsoft COCO: Common objects in context,” in Proceedings of the 13th European Conference on Computer Vision, Zurich, Switzerland, pp. 740–755, 2014.
[37]	E. Agustsson and R. Timofte, “NTIRE 2017 challenge on single image super-resolution: Dataset and study,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, pp. 1122–1131, 2017.
[38]	J. Ye, J. Q. Ni, and Y. Yi, “Deep learning hierarchical representations for image steganalysis,” IEEE Transactions on Information Forensics and Security, vol. 12, no. 11, pp. 2545–2557, 2017. doi: 10.1109/TIFS.2017.2710946
[39]	Y. M. Hei, L. H. Wang, J. W. Sheng, et al., “Label graph augmented soft cascade decoding model for overlapping event extraction,” International Journal of Machine Learning and Cybernetics, vol. 15, no. 1, pp. 79–95, 2024. doi: 10.1007/s13042-022-01760-y
[40]	M. Z. Konyar and S. Solak, “Efficient data hiding method for videos based on adaptive inverted lsb332 and secure frame selection with enhanced vigenere cipher,” Journal of Information Security and Applications, vol. 63, article no. 103037, 2021. doi: 10.1016/j.jisa.2021.103037
[41]	S. Kaçar, M. Z. Konyar, and Ünal Çavuşoğlu, “4D chaotic system-based secure data hiding method to improve robustness and embedding capacity of videos,” Journal of Information Security and Applications, vol. 71, article no. 103369, 2022. doi: 10.1016/j.jisa.2022.103369