Depth and Residual Images Based Rendering

DENG Xiangdong; CAO Xun; DAI Qionghai

doi:10.1049/cje.2016.01.020

Volume 25 Issue 1

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2016 > 25(1): 131-138

DENG Xiangdong, CAO Xun, DAI Qionghai, “Depth and Residual Images Based Rendering,” Chinese Journal of Electronics, vol. 25, no. 1, pp. 131-138, 2016, doi: 10.1049/cje.2016.01.020

Citation:

DENG Xiangdong, CAO Xun, DAI Qionghai, “Depth and Residual Images Based Rendering,” Chinese Journal of Electronics, vol. 25, no. 1, pp. 131-138, 2016, doi: 10.1049/cje.2016.01.020

DENG Xiangdong, CAO Xun, DAI Qionghai, “Depth and Residual Images Based Rendering,” Chinese Journal of Electronics, vol. 25, no. 1, pp. 131-138, 2016, doi: 10.1049/cje.2016.01.020

Citation:

DENG Xiangdong, CAO Xun, DAI Qionghai, “Depth and Residual Images Based Rendering,” Chinese Journal of Electronics, vol. 25, no. 1, pp. 131-138, 2016, doi: 10.1049/cje.2016.01.020

PDF( 8285 KB)

Depth and Residual Images Based Rendering

doi: 10.1049/cje.2016.01.020

1.
Academy of Broadcasting Planning, Beijing 100866, China;
2.
School of Electronic Science and Engineering, Nanjing University, Nanjing 210023, China;
3.
Department of Automation, Tsinghua University, Beijing 100084, China;
4.
Tsinghua National Laboratory for Information Science and Technology (TNList), Beijing 100084, China

Funds: This work is supported by the NSFC Project (No.61271445, No.61371166, No.61422107), the Natural Science Foundation of Jiangsu Province Project (No.BK20130583), and the Ministry of Science and Technology Project (No.2012BAH39F03).

Received Date: 2014-03-25
Rev Recd Date: 2015-08-10
Publish Date: 2016-01-10

Abstract

Abstract

This paper proposes a novel representation method which is called Depth and residual images based representation (DRIBR) for virtual image rendering in 3D video. The proposed DRIBR is composed of three parts: a single view image from one particular viewpoint, its corresponding depth map as well as a residual image. The residual image is pre-calculated from the combination of the above two components plus an image taken from the other viewpoint. The proposed DRIBR achieves low data redundancy, and resolves the occlusion problem which is one of the most annoying artifacts during the rendering of virtual image in traditional Depth image based rendering (DIBR). This paper addresses details of DRIBR and virtual image rendering algorithms based on the proposed format. We test the DRIBR on various stereo datasets and the experimental results demonstrate that our method can achieve better performance on both subjective and objective quality assessment.
- Depth map,
- Residual image,
- Depth image based representation (DIBR),
- Occlusion,
- Multi-view video (MVV)

FullText(HTML)

References(26)

References

C. Fehn, P. Kauff, M. Op de Beeck, et al., “An evolutionary and optimised approach on 3D-TV”, Proceedings of International Broadcast Conference, Amsterdam, Holland, pp.357-365, 2002.

X. Cao, Y. Liu and Q. Dai, “A flexible client-driven 3DTV system for real-time acquisition, transmission, and display of dynamic scenes”, EURASIP Journal on Applied Signal Processing, Article ID 351452, 2009.

A.A. Alatan, Y. Yemez, U. Gudukbay, et al., “Scene Representation Technologies for 3DTV A Survey”, IEEE Trans. on Circuits and Systems for Video Technology, Vol.17, No.11, pp.1587-1605, 2007.

M. Flierl and B. Girod, “Multi-view video compressionexploiting inter-image similarities”, IEEE Signal Processing, Vol.24, No.7, pp.66-76, 2007.

S. Shimizu, M. Kitahara, H. Kimata, et al., “View scalable multiview video coding using 3-D warping with depth map”, IEEE Trans. On Circuits and Systems for Video Technology, Vol.17, No.11, pp.1485-1495, 2007.

M.O. Beeck, E. Fert, C. Fehn, et al., “Broadcast requirements on 3D video coding”, ISO/IEC JTC1/SC29/WG11 M8040, 2002.

C. Fehn, “Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3DTV”, Proc. of SPIE, Electronic Imaging, California, Vol.5291, pp.93-104, 2004.

Zhixiang You, Yangzhou Du, E Li, et al., “Parallel Depthimage- based rendering (DIBR) algorithm for the next generation 3D-TV applications”, Workshop on Emerging Applications and Many-core Architecture, International Symposium on Computer Architecture (ISCA), 2008.

C. Vazquez, W.J. Tam and F. Speranza, “Stereoscopic imaging: Filling disoccluded areas in depth image-based rendering”, Proc. of SPIE, Three-Dimensional TV, Video and Display V, Vol.6392, pp.D.1-D.12, 2006.

L.M. Po, S. Zhang, X. Xu, et al., “A new multidirectional extrapolation hole-filling method for depth-image-based rendering”, Proceedings of International Conference on Image Processing (ICIP), pp.2637-2640, 2011.

J. Shade, S. Gortler, L.W. He, et al., “Layered depth images”, Computer Graphics, Proc. SIGGRAPH98, Vol.32, pp.231-242, 1998.

S.U. Yoon, E.K. Lee, S.Y. Kim, et al., “Coding of layered depth images representing multiple viewpoint video”, Proc. Picture Coding Symposium, pp.1-6, 2006.

S. Shimizu, H. Kimata, K. Kamikura, et al., “A backward compatible 3D scene coding using residual prediction”, IEEE ICASSP, pp.1141-1144, 2008.

G. Petrovic, P.H.N. deWith, “Near-future streaming framework for 3D-TV applications”, Proc. IEEE International Conference on Multimedia & Expo (ICME), Toronto, pp.1881-1884, 2006.

Yamei Feng, Dongxiao Li, Ming Zhang, et al., “Virtual visual point synthesizing method based on depth and block information”, Patent, CN101312540A, China, 2008-11-26.

D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms”, International Journal of Computer Vision (IJCV), Vol.47, No.1, pp.7- 42, 2002.

D. Scharstein and R. Szeliski, “High-accuracy stereo depth maps using structured light”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Vol.1, pp.195-202, 2003.

Xuyuan Xu, LaiMan Po, KwokWai Cheung, et al., “A foreground biased depth map refinement method for DIBR view synthesis”, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vol.37, No.4, pp.805- 808, 2012.

R.P.M. Berretty, F.J. Peters and G.T.G. Volleberg, “Real time rendering for multiview autostereoscopic displays”, Proc. of SPIE, Electronic Imaging, Vol.6055, pp.208-219, 2006.

The MathWorks, MATLAB, Image processing toolbox users guide, 2004.

A. Telea, “An image inpainting technique based on the fast marching method”, Journal of Graphics Tools, Vol.9, No.1, pp.25-36, 2004.

Liang Zhang and W.J. Tam, “Stereoscopic image generation based on depth images for 3D TV”, IEEE Transactions on Broadcasting, Vol.51, No.2, pp.191-199, 2005.

ITU-R Recommendation BT. 500-12, Methodology for the Subjective Assessment of the Quality of Television Pictures.

ITU-R Recommendation BT.1788, Methodology for the Subjective Assessment of Video Quality in Multimedia Applications.

Middlebury Stereo EvaluationWebsite, http://vision.middlebury. edu/stereo/data/, 2015-8-1.

A. Moorthy and A. Bovik, “A two-step framework for constructing blind image quality indices”, IEEE Signal Processing Letters, Vol.17, No.5, pp.513-516, 2010.

Relative Articles

Supplements(0)

Cited By

Proportional views