MULTI-VIEW 3D VIDEO ACQUISITION USING HYBRID CAMERAS WITH BEAM SPLITTER

BY JASEEM ROSHAN.B ROLL NO 22

1 College of engineering chengannur

CONTENTS
y What is 3D? y Multi-view 3D video acquisition system y Camera system setup y Multi view video system setup y Captured video processing y Conclusion y References

College of engineering chengannur

What is 3D?
y 3D means three-dimensional, i.e. something that has width,

height and depth (length). Our physical environment is threedimensional and we move around in 3D every day. y Humans are able to perceive the spatial relationship between objects just by looking at them because we have 3D perception, also known as depth perception.

College of engineering chengannur

Multi-view 3D video acquisition system

Depth acquisition

Passive approach
Image-based disparity estimation

Active approach
Time-of-flight (TOF) camera

College of engineering chengannur

Multi-view 3D video acquisition system

College of engineering chengannur

Camera System Setup

y The proposed hybrid camera system consists of three-color

cameras and one time of flight (TOF) camera. y A beam splitter is used between the color camera of the center view and the TOF camera to minimize the FOV difference between two cameras. y Capture three-view color and one-view depth videos

College of engineering chengannur

Camera System Setup

College of engineering chengannur

Multi-view Video Capture System Setup

y Use two capture servers with RAID HDD system, a

synchronization signal generator, and video capturing software y With a software, its able to capture three-view color videos and one-view depth videos with hardware synchronization y There are three kinds of images that can be captured from the TOF camera. 1. A depth image that is quantized gray image with 256 levels 2. An intensity image that represents a gray level image of the scene. 3. An amplitude image that reflects the signal strength of the active illumination.
8 College of engineering chengannur

Multi-view Video Capture System Setup

College of engineering chengannur

Multi-view Video Capture System Setup

10

College of engineering chengannur

Captured Video Processing

y In order to obtain better quality of depth for multi-view 3D

video. We first calibrate each camera in order to estimating extrinsic and intrinsic parameters y For camera calibration MATLAB toolbox is used

11

College of engineering chengannur

Color Video Processing

y Color correction y It analyze the differences between corresponding patches,

and calculate the correction curve using non-linear regression.

12

College of engineering chengannur

Color Video Processing

Captured color chart for color correction

13

College of engineering chengannur

Color Video Processing

Example of rectified multi-view images

14

College of engineering chengannur

Depth Calibration
y The depth values in the depth video from the TOF depth

camera are very sensitive to noises. y Compute the depth of the planar checker pattern within the limited space by increasing the distance from the checker pattern to the depth camera using our system

15

College of engineering chengannur

Depth Image Filtering

y For the removal of salt noise. y Salt noise is unusually very large compared to the depth

values of correct neighboring depth pixels. y These unexpected depth pixels should be removed and be substituted by new depth values. y Averaging filter together with median filter is used

16

College of engineering chengannur

Depth Image Filtering

(a) Original

(b) After filtering

17

College of engineering chengannur

3D Warping of Depth
y Its then generate the initial depth images for high-resolution

multi-view images by performing 3D warping of the depth values to the obtained from the TOF depth camera. y Project pixels of the depth images into the 3D world coordinate using the obtained depth values

(a) Color image

18 College of engineering chengannur

(b) 3D warped depth

Stereo Matching with Warped Initial Depth

y Utilize the warped depth values as initial depth values for the

stereo matching. y Thus can overcome the mismatching problem in the texture less regions y To reduce mismatched depth values along object boundaries, detect moving objects using color difference between frames.

19

College of engineering chengannur

Advantages
y It has a limited bandwidth increase compared to 2D so that it

can be used in existing distribution infrastructures. y Offers flexibility and compatibility with existing production equipment and compression tools.

20

College of engineering chengannur

Disadvantages
y It is not compatible with existing 2D or 3D-Ready displays. y 2d-plus-Depth cannot handle transparency.

21

College of engineering chengannur

REFERENCES
y MULTI-VIEW 3D VIDEO ACQUISITION USING HYBRID

CAMERAS WITH BEAM SPLITTER Gi-Mun Um1,Taeone Kim1, Gun Bang1, Namho Hur1, Eun-Kyung Lee,Jae-Il Jung,Yun-Suk Kang, Gyo-Yoon Lee, andYo-Sung Ho.
y J. Zhu, L. Wang, R.Yang, and J. Davis, Fusion of Timeof- Flight

Depth and Stereo for High Accuracy Depth Maps, Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2008, pp. 1-8, June 2008.
y 3D Interface Specifications White Paper Philips 3D Solutions
22 College of engineering chengannur

23

College of engineering chengannur

24

College of engineering chengannur