Submitted By :

Manish Tripathi
Enrollment No: A47171209021
Amity University
Human Depth Perception
The Overall System
Literature Review
Projective Geometry for 3D scene analysis
Camera Projection
I. Pin Hole Camera Model (Ideal Projection)
II. Actual Camera Model (Distortions Due to Camera and sensors)
III. Coordinate Transformation
Epipolar Geometry
I. Epipolar Constraint
II. Fundamental Matrix/ Homography Matrix / Essential Matrix
III. Stereo Camera Geometry
Correspondence Matching
Triangulation and 3D Reconstruction
Planar Segmentation
Landing Site Features
Projective Geometry for 3D scene analysis
Camera Projection
Pinhole Camera Model [ Ideal Projection ]
Zero Aperture
All rays follow a Straight Line and pass from one point only
Actual Camera
Tangential And Radial Distortions due to non zero lens Aperture
Can be removed using good quality Camera and sensor
Or by software
Projective Geometry for 3D scene analysis
Camera Projection Continued
Coordinate Transformation
Epipolar Geometry
Projective Geometry for 3D scene analysis
Epipolar Geometry Continued
Fundamental Matrix [ Relation In terms of World Coordinates]
Essential Matrix [ Relation In terms of Pixel Coordinates ]
Homography Matrix [ Planar Orientation ] [For Rectification]
Triangulation for 3D construction
3D Range Map of Every Pixel
Projective Geometry for 3D scene analysis
Correspondence Matching
Area Based Block Matching (Local Method)
Graph Cut ( Global Matching)
Based On Energy Minimization (Slower Method)
Planar Segmentation
Equation of Plane in 3D
ax+by+cz+d=0; [a, b, c] are Normal coefficients
[d] is distance from origin (Camera centre)
RANSAC Algorithm
Iteratively selects 3 Points and tries to fit the plane model using its distance
from the fourth randomly selected fourth point within a certain Threshold
and would join all the points within that range.
Used Extensively to fit Mathematical models( Planes, Lines etc).
Orientation -> Perpendicular to the View Axis within a certain Tolerance
angle
Landing Site Detection
Identify landing sites
Hazard free
Terrain is suitable
Large enough to fit UAV
Assumptions:
The camera mounted perpendicular to plane of the ground, pointing straight
down.
The vertical axis of the camera image plane is aligned with the principal axis of
the UAV.
General Approach:
Generate 3D terrain map from consecutive images
Determine surface roughness and slope
Choose area that fits footprint of helicopter and minimizes roughness and slope
using Plane Fitting on 3D Point cloud
Analysis and Method Used
Stereo Setup
Selection of Baseline
Too small: large depth error
Too large: difficult search problem, increase in outliers
Project analysis and Block Level Design
Disparity Estimation Results
Sequence of images captured by one
camera
Two Cameras with non-planar placement
Exact Co-planar positioning of Camera
Exact Co-planar positioning of Camera
High Accuracy
Disparity Estimation
Most important part of the whole process.
Comes Under The Process of Correspondence Matching
Area Based Block Matching based on minimisation of Sum of Absolute Differences
(SAD) i.e.
disparity() = arg[min(
w
(I
L
(i,j)-I
R
(x+i,y+j)]- (x,y)
L
Faster than the Graph Cut method.
Window Size Selection:
Small : Sensitive to noise; Thus less accurate but faster calculation
Large : Less sensitive to noise but Finer details get lost .
Thus I selected window size as 7X7 to get better results
Disparity Range should be optimum to get balance between time for execution
and Accuracy as each gray level value corresponds to one range
Disparity Estimation
Result using stereo dataset available at Vision.Middlebury website
Result using Raw images captured by my setup
CALIBRATION
Converting Disparity to Range requires the Projection
Matrix(f{focal length, S
x,y
,O
x,y
,relative orientation}
Calibration is done to find out:
Focal length of lens along X axis and Y axis
Lens displacement along X axis and Y axis
3 numbers that describe radial distortion
2 numbers that describe tangential distortion
The position and orientation of the camera in the real world (x, y and z) to
calculate real world distance from pixel values in m,cm,mmetc.
CALIBRATION
For Calibration we capture many images of a standard pattern like a Chessboard
Pattern at different orientations from the two lenses :
Straight line pattern and all individual corners can be easily detected.
Independent of nonlinearity in the image acquisition.
Rectification
To remove Image Distortions from the values calculated in Calibration
To achieve epipolar constraint that is all conjugate pairs should lie on
same rows so as to reduce search domain for point correspondences
to a single row .
We remap to remove distortions and find the new Camera Matrix.
Z-Coordinate Reconstruction
Input :
Disparity at each pixels of the overlapping region
X,Y coordinate of each pixel
Projection matrix Q calculated from calibration to convert disparity to
distance
Output:
Z coordinates at each pixel.
Formula :
Z=T*/d (=)
3D Reconstruction
Creating a Range Map of the image under consideration
Done using Point Cloud Library
Planar Segmentation
Objective :
Finding planes that are perpendicular to the view angle that is normal along
z-axis.
Uses RANSAC Algorithm
Calculate Distance to the plane detected.
Contour Generation
Outline representing the shape or form of the image.
Software Tools
C/C++ in Visual Studio 2010 Compiler.
OpenCV 2.4.7 Library to do computer vision like image acquisition ,
calibration, Rectification, Block Mathing
Point Cloud Library 1.6.0 to do 3D Reconstruction and planar
segmentation
The whole process can be implemented in MATLAB 2013a in its
computer vision toolbox but is slow comparatively.
MATLAB for plot analysis and contour generation.
Inferences
3D Range Map of the view under interest is generated.
Plane perpendicular to the z-axis and its distance is also
calculated.
The Contour of the Image is also created.
Future scope for research
Comparing the footprint of landing strip suitable for landing
with plane detected in the image.
Working on Multi-baseline Stereo imagery using single camera
to get higher accuracy in Depth Perception
Using Monocular cues to improve the depth.