Академический Документы
Профессиональный Документы
Культура Документы
5.
GENERATION
OF
3D POINT CLOUD
VIDEO
Given the
large variation in scene
composition, complexity, and imaging conditions, robust generation of detailed 3D scene
models is as hard a problem in computer vision as the capability to do so is compelling There
are variety of techniques for deriving 3D structure from motion, multi-view stereo, and the
associated sub problems of feature detection and correspondence matching.
Another way to generate 3D point cloud representations of a scene is to use a
laser ranging device. At this time these devices generally fall under one of two categories, a
single beam point-scanning technology or a focal plane array (FPA) device [4]. The former
technology is more mature, far more common and far more institutionalized than the latter.
Well-performing FPA flash LIDAR devices are just now becoming commercially available.
While commercially available single beam point-scanning LIDAR can generate very high
resolution 3D samplings of static scenes, moving sensor platforms necessitate the complexity
of independently registering each measured pixel using accurate record of the platforms
motion. Even when this is done properly, registering samples which fall on a moving target is
impossible if the precise motion description of the target is unknown. In many cases, the
motion description of a moving target is precisely the information we are trying to measure,
so in these cases point-scanning LIDAR systems are unsuitable.
Generally, a 3D flash LIDAR by contrast uses an array of independent LIDAR
receivers and focuses light returning from the scene onto the array with a lens system. In
many ways it has a user aesthetic which is exactly like any ordinary digital video camera. The
flash is generated by an on board laser module and a beam-spreading optical element. This is
analogous to flash photography with conventional 2D digital video cameras. Since all pixels
function in parallel to each other the motion of the platform and motion of the scene between
pixels samplings is zero.
Fig.7.1
Range: up to 1.5km
Size: 11x6x6inches
weight:10 lbs
Laser wave length is 1.57micro meter & pulsed for 5ns per video frame
Dealing with full motion flash LIDAR point cloud video sequences poses several
unique problems beginning first with how best to represent the data. While each pixel location
in the range image represents a location in 3D space, since a point behind a point cannot be
measured with the camera technology we are using, the data is not actually volumetric in
nature. For this reason some researchers describe this kind of data as 2.5D. The output of the
ASC flash LIDAR video camera is interpretable as a range measurement and an intensity
measurement for each pixel on the array. This can be conceptualized as a range image and a
co-registered intensity image. The intensity image is useful for fusion with 2D intensity
camera data. The range and intensity images are represented on a regularly spaced 2D grid
defined in pixel coordinates. Data in the form of 2D scalar arrays such as these are
convenient. They lend well to spatial filtering using finite length kernel operators, however
for our application this is not a necessary operation and not quite what we want. Another way
to represent the data is to use parameters from the camera and lens system to orthorectify the
data. Using a basic triangle approximation of the imaging system the data is sufficiently
orthorectified. After orthorectification, the samples in each range video frame are represented
as an irregularly spaced unoriented point cloud. The irregular spacing comes about as a result
of the spatial sampling frequency of the scene decreasing as a function of range. This is
illustrated in Figure2.
Fig.8.1
One approach to operating on irregularly spaced data is to interpolate the
data onto a regular grid and treat it as regularly spaced data. This method is not suitable for
our task because it produces undesirable artifacts at edges between foreground and
background objects which result in erroneously connecting the foreground with the
background later on. Two other methods for dealing with irregularly spaced data is to
interpolate it locally with piecewise continuous functions or to devise operators which can
work directly on irregularly sampled data. These two methods are more suitable for our task.
Local interpolation using a continuous model is used in step 1 of the algorithm and operators
which work on irregularly spaced data are used in step 2.
Fig.8.2
Fig.8.3
Figure 3 shows orthorectified point cloud data displayed on 3D axes. The
sampled scene depicts a man walking between two fixed lamp posts, and is an example of one
range image in a several second long sequence of range images sampled at 5 Hz. First we
would like to locate the ground plane. Without filtering or classifying the data points in any
way, we can do this by solving for the best fit 3D plane in the data. To do this we use a least
squares orthogonal distance minimization inside of a Random Sample Consensus (RANSAC)
construct. Orthogonal minimization is chosen because we assume error in all three dimensions
of our irregularly spaced data. RANSAC is used so the minimization process is robust to
outliers, which for this fitting would include all sample points not part of the ground. These
points include noise, but also the targets at this stage in the algorithm. Briefly, the RANSAC
procedure is to choose a random subset of the data and to fit a plane to this subset using least
squares. Next, a vote is computed by counting the number of original data points which fall
within some tolerance of the plane. The process is repeated for some number of iterations
using a different random subset of the data each time. Whichever model at the end of the
iterations has the highest vote is chosen as the best fit.
Fig.8.4
Fig.8.5
The 3D plane generated by this robust model fitting is shown superimposed on
the data in Figure 4. All sample points which fall within a small distance from this best fit
plane are removed. Figure 5 shows what is left over. These samples are the target candidates
plus noise; the objects not part of the ground plane. A neighbourhood filtering operation is
applied to the data, whereby at each remaining sample point, the number of neighbouring
points within some radius are determined. If the number of neighbouring points is less than
some small tolerance, the sample point is removed from the dataset. Since we already know
that our spatial sampling rate decreases as a function of range, the neighbourhood search
radius increases as a function of range. Due to this feature, the filter will operate evenly on an
irregularly spaced data such as this. The result of the cleaning filter is shown in Figure 6.
Fig.8.6
At this point we have isolated the sample points which are candidates
for being targets, but how many targets are there? A K-means clustering algorithm is used to
segment and label the candidate samples as separate targets. In statistics and data mining, kmeans clustering is a method of cluster analysis which aims to partition n observations into k
clusters in which each observation belongs to the cluster with the nearest mean. In this case
there are 3 targets detected, the two lamp posts and the man.
9. TARGET TRACKING
When the surface craft detector algorithm is performed on each range image in
a sequence of range images all of the necessary information for target tracking is provided.
The centroids of each target sample cluster can be used in conjunction with a tracking
algorithm to provide robust estimation of the target locations in a noisy environment and in
between measurements. The implementation of the surface
craft detector described in Section III-B and used to generate the result depicted in Figure 7
was written in MATLAB and was run on a low end consumer desktop computer. It was not
optimized for real time performance. However, the algorithm presented is simple enough that
it will function in real time if the range images are streamed at 5 Hz. The locations of each
target on the 2D plane can be considered noisy observations and can be refined by an
estimation filter. Figure 8 shows the results of filtering the noisy measurements with a basic
2D Kalman filter. A more sophisticated tracking algorithm and a higher sample rate would
refine this already useful result.
Fig.8.7
10. APPLICATIONS
Automotive
Autonomous Navigation
Collision avoidance
9
Aviation
Helicopter brownout
Wire detection
Defence
Laser trackers
Marine Robotics
Industrial Space
3D FLVC have been tested on STS-127, STS-133 and STS-134 and are planned for
deployment by SpaceX for Autonomous Rendezvous & Docking on their Dragon
Vehicle
Surveillance
Threat Detection
Object ID
10
11
3D Flash LIDAR camera with3 and 9 degree lenses and 15mJ laser (upper right)
Fig.11.1
ASC participated in NASA's Autonomous Landing and Hazard Avoidance
Technology (ALHAT) program where similar safe landing challenges are faced on a mission
to the Moon. The program's objective is to establish a capability to land near specific
resources that are located in potentially hazardous terrain on a future lunar mission. An active
sensor for measuring the topography of the landing site (3D Flash LIDAR) and terrain
analysis algorithms make this possible. The ALHAT program used field tests and high fidelity
Monte Carlo simulations of a lunar landing to evaluate the capability of these technologies
working in concert.
ASC participated in the COTS Autonomous rendezvous and docking (AR&D)
program. On July 15, 2009, ASCs 3D Flash LIDAR camera was tested on the Endeavor
Space Shuttle mission to the International Space Station becoming the 1st 3D Flash LIDAR in
space and again February 24, 2011 on STS-133 successfully demonstrating 3D Flash LIDAR
for AR&D applications. Combined with appropriate software to interpret the images obtained
from the Flash LIDAR, directions can be given to the spacecraft propulsion systems to guide
the spacecraft to a safe rendezvous and docking. This demonstration was supported by Space
X which is developing a vehicle for delivering supplies to the International Space Station.
12
ASC is enhancing its 3D Flash LIDAR sensors to meet the requirements for
Planetary Entry, Descent and Landing by (EDL) to produce the real-time 3D sensor that will
be deployed on future lunar, asteroid and planetary missions. Working on a Phase II SBIR,
ASC expects its new 3D sensor packaging will survive through 100kRad and be ready for
integration in 2012 to launch aboard upcoming space explorations, according to Advanced
Scientific Concepts, Inc. (ASC). Designed especially for Automated Rendezvous and Docking
(AR&D) and On-orbit Satellite Servicing (OSS), it was most recently tested on the STS-133
Discovery in February 2011 and has been listed as Technical Readiness Level (TRL) of 8.
11.1. DRAGON EYE-THE FIRST 3D FLASH LIDAR IN SPACE
DRAGON EYE
Fig.11.2
13
The Dragon Eye 3D Flash LIDAR camera opens many doors for 3D FLC in
space, making both manned and unmanned AR&D possible.ASC 3D FLC systems can be
found in a wide range of applications, including Automotive, Defence, Surveillance, Robotics
and Aviation. ASC's patented, unique technology brings tremendous value to space-based
applications.
13. CONCLUSION
14. REFERENCES
www.ieee.com
www.wikepedia.com
www.advancedscientificconcepts.com
www.vision-systems.com
R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision, 2nd ed.
United Kingdom: Cambridge University Press, 2003.
14