Obstacles detection in foggy environment

Sahil Dhawan a* and Jagdish Lal Raheja b

b

a
M.Sc,Tech Information Systems,BITS Pilani, Pilani Campus,Pilani,Rajasthan-333031, India
Machine Vision Lab, DSG, Council of Scientific & Industrial Research, CEERI, Pilani, Rajasthan

Abstract
In this paper, we propose a novel method of object detection in bad weather conditions like Fog and smoke. It is based on statistical
model of dark channel prior to estimate the thickness of the haze and physics based image restoration approach. Further applying simple
image enhancement techniques improve the visibility of an obstacle in the line of sight. Using this prior with this imaging model help
recover refined haze free image. Finally, moving objects are segmented by simple Kalman Filter background differencing algorithm.
Experimental results show that our method can be applied for building Intelligent Surveillance System to detect moving objects accurately
due to low scene visibility. Moreover, a high quality depth map can also be obtained as a by-product of haze removal.
Keywords: Foggy environment, image processing, surveillance, visibility, obstacle detection.

1.

is the scene radiance. The performance of vision algorithms

e.g., feature detection, haze removal can produce depth
information and benefit many vision algorithms an
advanced image editing. Haze or fog can be a useful depth
clue for scene understanding.
However, haze removal is a challenging problem
because the haze is dependent on the unknown depth
information [3]. Therefore, many methods have been
proposed by using multiple images or additional
information [4,5,6] for example Polarization methods and
dichromatic scattering model which fall into trouble when
the color of scene point is close to fog color and not to
mention but they are computationally expensive too.
Hence, we employ a simple defogging method using the
concept of dark channel prior [7]. It is based on a key
observation - most local patches in haze-free outdoor
images contain some pixels which have very low intensities
in at least one color channel. Using this prior with the haze
imaging model, we can directly estimate the thickness of
the haze and recover a high quality haze-free image. The
success of these methods lies in using stronger prior or
assumptions. Tan [10] observed that the haze-free image
must have higher contrast compared with the input haze
image. The results are visually compelling but may not be
physically valid. Fattal [4] estimates the albedo of the scene
based on prior that transmission and surface shading are
locally uncorrelated which fails in heavy haze images.

Introduction

Intelligent visual surveillance (IVS) is an active research

topic in computer vision. Similar to other outdoor visual
systems, IVS systems need to segment moving objects in
different environments correctly [1]. However, when it
comes to bad weather such as in foggy days, the scene
visibility is low and as a result it is difficult to detect
moving objects. Images of outdoor scenes are usually
degraded by the turbid medium (e.g., particles, waterdroplets) in the atmosphere. Haze, fog, and smoke are such
phenomena due to atmospheric absorption and scattering
[2]. The degraded images lose the contrast and color
fidelity, as shown in Figure1. Since the amount of
scattering depends on the distances of the scene points from
the camera, the degradation is spatial-variant.
Defogging of images is highly desirable in image
processing and real time surveillance systems. First,
removing haze can significantly increase the visibility of
the scene and correct the color shift caused by the air light.
In general, the haze-free image is more visually pleasuring.
Second, most computer vision algorithms, from low-level
image analysis to high-level object recognition, usually
assume that the input image (after radiometric calibration)
* Sahil Dhawan. Tel: +91-9988530434
Email: sahildhawan05@gmail.com
Jagdish Lal Raheja. Tel: +91-9602212817
Email: jagdish@ceeri.ernet.in

561

with the scene depth d. Obviously, attenuation light is

exponential to distance d. We assume here sky area of
scene is overcast and light intensity changes slowly, which
can both be satisfied in foggy day.
(a)

(b)

Figure 1. Haze Removal using single image. (a) Input haze image (b)
image after haze removal by our approach.

In this paper, we attempt dark channel prior single

image haze removal technique along with histogram
equalization contrast enhancement technique to detect the
obstacle in the path as shown in Figure 1. The dark channel
prior is based on the statistics of haze-free outdoor images.
We find that, in most of the local regions which do not
cover the sky, it is very often that some pixels called dark
pixels have very low intensity in, at least, one colour (rgb)
channel. In the haze image, the intensity of these dark
pixels in that channel is mainly contributed by the airlight.
Therefore, these dark pixels can directly provide accurate
estimation of the hazes transmission. We avoid
computationally expensive technique of soft matting
interpolation to make it ideal for surveillance. We believe
that developing priors from different directions and
combining them will yield impressive results.
In the second step, the scene background is subtracted
and moving objects are detected using background
differencing method. Assuming nonlinear model of scene is
build first, the dynamic characteristics are analyzed and
updated online using Kalman filter. This method enables
fast detection of motion of objects in the scene ideal for
video processing in foggy days.

Figure 2. Haze image formation model

Geometrically, the haze imaging Equation (1) means

that in RGB color space, vectors A, I(x) and R(x) are
coplanar and their end points are collinear shown in Figure
2. The transmission t is the ratio of two line segments:
t(x)=

where c {r, g, b} is color channel index. The transmission

t in a local patch is estimated by maximizing the visibility
of the patch and satisfying a constraint that the intensity of
R(x) is less than the intensity of A. An MRF model is used
to further regularize the result. This approach is able to
greatly unveil details and structures from the haze image.
However, because this approach is based on a
statistically independent assumption, it requires the
independent components varying significantly. Any lack of
variation or low signal-to-noise ratio (e.g., in dense haze
region) will make the statistics unreliable. Moreover, as the
statistics is based on color information, it is invalid for
grayscale images and difficult to handle dense haze which
is often colorless and noisy. To overcome this problem we
restore the output at each step to histogram equalization so
that the variation across all channels is distributed evenly
and the signal-to-noise ratio can be explored further.
The rest of the paper is organized as follows. Defogging
algorithm under Dark Channel Prior and Kalmin Filter
based background maintenance method are presented in
Section 2 and Section 3 respectively. Experimental results
are given and analyzed in Section 4. Finally, Section 5
concludes and presents promising direction of future work.

2. Background
In computer vision, the widely used model to describe
the formation of a haze image is as follows [8,9]:
I(x) = R(x) t(x) + A (1 t(x)),

(1)

where I is the observed intensity, R is the scene radiance, A

is the global atmospheric light, and t is the medium
transmission describing the portion of the light that is not
scattered and reaches the camera. The goal of haze removal
is to recover R, A, and t from I.
The first term R(x) t(x) on the right hand side of
Equation (1) is called direct attenuation, and the second
term A(1 t(x)) is called airlight. Direct attenuation
describes the scene radiance and its decay in the medium,
while airlight results from previously scattered light and
leads to the shift of the scene color. When the atmosphere
is homogenous, the transmission t can be expressed as
t(x) = ed(x) ,

(3)

3. Defogging Method
Assume that the atmospheric light A is given and that
the transmission (x) is constant for a local patch. Taking
min of equation (1) we have ,

(2)

where is the scattering coefficient of the atmosphere. It

indicates that the scene radiance is attenuated exponentially

min y(x) (Ic (y)) = t (x) min y(x)(Rc(y)) + (1-t(x))Ac (4)

562

complexity is linear to image size. The patch size is set to

15 15 for a 600 400 image. Avoiding the soft matting,
we use depth of a scene point acquired by prior knowledge.
It takes about 8ms to process a 320240 pixel image from
single video frame on a PC with a 3.0 GHz Intel Pentium 4
Processor.

Then, we take the min operation among three color

channels on the above equation and obtain
min c ( min y(x) (

)) = t (x) min c( min y(x) (

(1-t(x))

)) +
(5)

According to the dark channel prior, the dark channel

Jdark of the haze-free radiance J should tend to be zero
Jdark (x) = minc ( min y(x) (Jc (y))) = 0

(6)

As Ac is always positive, this leads to

minc ( min y(x) (

)) = 0

(7)

Using equation (7) and (5) we get,

Figure 3. Experiment of Dynamic scene defogging in PETS Dataset

t(x) = 1 minc ( min y(x) (

))

(8)

As can be seen in Figure 3, our approach can unveil the

details and recover vivid color information even in very
dense haze regions. The estimated depth maps are sharp
and consistent with the input images.
The colors of defogged by [10] result are often over
saturated, since that algorithm is not physically based and
may underestimate the transmission. Our method recovers
the structures without sacrificing the fidelity of the colors
(e.g., car). The halo artifacts are also significantly small in
our result.

this provides the dark channel of normalized haze image

. It directly provides the estimation of transmission.
Note since the sky is at infinite and tends to have zero
transmission, therefore it is valid to assume
min c ( min y(x) (

)) 1, and t(x) 0,

(9)
5. Experimental Results and Conclusion

hence Equation (8) accounts for both sky regions and

non-sky regions. Avoiding soft matting refinement, we find
some scene points with some depth using prior knowledge
about the scene. Through simple algebraic manipulation,
we obtain the depth of other points in the scene.

In our experiments, we performed both outdoor and

indoor to test the validity of our algorithm for visual
systems. The depth maps are computed up to an unknown
parameter . It can be seen in Figure 4 that with our
approach the edges and the profile of the obstacles in dense
fog is easily visible. The method preserves vivid color
information and successfully unveils major details in all
types of fog. The depth maps obtained are consistent with
the input images too.

(10)

4. Comparison
In order to testify the effectiveness and practicality of
our method, we carry out experiments under different
situations. In this section, our defogging method is first
employed to defog static and dynamic scenes. Then the
efficiency of the background maintenance algorithm is
verified. Finally, our moving object detection method is
used to subtract moving objects from a foggy day video. A
detailed picture below gives a comparison between the
methods we have applied and the traditional methods
followed. In our experiments, we perform the local min
operator using Marcel van Herks fast algorithm whose

Figure 4. Experiment of Static Scene defogging in FRIDA Image Dataset

The atmospheric light is automatically estimated along

with the dark channel prior approach, and then the output is
input to image enhancement techniques for edge detection

563

which makes the edges of the objects prominent. Using

standard techniques of filtration and histogram equalization
we obtained the results above. Our method recovers
structures without sacrificing the fidelity of colors and
works pretty well for both distant and near objects. The
halo artifacts are also significantly small in our result. Our
method even works for the gray scale images if there are
enough shadow regions in the image.
5.1. Moving Object Detection
Figure 5. Comparison with Fattals work on defogging

Building accurate background model is key step of

background subtraction. Thanks to Gaussian nonlinear state
function and measurement function to build as:

As it can be clearly seen on comparison with Fattals

work in Figure 5, from above that our defogging result
produces smooth edge detection of buildings and other
obstacles far away in the scene. The defogging intensity is
also slightly better in our case with better visual
appearance.
Since the dark channel prior is a kind of statistic, it may
not work when the scene objects are inherently similar to
the atmospheric light and no shadow is cast on them. The
haze imaging model may be invalid. We intend to study
and investigate these models in future and improve upon
priors in all directions. Improving segmentation of moving
objects under rainy and snowy weather is left as our future
work.

The variables have standard meanings here. To detect

changes quickly, dynamic characters are analyzed on two
levels, frame to frame and background differencing level.
Pixel movement matrix updated online can detect
background abrupt changes when reduced to zero. In such a
case, background value is set to pixel value in most recent
frame, as
This avoids ghost problem as
well. We use prior knowledge to decide threshold.
Our contribution can be summarized as (i) dark channel
prior to estimate the transmission. We show that this
approach is effective and improves scene visibility. (ii)
novel background differencing algorithm with online
parameter update. We show that it adapts to dynamic
change of background quickly, ideal for foggy day video
processing. (iii) Simplicity of the computational procedure,
which is appropriate for real time visual surveillance
systems.

References
[1] Gong Chen, Heqin Zhou, Jiefeng Yan. A Novel Method
for Moving
Object Detection in Foggy Day. IEEE 2007.
[2] P. Chavez. An improved dark-object substraction technique for
atmospheric scattering correction of multispectral data. Remote
Sensing of Environment, 24:450479, 1988
[3] Cozman.F and Krotkov.E, Depth from Scattering, In Proceedings for
IEEE International Conference on CVPR, San Juan, Puerto Rico,
1997, pp.801-806.
[4] R.Fattal. Single image dehazing. In SIGGRAPH, pages 1-9, 2008.
[5] E. B. Goldstein. Sensation and perception. 1980.
[6] E. Hsu, T.Mertens, S. Paris, S. Avidan, and F. Durand. Light mixture
estimation for spatially varying white balance. In SIGGRAPH, pages
17, 2008.
[7] Kaiming He, Jian Sun, Xiaoou Tang. The Chinese University of Hong
Kong. Microsoft Research Asia. Single Image Haze Removal Using
Dark Channel Prior. In CVPR, 2009.
[8] S. G. Narasimhan and S. K. Nayar. Chromatic framework for vision in
bad weather. CVPR, pages 598605, 2000.
[9] S. G. Narasimhan and S. K. Nayar. Vision and the atmosphere. IJCV,
48:233254, 2002.
[10] R. Tan. Visibility in bad weather from a single image. CVPR 2008.

564