IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 66, NO.
10, OCTOBER 2017 2585
Steel Surface Defect Detection Using a New
Haar–Weibull-Variance Model in
Unsupervised Manner
Kun Liu, Heying Wang, Haiyong Chen, Erqing Qu, Ying Tian, and Hexu Sun, Senior Member, IEEE
Abstract— Automatic defect detection on the steel surface is
a challenging task in computer vision, owing to miscellaneous
patterns of the defects, low contrast between the defect and
background, the existence of pseudo defects, and so on. In this
paper, a new Haar-Weibull-variance (HWV) model is proposed
for steel surface defect detection in an unsupervised manner.
First, an anisotropic diffusion model is utilized to eliminate
the influence of pseudodefects. Second, a new HWV model
is established to characterize the texture distribution of each
local patch in the image. The proposed model can project
the texture distribution of each patch into the low-dimensional
space with only two parameters. The parameter distribution
of the whole image can also be unified into the form of
linear radiation in an Euclidean space. The reliable background
can be extracted via the formation of parameter distribution,
by which the model parameter can be optimized further. Finally,
the adaptive threshold can be determined to distinguish the
defect from the background. Experimental results show that
the proposed method can detect an arbitrary type of defect
on the homogeneously textured surface and achieve an average
detection rate of 96.2% on the data set, which outperforms the
previous methods.
Index Terms— Defect detection, Haar wavelet coefficient,
unsupervised, variance, Weibull model.
I. I NTRODUCTION
S URFACE quality inspection is one of the important
aspects of industrial production. As one of the main
products of iron and steel class, the steel strip accounts for
Manuscript received December 11, 2016; revised April 13, 2017; accepted
May 14, 2017. Date of publication July 18, 2017; date of current version
September 13, 2017. This work was supported in part by the National Natural
Science Foundation of China under Grant 61403119 and in part by the
Natural Science Foundation of Hebei Province under Grant F201402166. The
Associate Editor coordinating the review process was Dr. Sergey Kharkovsky.
(Corresponding Author: Ying Tian.)
K. Liu, H. Wang, and H. Chen are with the School of Control Science
and Engineering, Hebei University of Technology, Tianjin 300130,
China (e-mail: liukun03@mails.thu.edu.cn; wangheying215@163.com;
haiyong.chen@hebut.edu.cn).
E. Qu is with Tangshan Iron and Steel Company Ltd.,
Tangshan 063000, China, and also with the School of Control Science
and Engineering, Hebei University of Technology, Tianjin 300130, China
(e-mail: querqing@126.com).
Y. Tian is with the School of Mechanical Engineering, Tianjin University,
Tianjin 300072, China, and also with the Key Laboratory of Equipment Design
and Manufacturing Technology, Tianjin University, Tianjin 300350, China
(e-mail: tianying09@163.com).
H. Sun is with the School of Control Science and Engineering, Hebei
University of Technology, Tianjin 300000, China, and also with the School
of Electrical Engineering, Hebei University of Science and Technology,
Shijiazhuang 050000, China (e-mail: hxsun@hebut.edu.cn).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TIM.2017.2712838
0018-9456 © 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
more than 80% of all products, which has become one of
the most important materials in aerospace, machinery, and
automobile industry. The surface quality of strip steel is an
important performance indicator. However, it is inevitable for
the surface of strip steel to contain different kinds of defects,
such as massive rupture, scratch, and rolled in scale. The
defects affect the appearance of the steel and reduce the
properties of the product, such as corrosion resistance and
fatigue strength.
In the past, the defect inspection was completed by human
vision. However, it is becoming more and more difficult for
human vision to meet the requirement of quickness, reliability,
and robustness. Thus, nondestructive testing technique (NDT)
has emerged in recent years. Qaddoumi et al. [1] has proposed
a novel near-field microwave NDT, which was employed to
detect defects in nonceramic insulators. Foudazi et al. [2], [3]
have applied the active microwave thermography technique to
detect the defects on steel surface and carbon-fiber-reinforced
polymer-strengthened cement-based materials successfully. In
this paper, we mainly discuss the detection approach based on
the computer vision technology. It facilitates the collection of
surface quality information and finds the relevant failures in
time during the production process. Defect detection methods
can be classified into the supervised and the unsupervised
manner. Supervised methods mainly rely on an adequate and
representative set of training data [4]–[6]. However, in rare
cases, there exists a complete set of defect measurements in
an actual production environment. In this case, unsupervised
methods based on anomaly detection can be of great practical
value, which can locate the abnormal regions by evaluating
and comparing the local patches in the image [7], [8].
Generally, current anomaly detection methods can be
classified into three groups: statistical-, spectral-, and model-
based techniques. Historically, statistical approaches were used
to analyze texture by evaluating the spatial distribution of pixel
intensities. In those approaches, the defects were detected by
using the first-order statistics, such as mean-, variance-, and
histogram-based computations together with the second-order
statistics based on the cooccurrence matrix. Reference [9]
utilized the histogram of gray information to detect the surface
defects of an aluminum disk. The local binary pattern (LBP)
was applied to achieve real-time defect detection of metallic
surfaces [10], [11]. However, it is sensitive to noise and
has a limited ability to represent miscellaneous textures.
Song and Yan [12] proposed a robust feature descriptor
against noise named the adjacent evaluation completed
2586 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 66, NO. 10, OCTOBER 2017
LBPs (AECLBP). The second-order statistics based on the
cooccurrence matrix were applied to a visual inspection
system [13], [14] with a high computational load.
Tsai et al. [15] proposed a measure based on the weighted
covariance matrix to detect the defects on homogeneously
textured surfaces, such as LCD panel and deer-skin leather.
And the integral image technique was also applied to reduce
the computation complexity.
Spectral approaches imply a bank of filters applied to the
image, in which the energy of the filter responses is utilized
as a feature to distinguish the defects from the background.
The filter operation is often done by transforming the image
into a frequency domain. Li and Tsai [16] developed the
Fourier image reconstruction technique for the solar cells
inspection. Aiger and Talbot [17] proposed the phase only
transform, individually algorithm, which utilized only the
phase information of the Fourier transform to detect the
defects on periodic textured surfaces. Choi and Kim [18]
proposed a two-step approach (TS) to extend the PHOT
method by increasing a phase of local refinement. Over-
all, the methods based on the Fourier transform are global
approaches, which lack the capability for identifying local
features. The methods based on the Gabor transform give
consideration to the local texture property besides the global
statistical property. Bi et al. [19] applied the Gabor transform
in the multiple fixed orientations and scales to detect the
defects in a liquid crystal display. In addition, the wavelet
transform can describe the texture flexibly and can auto-
matically adjust the spatial resolution according to the fre-
quency characteristic. The wavelet transform was utilized to
describe the texture of the semiconductor wafer dies [20].
Li and Tsai [21] extracted features by the wavelet coefficients
to identify defects in the solar wafers successfully.
Model-based approaches can achieve a better detection
performance by the texture analysis if the prior knowledge
about the texture pattern is available. Gan and Zhao [22]
proposed a modified local binary fitting model for LCD
detection, in which the initial contour needed to be preset.
Xu and Huang [23] provided the Gaussian Markov ran-
dom field for assessing the quality of nanomaterial and
Gu et al. [24] proposed a new Mumford–Shah model for multi-
phase image segmentation. Besides, recent studies have shown
that Weibull distribution can accurately model a wide variety
of natural and handmade images [25]. Anomaly detection on
the textured surface, such as textile, stone, or wood, has been
realized based on the Weibull model [26]. Two parameters
of a Weibull fit were estimated for the distribution of image
gradients in local patches. By taking the mean or median of
all samples in the Weibull space as the reference point and
evaluating the distance of each sample to the reference point,
the defective regions were detected. The method is mainly
applicable to the subtle defect detection.
In this paper, we present a new defect detection algorithm
based on the Weibull model. Different from the method in [26],
the Haar feature is utilized to replace the local gradient
magnitude, which can extract more complete texture infor-
mation besides the edge response. More importantly, a new
Haar–Weibull-variance (HWV) model is developed to
characterize the stochastic texture distribution. By the pro-
posed model, the parameter distribution on all local patches
can be unified into the form of linear radiation in an Euclidean
space. By taking the origin as the reference point in the
proposed model space and evaluating the distance of each
sample to the origin, the defects can be located accurately.
Our approach is independent of any defect type.
Briefly, the main contribution of this paper is to develop a
new model-based approach to detect the steel surface defects.
The advantages of the proposed approach are shown as
follows.
1) An arbitrary type of defect on the homogeneously
textured surface can be detected, including subtle and
miscellaneous defects, even under the low contrast
condition.
2) The proposed model can project the stochastic texture
distribution to the low-dimensional space with compact
representation.
3) No massive labeled images and exhaustive training data
are required, and the adaptive threshold can be set for
each image.
The remaining part of this paper is organized as follows.
Section II presents the new defect detection scheme concretely.
Section III gives the experimental results on the defective
images collected from the practical production line and the
comparison with other methods. Last, the conclusion is given
in Section IV.
II. D EFECT D ETECTION S CHEME
The proposed defect detection scheme includes four
steps, which are image preprocessing, building the original
HWV (O-HWV) model on the local patches in the image,
modifying and optimizing HWV model, and locating defects
by an adaptive threshold. First, the anisotropic diffusion
process is implemented to eliminate the effect of the pseudode-
fects in the background. Second, the Haar feature is extracted
from local patches on which the Weibull fit is computed. Third,
the O-HWV model can be built by combining the Weibull
parameter with variance of each local patch after normaliza-
tion. Next, the HWV model is optimized after locating the
reliable defect-free patches. Finally, the defects can be detected
by evaluating the distance of each sample to the origin in the
HWV space. The total framework is shown in Fig. 1.
A. Anisotropic Diffusion Processing
Due to the rough background texture and the similar texture
of local defects to the background, there are some pseudo-
defects within the background. To avoid the disturbance of
the pseudodefects, an effective mechanism must be applied
to smooth the background and preserve the defects simul-
taneously. The anisotropic diffusion model was introduced
by Perona and Malik [27] in image processing for scale-
space description and edge detection. Here, it is utilized
to suppress the interference of the pseudodefects within the
background. The defect-free area can be smoothed, while the
defective region can be preserved by assigning different diffu-
sion coefficients according to the distribution of the intensity.
LIU et al.: STEEL SURFACE DEFECT DETECTION USING A NEW HWV MODEL IN UNSUPERVISED MANNER
Fig. 1. Total framework of the proposed method.
The anisotropic diffusion model is given by
1  x
4
It +1 (i, j ) = It (i, j ) + ct (i, j ) · ∇ Itx (i, j )
4 
x=1
where t refers to the number of iterations, I0 (i, j ) repre-
sents the input image, It (i, j ) is the intensity at the coor-
dinates (i , j ) of input image at the tth iteration, ∇ Itx (i, j ),
x = 1, . . . , 4 represents the gradient of intensity in four-
connected directions, and ctx (i, j ) represents the diffusion
coefficient which has a negative correlation with ∇ Itx (i, j ),
that is
    2 
ctx (i, j ) = g ∇ Itx (i, j ) = 1/ 1 + |∇ Itx (i, j )|K
where K acts as an edge strength threshold and it is a constant.
The diffusion coefficient will take lower value at high-gradient
edges and take higher value within the isotropic background.
B. Building the HWV Model
1) Haar Feature Extraction: In [26], the Weibull model
was utilized to evaluate the stochastic texture by performing
a Weibull fit to the distribution of local gradient magnitude
obtained by a Gaussian derivative filter. However, it is unsuit-
able for the defects with gradual intensity or with the low
contrast to the background, since there is no noticeable edge
response in the local range. Here, Haar feature is utilized to
replace the local gradient magnitude, which can enhance the
defective regions globally and extract more complete texture
information besides the edge response.
In this paper, we use the Haar decomposition coefficient
as the texture feature. The process of obtaining Haar feature
set is performed as the following steps. First, a low-pass filter
combined with a high-pass filter is applied to each row of the
steel image, respectively. Then, the similar filter is applied to
each column in the above resulted image. Four subimages can
2587
be obtained by different combinations of high-pass and low-
pass filter on row and column. For an image of size M × N,
 the row transformation can be obtained by
(1)
√
I R (i, j ) = [I (i, 2 j ) + I (i, 2 j + 1)]/ 2 √ (3)
I R (i, j + (N/2)) = [I (i, 2 j ) − I (i, 2 j + 1)]/ 2
where I (i, j ) is the intensity of the input image, I R (i, j ) is the
row-transformed result, i ranges from 0 to M −1, and j ranges
from 0 to (N/2) − 1. Next, the column transformation can be
obtained by
 √
IC (i, j ) = [I R (2i, j ) + I R (2i + 1, j )]/ 2 √ (4)
IC (i + (M/2), j ) = [I R (2i, j ) − I R (2i + 1, j )]/ 2
(2)
where IC (i, j ) is the column-transformed result of I R (i, j ),
i ranges from 0 to (M/2) − 1, and j ranges from 0 to N − 1.
By the similar transformation, the four subimages can be
defined as
⎧
⎪
⎪ A(i, j ) = IC (i, j )
⎨
D1 (i, j ) = IC (i, j + (N/2))
(5)
⎪
⎪ D2 (i, j ) = IC (i + (M/2), j )
⎩
D3 (i, j ) = IC (i + (M/2), j + (N/2))
where A(i, j ) is the approximate coefficient matrix of the
given image, D1 (i, j ), D2 (i, j ), and D3 (i, j ) are the hori-
zontal, vertical, and diagonal coefficient matrix individually,
i ranges from 0 to (M/2)−1, and j ranges from 0 to (N/2)−1.
For a given defective image with an M × N size, the approx-
imate, horizontal, vertical, and diagonal coefficient matrix are
shown in Fig. 2(b)–(d), which get the size of (M/2) × (N/2)
individually. It can be observed that the approximate coeffi-
cient matrix includes most energy of the image and it can
enhance the defective regions globally. In addition, a given
image can be decomposed into different resolution levels to
extract Haar feature set. Here, we choose the approximate
coefficient matrix in one-level decomposition as the original
feature.
2588 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 66, NO. 10, OCTOBER 2017
Fig. 2. Haar-decomposed results in one-level for a given image. (a) Input
image. (b) Approximate coefficient matrix. (c) Horizontal coefficient matrix.
(d) Vertical coefficient matrix. (e) Diagonal coefficient matrix.
2) Weibull Parameter Extraction: In recent studies, the
Weibull distribution has been utilized to model various types
of signals, such as stochastic textures [28], radar [29], and
sonar [30]. The Weibull model can yield a spectrum of
distributions, including the exponential, Gaussian, and Raleigh
distribution by adjusting model parameters. For the texture
obeying random distribution, Weibull parameters can build a
complete orthogonal basis, which can depict image contrast,
roughness, orientation, scale, and shape effectively. The para-
meterized Weibull distribution is given with the probability
density function as follows:
colors. From the parameter distribution in the Weibull space,
we can find that the scale parameter in the Weibull model is
mainly related to brightness of the given patch. The region
with lower brightness will generate a smaller scale parameter.
Meanwhile, the shape parameter in the Weibull model is
mainly related to the richness of the textures, such as contrast,
roughness, and orientation. The region with richer texture
generates a smaller shape parameter.
Although it seems reasonable to analyze the texture of
the steel surface by Weibull fit, it is difficult to locate the
defects accurately by the Weibull model. From the results in
Fig. 3(d1) and (d3), it is observed that there exists a blurry
boundary between the defective and defect-free samples in the
Weibull space.
3) Weighting With the Variance Information: To address
the above problem, we introduce the variance parameter to
the Weibull model. To some extent, variance represents the
uniformity or richness of the texture within a region, which
has a similar meaning to the shape parameter in the Weibull
model. Since the defective regions contain richer texture than
the homogeneous background in most cases, the variance
of the defective region is larger than that of the defect-free
region. Therefore, by weighting the shape parameter with
the variance, the interclass distance between the defect and
the background can be increased further. Furthermore, the
parameter distribution of the whole image can be transformed

 β
β I (x) β−1 − I (x) into a uniform form for different types of defects by weighting
f α,β (I (x)) = e α
(6) the scale parameter with the variance in the similar way.
α α
Before the weighting operation, the scale, shape, and vari-
where I (x) is intensity of the given image, α > 0 represents ance parameter must be normalized, since they may be of a
the scale parameter, and β > 0 represents the shape parameter. different scale. Particularly, the variance parameter is mapped
We model the texture distribution of each Haar-decomposed inversely to preserve a relatively consistent relationship with
patch in the given image by determining Weibull parameters shape parameter in the Weibull model. Then, the normalization
with maximum likelihood estimation. The maximum likeli- operation is performed as follows:
hood estimation of α and β can be derived by
n αci = (αi − αmin )/(αmax − αmin ) (9)
1
n β
1 i=1 I (x i ) ln I (x i ) βci = (βi − βmin )/(βmax − βmin )
+ ln I (x i ) = n β
(7) (10)
β h i=1 I (x i )
i=1
 n  σci = (σmax − σi )/(σmax − σmin ) (11)
1 
αβ = I (x i )β (8) where αi , βi , and σi (i = 1, 2, . . . , n) are the scale, shape,
h and variance extracted from the i th Haar-decomposed patch,
i=1
respectively, n is the total number of patches in the given
where h is the number of pixels in each patch.
image, α, β, and σ are parameter matrices formed by αi , βi ,
Several defective and defect-free patches are selected ran-
and σi individually, αmin , βmin , and σmin are the minimum
domly from several defective images containing the oil
values of the corresponding parameter matrices, αmax , βmax ,
stain, white dot, roll marks, and pitted surface individually
and σmax are the maximum values of the corresponding
in Fig. 3(a). The corresponding probability density curves by
parameter matrices, and αci , βci , and σci are the normalized
Weibull fit on these patches are shown in Fig. 3(b). As a
values of αi , βi , and σi , which are mapped to the range [0,1].
result, the probability density curves of the defect-free patches
Fig. 4 explains the normalization procedure for a given
own similar sharp peaks and overlap each other, which are
image that is divided into 5 × 5 overlapping patches. Scale,
noticeably different from those of the defective patches. At the
shape, and variance parameter can be extracted from each local
same time, the curves of defective patches are different from
patch to form the corresponding matrices α, β, and σ . αc , βc ,
each other.
and σc represent the parameter matrices after normalization.
To analyze the statistic characteristic of Weibull parameters
After the normalization, a new HWV model can be built
further, the Weibull space determined by the scale and shape
by weighting the scale, shape parameter with the variance
parameter on all local patches is built. For three defective
parameter according to
images in Fig. 3(c1)–(c3), the corresponding Weibull spaces  
are shown, respectively, in Fig. 3(d1)–(d3), in which the αi = ξ αci + ζ σci
(12)
defective and defect-free samples are labeled with different βi = ξβci + ζ σci
LIU et al.: STEEL SURFACE DEFECT DETECTION USING A NEW HWV MODEL IN UNSUPERVISED MANNER
Fig. 3. Statistic results of the Weibull distribution. (a) Four defective images and selected patches. (b) Corresponding Weibull parameters and probability
density curves, in which the defective patches are labeled as red and defect-free patches are labeled as other colors. (c1)–(c3) Three defective images.
(d1)–(d3) Corresponding Weibull spaces, in which the red points correspond to the defective patches and the blue points correspond to defect-free patches.
Fig. 4. Normalization process for a given image.
where αi is the weighted scale (w-scale) parameter and βi
is the weighted shape (w-shape) parameter for the i th local
patch, ξ and ζ are the weights, and ξ + ζ = 1.
The HWV space determined by the w-scale and w-shape
parameters on all sliding patches is shown for several given
images in Fig. 5(a1)–(a6). In the HWV space, two clusters are
formed by defective and defect-free samples, which are totally
away from each other. Furthermore, the parameter distribution
of HWV model can be unified into the form of linear radiation
for different defect types. The defective samples are closer
to the origin, whereas the defect-free samples are further
away from the origin in the HWV space. In a few cases,
2589
there still exists a blurry boundary between the two clusters.
For example, for the defects with low-contrast or containing
similar texture with the background in Fig. 5(a5) and (a6),
the two clusters overlap each other to some extent. In order
to detect an arbitrary type of defect, the HWV model will
be modified and optimized further. For a clear description,
the above-mentioned model is named the O-HWV model.
C. Modifying HWV Parameters
The initial assumption in the HWV model is that the defec-
tive regions involve richer texture than the background, which
cannot be applicable to all defects. There exist some defects
with different intensities from the background, whereas the
texture distributions are similar to the background. Therefore,
we expect to find a parameter for each local patch, which can
represent the intensity deviation with the background besides
describing the texture distribution.
To achieve the goal, we update the variance of each local
patch by referring the intensity of the background. In this way,
the defective patches will get larger variance than the defect-
free patches in all cases. Concretely, the reliable background
patches can be extracted by selecting the farthest sample from
the origin in the O-HWV space. To avoid the disturbance,
multiple samples close to the farthest sample are collected
and the average intensity of the corresponding patches m is
computed by
⎛ ⎞
 c s/2 
s/2
m= ⎝ A g (x, y)⎠ (c(s/2)2 − 1) (13)
g=1 x=1 y=1
2590 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 66, NO. 10, OCTOBER 2017
Fig. 5. Parameter distribution in the O-HWV space. (a1)–(a6) Input images. (b1)–(b6) Corresponding parameter distributions, in which red points represent
the defective patches and blue points represent the defect-free patches.
where c is the number of the collected samples, A g is the Haar
feature of the gth background patch (g = 1, . . . , c), (x, y)
denotes the pixel coordinates in the given patch, and s is the
size of each patch.
Applying the above mean m, the updated variance parameter
is computed on all local patches in the image by

s/2 
s/2
σi = (Ai (x, y) − m)2 ((s/2)2 − 1) (14)
x=1 y=1
δi = (1/2) log σi
where Ai is the Haar feature of the i th local patch in the image,
σi is the updated variance parameter, and δi is the logarithmic
transformation of σi .
Next, the normalization and weighting operation on all
local patches are performed again to get the modified HWV
(M-HWV) model. The functions are formalized using
δci = (δmax − δi )/(δmax − δmin )
 
αi = ξ  αi + ζ  δci
βi = ξ  βi + ζ  δci
where δci is the normalized value of δi on the i th local patch,
which is in the range of 0 and 1, δ is the parameter matrix
formed by δi (i = 1, 2, . . . , ), δmax and δmin are the maximum
and minimum values of updated variance parameter matrix,
ξ  and ζ  are the weighted coefficients, ξ  + ζ  = 1, and
αi and βi represent the modified scale and shape parameter,
which are abbreviated as m-scale and m-shape individually.
To verify the performance of the proposed model, the
comparison of parameter distribution by the Weibull model,
O-HWV model, and M-HWV model is accomplished. For
several representative defective images, the parameter distri-
butions in the Weibull space, O-HWV space, and M-HWV
space are shown individually in Fig. 6. From the results, it can
be observed that each cluster can get a smaller within-class
distance in the O-HWV space than in the Weibull space.
A uniform distribution pattern with a fixed reference point
(i.e., the origin) is formed for different defect types. Further-
more, for the defects with low-contrast or containing similar
texture with the background in Fig. 6(e) and (f), there exists
a larger gap between two clusters in the M-HWV space than
in the other spaces.
Algorithm 1 Feature Extracting Process of HWV Model
Initialize:
Pre-processing of the anisotropic diffusion for image I0 .
Choice patch size:
Patches with a size of s × s pixel and an overlap of 50%.
(15) Build HWV model:
For the i th patch:
1) Extract the Haar feature Ai ;
2) Obtain parameters αi , βi and σi ;
3) Calculate original HWV parameters αi and βi after
normalization.
Modify HWV parameters:
1) Compute the value m of the background;
(16) For the i th patch:
2) Compute the updated variance parameter σi ;
(17)
3) Assign the new feature δi to the current patch, let δi =
(1/2) log σi ;
4) Obtain modified HWV parameters αi and βi ;
5) Measure the Euclidean distance di between (αi , βi ) and
the origin.
Determine the adaptive threshold.
D. Defect Detection Based on Adaptive Threshold
By the M-HWV model, the parameter distribution can be
unified into the form of linear radiation for different defect
types. The defective samples are closer to the origin in the
M-HWV space, whereas those of defect-free patches are
further away from the origin. There exists a larger gap between
two clusters representing defective and defect-free region.
Intuitively, by evaluating the distance of samples with the
origin in the M-HWV space, the defective regions can be
located.
LIU et al.: STEEL SURFACE DEFECT DETECTION USING A NEW HWV MODEL IN UNSUPERVISED MANNER
Fig. 6. Comparison of the parameter distributions. (a) six defective images. (b)–(f) Corresponding parameter distributions in the Weibull space,
O-HWV space, and M-HWV space, in which the samples of defective patch are labeled as red and the samples of defect-free patch are labeled as blue.
In order to distinguish the defective patches from the defect-
free patches, the adaptive threshold is obtained in our appli-
cation. Here, we apply Otsu’s algorithm based on the cluster
theory to determine the threshold. The traditional Otsu’s algo-
rithm is used to gain optimal image segmentation result by
maximizing the gray-level difference between the objects and
the background. Based on the Otsu theory, we maximize the
between-class distance between the defect and the background
after evaluating the distance of the samples with the origin in
the M-HWV space. The proposed method can get the optimal
threshold and defective regions can be located accurately.
In summary, the procedure of our algorithm can be
described in Algorithm 1.
III. E XPERIMENTAL R ESULTS
A. Defect Detection System
This section presents the total framework of steel surface
inspection system. The inspection system consists of the image
collection device, light source, and conveyor unit. The image
acquisition system is one of the important subsystems, which
can guarantee that the clear defective images are collected
in a well-defined manner. There are two groups of cameras
installed above and below the steel surface, respectively. Each
group contains two cameras for real-time image acquisition.
The cameras used here are the monochrome linear array
camera made by DALSA, with 2048 × 2 pixel resolution and
2591
36-kHz horizontal scanning frequency. The field of view of
each camera is 1000 mm. There exists an overlap of 120 mm
between the fields of view of two cameras. The distance
between the camera and the surface of strip steel is 1130 mm.
The width of the strip steel is 1748 mm. The LED bar light
illuminates the steel surface in the frequent flash mode, which
can ensure the collection of clear and high-contrast images.
The conveyor unit utilizes the combination of multiple
rollers, which can increase the thread to avoid the equipment
disjoint in some emergent cases and increase the tension to
avoid the fluctuation caused by long-distance transmission.
The sheet thickness of the steel varies from 1.5 to 10 mm.
According to the required thickness, the rolling speed of the
steel plate ranges from 3 to 20 m/s. The image processing
unit is performed by the high-performance server. It contains
2 × 1.6 GHz processor with 48-GB memory, which can
satisfy the demand of high-speed image processing. Parts of
the hardware system and the processing procedure are given
in Fig. 7.
B. Data Set and Parameter Setting
In order to verify the performance of the proposed method,
we collect the defective images captured by industrial charge-
coupled device cameras on the actual production line in Han-
dan Steel Factory. The data set includes almost 100 000 defec-
tive images. The defects can be divided into different types
2592 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 66, NO. 10, OCTOBER 2017
Fig. 7. Hardware system and the detection processing. (a) Parts of the hardware. (b) Procedure of defect detection.
Fig. 8. Some representative defective images. (a1)–(a7) Block-type defects.
(b1) and (b2) Line-type defects. (c1) Dot-type defects. (d1)–(d4) Embedded-
type defects. (e1) Gradual-type defects.
according to the texture and geometric property. Block-type
defects have complicated and miscellaneous patterns, such as
massive rupture, drops tar, crush, floating, oil stain, and roll
marks. Line-type defects are thin, such as scratch. Dot-type
defects are subtle, such as white dot. Embedded-type defects
have no noticeable edge, such as pitted surface and mountain.
For the gradual-type defects such as shadow, there exists no
noticeable intensity change within the local regions. Some
representative defective images are shown in Fig. 8. Here,
1200 defective images, including different types, are used to
verify the performance of the proposed method.
In the stage of image preprocessing, we have to determine
two parameters: the number of iterations t and the diffusion
coefficient K for the anisotropic diffusion model. If K or t is
set too large, the diffusion process will result in a blurred
image. Conversely, if K or t is set too small, the diffusion
process will stop smoothing in earlier iterations. The optimal
performance is achieved when t is 5 and K is 10.
The size of local patches needs to be determined before
building the model, which influences the accuracy of defect
detection. Furthermore, it is necessary to determine the overlap
of local patches in order to avoid border artifacts. The patch
size s is fixed at 16 with an overlap of 50% for a compromise
with the computational efficiency.
Besides, the two groups of weights need to be determined,
which include ξ and ζ in the O-HWV model and ξ  and ζ 
in the M-HWV model. For the O-HWV model, the constraint
is that the weighted coefficient ζ is larger than ξ , which can
guarantee the uniform linear radiation formation for different
kinds of defects in the Euclidean space. Here, αi is set to
0.3 and ζ is given by 0.7. In the M-HWV model, the variance
of each patch is updated in order to detect the defective patches
containing a similar texture to the background. ξ  and ζ  are
utilized to weight the w-scale and w-shape parameter with the
updated variance. Since the intensity of the background is not
completely uniform, the updated variance cannot account for
a too large proportion in the M-HWV model. Here, αi is set
to 0.8 and ζ  is given by 0.2.
C. Performance Estimation
In this paper, our method is compared with the TS [18],
Gabor [19], AECLBP [12], and Weibull-based (WB) meth-
ods [26] on the given data set. According to [18], [19],
and [26], the optimal threshold is set manually for the WB,
TS, and Gabor methods. As an illustration, Fig. 9 shows
six representative defective images and the corresponding
detection results. The original images are shown in the first
column and the corresponding results given from the second
column to seventh column, respectively.
The spectral-based TS method can detect defects by sup-
pressing the periodic texture of the background. The detection
results are shown in Fig. 9(b1)–(b6), from which it is observed
that the TS method can get better detection results for defec-
tive regions with noticeable edge response, such as shown
in Fig. 9(b2) and (b6). However, for the defects with regular
texture such as the gradual-type defect in Fig. 9(a1), the line-
type defect in Fig. 9(a5), and the internal defect in Fig. 9(a3),
it achieves poor performance. It can be explained by the fact
that the regular texture of the defects may be considered as
periodic signal to some extent, which is suppressed during the
LIU et al.: STEEL SURFACE DEFECT DETECTION USING A NEW HWV MODEL IN UNSUPERVISED MANNER 2593

Fig. 9. Comparison of the proposed method with traditional methods. a(1)–a(6) Six defective images. b(1)–b(6) Detection results of the TS method.
c(1)–c(6) Detection results of the Gabor method. d(1)–d(6) Detection results of the AECLBP method. e(1)–e(6) Detection results of the WB method. f(1)–f(6)
Detection results of the O-HWV based method. g(1)–g(6) Detection results of the M-HWV based method. The defective regions are labeled as red points or
blocks.

detection process. In addition, false positives are detected at
the same time owing to the disturbance of the regular texture.
The wavelet-based Gabor method can be used to detect
defects by filtering the given image in different directions.
As suggested in [19], there are several parameters selected as:
a mask size of 11 × 11 pixels, a central frequency of 1/8, and
eight directions started from 0° to 180° with the 22.5° interval.
The detection results are shown in Fig. 9(c1)–(c6). For the
Gabor method, the number of red points representing the
defective pixels decreases when compared with the TS method.
In other words, the Gabor method produces less false positives.
The Gabor method mainly detects the edges of the defects
with high contrast, which results in more false negatives,
as shown in Fig. 9(c2). In addition, in terms of the defects with
low contrast to the background, the edge cannot be detected
effectively, such as shown in Fig. 9(c3).
The AECLBP method is a supervised method, which
extracts the texture features on local patches. Based on
the relationship among the central pixel, its neighbors,
and adjacent evaluation windows of each local region,
three operators are established, including AECLBP_Sign,
AECLBP_Magnitude, and AECLBP_Center. In this exper-
iment, the size of local patch is 5 × 5 and the size of
adjacent evaluation window is 3 × 3. The training data
2594 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 66, NO. 10, OCTOBER 2017
TABLE I
C OMPARISON OF P ERCENTAGE P RECISION , R ECALL , AND F -M EASURE OF D EFECT D ETECTION ON 12 D IFFERENT
T YPES OF D EFECTS U SING O UR M ETHODS AND T RADITIONAL M ETHODS
set consists of patches with a 25 × 25 pixel size, which
includes 3600 defective patches with 12 types of defects
and 1600 defect-free patches. The classifier used here is the
support vector machine. The final detection results are shown
in Fig. 9(d1)–(d6). When compared with the TS method and
Gabor method, the AECLBP method can get better detection
results. However, it is difficult to detect the defects with the
similar intensity to the background.
In the WB method, the scale and the shape parameter are
extracted from the local patch with a 16 × 16 pixel size.
The detection results are shown in Fig. 9(e1)–(e6). Based
on the gradient magnitude, the method cannot perform well
for the defects with no noticeable edge, such as shown
in Fig. 9(e1)–(e3). In addition, there are some false positives
for the defects in Fig. 9(a4)–(a6) owing to the uncertain
reference point.
The detection results based on our methods are shown
in Fig. 9(f1)–(f6) and (g1)–(g6). For the O-HWV method, most
of the defects can be detected accurately except those with the
uniform texture distribution. Moreover, less false positives are
detected by the O-HWV method than TS, Gabor, AECLBP,
and WB methods. Furthermore, the M-HWV method can
decrease the false negatives compared with the O-HWV
method. Overall, it achieves much better performance than the
other five methods.
Furthermore, a quantitative evaluation based on the statistic
results is given to compare the proposed method with the
traditional methods. In the case of defect detection application,
only accuracy is not an appropriate performance measure on
a particular data set. Therefore, two performance indexes,
including Precision (Pr) and Recall (Re), are defined as
follows:
TP
Precision =
TP + FP
TP
Recall = (18)
TP + FN
where TP, FP, and FN represent the number of true posi-
tives, false positives, and false negatives, respectively. The
F-measure (Fm) is also calculated to evaluate the overall
performance of defect detection based on precision and recall.
The F-measure is defined as
2 × Precision × Recall
F − measure = . (19)
Precision + Recall
For 12 different types of defective images, the quantitative
evaluation based on the statistic results is shown in Table I. The
results obtained by the TS, Gabor, AECLBP, WB, O-HWV,
and M-HWV methods are shown, respectively. For example,
the first row of Table I shows the experimental results obtained
on the white dot type of defective images using six different
methods. The precision, recall, and F-measure are given in
different columns for each method individually. The last row
shows the overall performance of different methods.
From Table I, it can be observed that the TS method
has a lower recall rate than the other methods because of
suppressing the regular texture. Especially, the TS method
performs badly for scratch, floating, and shadow. Moreover,
the TS method and the Gabor method get lower precision rate
LIU et al.: STEEL SURFACE DEFECT DETECTION USING A NEW HWV MODEL IN UNSUPERVISED MANNER
Fig. 10. Variation of percentage F-measure by different methods.
TABLE II
C OMPARISON OF AVERAGE C OMPUTATIONAL T IME
than the other methods for shadow, since there is no noticeable
edge information. The WB method gets the similar precision
rate to the TS method and Gabor method, whereas it has a
higher recall rate than them. It can be explained by the fact
that the WB method detects defects by evaluating the local
patches in the global range.
Comparatively, the AECLBP method has a better preci-
sion and recall rate than the TS, Gabor, and WB methods.
However, it is insensitive to the defects with the similar texture
and intensity to the background. The O-HWV and M-HWV
methods can give a better precision rate than the other four
methods. Furthermore, the M-HWV method achieves the best
performance in both terms of recall and F-measure.
In terms of computational efficiency, the average processing
time per test image with the size 864 × 864 pixels of all these
methods is summarized in Table II for comparison. Besides the
size of the image, the computational efficiency is also related
to the size of the local patch for the AECLBP, WB, O-HWV,
and M-HWV methods and the number of filter directions for
the Gabor method.
As an illustration, a graphical representation of percent-
age F-measure based on the above six methods is shown
in Fig. 10. From the experimental results in Fig. 10, it is
observed that the O-HWV and M-HWV methods give better
detection results than the others. For line-type, embedded-
type, and dot-type defects such as white dot, mountain, and
scratch, the similar results are obtained using the two proposed
2595
methods. For the block-type and gradual-type defect such as
massive rupture, floating, and shadow, the M-HWV method
can give more accurate results. Overall, the performance of
the proposed M-HWV method is more competitive and better
than that of the other methods.
IV. C ONCLUSION
We proposed a novel approach to detect the steel surface
defects in an unsupervised manner. The performance was
exhaustively evaluated on the challenging data set collected
from steel production lines, which includes massive defective
images with different patterns. A new HWV model was built
to characterize the stochastic texture of the steel surface. The
defective regions can be located by the adaptive threshold,
which is computed after evaluating the distance of each sample
with the origin in the M-HWV space.
The experimental results have shown that the proposed
model has stronger discriminatory power than the traditional
Weibull model in terms of evaluating the texture distribution
of the steel images. Using the proposed model, an arbitrary
type of defect on the homogeneously textured background
can be detected, including subtle and miscellaneous defects,
even under the low contrast condition. It can achieve high
performance (both precision and recall are higher than 0.96).
In addition, it is worth noting that the proposed novel
method does not involve any sophisticated learning algorithm.
Hence, the training and the exhaustive parameter optimization
are not required. Although the HWV model has been proposed
to detect the defects of steel surface, it is promising to extend
it to other similar application, such as the LCD and stone.
R EFERENCES
[1] N. Qaddoumi, A. H. El-hag, M. A. Hosani, I. A. Mansouri, and
H. A. Ghufli, “Detecting defects in outdoor non-ceramic insulators using
near-field microwave non-destructive testing,” IEEE Trans. Dielectr.
Electr. Insul., vol. 17, no. 2, pp. 402–407, Apr. 2010.
[2] A. Foudazi, C. A. Edwards, M. T. Ghasr, and K. M. Donnell, “Active
microwave thermography for defect detection of CFRP-strengthened
cement-based materials,” IEEE Trans. Instrum. Meas., vol. 65, no. 11,
pp. 2612–2620, Nov. 2016.
[3] A. Foudazi, M. T. Ghasr, and K. M. Donnell, “Characterization of
corroded reinforced steel bars by active microwave thermography,” IEEE
Trans. Instrum. Meas., vol. 64, no. 9, pp. 2583–2585, Sep. 2015.
[4] Y. Tian, M. Y. Fu, and F. Wu, “Steel plates fault diagnosis on the
basis of support vector machines,” Neurocomputing, vol. 151, no. 1,
pp. 296–303, Mar. 2015.
[5] S. Ghorai, A. Mukherjee, M. Gangadaran, and P. K. Dutta, “Automatic
defect detection on hot-rolled flat steel products,” IEEE Trans. Instrum.
Meas., vol. 62, no. 3, pp. 612–621, Mar. 2013.
[6] S. C. Turaga, J. F. Murray, V. Jain, and F. Roth, “Convolutional networks
can learn to generate affinity graphs for image segmentation,” Neural
Comput., vol. 22, no. 2, pp. 511–538, Feb. 2010.
[7] T. Y. Li, J. Z. Tsai, R. S. Chang, L. W. Ho, and C. F. Yang, “Pretest
gap Mura on TFT LCDs using the optical interference pattern sensing
method and neural network classification,” IEEE Trans. Ind. Electron.,
vol. 60, no. 9, pp. 3976–3982, Sep. 2013.
[8] M. H. Karimi and D. Asemani, “Surface defect detection in tiling indus-
tries using digital image processing methods: Analysis and evaluation,”
ISA Trans., vol. 53, no. 3, pp. 834–844, 2014.
[9] M. Win, A. R. Bushroa, M. A. Hassan, N. M. Hilman, and
A. Ide-Ektessabi, “A contrast adjustment thresholding method for surface
defect detection based on mesoscopy,” IEEE Trans. Ind. Informat.,
vol. 11, no. 3, pp. 642–649, Jun. 2015.
[10] M. Mansano, L. Pavesi, L. S. Oliveira, A. Britto, and A. Koerich,
“Inspection of metallic surfaces using local binary patterns,” in Proc.
IECON, Melbourne, VIC, Australia, 2011, pp. 2227–2231.
2596 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 66, NO. 10, OCTOBER 2017
[11] S. R. Aghdam, E. Amid, and M. F. Imani, “A fast method of steel surface
defect detection using decision trees applied to LBP based features,” in
Proc. ICIEA, Singapore, 2012, pp. 1447–1452.
[12] K. Song and Y. Yan, “A noise robust method based on completed local
binary patterns for hot-rolled steel strip surface defects,” Appl. Surf. Sci.,
vol. 285, no. 21, pp. 858–864, Nov. 2013.
[13] H. Zuo, Y. Wang, X. Yang, and X. Wang, “Fabric defect detection
based on texture enhancement,” in Proc. CISP, Chongqing, China, 2012,
pp. 876–880.
[14] L. Liu, F. Zhou, and Y. He, “Automated visual inspection system for
bogie block key under complex freight train environment,” IEEE Trans.
Instrum. Meas., vol. 65, no. 1, pp. 2–14, Jan. 2016.
[15] D.-M. Tsai, C.-M. Chen, W.-C. Li, and W.-Y. Chiu, “A fast regularity
measure for surface defect detection,” Mach. Vis. Appl., vol. 23, no. 5,
pp. 869–886, Sep. 2012.
[16] W. C. Li and D. M. Tsai, “Automatic saw-mark detection in multicrys-
talline solar wafer images,” Solar Energy Mater. Solar Cells, vol. 95,
no. 8, pp. 2206–2220, Aug. 2011.
[17] D. Aiger and H. Talbot, “The phase only transform for unsupervised
surface defect detection,” in Proc. CVPR, San Francisco, CA, USA,
2010, pp. 295–302.
[18] J. Choi and C. Kim, “Unsupervised detection of surface defects: A two-
step approach,” in Proc. ICIP, Orlando, FL, USA, 2012, pp. 1037–1040.
[19] X. Bi, X. Xu, and J. Shen, “An automatic detection method of Mura
defects for liquid crystal display using real Gabor filters,” in Proc. CISP,
Shenyang, China, 2015, pp. 871–875.
[20] C. H. Yeh, F. C. Wu, W. L. Ji, and C. Y. Huang, “A wavelet-based
approach in detecting visual defects on semiconductor wafer dies,” IEEE
Trans. Semicond. Manuf., vol. 23, no. 2, pp. 284–292, May 2010.
[21] W. C. Li and D. M. Tsai, “Wavelet-based defect detection in solar wafer
images with inhomogeneous texture,” Pattern Recognit., vol. 45, no. 2,
pp. 742–756, Feb. 2012.
[22] Y. Gan and Q. Zhao, “An effective defect inspection method for LCD
using active contour model,” IEEE Trans. Instrum. Meas., vol. 62, no. 9,
pp. 2438–2445, Sep. 2013.
[23] L. Xu and Q. Huang, “Modeling the interactions among neighboring
nanostructures for local feature characterization and defect detection,”
IEEE Trans. Autom. Sci. Eng., vol. 9, no. 4, pp. 745–754, Oct. 2012.
[24] Y. Guet al, “A new Mumford-Shah type model involving a smooth-
ing operator for multiphase image segmentation,” in Proc. ICIP,
Quebec City, QC, Canada, 2015, pp. 1990–1994.
[25] I. B. Ayed, N. Hennane, and A. Mitiche, “Unsupervised variational
image segmentation/classification using a Weibull observation model,”
IEEE Trans. Image Process., vol. 15, no. 11, pp. 3431–3439, Nov. 2006.
[26] F. Timm and E. Barth, “Non-parametric texture defect detection using
Weibull features,” in Proc. SPIE., vol. 7877, p. 78770J, Feb. 2011.
[27] P. Perona and J. Malik, “Scale-space and edge detection using
anisotropic diffusion,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 12,
no. 7, pp. 629–639, Jul. 1990.
[28] J. M. Geusebroek and A. W. M. Smeulders, “A six-stimulus theory for
stochastic texture,” Int. J. Comput. Vis., vol. 62, no. 1, pp. 7–16, 2005.
[29] T.-T. V. Cao, “Modelling of inhomogeneity in radar clutter using
Weibull mixture densities,” IET Radar, Sonar Navigat., vol. 8, no. 3,
pp. 180–194, Mar. 2014.
[30] D. A. Abraham, J. M. Gelb, and A. W. Oldag, “Background and clutter
mixture distributions for active sonar statistics,” IEEE J. Ocean. Eng.,
vol. 36, no. 2, pp. 231–247, Apr. 2011.
Kun Liu received the M.S. degree from the Harbin
Institute of Technology, Harbin, China, in 2003,
and the Ph.D. degree in automation from Tsinghua
University, Beijing, China, in 2009.
She is currently an Associate Professor with the
School of Control Science and Engineering, Hebei
University of Technology, Tianjin, China. Her cur-
rent research interests include image processing,
computer vision, and pattern recognition.
Heying Wang received the B.S. degree from the
Hebei University of Technology, Tianjin, China,
in 2014, where she is currently pursuing the M.S.
degree in automation with the School of Control
Science and Engineering.
Her current research interests include computer
vision pattern recognition and machine learning.
Haiyong Chen received the M.S. degree from
the Harbin University of Science and Technology,
Harbin, China, in 2005, and the Ph.D. degree from
the Institute of Automation, Chinese Academy of
Sciences, Beijing, China, in 2008.
He is currently an Associate Professor with the
School of Control Science and Engineering, Hebei
University of Technology, Tianjin, China. His cur-
rent research interests include image processing,
robot vision, and pattern recognition.
Erqing Qu received the B.S. degree from the
Beijing Institute of Technology, Beijing, China,
in 1999, and the M.S degree in control engineering,
Yanshan University, Hebei, China, in 2009. He is
currently pursuing the Ph.D. degree in automation
with the School of Control Science and Engineering,
Hebei University of Technology, Tianjin, China.
He is currently a Senior Automation Engineer with
Tangshan Iron and Steel Company Ltd., Tangshan,
China. His current research interests include steel
inspection and its automation research.
Ying Tian received the M.S. degree in mechanical
engineering from the Harbin Institute of Technol-
ogy, Harbin, China, in 2003, and the Ph.D. degree
in mechanical engineering from Tianjin University,
Tianjin, China, in 2010.
She is currently a Lecturer with the School
of Mechanical Engineering, Tianjin University,
Tianjin, China, and the Key Laboratory of Equip-
ment Design and Manufacturing Technology, Tianjin
University. Her current research interests include
product’s energy modeling and simulation, inte-
grated system modeling, and product’s innovative and creative design
methodology.
Hexu Sun (M’90–SM’96) received the Ph.D.
degree in automation from Northeastern University,
Shenyang, China, in 1993.
He has been a Professor with the School of Control
Science and Engineering, Hebei University of Tech-
nology, Tianjin, China, and the School of Electrical
Engineering, Hebei University of Science and Tech-
nology, Shijiazhuang, China. He has authored five
books and more than 130 journal and conference
papers, and holds 13 U.S. patents and five computer
software copyrights. His current research interests
include robotics and complex engineering system.
Dr. Sun was the recipient of many prestigious national awards from China.
He was the Director in many societies and committees in China. He is
currently the invited Plenary Speaker and a General Co-Chair of many
international conferences.