Computers and Electronics in Agriculture 151 (2018) 416–425

Contents lists available at ScienceDirect

Computers and Electronics in Agriculture

journal homepage: www.elsevier.com/locate/compag

An automatic sorting system for fresh white button mushrooms based on T

image processing
⁎
Fengyun Wanga, Jiye Zhenga, Xincheng Tianb, Jianfei Wanga, Luyan Niua, Wenjie Fenga,
a
S&T Information Institution, Shandong Academy of Agricultural Sciences, Jinan, China
b
Control Science and Engineering School, Shandong University, Jinan, China

A R T I C LE I N FO A B S T R A C T

Keywords: White button mushrooms, one of the most commonly and widely consumed mushrooms worldwide, require
Online sorting grading before entering the market. The diameter of the pileus is an important factor that affects the selling
White button mushroom price. The aim of this study is to develop an automatic sorting system for fresh white button mushrooms based on
Image processing image processing according to the pileus diameter. First, an automatic sorting hardware system is designed that
Automatic control
consists of a conveying mechanism, an image acquisition system, a control module and an actuator. Second, we
present an image algorithm based on the watershed method, Canny operator, OR operation, and closed op-
eration to determine the pileus diameter of fresh white button mushrooms. The algorithm eliminates the in-
fluence of the shadow and petiole on the image. Third, a precise control strategy is introduced based on the
conveyor speed, distance between the trigger and the flap piece, trigger time and algorithm processing time.
Finally, we present analysis and control software based on OpenCV 2.4.10 and Visual Studio 2010. In the ex-
perimental portion, a prototype system was tested to validate its applicability and reliability by the grading
performance and results. The experimental results showed that the average maximum grading speed was
102.41 mushrooms/minute, the accuracy of grading was 97.42%, the damage rate was 0.05%, and the un-
detected rate was 0.96%. Relative to manual grading, the grading speed was improved by 38.86%, and the
accuracy was improved by 6.84%. The system can operate stably and continuously. For long-duration grading in
particular, the advantage of the automatic sorting system is apparent because it eliminates labourer fatigue. The
system achieves online automatic sorting and grading of fresh white button mushrooms with minimal de-
struction.

1. Introduction cholesterol while being high in protein, carbohydrate, fibre, vitamins

An important procedure in any commercial production line is to sort
out any inferior-quality or defective products (Jian et al., 2017; Duan
et al., 2007). Sorting industrial products is easier than sorting agri-
cultural products due to the regularity and uniformity of industrial
products (Yang et al., 2016; Nashat et al., 2014). For fresh agricultural
products, sorting is a fundamental problem because of the wide varia-
tion and individual differences that arise from differences in the culti-
vation environment, nutrition, planting mode and other factors
(Jarimopas and Jaisin, 2008; Elmasry et al., 2012).
White button mushrooms (Agaricus bisporus) are cultivated in more
than seventy countries (Cappelli, 1984) and are one of the most com-
monly and widely consumed mushrooms in the world (Pasban et al.,
2014). Recently, these mushrooms have become increasingly popular
because of their pleasing taste, broad availability, and popularity as
functional foods, as they are low in calories, sodium, fat, and
and essential amino acids (Mattila et al., 2000). Thus, the amount
consumed has increased, and production has been industrialised. The
market price varies depending on the pileus size of the fresh white
button mushroom. Therefore, these mushrooms require grading at the
postharvest stage before being marketed to achieve the maximum
economic benefit.
Currently, the industrialised production of white button mushrooms
is mature in China. The daily production for larger factories exceeds ten
tons. However, production depends on the labour required to grade the
fresh white button mushroom, which leads to unstable grading accu-
racy, a high workload, and low productivity. With increasing produc-
tion demands and strict quality requirements, manually grading has
become a bottleneck for the white button mushroom industry, which
would benefit from a real-time, high-speed, non-destructive automatic
sorting system to improve the commercial production capacity.
With the development of intelligent control and image-processing

⁎
Corresponding author.
E-mail address: 34941269@qq.com (W. Feng).

https://doi.org/10.1016/j.compag.2018.06.022
Received 12 January 2018; Received in revised form 14 April 2018; Accepted 10 June 2018
0168-1699/ © 2018 Published by Elsevier B.V.
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425

technology, many scholars have studied the sorting of agricultural

products using image processing. For example, Mizushima and Lu
(2013) proposed robust image segmentation, a support vector machine,
and Otsu’s method in a multi-channel colour space for apple sorting and
grading. Kurtulmuş and Kavdir (2014) used regular colour images to
detect the cutting location of a corn tassel. A support vector machine
was used by Yorulmaz et al. (2012) to classify popcorn kernels by
calculating covariance and correlation matrices. Ambrose et al. (2016)
classified treated and untreated corn seeds using an image-processing
method with partial least squares discriminant analysis.
Multiple computer vision-based sorting systems have been studied
for agricultural products. A computer vision-based tomato grading
system was proposed by Arakeri and Lakshmana (2016) that achieved
96.47% accuracy in evaluating tomato quality. Pourdarbani et al.
(2015) designed an automatic date fruit sorting system with a 15.45 kg/
h capacity. Abbasgholipour et al. (2011) developed a raisin sorting
apparatus using a permutation-coded genetic algorithm to identify re-
gions in the hue-saturation-intensity colour space for desired and un-
desired raisin detection in various conditions.
Non-destructive testing for mushroom using an imaging method has
focused on colour, freshness, disease (Vı́Zhányó and Felföldi, 2000),
size, and shape. For example, the application of image analysis for
variety testing of mushrooms to grant plant breeders' rights was in-
vestigated by Vooren et al. (1991). In their paper, the length, width,
and a range of several other more-or-less complex shape descriptors Fig. 1. Structural diagram of the automatic sorting system.
were used to recognise the variety. A total of 460 observations were
tested with an 80% distinction rate. Tillett and Batchelor (1991) de- 2.1. Conveying mechanism
scribed an image algorithm to identify the size and position of in-
dividual mushrooms in the growing bed. A robotic harvesting system The conveying mechanism included a fixed support (1 in Fig. 1),
was designed and tested by Reed and Tillet (1994), Reed et al. (2001), conveyor (2 in Fig. 1), roller, driving unit, tension unit, cleaner and
who used computer vision and image processing to determine the po- guide plate (16 in Fig. 1). The conveyor was made of polyurethane. The
sition and size of the mushrooms to realise automatic harvesting. In conveyor was 200 mm wide and 1.8 mm deep with a 3200 mm peri-
China, Zhou et al. (1995) introduced the working process of a picking meter. The roller diameter was 80 mm. The driving unit used a 0.18 kW
robot for mushrooms and studied the numerical characteristics of gear motor and Schneider ATV12H037M2 transducer with 0.1–30 m/
mushrooms and growing beds and the extraction algorithm of the min adjustable speed. A stop button was incorporated for emergencies.
image border. Yu et al. (2005) studied a contour description and lo-
cating algorithm for mushrooms that supplied the theoretic basis for 2.2. Image acquisition system
image processing to locate a mushroom in a growing bed. A grading
system with a 4.8 s grading cycle was developed by Chen and Ting To acquire the image online for the continuously moving mushroom
(2004) to classify shiitake mushrooms automatically into various grades on the conveyor, a line scan camera MV-LC2K40 (12 in Fig. 1) was
of quality in terms of size and shape. chosen with a resolution ratio of 20,481× and a line rate of 1–18 kHz.
Sorting technology based on machine vision has become mature; The lens chosen was the AFT-LCL5 with a 50 mm focal length and F
however, there have been few previous studies of its application to extension tube. The linear light source (13 in Fig. 1) and its controller
mushroom sorting. The studies regarding intelligent control and image- were the AFT-LL86232W and AFT-ALP24150-01, respectively. The
processing technology on other agricultural products sorting provide image acquisition system was connected with an industrial personal
support for our study. The modern production of fresh white button computer (IPC) using the image capture card LIC-2KB02.
mushroom urgently needs an automatic classification technology to
improve the commercial production capacity. Accordingly, we present
an automatic sorting system for fresh white button mushrooms based 2.3. Control section
on image processing. This study included hardware and software de-
sign. The performance of the developed system was tested using a large The control section included two parts. One part controlled the
quantity of fresh white button mushroom samples. camera that scans using a photoelectric sensor (15 in Fig. 1) that trig-
gered the camera. The other part controlled the actuator using elec-
2. Hardware design tromagnetic valve 4V220-06 and relay starter USB-4761, which re-
ceived the signals from the IPC. The valve 4V220-06 had five ports and
According to China’s industrial standard NY/T 1790–2009, fresh two positions with a maximum frequency of five times/s. The relay
white button mushrooms are classified into three grades by 25 mm and starter was based on a universal serial bus (USB) and had eight digital
45 mm according to the pileus diameter. Therefore, we designed two outputs with a LED indicator to display the ON/OFF status of each
identical mechanisms to classify in our study. For more than two channel.
grades, actuators and minor modification of the programs are required.
The automatic sorting system consisted of a conveying mechanism, 2.4. Actuator and mechanical design
image acquisition system, control section and an actuator. The diagram
in Fig. 1 shows a schematic of the system. The actuator included two The actuator had two identical mechanisms (3 and 4 in Fig. 1). Each
identical mechanisms, 3 and 4, for two grades, and mushrooms of a mechanism consisted of an air compressor OLF-2524/7, duplex pieces
third grade were placed into a corresponding container through the GFC20008F1, a muffler BSLM01, an air cylinder RMTL10X200A, joints,
movement of a conveyor. a gas channel, a side board (5 in Fig. 1), a guide rod (6 in Fig. 1), a slider

417
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425

White button Pressurize to set Adjust light seven different threshold segmentation algorithms (shown in Fig. 3).
mushroom Conveyor start value source controller As shown in Fig. 3, only one of the methods could not effectively
remove the shadow and petiole while maintaining the pileus. The factor
Trigger that affected the maximum diameter was the petiole.
Accordingly, an image algorithm was designed to determine the
1st Electromagnetic 1st pneumatic
valve act actuator move 1st Container diameter of the white button mushroom based on the watershed
Line scan method, Canny operator, OR operation and closed operation. First, the
first watershed algorithm (FWS) combining the global threshold seg-
2nd Electromagnetic 2nd pneumatic
Acquire, analyze valve act actuator move 2nd Container mentation method and maximum entropy threshold segmentation
and process method was used to remove the shadow from the image. Then, the
second watershed algorithm (SWS) based on the Canny operator, the
No Electromagnetic No pneumatic OR operator and the closed operation was used to remove the petiole
Digital output valve act actuator move 3rd Container
noise. Finally, the minimum enclosing rectangle method was used to
Fig. 2. Workflow. determine the diameter of the white button mushroom.

3.1. First watershed method

(7 in Fig. 1), a connector (8 in Fig. 1), a flap (9 in Fig. 1), a baffle (10 in
Fig. 1), an outlet (11 in Fig. 1), and containers (17 in Fig. 1), among
other components.
2.5. Operating procedure
As shown in Fig. 2, during system operation, the light source con-
troller can be adjusted to make the image clear and stable. The air
pressure is pressurised to 0.3 MPa using an air compressor. The con-
veyor motor is started and the speed adjusted based on the desired
grading speed. A white button mushroom enters the guide plate and
triggers the camera to scan the mushroom. The IPC analyses the image
after acquiring the data from the image capture card and converts the
grade result into an instruction. After the relay starter receives the in-
struction, the corresponding valve drives the slider of the air cylinder to
move to push the mushroom into the container.
3. Image-processing algorithms
Based on China’s industrial standard NY/T 1790–2009, we chose
the pileus diameter as the characteristic parameter in our study. When
the image was analysed, we found that the important factors affecting
the diameter were the shadow and petiole in the white button mush-
room image. We comparatively analysed the segmentation effects using
The FWS removes the shadow of the image by combining the global
threshold segmentation method and the maximum entropy threshold
segmentation method.
3.1.1. Region of interest extraction
The acquired original image includes the conveyor and its border,
which is removed through region-of-interest extraction. ROI_image is
returned after the mushroom image is separated from the background
and irrelevant elements have been eliminated.
3.1.2. Basic global threshold method (BGM)
The BGM is important in image segmentation because of its direct
visual processing, simple realisation and quick computation speed
(Abera et al., 2017; Medina-Carnicer et al., 2005). We applied the BGM
on ROI_image to determine the threshold value T1. The method is as
follows:
(1) Select an initial estimated value T (average grey degree value) for
the global threshold value.
(2) Segment the image by T to obtain two sets, of which the G1 set is
composed of pixels with a grey value larger than T and the G2 set is
composed of pixels with a grey value less than or equal to T.
(3) Compute the average degree value of G1 and G2 separately to obtain

(a) (b) (c) (d)

(e) (f) (g) (h)

(a) Original image; (b) BGM; (c) Iteration threshold method; (d) OTSU method; (e) Self-adaption
threshold method; (f) Top and bottom threshold method; (g) Manual set threshold value 71; (h)
MEM
Fig. 3. Comparison of seven image-processing effects.

418
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425

m1 and m2. 3.2. Second watershed method

m +m
(4) Compute a new threshold using the formula T = 1 2 2 .
(5) Repeat steps (2) through (4) until the difference between succes- The SWS removes the noise of the petiole based on the Canny op-
sive iteration T values is zero. erator, the OR operator and the closed operation.

The final T is the required threshold value T1. ROI_image is con-

3.2.1. Silhouette extraction
verted into a greyscale image before binarisation processing. The grey
The Canny operator is used for edge detection (Jiang et al., 2015;
degree value of pixels larger T1 are assigned to 255 (i.e., white), and
Tang et al., 2016). The upper and lower threshold values are above T1
those less than or equal to T1 are assigned to 0 (i.e., black). Accordingly,
and T2. Only the pixels with a continuous connection path in the edge
ROI_image is converted into a binary image with 0 or 255 grey values
distribution diagram with the lower threshold value are retained and
and designated image1.
connected to the edge with the upper threshold value to obtain the first
silhouette picture contour1. The silhouette picture contour1 retains all
3.1.3. Maximum entropy threshold method (MEM)
the edge pixels on the upper threshold value distribution diagram and
Entropy refers to the uncertainty of a random variable. When the
removes all the isolated pixels on the lower threshold value distribution
entropy is largest, the random variable is the most uncertain. The
diagram.
principle of maximum entropy states that the probability distribution
that best represents the current state of knowledge is the one with the
largest entropy. In an image, the information at the junction between 3.2.2. Logical OR operation and closed operation
the foreground and background is the highest, corresponding to the Convert the foreground of watershed1 (i.e., the white button mush-
largest entropy (Sun and Zhao, 2015). The KSW-entropy algorithm room) into black (level 0) and the background of watershed1 into white
seeks the threshold that maximises the entropy. (level 255). Set the foreground of contour1 (i.e., the white button
Assume that the segment threshold value is t and the image has L mushroom silhouette obtained using the Canny operator) as white
grey levels: (level 255) and the background of contour1 as black (level 0). Perform
L−1 the logical OR operation for watershed1 and contour1 to obtain result1,
∑ Pi = 1, i = 0, 1, 2, …, L−1 which removes the edge not belonging to the white button mushroom
i=0 (1) and retains only the inner edge of the white button mushroom in con-
tour1 (e.g., the connection part between pileus and petiole). Perform the
In formula (1), Pi is the probability for the ith grey scale. Assume
closed operation (expansion-corrosion) for result1. The resulting picture
that T is the grey level distribution belonging to {0, 1, 2, …, t } and that B
image_closed with an improved processing effect is obtained after eight
is the grey level distribution belonging to {t + 1, t + 2, …, L−1} ; thus, the
iterations of the closed operation.
probability distribution is
P0 P1 P
T: , , …, t 3.2.3. Second watershed algorithm
Pn Pn Pn (2)
Use the white button mushroom in image_closed as the foreground of
Pt + 1 Pt + 2 P image marker2 in which the grey level is marked as 255 (white). Use the
B: , , …, L − 1
1−Pn 1−Pn 1−Pn (3) background of watershed1 as the background of image marker2, in which
t the grey degree is marked as 128 (grey). The grey degree of the other
In formula (2) and (3), Pn = ∑ Pi . pixels in image marker2 is marked as 0 (black). Set marker2 as the
i=0
The entropy related to the two probability densities is as follows: marked image of the watershed to process ROI_image to obtain the re-
sulting image watershed2 from the SWS.
t
Pi ⎛ Pi ⎞
H (T ) = − ∑ ln⎜ ⎟

i=0
Pn ⎝ Pn ⎠
L−1
Pi P
H (B ) = − ∑ ln ⎛ i ⎞
⎜ ⎟
1−Pn ⎝ 1−Pn ⎠
i=t+1
In formula (4) and (5), H(T) and H(B) are the entropies related to
the probability and density for the T and B distributions, respectively.
Define the entropy function φ(t) as follows:
φ (t ) = H (T ) + H (B )
The optimal threshold value T2 is the grey degree grade t when the
value of φ(t) is maximised. ROI_image should be converted into a
greyscale image before binarisation processing. The grey degree value
of a pixel larger T2 is assigned to 128 (grey) and less than or equal to T2
is assigned to 0 (black). Accordingly, ROI_image is converted into a
binary image with a 0 or 128 grey level and designated image2.
3.1.4. First watershed algorithm
Use the foreground of image1 as the foreground of image marker1,
which has a grey level marked as 255 (white). Use the background of
image2 as the background of image marker1, which has a grey level
marked as 128 (grey). The grey levels of the other pixels in image
marker1 are marked as 0 (black). Set marker1 as the marked image of the
watershed to identify the certain foreground region, certain background
region and uncertain region. Then, use the watershed algorithm to
process ROI_image to obtain the resulting image watershed1 of the FWS.
(4) 3.3. Minimum enclosing rectangle method
Extract the silhouette for watershed2, and remove the larger or
(5)
smaller silhouette to obtain the second silhouette picture contours2. Use
the Graham algorithm to scan the convex hull of the white button
mushroom, and use the even interval rotation method to identify the
enclosing rectangle with the minimum area to obtain the final picture
result2.
(6)
(1) Find an enclosing rectangle M1 of silhouette picture contours2 and
compute its area S1. Let SMIN = S1 and MMIN = M1 and set the po-
sition angle α = β = 0.
(2) Rotate the silhouette clockwise with an angle θ, identify its en-
closing rectangle M2, and compute its area S2. The new position
angle is α = α + θ.
(3) Compare S2 with SMIN. If S2 < SMIN, let SMIN = S2, β = α, and
MMIN = M2.
( 90
)
(4) Iterative N N = θ times for (2) and (3). Rectangle MMIN is the
required minimum enclosing rectangle. Rotate the silhouette and
enclosing rectangle anticlockwise by an angle β.
The longer side of the minimum enclosing rectangle is the required
diameter of the white button mushroom. Variations in the mushroom
position affect the shorter side of the minimum enclosing rectangle with
no change to the longer side due to the circular symmetry of the pileus.

419
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425

ROI_image image1 image2 marker1 watershed1 contours1

result1 image_closed marker2 watershed2 contours2 result2

Fig. 4. Algorithm processing results.

Table 1 Original image

Relationship between the speed and line frequency.
ROI extraction
Speed (Hz) 35 40 45 50
ROI_image
Line frequency (lines) 1330 1525 1700 1900
BGM MEM

T1 T2

Binarization Binarization

image1 image2

Combination

marker1

Canny operator
FSD

contours1 watershed1

OR operation

result1

Eight Closing operations

Fig. 5. USB-4761 module.
image_closed Combination
3.4. Image calibration
marker2
The unit of the diameter is the pixel, which must be converted into SWS
the actual length. We use a steel ruler as a reference. We record the
Silhouette
actual coordinates for two marked points and measure the pixel values watershed2 extraction
for two marked points to compute the proportional factor using the
following formula: contours2
|X1−X2 | Minimum enclosing rectangle method
κ =
L (7)
result2
where κ is the pixel value per unit length, X1 and X2 are the pixel co-
ordinates for the two marked points, and L signifies the actual length. Calibration
The diameter in pixels is divided by κ to determine the actual diameter
Diameter R
of the white button mushroom.
The output of each algorithm is shown in Fig. 4. Fig. 6. Flow diagram of the image-processing algorithm.

4. System control principle container.

When the white button mushroom arrives at the photoelectric
sensor, the sensor triggers the industrial camera to scan the mushroom.
The image capture card acquires the data and sends this information to
the IPC. The IPC processes the image data and sends the instruction of
the grading result to the electromagnetic valve through the USB control
module. Finally, the mushroom is pushed into the corresponding
The speed of the conveyor affects the line frequency of the camera
and the accuracy of the actuator. The line frequency should match the
speed of the conveyor to ensure a clear picture. The faster the speed, the
higher the line frequency and the more frequently the actuator moves.
If the speed changes but the line frequency does not, the image will be
distorted. The relationship between the speed and the line frequency is

420
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425

Set DO as a control control mode optimises the execution efficiency.

byte Assume that the speed of the conveyor is v and that the distance
from the photoelectric sensor to the centre of the flap piece (9 in Fig. 1)
Initialize DO = 0x 00, write is d; then, the time t from the triggering to sorting the mushroom is as
DO into USB port follows:
d
Assign DO = 0x 05, t=
Write DO into USB port v (8)
The time t includes the image capturing and processing time ta
which is approximately 120 ms as measured automatically by the pro-
gram. Thus, the exact sorting time te is as follows:

Grading type? te = t −ta (9)

Because the distance from the photoelectric sensor to the centre of
Grade-1 Grade-3
the flap piece is fixed, the time te depends on the conveyor speed and
Grade-2
the IPC operating speed.
DO = DO & 0x FF DO = DO ^ 0x 03 DO = DO ^ 0x 0C The valve is controlled by the Advantech USB-4761 module (Fig. 5)
for the USB port, which has eight relay output channels. Each channel
has an LED indicator to show its current status. Each valve has two
Delay te1 Delay te2 Delay te3 positions, corresponding to two output channels. In the system, the four
relay output channels D0–D3 are used to control two electromagnetic
Write DO into valves using a rising edge-trigger. When initialised, set D0–D3 as 0, and
USB port set D0 and D2 as 1. During operation, the corresponding channels of the
valve perform the logical NOT operation.
Fig. 7. Flow diagram of the control algorithm.
5. Software system

shown in Table 1.
The container farthest away from the photoelectric sensor was de-
signed for the grade of mushroom with the highest quantity; these
mushrooms enter the container immediately and do not require elec-
tromagnetic valve action. The container farther from the photoelectric
sensor is designed for the grade of mushroom with a lower quantity;
these mushrooms are sorted using the second actuator (4 in Fig. 1). This
actuator is farther from the photoelectric sensor, which results in a
longer delay for its valve action. The container nearest to the photo-
electric sensor is for the grade of mushroom with the lowest quantity to
reduce the frequency of the action of the first actuator (3 in Fig. 1). This
The software system is based on OpenCV 2.4.10 and Visual Studio
2010 to acquire, analyse and process images and output control in-
structions to control the valve using the USB-4761 module. The algo-
rithm flow diagram of the image processing is shown in Fig. 6.
The control algorithm flow diagram is shown in Fig. 7. We set DO as
a control variable byte. Grade-1 corresponds to the grade of mushroom
with the highest quantity and two valves without actuation. Grade-2
corresponds to the grade of mushroom with a lower quantity and the
second actuator (4 in Fig. 1) controlled by D0 and D1 bits of DO. Grade-
3 corresponds to the grade of mushroom with the lowest quantity and
the first actuator (3 in Fig. 1) controlled by D2 and D3 bits of DO.

Fig. 8. System interface.

421
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425

Table 2
Static experimental data and analysis.
No. 1 2 3 4 5 6 7 8 9 10

AV (mm) 49.00 59.80 37.50 49.80 44.00 45.80 54.60 35.40 46.70 44.40
EV (mm) 49.20 60.45 37.00 50.20 44.50 45.05 54.70 35.65 46.74 44.10
Error (%) 0.41 1.08 1.35 0.80 1.12 1.66 0.18 0.70 0.09 0.68

No. 11 12 13 14 15 16 17 18 19 20
AV (mm) 47.50 35.40 46.00 45.10 46.30 37.00 46.00 49.40 40.00 47.60
EV (mm) 48.00 35.80 46.10 45.50 47.80 37.55 46.25 49.85 39.55 47.00
Error (%) 1.04 1.12 0.22 0.88 3.14 1.46 0.54 0.90 1.14 1.28

No. 21 22 23 24 25 26 27 28 29 30
AV (mm) 40.00 37.00 40.30 45.80 36.80 54.00 48.00 43.00 47.00 35.70
EV (mm) 39.60 36.65 40.90 45.98 36.30 54.30 47.35 42.55 47.65 34.75
Error (%) 1.01 0.95 1.47 0.39 1.38 0.55 1.37 1.06 1.36 2.73

No. 31 32 33 34 35 36 37 38 39 40
AV (mm) 34.90 45.40 42.00 35.80 38.10 39.50 42.00 43.10 42.10 56.00
EV (mm) 34.65 45.80 42.15 36.20 38.55 39.85 42.15 43.45 42.40 55.70
Error (%) 0.72 0.87 0.36 1.10 1.17 0.88 0.36 0.81 0.71 0.54

Average error: 0.99%

probability of the t-test for the AV and EV values was 0.9405, which is
larger than 0.05. The results of the t-test show that there was no sig-
nificant difference between the AV and EV values. The above analysis
shows that the image-processing algorithm is appropriate for static
measurement.

6.2. Dynamic test

According to China’s industrial standard NY/T 1790–2009, we

Fig. 9. Experimental prototype.
The software also has a visual parameter configuration function for
the camera, control module and grading standard. The camera para-
meter configuration includes a communication port, line scan fre-
quency and exposure time. The control module configuration consists of
the conveyor speed, distance from the photoelectric sensor to the first
flap piece, distance from the photoelectric sensor to the second flap
piece and the calibration factor. The diameter corresponding to the
various grading standards is set in the software interface. Different
grading results are also counted by the system automatically. The
system interface is shown in Fig. 8.
6. Test results and discussion
To validate the effect of the algorithm, applicability, and reliability
of the system, static and dynamic tests were performed separately.
6.1. Static test
The area-array camera was used to acquire the images of 40
mushroom samples and convert the image data into length data in mm
units as the experimental value (EV) by factor κ (formula (7)). A Vernier
calliper was used to measure the diameter of the actual value (AV). The
absolute differential value between EV and AV was divided by AV to
compute the error. The results are shown in Table 2.
The maximum error was 3.14% (in bold), the minimum error was
0.09% (in bold) and the average error was 0.99% (in bold). The
classified the pileus diameter into three grades using thresholds at
25 mm and 45 mm. In our experiment, we found that most mushrooms
had a pileus diameter larger than 45 mm. Mushrooms sized between
25 mm and 45 mm composed the second largest group, and mushrooms
smaller than 25 mm corresponded to the lowest quantity. Therefore, we
specified Grade-1, Grade-2 and Grade-3 to correspond to mushrooms
with a pileus diameter larger than 45 mm, between 25 mm and 45 mm
and less than 25 mm, respectively.
The dynamic test was used to determine the grading efficiency,
accuracy rate, damage rate and undetected rate. Each experiment in-
cluded automatic system classification and manual classification.
Automatic classification was performed on the prototype of the sorting
and grading system as shown in Fig. 9.
The manual classification was done by skilled workers through vi-
sual inspection. Finally, a Vernier calliper was used to determine the
actual value of the diameter in order to evaluate the accuracy of the
automatic and manual results. The duration was recorded separately for
automatic classification and manual classification. The quantity was
recorded separately for Grade-1, Grade-2 and Grade-3 by system
counting, manual counting, calliper-checked counting and damage
counting. The recorded data for 1160 samples tested on 18 October
2017 are shown in Table 3.
In Table 3, the sample quantity for No. 1 to No. 10 was approxi-
mately 100 and for No. 11 was approximately 500. For each automatic
grading record in Table 3, Ma, Ga1, Ga2, Ga3, Ha1, Ha2, Ha3, Ca1, Ca2, Ca3,
Da1, Da2, Da3 were used to represent the automatic grading duration,
system counting for Grade-1, system counting for Grade-2, system
counting for Grade-3, manual counting for Grade-1, manual counting
for Grade-2, manual counting for Grade-3, calliper-checked counting
for Grade-1, calliper-checked counting for Grade-2, calliper-checked
counting for Grade-3, damaged quantity for Grade-1, damaged quantity
for Grade-2, and damaged quantity for Grade-3, respectively. Then, the
automatic grading efficiency Ea, accuracy Aa, damaged rate Da and
undetected rate La were separately computed using formulas (10)–(13):

422
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425

Table 3
Original experiment data.
No. Automatic grading Manual grading

Duration Count Grade-1 Grade-2 Grade-3 Duration Grade-1 Grade-2 Grade-3

(second) (> 45 mm) (25–45 mm) (< 25 mm) (second) (> 45 mm) (25–45 mm) (< 25 mm)

1 63 System 48 47 0 63
Manual 48 47 0 52 42 1
By calliper 47 48 0 48 47 0
Damaged 0 0 0 0 0 0

2 61 System 48 54 0 78
Manual 49 55 0 68 35 1
By calliper 49 55 0 48 56 0
Damaged 0 0 0 0 0 0

3 66 System 50 61 2 91
Manual 53 61 1 40 73 2
By calliper 48 67 0 49 66 0
Damaged 0 0 0 0 0 0

4 67 System 55 58 1 93
Manual 61 54 1 31 82 0
By calliper 58 57 1 55 58 0
Damaged 0 0 0 0 2 0

5 61 System 58 46 0 89
Manual 58 46 0 43 61 0
By calliper 57 47 0 58 46 0
Damaged 0 0 0 0 0 0

6 60 System 48 60 1 84
Manual 49 60 1 32 77 0
By calliper 51 58 1 36 73 0
Damaged 0 0 0 0 0 0

7 121 System 78 117 7 177

Manual 86 116 7 91 115 3
By calliper 87 117 5 92 112 5
Damaged 1 0 0 0 0 0

8 60 System 71 30 1 95
Manual 71 30 1 45 57 0
By calliper 65 36 1 46 55 1
Damaged 0 0 0 0 0 0

9 61 System 79 23 0 101
Manual 79 23 0 52 50 0
By calliper 76 26 0 59 43 0
Damaged 0 0 0 0 0 0

10 60 System 77 26 0 86
Manual 77 26 0 54 49 0
By calliper 72 31 0 56 47 0
Damaged 0 0 0 0 0 0

11 315 System 279 229 4 472

Manual 291 226 3 179 337 3
By calliper 286 230 3 193 351 3
Damaged 2 0 0 0 5 0

Note:
(1) The first line of each record is the automatic statistic quantity using the system.
(2) The second line of each record was counted by a person to compute the undetected rate. “Manual” in the rows refers to counting by a person. “Manual” in the
columns refers to grading by a person.
(3) The third line of each record is the quantity measured using the calliper to compute the grading accuracy.

Ha1 + Ha2 + Ha3 efficiency, accuracy and damaged rate were similar to the method for
Ea = × 100%
Ma (10) automatic classification. The analysis of the data in Table 3 is shown in
Table 4.
Max {|Ha1−Ca1|, |Ha2−Ca2|, |Ha3−Ca3 |} ⎞ Table 4 shows that the average automatic grading speed was 102.41
Aa = ⎛1−⎜ × 100%
⎟

⎝ Ha1 + Ha2 + Ha3 ⎠ (11) mushrooms per minute, the average automatic accuracy was 97.42, the
average automatic damage rate was 0.05% and the average automatic
Da1 + Da2 + Da3
Da = × 100% undetected rate was 0.96%. Relative to manual classification, the effi-
Ha1 + Ha2 + Ha3 (12) ciency improved by 38.86%, and the accuracy improved by 6.84%. For
classification over a long duration, the advantage of the automatic
|(Ga1 + Ga2 + Ga3)−(Ha1 + Ha2 + Ha3 )|
La = × 100% classification is clear. In actual production, the daily manual classifi-
Ha1 + Ha2 + Ha3 (13)
cation (for eight hours) capability is from 60 kg to 100 kg depending on
The average value was computed using ten records. For manual the sorting individual’s working efficiency. For the automatic system,
classification, undetected cases did not exist, and the computations for the average efficiency is 102.41 mushrooms per minute, equivalent to

423
F. Wang et al. Computers and Electronics in Agriculture 151 (2018) 416–425

Table 4 developed by Chen and Ting (2004) for dried shiitake mushrooms. The
Data analysis. present research improved the grading efficiency relative to the grading
No. Type Efficiency Accuracy (%) Damaged Undetected cycle of 4.8 s in that research. An important factor is the efficiency.
(quantity/ rate (%) rate (%) Although the daily automatic efficiency improved five times over
minute) manual efficiency, the manual feeding mode limited the commercial
application. Therefore, we plan to address the feeding mechanism in
1 Automatic 90.48 98.95% 0.00% 0.00%
Manual 90.48 93.68% 0.00%
future studies.
2 Automatic 102.30 100.00% 0.00% 1.92%
Manual 80.00 78.85% 0.00% 7. Conclusions
3 Automatic 104.55 94.78% 0.00% 1.74%
Manual 75.82 92.17% 0.00%
In this study, an automatic sorting system for fresh white button
4 Automatic 103.88 97.41% 0.00% 1.72%
Manual 74.84 79.31% 0.00% mushrooms was designed, realised and tested. One important devel-
5 Automatic 102.30 99.04% 0.00% 0.00% opment was the image-processing model and the algorithm based on
Manual 70.11 85.58% 0.00% the watershed method, Canny operator, OR operation and closed op-
6 Automatic 110.00 98.18% 0.00% 0.91%
eration to determine the pileus diameter of fresh white button mush-
Manual 78.57 96.36% 1.82%
7 Automatic 103.64 99.52% 0.48% 3.35%
rooms. Another development was a precise control strategy based on
Manual 70.85 97.61% 0.00% the conveyor speed, distance between the trigger and flap piece, trigger
8 Automatic 102.00 94.12% 0.00% 0.00% time and algorithm processing time. The hardware consisted of me-
Manual 64.42 97.06% 0.00% chanical parts, including a conveyor, roller, cleaner guide rod, and
9 Automatic 102.00 97.06% 0.00% 0.00%
slider. Additional hardware included electrical and electronic parts,
Manual 60.59 93.14% 0.00%
10 Automatic 103.00 95.15% 0.00% 0.00% such as a line-scan camera, motors, photoelectric sensor, light source
Manual 71.86 98.06% 0.00% and its controller, electromagnetic valve, USB relay control module,
11 Automatic 98.86 99.04% 0.39% 1.35% IPC, and image capture card. The software function consisted of the
Manual 65.97 97.30% 0.96% camera configuration, parameter configuration, image acquisition,
Average Automatic 102.41 97.42% 0.05% 0.96%
value Manual 73.75 91.18% 0.18%
image processing, output control, data display and statistical analysis.
Our goal was to design a system that improves the commercial pro-
duction capacity. The proposed system can process more than 600 kg of
approximately 1.29 kg per minute is approximately 619.31 kg for eight mushrooms in eight working hours. The system can be used for other
hours. Thus, the daily automatic efficiency is improved five times over fruits and vegetables classified by their size.
the manual efficiency.
Acknowledgements

6.3. Discussion
The high efficiency is clearly shown through the experiment de-
scribed above. The maximum action frequency of the valve is five times
per second. In theory, one valve can sort 300 mushrooms per minute,
and the efficiency is doubled for two valves. In our experiment, the
efficiency was 102.41 mushrooms per minute and was a function of the
mushroom feeding mode and the image-processing speed. A vibration
disc was designed to guide the mushrooms individually onto the con-
veyor along the guide plate. However, the fresh mushrooms were often
blocked, overlapped, or damaged due to their high moisture content.
The manual feeding mode was adopted in the dynamic test. The image-
processing speed can be improved by adding cameras.
The accuracy depends on the image-processing algorithm and the
control algorithm. We found in the static test that the average accuracy
for the image-processing algorithm was 99.01%. When there were
brown spots or scars leading to large connection areas on the image, the
accuracy of the algorithm was affected. However, the occurrence of
brown spots or scars is unusual for fresh white button mushrooms.
Therefore, the accuracy is predominantly affected by the control algo-
rithm, which can be improved using variable air pressure with the size
of mushroom and an exact variable time delay with the IPC processing
speed.
Any damage caused by the system on the product arose through
improper air pressure and the lack of buffering between the container
and the actuator. The damage rate was reduced by adjusting the air
pressure. The mushroom was directly pushed into the container without
a buffering slideway in the hardware design, which was a shortcoming
of this design. Any damage done by manual classification arose through
unsuitable sorting force.
For certain conveyor speeds, the distance between two adjacent
image-processed mushrooms had a minimum value. If the distance was
less than the minimum value, the mushroom was undetected.
A similar research study was conducted on the grading system
This research was funded by the Department of Science &
Technology of Shandong Province, China, project no. 2016GNC11008,
and by the Shandong Academy of Agricultural Sciences, China, project
no. CXGC2017B04. The authors declare that they have no conflict of
interest.
Appendix A. Supplementary material
Supplementary data associated with this article can be found, in the
online version, at https://doi.org/10.1016/j.compag.2018.06.022.
References
Abbasgholipour, M., Omid, M., Keyhani, A., Mohtasebi, S.S., 2011. Color image seg-
mentation with genetic algorithm in a raisin sorting system based on machine vision
in variable conditions. Expert Syst. Appl. 38 (4), 3671–3678.
Abera, K.A., Manahiloh, K.N., Nejad, M.M., 2017. The effectiveness of global thresholding
techniques in segmenting two-phase porous media. Constr. Build. Mater. 142,
256–267.
Ambrose, A., Kandpal, L.M., Kim, M.S., Lee, W.H., Cho, B.K., 2016. High speed mea-
surement of corn seed viability using hyperspectral imaging. Inf. Phys. Technol. 75,
173–179.
Arakeri, M.P., Lakshmana, 2016. Computer vision based fruit grading system for quality
evaluation of tomato in agriculture industry. Procedia Comput. Sci. 79, 426–433.
Cappelli, A., 1984. Fungi Europaei: Agaricus (in Italian). Giovanna Biella, Saronno, Italy,
pp. 123–125.
Chen, H.H., Ting, C.H., 2004. The development of a machine vision system for shiitake
grading. J. Food Qual. 27 (5), 352–365.
Duan, F., Wang, Y., Liu, H., Li, Y., 2007. A machine vision inspector for beer bottle. Eng.
Appl. Artif. Intell. 20 (7), 1013–1021.
Elmasry, G., Cubero, S., Moltó, E., Blasco, J., 2012. In-line sorting of irregular potatoes by
using automated computer-based machine vision system. J. Food Eng. 112 (1–2),
60–68.
Jarimopas, B., Jaisin, N., 2008. An experimental machine vision system for sorting sweet
tamarind. J. Food Eng. 89 (3), 291–297.
Jian, C., Gao, J., Ao, Y., 2017. Automatic surface defect detection for mobile phone screen
glass based on machine vision. Appl. Soft Comput. 52, 348–358.
Jiang, W., Zhou, H., Shen, Y., Liu, B., Fu, Z., 2015. Image segmentation with pulse-cou-
pled neural network and canny operators. Comput. Electron. Agric. 46 (C), 528–538.
Kurtulmuş, F., Kavdir, İ., 2014. Detecting corn tassels using computer vision and support

424
F. Wang et al.
vector machines. Expert Syst. Appl. 41 (16), 7390–7397.
Mattila, P., Suonpää, K., Piironen, V., 2000. Functional properties of edible mushrooms. J.
Nutr. 16 (7–8), 694–696.
Medina-Carnicer, R., Madrid-Cuevas, F.J., Carmona-Poyato, A., 2005. Evaluation of
global thresholding techniques in non-contextual edge detection. Pattern Recognit.
Lett. 26 (10), 1423–1434.
Mizushima, A., Lu, R., 2013. An image segmentation method for apple sorting and
grading using support vector machine and Otsu's method. Comput. Electron. Agric.
94, 29–37.
Nashat, S., Abdullah, A., Abdullah, M.Z., 2014. Machine vision for crack inspection of
biscuits featuring pyramid detection scheme. J. Food Eng. 120, 233–247.
NY/T 1790–2009, Grades and specification of Agaricus bisporus.
Pasban, A., Mohebbi, M., Pourazarang, H., Varidi, M., 2014. Effects of endemic hydro-
colloids and xanthan gum on foaming properties of white button mushroom puree
studied by cluster analysis: a comparative study. J. Taibah Univ. Sci. 8 (1), 31–38.
Pourdarbani, R., Ghassemzadeh, H.R., Seyedarabi, H., Nahandi, F.Z., Vahed, M.M., 2015.
Study on an automatic sorting system for date fruits. J. Saudi Soc. Agric. Sci. 14 (1),
83–90.
Reed, J.N., Tillett, R.D., 1994. Initial experiments in robotic mushroom harvesting.
Mechatron 4 (3), 265–279.
Reed, J.N., Miles, S.J., Butler, J., Baldwin, M., Noble, R., 2001. Ae-automation and
425
Computers and Electronics in Agriculture 151 (2018) 416–425
emerging technologies: automatic mushroom harvester development. J. Agric. Eng.
Res. 78 (1), 15–23.
Sun, R.Y., Zhao, C.Y., 2015. Restoration of space object images by using a maximum
entropy method. Chin. Astron. Astrophy 39 (1), 89–99.
Tang, Z., Huang, L., Zhang, X., Lao, H., 2016. Robust image hashing based on color vector
angle and canny operator. AEU – AEU Int. J. Electron. Commun. 70 (6), 833–841.
Tillett, R.D., Batchelor, B.G., 1991. An algorithm for locating mushrooms in a growing
bed. Comput. Electron. Agric. 6 (3), 191–200.
Vı́Zhányó, T., Felföldi, J., 2000. Enhancing colour differences in images of diseased
mushrooms. Comput. Electron. Agric. 26 (2), 187–198.
Vooren, J.G.V.D., Polder, G., van der Heijden, G.W.A.M., 1991. Application of image
analysis for variety testing of mushroom. Euphytica 57 (3), 245–250.
Yang, Y., Zha, Z., Gao, M., He, Z., 2016. A robust vision inspection system for detecting
surface defects of film capacitors. Signal Process. 124, 54–62.
Yorulmaz, O., Pearson, T., Çetin, A., 2012. Detection of fungal damaged popcorn using
image property covariance features. Comput. Electron. Agric. 84 (2), 47–52.
Yu, G.H., Zhao, Y., Li, G., Shi, H., 2005. Algorithm for locating individual mushroom and
description of its contour using machine vision. Trans. CSAE 21 (6), 101–104.
Zhou, Y.S., Li, Q., Li, H.Y., Wang, R.B., He, W.B., 1995. Application of computer vision in
mushroom picking robot. Trans. CSAE 11 (4), 27–32.