Mango Variety Recognizer Using Image Processing and Artificial
Neural Network
Winston M. Tabada †
Dept. of Computer Science and Technology
Visayas State University
Baybay City, Leyte, Philippines
winston.tabada@vsu.edu.ph
ABSTRACT
Mango (Mangifera indica L.) is considered in the Philippines as
one of its most important fruit crops in terms of values and
exports. Selecting the right variety of mango to plant is critical in
mango production industry. Variety recognition is considered an
essential tool for ensuring quality product. Human experts can do
recognition but in their absence, technology cannot be put into
good use. This paper developed a multilayer perceptron (MLP)
type of neural network application that can identify a mango
variety from a leaf image of any of the four predominant mango
varieties of the Philippines: Carabao, Pico, Pahutan and
Katchamita. Nine color features, nine textural features and seven
Hu moments morphological features were extracted from each
leaf image samples using different image processing techniques
such as the automatic thresholding method of segmentation,
median filter, dilation and erosion. The MLP has 25 neurons in the
input layer, 50 neurons in the hidden layer and 4 neurons in the
output layer. The recognizer was tested with 40 leaf images (10
samples for each variety) from the samples used in the training
and 40 leaf images that were not used in the training. The test
obtained an accuracy of 96%.
CCS CONCEPTS
I.4.1 [Digitization and Image Capture]: Imaging
geometry,Quantization;I.4.3 [Enhancement]: Filtering, Grayscale
manipulation; I.4.6 [Segmentation]: Edge and feature detection;
I.4.7 [Feature Measurement]: Feature representation, Invariants,
Moments, Projections, Size and shape, Texture; I.5.2 [Design
Methodology]: Classifier design and evaluation, Feature
evaluation and selection, Pattern analysis
KEYWORDS
Backpropagation, moment invariants, texture analysis,
thresholding
† Author Note
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or
distributed for profit or commercial advantage and that copies bear this notice and
the full citation on the first page. Copyrights for third-party components of this
work must be honored. For all other uses, contact the owner/author(s).
PCSC2019, March 2019, City of Manila, Philippines
© 2019 Copyright held by the owner/author(s).
https://doi.org/10.1145/1234567890
Jerik G. Beltran
Dept. of Computer Science and Technology
Visayas State University
Baybay City, Leyte, Philippines
jgbeltran.0626@gmail.com
1 INTRODUCTION
Mango (Mangifera indica L.) originated in India and and were
declared as the national fruit of India, Pakistan and the
Philippines. It is also considered as one of the most highly prized
fruit of South Asia [1]. In the Philippines, there is one variety that
stands among the rest and that is the "Carabao Mango”. Mango is
the third most important fruit crop of the Philippines based on
export volume and value next to banana and pineapple (2).
Determining the variety of mango requires some knowledge,
skills and experience. Mango variety identification is manually
done by mango experts. Mango experts may be efficient in
recognizing variety; however, they have some associated
disadvantages that may harm the effort in many cases. Experts
may lose concentration, which will diminish their accuracy to
identify. Accuracy to identify can be affected by inter-and-intra
expert variability or subjectivity. An alternative solution in
identifying a mango variety is to automate the process of
identification to minimize subjectivity or human error.
Automation of the process of identification is done through image
processing in extracting the different features of the mango leaf
image and loaded to the artificial neural network (ANN) for
recognition. An ANN is a computational structure inspired by the
study of biological neural processing, specifically to an important
class of neural network that perform useful computations through
a process of learning [3]. The automation of identifying a mango
variety through its leaf digital image is the main goal of this study.
2 RELATED WORKS
Research on plant recognition is complex and difficult because
there are so many different plant species. Plant leaf is widely used
for plant recognition because it is the easiest to collect and the
most abundant type of data available [4]. In this study, they
introduced plant identification application on Android devices.
They proved the robustness of kernel descriptor (KDES) for many
object recognition applications.
The work of [5] proposed a strategy in recognizing plants from
their leaf images utilizing the different visual features to manage
non-homogeneous types of plants. Considering that plant leaves
can have different identifiable features like color, shape and
 PCSC2019, March 2019, City of Manila, Philippines W. Tabada and J. Beltran

texture, a single set of features may not be sufficiently enough for
complete classification of non-homogeneous types of plants. In a
layered approach, each layer manipulates a specific set of features
to produce its data model. Features from the different layers are
then fed to the classifier layer for a more vigorous classification.
The work of [6] suggested to use shape, vein, color and texture
features combined with Zernike movements to classify plants
using the Radial Basis probabilistic neural network (RBPNN). The
RBPNN is trained using the dual stage training procedure, which
significantly improved the performance of the classifier.
3 PROPOSED MECHANISM
3.1 System Architecture
Figure 1 shows the system architecture of the mango variety
recognizer using image processing and neural network.
to the grayscale image to completely filter out noise interference
caused by sharp wave, while protecting the edge of the target
image. The filtered grayscale image is then segmented into two:
the leaf image in black and the background in white. Dilation and
erosion were then applied to the binary image. Dilation is applied
for enlarging the boundaries of regions of foreground pixels while
holes within those regions become smaller. Erosion is applied for
shrinking the binary image. Median filter, erosion and dilation are
methods available in imageJ[7].
3.4 Image segmentation
Image segmentation is the partition of an image into regions or
categories, which correspond to different objects or parts of
objects [8]. The region of interest (ROI) in this study is the leaf
image, which is segmented from its background using the Otsu
thresholding method [9]. The optimum threshold value in Otsu's
thresholding method is established by maximizing the weighted
sum of between-class variances of foreground and the background
pixels.

3.5 Feature Extraction

After segmentation, the color, morphological and texture features
are extracted from the ROI. In the RGB model, a color is expressed
in terms of the amounts of Red, Green and Blue light in the pixel.
In the Hue, Saturation, and Value (HSV) model, a pixel’s
brightness (V) is calculated using the following equations [10]:
Max = Max (R, G, B) (1)
Min = Min (R, G, B) (2)
V = Max (3)
𝑀𝑎𝑥−𝑀𝑖𝑛
S= (4)
𝑀𝑎𝑥
𝐺−𝐵
when V = R
𝑀𝑎𝑥−𝑀𝑖𝑛
𝐵−𝑅
Figure 1. System architecture of the mango variety H= 2+ when V = G (5)
𝑀𝑎𝑥−𝑀𝑖𝑛
recognizer 4+
𝑅−𝐺
when V = B
𝑀𝑎𝑥−𝑀𝑖𝑛

3.2 Image Acquisition

A Canon IXY digital camera, which has a 14.1MP resolution, was
used in capturing digital images of mango leaves. An 8" x 10"
illustration board with the white side facing the camera was
placed at the back of the selected leaf. The camera is positioned
approximately perpendicular to the plane of the leaf and close
enough such that the camera captures the entire leaf. The ten leaf
samples for each of the four mango varieties were taken from the
different branches of the same mango tree. The third or fourth leaf
from the apex is the one selected as the sample leaf.
In YCbCr color model, ‘Y’ indicates luminance component and Cb,
Cr indicates color component.
Y = 0.299R + 0.587G + 0.114B
Cb = -0.1687R – 0.3313G + 0.500B + 128 (6)
Cr = 0.500R – 0.4187G – 0.0813B + 128
Figure 2 shows the color features comparison of the four mango
varieties. It showed that there were differences of color features
among the four varieties of mango.

3.3 Image Preprocessing

Image preprocessing steps were carried out before the actual
analysis of the image data to eliminate the noise and correct the
distorted or degraded data.
The leaf image whose pixel has three RGB components is
transformed to a grayscale value. Median filter was then applied
 Mango Variety Recognizer Using Image Processing and Artificial
PCSC2019, March 2019, City of Manila, Philippines
Neural Network

𝑁𝑔 −1
𝑁𝑔 −1
𝑐𝑜𝑣 = ∑𝑖=0 ∑𝑗=0 (𝑖 − 𝑚𝑒𝑎𝑛)(𝑗 − 𝑚𝑒𝑎𝑛)𝑃(𝑖, 𝑗) (11)
Inverse Difference Moment measures the contrast of an image.
𝑁𝑔 −1
𝑁𝑔 −1 𝑃(𝑖,𝑗)
𝑖𝑑𝑚 = ∑𝑖=0 ∑𝑗=0 (12)
1+(𝑖−𝑗)2

Entropy measures the amount of order in an image.

𝑁𝑔 𝑁𝑔−1 −1
𝑒𝑛𝑡 = − ∑𝑖=0 ∑𝑗=0 𝑃(𝑖, 𝑗) x log(P(i, j)) (13)
Information measures of correlation do not exhibit any apparent
physical interpretation.
𝑒𝑛𝑡𝑟𝑜𝑝𝑦−𝐻𝑋𝑌1
𝑖𝑚𝑐1 = (14)
max{𝐻𝑋,𝐻𝑌}
1
𝑖𝑚𝑐2 = [1 − 𝑒 −2(𝐻𝑋𝑌2− 𝑒𝑛𝑡) ]2 (15)
where:
Figure 2. Color features comparison among the four 𝑁𝑔 𝑁𝑔 −1 −1

mango varieties 𝐻𝑋𝑌1 = − ∑𝑖=0 ∑𝑗=0 𝑃𝑥 (𝑖, 𝑗) 𝑙𝑜𝑔[𝑃𝑥 (𝑖)𝑃𝑥 (𝑗)]

−1 −1
𝐻𝑋𝑌2 = − ∑𝑁𝑔 ∑𝑁𝑔 𝑃𝑥 (𝑖)𝑃𝑥 (𝑗) 𝑙𝑜𝑔[𝑃𝑥 (𝑖)𝑃𝑥 (𝑗)]
Spatial arrangement of color in an image is called the textural 𝑖=0 𝑗=0

feature of an image [6]. The identification of explicit textures in 𝑁𝑔−1

𝐻𝑋 = ∑𝑖=0 𝑃𝑥 (𝑖)𝑙𝑛𝑃(𝑖)
an image is determined by modeling texture as a two-dimensional
gray level variation known as Gray Level Co-occurence Matrix 𝑁𝑔−1
𝐻𝑌 = ∑𝑖=0 𝑃𝑦 (𝑖)𝑙𝑛𝑃(𝑖)
(GLCM). Spatial Gray Level Dependency Matrix (SGLDM) is a
statistical method of constructing co-occurrences matrices to Figure 3 shows the comparison of the normalized texture features
show the spatial distribution of gray levels in the ROI. Properties among the four varieties of mango. It showed that there were
of SGLDM like angular moment, mean intensity level, variance, differences of texture values among the four varieties of mango.
covariance correlation, inverse difference moment, entropy and
information correlations 1 and 2 are computed from the hue
content of the image using equations 7 to 16 [11].
Angular moment is a measure of image homogeneity.
𝑁𝑔 −1
𝑁𝑔 −1
asm = ∑𝑖=0 ∑𝑗=0 [𝑝(𝑖, 𝑗)]2 (7)

Mean intensity level is a measure of image brightness.

mean = (𝜇𝑥 + 𝜇𝑦 )/2 (8)
𝑁𝑔−1 𝑁𝑔−1

𝑤ℎ𝑒𝑟𝑒 𝜇𝑥 = ∑ 𝑖𝑃𝑥 (𝑖) 𝑎𝑛𝑑 𝜇𝑦 = ∑ 𝑗𝑃𝑦 (𝑗)

𝑖=0 𝑗=0

Variance texture is a measure of variation of intensity.

−1 −1
var = ∑𝑖=0
𝑁𝑔
∑𝑁𝑔 2
𝑗=0 (𝑖 − 𝑚𝑒𝑎𝑛) 𝑃𝑥(𝑖,𝑗) (9)
Figure 3. Textural features comparison among the four
Correlation is a measure of intensity linear dependence. mango varieties

𝑁𝑔−1 𝑁𝑔−1 𝑖∙𝑗∙𝑃(𝑖,𝑗)−𝜇𝑥 ∙𝜇𝑦

𝑐𝑜𝑟 = ∑𝑖=0 ∑𝑗=0 (10) Image moments have been used in computer vision as feature
𝜎𝑥 ∙𝜎𝑦
descriptor as cited in [12]. Seven moment invariants from
where : geometric moments that are invariants to rotation were defined
𝑁𝑔−1 in [13]. These orthogonal invariants are independent of position,
𝜎𝑥2 = ∑𝑖=0 [𝑃𝑥 (𝑖) − 𝜇𝑥 (𝑖)]2 𝑃(𝑖) size, orientation and parallel projection. [13] defines the formula
𝑁𝑔−1 2 for each of the seven moment invariants as follows:
𝜎𝑦2 = ∑𝑖=0 [𝑃𝑦 (𝑖) − 𝜇𝑦 (𝑖)] 𝑃(𝑖)
𝑀1 = 𝑁20 + 𝑁02 (16)
Covariance of the intensity (cov) is obtained by
 PCSC2019, March 2019, City of Manila, Philippines W. Tabada and J. Beltran

2
𝑀2 = (𝑁20 − 𝑁02 )2 + 4𝑁11 (17) varieties. These 25 features were mapped into the 25 nodes of the
input layer, 50 nodes in the hidden layer and 4 nodes in the output
𝑀3 = (𝑁30 − 3𝑁12 )2 + (3𝑁21 − 𝑁03 )2 (18)
layer. The four nodes in the output layer correspond to the four
𝑀4 = (𝑁30 − 𝑁12 )2 + (𝑁21 − 𝑁03 )2 (19) varieties of mango. Input layer nodes are connected to the hidden
𝑀5 = (𝑁30 − 3𝑁12 )2 𝑥 (𝑁30 − 𝑁12 )2 𝑥 layer nodes with weights. The initial values of weights were
chosen randomly between -1.0 and 1.0, and the backpropagation
((𝑁30 + 𝑁12 )2 − 3(𝑁21 − 𝑁03 )2 ) algorithm was used to adjust the weights until it reached the
+ (3𝑁21 − 𝑁03 )2 𝑥 (𝑁21 − 𝑁03 )2 𝑥 predetermined minimum error or reached the maximum epochs.

(3(𝑁30 + 𝑁12 )2 − (𝑁21 − 𝑁03 )2 ) (20)

𝑀6 = (𝑁20 − 𝑁02 ) 𝑥 ((𝑁30 + 𝑁12 )2 − (𝑁21 − 𝑁03 )2 )
+4𝑁11 𝑥 (𝑁30 + 𝑁12 ) 𝑥 (𝑁21 + 𝑁03 ) (21)
𝑀7 = (3𝑁21 + 𝑁03 ) 𝑥 (𝑁30 + 𝑁12 ) 𝑥
((𝑁30 − 𝑁12 )2 − 3(𝑁21 − 𝑁03 )2
−(𝑁30 + 3𝑁12 ) 𝑥 (𝑁21 + 𝑁03 )𝑥
(3(𝑁30 − 𝑁12 )2 ) − (𝑁21 − 𝑁03 )2 ) (22)
where:
𝜇𝑝𝑞
𝑁𝑝𝑞 = 𝛾
𝜇00

p, q is the order and γ = (p+q+1) is a normalization factor

𝜇𝑝𝑞 = ∑𝑀−1 𝑁−1 𝑝
̅)𝑞 𝑓(𝑥, 𝑦)
𝑥=0 ∑𝑦=0 (𝑥 − 𝑥̅ ) (𝑦 − 𝑦
𝑚10 𝑚01
𝑥̅ = 𝑦̅ =
𝑚00 𝑚00

𝑚𝑝𝑞 = ∑𝑀−1 𝑁−1 𝑝 𝑞

𝑥=0 ∑𝑦=0 𝑥 𝑦 𝑓(𝑥, 𝑦)

Figure 5. The neural network architecture of the mango

Figure 4 shows the comparison of the seven (7) moments
invariants among the four mango varieties. The differences of
moments invariants for each mango varieties are clearly visible.
variety recognizer
4 EXPERIMENTAL RESULT
4.1 Training the Mango Variety Recognizer
The artificial neural network was trained using 10 samples of leaf
image for each of the 4 mango varieties. Different values of the
training parameters were tried to obtain the minimum error at an
acceptable duration. The choice of the values of learning rate, and
momentum was based upon on the five trials made on the
different values of the two parameters whose results are shown in
Table 1. Trial 3 whose learning rate is 0.2 and a momentum of 0.03
gave a reasonable mean error of 0.0032 and accuracy of 96%.

Table 1. Trials on the different values of learning rate and

momentum.

Figure 4. Moment invariants comparison among the four

mango varieties.

3.6 Neural Network Architecture

Figure 5 shows the multilayer perceptron neural network
architecture of the mango variety recognizer. There were 25
features extracted from each of the sample leaf images of 4 mango
 Mango Variety Recognizer Using Image Processing and Artificial
PCSC2019, March 2019, City of Manila, Philippines
Neural Network

Learning rate is the parameter that controls the adjustment of
weights of the neural network to minimize the loss gradient
function [14]). Momentum updates removes some of the noises
and oscillations that gradient descent has, in particular in the
directions of high curvature of the loss function [15].
Figure 6 shows the mean square error convergence curve of the
mango variety recognizer.
leaves of mangoes, the recognizer mistook the two Pahutan
variety leaves as Pico variety leaves while a Pico variety leaf was
also mistaken as a Pahutan leaf. The leaves of two mango varieties
almost look the same as shown in Figure 7. Overall, the accuracy
of the mango variety recognizer is 96%.

Figure 6. Mean square error convergence curve of the

mango variety recognizer with a learning rate of 0.2 and
momentum of 0.03 which obtain a minimum error of Figure 7. Similarity of the Pahutan and Pico mango leaves
0.0032.
4.3 User-interface of the Recognizer
The mango recognizer was developed using the Java
4.2 Accuracy of the Recognizer programming language with some of the image processing
The mango variety recognizer was tested with 40 samples of plugins of imageJ. Figure 8 shows the screenshot of the mango
mango leaves images that were used to train the recognizer and recognizer user-interface for loading leaf image, extract the
40 samples of newly acquired mango leaves images. The accuracy twenty five features, save the features to the database and run the
of the recognizer was computed using the formula [16]: supervised training process when the sample leaf images are
𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒
already sufficient.
𝑎𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = × 100 (23)
𝑇𝑜𝑡𝑎𝑙 𝑆𝑎𝑚𝑝𝑙𝑒𝑠

where : True Positive = number of samples whose variety was

correctly identified.
Table 2. Results of testing the accuracy of the mango
variety recognizer.
Correctly Falsely
Leaf
Variety Identified Identified
samples
N % N %
Training data
Carabao 10 10 100% 0 0%
Indian 10 10 100% 0 0%
Pahutan 10 10 100% 0 0%
Pico 10 10 100% 0 0%
Test Data
Carabao 10 10 100% 0 0%
Figure 8. A screenshot of the Training tab.
Indian 10 10 100% 0 0%
Pahutan 10 8 80% 2 20% Figure 9 shows the screenshot of the mango recognizer user-
Pico 10 9 90% 1 10% interface in loading the mango leaf image whose variety is to be
Grand Total 80 77 96% 3 4% identified yet, extract its features and display the name and
description of the variety that matched the mango leaf image.
Table 2 shows the results of the test. The mango variety
recognizer was able to identify all the sample leaves that were
used in the training, however, on the newly acquired sample
 PCSC2019, March 2019, City of Manila, Philippines W. Tabada and J. Beltran

[9] Otsu, Nobuyuki. 1979. A threshold selection method from gray-level

histograms. 1, s.l. : IEEE Transactions on Systems, Man, and Cybernetics, Vol.
9.
[10] Waldman, Nikolai. Math behind colorspace conversions, RGB-HSL. Waldman
Media. [Online] 2013. [Cited: February 18, 2017.]
http://www.niwa.nu/2013/05/math-behind-colorspace-conversions-rgb-hsl/.
[11] Gadkari, Dhanashree. 2000. Image Quality Analysis Using GLCM. Electronic
Theses and Dissertations. 187. Retrieved from
http://stars.library.ucf.edu/etd/187 on May 22, 2017
[12] Chaumette, François. Image moments: a general and useful set of features for
visual servoing. 4, s.l. : IEEE, 2004, IEEE Transactions on Robotics, Vol. 20, pp.
713-723.
[13] HU, MING-KUEI. 1962. Visual Pattern Recognition by Moment Invariants. IRE
TRANSACTIONS ON INFORMATION THEORY, pp. 179-187.
[14] Mirjalili, Vahid. (2017). "What is the learning rate in neural network",
Retrieved on May 22, 2017 from www.qoura.com:
https://www.qoura.com/What-is-the-learning-rate-in-neural-networks
[15] Yoshua Bengio. 2012. " Practical Recommendations for Gradient-Based
Training of Deep Architectures", Retrieved on May 18, 2018 from
https://arxiv.org/pdf/ 1206.5533v2.pdf
Figure 9. A screenshot of the Recognition tab. [16] Aswathi, B.L. Sensitivity, Specificity, Accuracy and the relatioship between
them. [Online] 2009. [Cited: May 31, 2018.] http://www.lifescience.com/
bioinformatics/sesitivity-specificity-accuracy-and.
5 CONCLUSION AND RECOMMENDATIONS
A software that can identify a mango variety using its leaf image
was successfully developed in this study. The Otsu thresholding
method was used in segmenting the region of interest from the
background. Nine color features, seven moments invariant
features and nine textural features were extracted using image
processing techniques. The software was successfully trained
using the back propagation algorithm with ten samples of leaf
images for each of the four mango varieties. The accuracy of the
software to identify a variety is 96%.
The accuracy of the mango variety recognizer can possibly be
improved by increasing the number of leaf image samples during
the training of the recognizer and to make sure that the different
growth stages of leaves are well represented. Accuracy can be also
improved by adding more features such as the Zernike Moments
which can classify complex images structurally [7].

REFERENCES
[1] Mehta, Indu. History of Mango – ‘King of Fruits’. 7, Nainital, Uttarakhand
(India) : s.n., July 2017, International Journal of Engineering Science Invention,
Vol. 6, pp. 20-24. ISSN (Online): 2319 – 6734, ISSN (Print): 2319 – 6726.
[2] Fresh Tropical Fruits. [Online] May 14, 2017. http://investphilippines.gov.ph.
[3] Rao, Valluru B. and Rao, Hayagriva. C++ Neural Networks and Fuzzy Logic.
s.l. : MTBooks, IDG Books Worldwide, Inc., 1995. ISBN: 1558515526.
[4] Le, Thi-Lan and Tran, Duc-Tuan and Hoang, Van-Nam. 2014. Fully automatic
leaf-based plant identification, application for Vietnamese medicinal plant
search. Hanoi, Vietnam : Proceedings of the Fifth Symposium on Information
and Communication Technology. pp. 146-154. ISBN: 978-1-4503-2930-9.
[5] Chaki, Jyotismita, Parekh, Ranjan and Bhattacharya, Samar. Kolkata. 2016.
Plant Leaf Recognition Using a Layered Approach. India : 2016 International
Conference on Microelectronics, Computing and Communications
(MicroCom). ISBN: 978-1-4673-6621-2.
[6] A.H. Kulkarni, Dr. H.M.Rai, Dr. K.A.Jahagirdar, P.S.Upparamani. 2013. Leaf
Recognition Technique for Plant Classification Using RBPNN and Zernike
Moments. Belgaum, India : s.n., 2013. International Journal of Advanced
Research in Computer and Communication Engineering. ISSN (Print) : 2319-
5940.
[7] Schneider, C. A.; Rasband, W. S. & Eliceiri, K. W. (2012), "NIH Image to ImageJ:
25 years of image analysis", Nature methods 9(7): 671-675, PMID 22930834 (on
Google Scholar).
[8] Horgan, C. A. Glasbey & G. W. IMAGE ANALYSIS FOR THE BIOLOGICAL
SCIENCES. s.l. : Wiley, 1 Edition, 1995. ISBN-13: 978-0471937265.