Available online at www.sciencedirect.

com
ScienceDirect
ScienceDirect
Procedia online
Available Computer
at Science 00 (2018) 000–000
www.sciencedirect.com
Procedia Computer Science 00 (2018) 000–000 www.elsevier.com/locate/procedia
www.elsevier.com/locate/procedia
ScienceDirect
Procedia Computer Science 135 (2018) 214–221

3rd International Conference on Computer Science and Computational Intelligence 2018

3rd International Conference on Computer Science and Computational Intelligence 2018
Thai License Plate Recognition Based on Deep Learning
Thai License Plate Recognition Based on Deep Learning
Wichai Puarungroj*, Narong Boonsirisumpun
Wichai Puarungroj*, Narong Boonsirisumpun
Computer Science Department, Faculty of Science and Technology, Loei Rajabhat University, Loei,42000, Thailand
Computer Science Department, Faculty of Science and Technology, Loei Rajabhat University, Loei,42000, Thailand

Abstract
Abstract
Recognizing vehicle’s license plate is necessary because the number of vehicles is increasing and it goes beyond human’s ability
Recognizing
to complete this vehicle’s license
task. The vehicle
platelicense recognition
is necessary because system commonly
the number combines
of vehicles and itlicense
2 sub-systems:
is increasing goes beyond human’s ability
plate detection, which
aims to locate
to complete thetask.
this vehicle
The and its license
vehicle licenseplate; and license
recognition systemplate recognition,
commonly which 2aims
combines to recognize
sub-systems: the characters
license on the which
plate detection, plate.
This
aims research
to locate focuses
the vehicleon the
andlatter part by
its license researching
plate; and licenseintoplate
Thairecognition,
motorcycle which
licenseaims
plate.toThis type of
recognize theplate is a triple-line
characters type
on the plate.
containing
This research 3 lines
focusesof character sets,part
on the latter which represent category,
by researching into Thairegistered
motorcycleplace, and registered
license plate. Thisnumbers respectively
type of plate from top
is a triple-line to
type
bottom.
containing Differently
3 lines offrom a one-line
character sets, type
whichlicense plate,category,
represent the research proposed
registered an approach
place, of two-step
and registered numbers training and recognizing
respectively from top byto
firstly
bottom.segmenting
Differentlyand fromtraining the type
a one-line 3 lines of each
license license
plate, plate byproposed
the research using deep learning of
an approach (MobileNets and Inception-v3);
two-step training and recognizingin the
by
second step, Thai letters
firstly segmenting and numbers
and training on theofplate
the 3 lines eachwere segmented
license plate byand trained.
using deep The models
learning from theseand
(MobileNets trainings were deployed
Inception-v3); in the
with
secondthestep,
whole license
Thai lettersplate
and testing.
numbersA on testthe
was carried
plate wereout on LP images
segmented with varied
and trained. angles of
The models orientation
from and thewere
these trainings accuracy rate
deployed
of line
with theand character
whole licenserecognition wereA96.94%
plate testing. test wasand 91.76%
carried out respectively.
on LP images with varied angles of orientation and the accuracy rate
of line and character recognition were 96.94% and 91.76% respectively.
© 2018 The Authors. Published by Elsevier Ltd.
© 2018
© 2018
This is anThe
The Authors.
open Published
accessPublished
Authors. article underby Elsevier
by Elsevier Ltd.
the CC BY-NC-ND
Ltd. license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Selection
This is an and
openpeer-review
access articleunder
underresponsibility
the CC BY-NC-NDof the 3rdlicense
International Conference on Computer Science and Computational
(https://creativecommons.org/licenses/by-nc-nd/4.0/)
Selection and peer-review under responsibility of the 3rd International Conference on Computer Science and Computational
Intelligence
Selection
Intelligence and2018.
peer-review under responsibility of the 3rd International Conference on Computer Science and Computational
2018.
Intelligence 2018.
Keywords: Deep Learning, License Plate Recognition, Thai Motorcycle, MobileNets, Inception, Convolutional Neural Networks
Keywords: Deep Learning, License Plate Recognition, Thai Motorcycle, MobileNets, Inception, Convolutional Neural Networks

1. Introduction
1. Introduction
License plate recognition (LPR) has received tremendous interest in recent years as a challenging research
License
topic 1,7,11 plateis recognition
. This (LPR)
due to the fact that has
the received
conditionstremendous
(e.g. light, interest in recent
color, dirt, years
shadows, as a challenging
character research
sharpness, language
topic1,7,11. This is due to the fact that the conditions (e.g. light, color, dirt, shadows, character sharpness, language

* Corresponding author. Tel.: +668-4160-1033; fax: +66-42-81-1072.

* E-mail address:author.
Corresponding wichai@lru.ac.th
Tel.: +668-4160-1033; fax: +66-42-81-1072.
E-mail address: wichai@lru.ac.th
1877-0509 © 2018 The Authors. Published by Elsevier Ltd.
This is an open
1877-0509 access
© 2018 Thearticle under
Authors. the CC BY-NC-ND
Published license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
by Elsevier Ltd.
Selection
This is an and
openpeer-review under
access article responsibility
under of the 3rdlicense
the CC BY-NC-ND International Conference on Computer Science and Computational Intelligence 2018.
(https://creativecommons.org/licenses/by-nc-nd/4.0/)
Selection and peer-review under responsibility of the 3rd International Conference on Computer Science and Computational Intelligence 2018.

1877-0509 © 2018 The Authors. Published by Elsevier Ltd.

This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Selection and peer-review under responsibility of the 3rd International Conference on Computer Science and Computational
Intelligence 2018.
10.1016/j.procs.2018.08.168
 Wichai Puarungroj et al. / Procedia Computer Science 135 (2018) 214–221 215
2 Wichai Puarungroj and Narong Boonsirisumpun / Procedia Computer Science 00 (2018) 000–000

and etc.) and types of license plates are varied from place to place4,17. LPR has become an important part of many
applications, for example road safety enforcement8, automatic parking lot control6,8, automatic toll collection7,12,
speed limit enforcement, and vehicle tracking and identification. LPR system may be installed as a part of traffic
monitoring systems working together with traffic light in order to identify the car that break traffic rules or detect
prohibited vehicles. In many cases, LPR system is useful for recognizing motorcycles with harmful behaviors, for
example, riding against the traffic direction, riding over speed limit, and not wearing helmet. A number of
motorcycles have been used in Thailand, where these behaviors are regularly found and the accidents occurring from
these behaviors are extremely high. Motorcycles have been widely used as household vehicles and used by students
as vehicles to go to school. Therefore, road safety enforcement is now getting high attention as an important issue in
order to reduce the possibility of accidence from irresponsible motorists. This research aims to apply a deep learning
approach to recognize motorcycle license plates in Thailand. Although many robust approaches have been employed
by prior research, deep learning approach has gain dramatic attention in recent days in the field of pattern
recognition. Most of Thai LPR research has focused on recognizing car LPs23,24. In recognizing the characters, many
algorithms have been found in the literature. For example, Hidden Markov model19; hybrid discriminative restricted
Boltzmann machine5, Support Vector Machine (SVM), which has been employed for Chinese license plates8, Iranian
license plates6 and Thai license plates24; template matching9,10,13 is a technique suitable for recognizing characters
that have single-font, not rotated, and fixed-size properties3,4. Additionally, artificial neural network has also
received high attention due to its great capability in classifying license plates2.

2. Literature Review

2.1. License Plate Recognition Methods

There have been many robust approaches for recognizing letters and numbers on the license plates after passing
the process of license plate detection. Anagnostopoulos18 has classified approaches for conducting character
recognition into two main categories: classifiers and pattern matching as shown in Table 1.

Table 1. Approaches for LPR18.

Category Approach Description of Algorithm Applied
Classifiers Statistical classifiers Hidden Markov models
Support Vector Machines (SVM)
Likelihood models in tree hierarchy
Computational Multi layered feed-forward neural networks
Intelligence Adaptive resonance theory neural networks
Self organized neural networks
Probabilistic neural networks
Learning vector quantization neural network
Pattern matching Template matching Normalized cross correlation
Hausdorff distance
Root mean squared error (RMSE) for all the template shifts
Image partition: zoning, projections, contour distance,
segment count

From Table 1., various algorithms have been applied for LPR. The approaches and relevant algorithms have
proved their high performance. Hidden Markov model, for example, is one of the robust classifiers that have been
deployed for recognizing the license plates19 so as to SVM6,8. Additionally, artificial neural network has also
received high attention from its great capability in classifying license plates 2. In case of template matching, it has
been implemented widely9,10,13,14. This approach generates templates of each letter (e.g. A-Z or letters in other
216 Wichai Puarungroj et al. / Procedia Computer Science 135 (2018) 214–221
Wichai Puarungroj and Narong Boonsirisumpun / Procedia Computer Science 00 (2018) 000–000 3

languages) and number from the training dataset. These templates are then used to match the characteristic of each
letter and number on the plate. Although this approach has shown great performance in LPR, it still lacks capability
in handling characters with various positions3,4,6. Apart from these approaches, a recent deep learning approach
based on Convolutional Neural Networks (CNN) is receiving high attention due to its great classification and
prediction performance. This deep learning method has been in its beginning state of applying in an LPR research
field. Some papers have reported CNN experiments for license plate recognition11,15,17.

2.2. Inception-v3 and MobileNets

Some examples of deep learning method are Inception-v321 and MobileNets16. Inception-v3 was an improved
version of the original model called GoogLeNet or Inception-v1 20. GoogLeNet was the new idea of CNN with an
addition module call inception module, the module that developed to find the optimal local construction in each
layer and to repeat it spatially. Each inception layer was constructed by the different sizes of convolution node (1x1,
3x3, and 5x5) and 3x3 max pooling node (See Fig. 1).

Fig. 1. Structure of Inception module from GoogLeNet20.

Inception-v3 implemented a new idea into inception node by factorizing the convolution node and made them
smaller. For example, they factorize one nxn kernel into nx1 and 1xn kernels (See Fig. 2). The new idea of
Inception-v3 achieves better performance than the original one both in term of image recognition accuracy and time
consuming21.

Fig. 2. Factorization process in Inception-v321.

 Wichai Puarungroj et al. / Procedia Computer Science 135 (2018) 214–221 217
4 Wichai Puarungroj and Narong Boonsirisumpun / Procedia Computer Science 00 (2018) 000–000

However, the problem of GoogLeNet (inception) was the size of their training model. There is another
comparable model called MobileNets16 that surpass GoogLeNet in term of scale. MobileNets was developed by
Andrew G. Howard et al in 201716 with the objective to reduce the size of deep learning model and can be used for a
mobile version by using the technique call Depthwise Separable Convolution (See Fig.3). MobileNets can achieve
the close accuracy to Inception-v3 but come with a lot smaller size.

Fig. 3. Depthwise Separable Convolution in MobileNets16.

2.3. SSD (Single Shot multibox Detector)

Inception-v3 and MobileNets are the examples of deep learning methods for image recognition. However, the full
process in image recognition needs one more step called image segmentation and those two models alone could not
achieve both processes at the same time. Another technique called SSD (Single Shot multibox Detector) 22 has been
used to solve this problem. SSD is the technique to create the bounding box (an area) of each segment with high
probability to be the same object in the image. SSD can work together with other image recognition models
(Inception-v3, MobileNets or else) to detect multiple object bounding boxes and recognize their categories at the
same time (See Fig.4).

Fig. 4. Example result of SSD and MobileNets16.

3. Research Methodology

The research process mainly comprised training and testing phases in order to make the system to be able to
classify and recognize characters and number on the LP. The methods carried out in this research can be illustrated
as shown in Fig. 5.
218 Wichai
Wichai Puarungroj Puarungroj
and Narong et al. / Procedia
Boonsirisumpun Computer
/ Procedia ScienceScience
Computer 135 (2018) 214–221
00 (2018) 000–000 5

Fig. 5. The research process.

The first step was an LP image collection. In this step, 710 real high-resolution images of motorcycle LPs with
varied angles of orientation were collected by photo taking. The image angles were not fixed in order to support the
characteristics of the motorcycles when moving past the camera. Examples of LP images are shown in Fig. 6.

Fig. 6. Examples of LP images.

In the second step, each LP image was then segmented into 3 parts according to its triple-line type of LP by using
SSD. The SSD processes have been carried out separately for Inception-v3 and MobileNets. The character sets on
Motorcycle LPs in Thailand are divided into 3 lines as illustrated in Fig. 7. The first line comprises three digits of a
combination of numbers and Thai letters by beginning with one number (from 1 to 9) and then followed by two
letters, which represent LP categories. The letters come from any of 35 Thai characters out of the whole 44 Thai
characters. From these 44 characters, nine characters are excluded from use due to their similarities to some of the
first 35 characters. This exclusion is intended to reduce confusion and blurred vision when viewing remotely. The
second line contains the name of the registered province. In this line, there can be 78 distinct names, which come
from 77 provinces and one special administrative region. The third line represents registered LP number, which
contains no more than 4 digits.
In the third step, the letters, numbers, and the name of registered places in LPs were segmented by using SSD. At
least 100 of images of each letter and number were needed for putting forward into training. In the fourth step, the
trainings were conducted in two parts: the first part used the results of three lines segmentation from the second step
to make the system distinguish the lines of LP to train by using Inception-v3 and MobileNets then the performance
of the model from this training was evaluated and the better model was selected for further use. After that the
selected model was used to train and test the letters, numbers, and the name of registered places from the third step.
 Wichai Puarungroj et al. / Procedia Computer Science 135 (2018) 214–221 219
6 Wichai Puarungroj and Narong Boonsirisumpun / Procedia Computer Science 00 (2018) 000–000

Fig. 7. An example of Thai motorcycle LP.

4. Results

From the total of 710 license plate images, 590 images were used to train by MobileNets and Inception-v3, while
the rest (120 images) were used to test in our experiment. The first test was conducted to segment and detect the 3-
lines of characters in each LP. After training with 590 images of input dataset, the file sizes of predictive models
produced by Inception-v3 and MobileNets were 53,236 kB, and 22,269 kB respectively. The size of MobileNets was
a lot smaller than that of Inception-v3. The results of the experiment with 120 images using the two algorithms are
shown in Table 2. There were 360 lines in total from 120 images.

Table 2. Results of 3-Lines Segment and Detection.

Inception-v3 Percent MobileNets Percent
Number of Lines Detected 349 96.94 349 96.94
Number of Lines Undetected 11 3.06 11 3.06
Number of Lines Detected and Correct 349 96.94 349 96.94
Number of Lines Detected, but Incorrect 0 0 0 0

From Table 2., line segmentation and detection using Inception-v3 and MobileNets give the same accuracy
result. Out of 360 lines, 349 are detected and correct (96.94%), while 11 lines are not detected (3.06%). The second
test was conducted to segment and recognize characters in each LP. In this test, MobileNets was chosen due to its
smaller size model. The results from the experiment with 120 images, which contained 886 characters (Thai letters
and numbers) are shown in Table 3 and an example of the outcomes from the experiment is shown in Fig.8.

a b

Fig. 8. (a) 3-line recognition; (b) character recognition.

From Table 3., the accuracy of character recognition by MobileNets was high (91.76%); however, some
characters were not detected (6.09%). Most of the problems found in this experiment came from the lack of enough
training dataset of some characters (e.g. ณ, ร, ด, ว, ย, ธ, ฉ) and some registered provinces (e.g. เพชรบูรณ์, ลำพูน,
นครปฐม) and problems of too dark images, and containing nuts and bolts on the images. Another challenge of this
220 Wichai Puarungroj et al. / Procedia Computer Science 135 (2018) 214–221
Wichai Puarungroj and Narong Boonsirisumpun / Procedia Computer Science 00 (2018) 000–000 7

research was that many of LP frames were decorated with images and characters. Some parts of the data on the plate
were covered by the improper LP frame as illustrated in Fig 9. The results of our experiment, however, provide
satisfactory results as the model could recognize the unclear numbers as illustrated in Fig 10.

Table 3. Results of Character Recognition.

MobileNets Percent
Number of Characters Detected 832 93.91
Number of Characters Undetected 54 6.09
Number of Characters Detected and Correct 813 91.76
Number of Characters Detected, but Incorrect 19 2.14

Fig. 9. Examples of data on LP hidden by improper LP frames.

a b

Fig. 10. (a) 3-line recognition of LP in an improper frame; (b) character recognition of LP in an improper frame.

5. Conclusions

In this paper, the process of license plate recognition based on deep learning has been presented. The research
focuses on recognizing Thai motorcycle license plate, which is a triple-line plate type. The lines from top to bottom
represent categories, registered places, and registered numbers respectively. 710 real license plate images with
varied angles of orientation were collected. In this research, the two-steps segmentation and training were proposed.
Firstly, the three lines of license plates were segmented by using SSD and trained by using Inception-v3 and
MobileNets and secondly, the characters on license plates were segmented and trained by the same set of
approaches. The models from the training steps were employed to test the whole set of license plate images and the
accuracy rates for line and character recognition were 96.94% and 91.76% respectively, which is suitable for further
deployment of LPR. The challenge in this research was that many license plates were partly covered by their
improper frames. The frame hid some parts of the plate even viewed by human’s eyes. From this case, the research
found that the characters in this case were still detected and recognized. For future work, the models will be trained
further with more datasets on various conditions and the developed LPR will be combined with the motorcycle and
helmet wearing system to work with surveillance video camera system in order to conduct traffic safety
enforcement.
 Wichai Puarungroj et al. / Procedia Computer Science 135 (2018) 214–221 221
8 Wichai Puarungroj and Narong Boonsirisumpun / Procedia Computer Science 00 (2018) 000–000

References

[1] Du, Shan, Ibrahim, Mahmoud, Shehata, Mohamed, and Wael Badawy. (2013) “Automatic license plate recognition (ALPR): A state-of-the-
art review.” IEEE Transactions on Circuits and Systems for Video Technology 23 (2): 311-325.
[2] Tarigan, Joseph, Nadia, Diedan, Ryanda, and Yaya Suryana. (2017) “Plate recognition using backpropagation neural network and genetic
algorithm.” Procedia Computer Science 116: 365-372.
[3] Anagnostopoulos, Christos-Nikolaos E, Anagnostopoulos, Ioannis E, Loumos, Vassili, and Eleftherios Kayafas (2006) “A license plate-
recognition algorithm for intelligent transportation system applications” IEEE Transactions on Intelligent Transportation Systems 7(3): 377-
392.
[4] Anagnostopoulos, Christos-Nikolaos E., Anagnostopoulos, Ioannis E., Psoroulas, Ioannis D. Loumos, Vassili, and Eleftherios Kayafas (2008)
“License plate recognition from still images and video sequences: A survey.” IEEE Transactions on Intelligent Transportation Systems 9(3):
377-391.
[5] Gou, Chao, Wang, Kunfeng, Yao, Yanjie, and Zhengxi Li. (2016) “Vehicle license plate recognition based on extremal regions and restricted
Boltzmann machines.” IEEE Transactions on Intelligent Transportation Systems 17(4): 1096-1107.
[6] Ashtari, Amir Hossein, Nordin, Md. Jan, and Mahmood Fathy (2014) “An Iranian license plate recognition system based on color features.”
IEEE Transactions on Intelligent Transportation Systems 14(4): 1690-1705.
[7] Gonçalves, Gabriel Resende, Menotti, David, and William Robson Schwartz (2016) “License plate recognition based on temporal
redundancy.” IEEE 19th International Conference on Intelligent Transportation Systems (ITSC2016): 2577-2582.
[8] Wen, Ying, Lu, Yue, Yan, Jingqi, Zhou, Zhenyu, von Deneen, Karen M., and Pengfei Shi (2011) “An algorithm for license plate recognition
applied to intelligent transportation system.” IEEE Transactions on Intelligent Transportation Systems 12(3): 830-845.
[9] Rahman, Choudhury A., Badawy, Wael, and Ahmad Radmanesh (2003) “A real time vehicle’s license plate recognition system.” IEEE
Conference on Advanced Video and Signal Based Surveillance (AVSS’03): 163-166.
[10] Comelli, P., Ferragina, P., Granieri, M.N. and F. Stabile (1995) “Optical recognition of motor vehicle license plates.” IEEE Transactions on
Vehicular Technology 44(4): 790-799.
[11] Jain, Vishal, Sasindran, Zitha, Rajagopal, Anoop, Biswas, Soma, Bharadwaj, Harish S., and K. R. Ramakrishnan (2016) “Deep automatic
license A real time vehicle’s license plate recognition system.” IEEE Conference on Advanced Video and Signal Based Surveillance
(AVSS’03): 163-166.
[12] Chang, Shyang-Lih, Chen, Li-Shien, Chung, Yun-Chung, and Sei-Wan Chen (2004) “Automatic license plate recognition.” IEEE
Transactions on Intelligent Transportation Systems 5(1): 42-53.
[13] Yu, Mei, and Yong Deak Kim (2000) “An approach to Korean license plate recognition based on vertical edge matching.” IEEE
International Conference on Systems, Man, and Cybernetics: 2975-2980.
[14] Sarfraz, Muhammad, Ahmed, Mohammed Jameel, and Syed A. Ghazi (2003) “Saudi Arabian license plate recognition system.”
International Conference on Geometric Modeling and Graphics: 36-41.
[15] Wang, Xinlong, Man, Zhipeng, You, Menglong, and Chunhua Shen (2017). “Adversarial Generation of Training Examples: Applications to
Moving Vehicle License Plate Recognition.” arXiv:1707.03124v3.
[16] Howard, Andrew G., Zhu, Menglong, Chen, Bo, Kalenichenko, Dmitry, Wang, Weijun, Weyand, Tobias, Andreetto, M., and Hartwig Adam
(2017). “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications.” CoRR, abs/1704.04861.
[17] Masood, Syed Zain, Shu, Guang, Dehghan, Afshin, and Enrique G. Ortiz (2017) “License Plate Detection and Recognition Using Deeply
Learned Convolutional Neural Networks.” arXiv:1703.07330v2.
[18] Anagnostopoulos, Christos-Nikolaos E (2014) “License plate recognition: a brief tutorial.” IEEE Intelligent Transportation Systems
Magazine 6(1): 59-67.
[19] Duan, Tran Duc, Hong Du, Tran Le, Phuoc, Tran Vinh, and Nguyen Viet Hoang (2005) “Building an automatic vehicle license plate
recognition system.” The International Conference in Computer Science: 59-63.
[20] Szegedy, Christian, Liu, Wei, Jia, Yangqing, Sermanet, Pierre, Reed, Scott, Anguelov, Dragomir, Erhan, Dumitru, Vanhoucke, Vincent, and
Andrew Rabinovich (2014) “Going deeper with convolutions.” arXiv preprint arXiv:1409.4842, 7.
[21] Szegedy, Christian, Vanhoucke, Vincent, Ioffe, Sergey, Shlens, Jonathon, and Zbigniew WojnaHoang (2016) “Rethinking the inception
architecture for computer vision.” The International Conference in Computer Vision and Pattern Recognition: 2818-2826.
[22] Liu, Wei, Anguelov, Dragomir, Erhan, Dumitru, Szegedy, Christian, Reed, Scott, Fu, Cheng-Yang, and Alexander C. Berg (2015) “SSD:
Single shot multibox detector.” arXiv:1512.02325, 2015. 7.
[23] Sa-ngamuang, P., Thamnittasana, C., and T. Condo (2007) “Thai car license plate recognition using essential-elements-based method.” Asia-
Pacific Conference on Communications: 41-44.
[24] Ogiuchi, Yasua, Higashikubo, Masakatsu, Panwai, Sakda, and Ekarin Luenagvilai (2014) “Automatic license plate detection and recognition
in Thailand.” SEI Technical Review 78: 39-43.