Академический Документы
Профессиональный Документы
Культура Документы
DOI 10.1007/s00138-008-0133-3
ORIGINAL PAPER
A hybrid system for embedded machine vision using FPGAs
and neural networks
Miguel S. Prieto Alastair R. Allen
Received: 26 February 2007 / Accepted: 25 February 2008 / Published online: 27 March 2008
Springer-Verlag 2008
Abstract This paper presents a hybrid model for embedded
machine vision combining programmable hardware for the
image processing tasks and a digital hardware implementa-
tion of an articial neural network for the pattern recognition
and classication tasks. A number of possible architectural
implementations are compared. A prototype development
system of the hybrid model has been created, and hardware
details and software tools are discussed. The applicability of
the hybrid design is demonstrated with the development of a
vision application: real-time detection and recognition
of road signs.
Keywords Embedded machine vision FPGA ANN
SOM
1 Introduction
The types of vision application which will form the basis for
mass implementationover the next fewyears are those requir-
ing the embedding of intelligent pattern recognition sys-
tems into industrial processes, instrumentation and portable
systems. These systems will make extreme demands on the
hardware, requiring the integration of traditional image pro-
cessing (IP) with articial intelligence (AI) techniques, and
be implemented in designs with small footprint and low
power consumption. In these kind of designs, there is often
the need for ltering and preprocessing of the images,
M. S. Prieto A. R. Allen (B)
School of Engineering and Physical Sciences,
University of Aberdeen,
Aberdeen AB24 3UE, Scotland, UK
e-mail: a.allen@abdn.ac.uk
preparing them for high-level analysis which frequently
involves the recognition or classication of certain features
which may be present in the image.
This paper investigates possible architectures of a hybrid
system combining programmable hardware (for IP tasks)
with a neural array (for classication tasks). Additionally,
this paper also presents a hybrid prototype system (HPS)
that facilitates the development and test of vision applica-
tions on the hybrid system. Designing a prototype of the
hybrid system has two main purposes. The rst purpose is
to reduce the cost in time and effort of implementing an
embedded vision application. Following the methodology
described and using the suggested set of tools, the imple-
mentation of an application is greatly simplied. The second
purpose of creating the HPS is that the specic requirements
of an application can be better studied through testing its
performance on the HPS. Once the tests have been analysed,
the choice of architecture to be used in the nal implemen-
tation of the hybrid system, for that particular application,
becomes clear. In this way, a new vision application would
rst be implemented and tested on the HPS, and then a suit-
able architecture would be chosen according to the results
obtainedduringthe tests. This process is illustratedduringthe
development of the demonstrator application included in this
paper.
The next section presents an overviewof previous work in
the elds related to this project. Section3 studies the advan-
tages and disadvantages of several architectures of the hybrid
system, and Sect. 4 presents a prototype systemfor the devel-
opment of applications based on any of the hybrid system
architectures presented in Sect. 3. As a demonstrator of the
hybrid system, a road sign recognition (RSR) application has
been implemented, and this is outlined in Sect. 5. Finally,
Sect. 6 draws some conclusions and proposes some future
research directions of this work.
1 3
380 M. S. Prieto, A. R. Allen
2 Background
The rst part of this section reviews the work that has been
carried out towards the development of hardware systems
for the pre-processing of images. Section2.2 introduces var-
ious techniques of post-processing analysis of the images,
and concentrates on the application of articial neural net-
works (ANN) to IP, and the advances in the development
of hardware systems for ANN. To conclude this section,
Sect. 2.3 discusses some of the issues related to combining
these two technologies and gives a short introduction to the
hybrid model chosen as the main theme of this paper.
2.1 Pre-processing
Most machine vision systems use a combination of IP tech-
niques to perform a complete inspection. Amongst the more
common IPtechniques are: thresholding, segmentation, mea-
surement, ltering, texture and colour analysis, and edge
detection. Some of these techniques require complex and
time-consuming calculations. The large amount of compu-
tation required by some techniques can easily exceed the
capacity of a conventional microprocessor, when the system
is expected to process images fed by a camera at real-time
speed.
In the past, the most common solution to this problemwas
the use of application specic integrated circuits (ASIC),
hardware designed specifically for each application. How-
ever, this solution is not very practical, specially for low
volume applications, since ASICs are expensive to produce
in small quantities, require a great amount of experience in
microelectronics, and their development is a very time-con-
suming task.
Another common approach for real-time image process-
ing has been the use of parallel computing [59]. In IP, due to
the nature of its operations, it is often possible to partition the
image across several processors, and to process each parti-
tion in parallel. This mechanism requires additional commu-
nication between the processors to share the data, which can
be a problem for high-latency networks. As a result, many
multiprocessor systems have difculties with real-time video
processing due to communication overheads. Additionally,
parallel computing is costly, and the lack of stability and
software support for the parallel machines can also be a dis-
advantage [7].
Fortunately, there have recently been great advances in
the development of programmable hardware, which substan-
tially reduces the time spent in the design phase of an appli-
cation.
Fieldprogrammable gate arrays. Aeldprogrammable gate
array (FPGA) consists of an array of logic and routing resour-
ces that canbe reprogrammedafter it is manufactured. FPGAs
are generally slower than purpose-built hardware, and draw
more power. On the other hand, the advantages that they
offer are of great importance: a shorter time-to-market, lower
development costs, and allow the reprogramming of the
device [12, 57].
The greatest problem of FPGAs used to be their very lim-
ited size. Complex operations, such as some IP algorithms,
required impractical designs implemented across multiple
devices, with the communication problems that this implied.
Even though by the early 1990s there were already some
FPGA-based IP systems [1, 27], it was not until the end of the
1990s, that the significant improvements in FPGAcapacities
(and FPGA technology in general) made this technology a
serious candidate for hardware, and more applications started
to appear [17, 26]. Since then, FPGA technology has only
improved, with a consequent increase in the number of FPGA
implementations of IP. Various researchers have developed
FPGA-based systems for IP applications. Salcic et al. [53]
and Bouridane et al. [7] produced real-time co-processors
based on FPGA. Bouridane et al. also created a library of
common IP algorithms that can be called fromthe PChosting
the co-processor. This original work evolved into a different
kind of library in [5], which is a collection of reusable hard-
ware skeletons of common IP operations, which can be com-
bined through high-level algorithmic descriptions to obtain
hardware solutions. Another processing system for machine
vision applications was developed by Dunn et al. [18]. An
important part of their workis the development of a high-level
language that could be compiled into hardware by automated
techniques. More details on these techniques follow in the
next section. Similar to these works, there is a growing num-
ber of other kinds of architectures based on FPGAs designed
for real-time IP [4, 34, 42].
Recent technological advances have provided new func-
tionalities for the FPGAs of latter generations, such as
dynamic reconguration. Dynamic reconguration is the
ability to recongure a part or the whole of the FPGA dur-
ing execution time. Tanougast et al. [60] have introduced
methods to optimise the use of the FPGA, exploiting this
new functionality, and created a dynamically recongurable
embeddedreal-time system. Nevertheless, dynamic recong-
uration is still a very new area of research that still requires
some improvement to facilitate its use.
Despite the great advantages of the new functionalities of
FPGAs, such as dynamic reconguration, possibly the most
important area of research around FPGA technologies at the
moment is centred on the programming of the FPGAs. More
specifically, the development of high-level languages that can
be compiled directly into hardware by automatic techniques
can neutralize the main disadvantage of FPGAs, i.e. the pro-
gramming model for FPGAs has been up to now at gate
level. This issue is examined in more detail in the following
section.
1 3
A hybrid system for embedded machine vision using FPGAs and neural networks 381
Hardware compilation. The development and testing of
algorithms at gate level is a very time-consuming task and
requires a considerable expertise in microelectronics. Hard-
ware compilation [14, 57] is a recent approach to this problem
in which algorithms are rst written in a high-level language,
and then compiled into hardware by automatic techniques.
Some high-level languages usedinhardware compilationare:
the CCGL designed specifically for Xilinx FPGAs by Dunn
et al. [18], the SA-C developed by Draper et al. [16], and
Handel-C [10], which has been extended considerably more
than other languages. The familiarity of writing in a high-
level language, combined with the simple compilation and
simulation environment, makes hardware compilation very
suitable for the adaptation of IP algorithms into hardware
implementations [44, 57, 58].
Handel-C is a programming language that includes most
of the capabilities of conventional C. In addition, Handel-
C also includes some parallel constructs, which enable the
exploitation of the inherent parallelism of algorithms. In this
way, the implementation of an algorithmin hardware follows
three steps: translation of the C/C++ programinto Handel-C,
parallelisation of the code, and adaptation of the code to the
hardware platform that it is going to be running on (by x-
ing variable widths, high-level interfacing of devices, and so
on). After this step, all that is needed is to use the Handel-C
hardware compiler to automatically generate the gate-level
hardware.
2.2 Post-processing
The computational complexity of recent machine vision
applications has demanded new AI techniques to be added
to the traditional methods of IP. Two common approaches
to analysing the data are from the point of view of statistics
or ANN. In general, there are many parallelisms between the
twoelds, andthere is a direct correspondence betweenmany
of the techniques. An excellent summary of the similarities,
differences, and the ideal situations in which different tech-
niques should be used, can be found in the Neural Network
FAQ by Sarle [54].
This section is concerned mainly with the use of ANNs
in the context of IP, and the development of hardware imple-
mentations of ANN. One of the reasons whyANNare appeal-
ing to this project is because they are biologically inspired
systems. Since one of the global aims of this project is an
attempt to produce a suitable system for the simulation of
cognitive models of the human mind, it seems appropriate to
use methods that are biologically inspired.
Concerning the use of ANNs in IP applications, there has
been a large amount of work to evaluate the performance of
different networks for each kind of IP task. A very compre-
hensive review of these studies can be found in [19]. In this
study, Egmonton-Petersen et al. point out that the most fre-
quently applied architectures are: feed-forward ANNs, self-
organizing maps (SOM) and Hopeld ANNs. From these
three, feed-forward ANNs seem to be less useful for some
very important IP tasks, and Hopeld ANNs are sometimes
difcult to apply to a particular problem and convergence to
a global optimum cannot be guaranteed [19].
The major problem of using ANNs in real-time IP is that
most of them are very costly computationally and therefore
unsuitable for real-time processing, unless implemented in
hardware. For this reason, there has been a great effort on
producing suitable architectures for ANN[43]. Neural archi-
tectures are commonly classied into the following main
categories [28, 39]: accelerator boards, multiprocessor sys-
tems, and digital, analogue and hybrid (digital and analogue)
neurochips. Heemskerk [28] mentions that the successive
order of these categories corresponds to increasing speed-up
but decreasing maturity of the techniques used. In general,
hybrid neurochips [6, 23] have only been used in laboratories,
while accelerator boards have been commercially available
for some time. At the moment, digital techniques seem to
offer the best possibilities for implementing exible, gen-
eral-purpose neurocomputers.
There are many reviews of neural hardware in the litera-
ture. Some very complete reviews are [28, 31, 39]. In this last
work, Ienne et al. argue that the quest for generality has inu-
enced the design of many new processing elements, making
new architectures tending to be less attached to a particular
algorithm or ANN, and looking for a more general design.
Other reviews can be found in [13, 55, 56]. An interesting
comment by Dias et al. in their recent work [13] is that many
neurochips are being removed from the market, and very
few new ones appearing. They suggest that this might be
because neurochips are expensive to build and there is still
little known about the real commercial prospects for working
implementations, but that the appearance of new hardware
solutions in the coming years may change the present state
of the ANN hardware market.
Having those considerations in mind, in order to produce
a system with real hardware, the ANN chosen for the system
developed in this project is the SOM, for various reasons.
There seems to be a significant amount of research on appli-
cations of the SOM to IP and with apparently very positive
results [3, 8, 9, 19, 38]. Another reason is that there are avail-
able hardware implementations of the SOM (more details in
the next sections). The following section presents the basic
theory of the SOM and introduces the VindAX processor, a
digital implementation of this kind of network.
Self-organising maps. In 1982, Tuevo Kohonen presented a
new model of unsupervised competitive ANN called Self-
Organising Map (SOM) [24, 35]. The model consists of an
array of neurons, each of which is connected to all the inputs
and to some neighbourhood of the surrounding neurons.
1 3
382 M. S. Prieto, A. R. Allen
This architecture resembles the way biological nets are
organised. In the brain, the number of connections within
each group is much greater than the connections to outside
of the group. Moreover, the physical proximity of two bio-
logical neurons reects some kind of similarity between the
impulses that activate them. In order to implement this fea-
ture, while classical competitive learning only updates the
weights of the winning neuron, the Kohonen learning algo-
rithm also extends the competition over spatial neighbour-
hoods. This extension is achieved by updating the weights
of the neurons within the proximity of the winner neuron,
thus allowing the formation of clusters of nodes within the
array. As in the biological system, the neurons grouped in
a cluster share some sort of similarity between the features
of the inputs that activate them. In this way, it is possible to
perceive the underlying structure of multidimensional data
by projecting it over the array of neurons and observing the
array auto-organise in clusters.
The SOM algorithm. In the SOM algorithm, the multidi-
mensional Euclidean input space
n
, is mapped into a two-
dimensional output space
2
. The reference vector of each
neuron in the network is m
i
= [
i 1
,
i 2
, . . . ,
i n
]
n
,
where
i j
are scalar weights, i is the neuron index and j the
weight index. An input vector x = [
1
,
2
, . . . ,
n
]
n
is presented to all neurons in the network, and the neuron
with the closest matching (i.e. greatest similarity) vector c
becomes the active neuron, i.e.
c = argmin
i
{|x m
i
|}, (1)
which means the same as
x m
c
= min
i
{x m
i
} ,
where x m
i
is the distance between the input vector x
and the reference vector m
i
, using a similarity metric such
as the Euclidean distance.
During the training of the ANN, after the active neuron
has been identied, the reference vectors of the neurons in
the neighbourhood of the active neuron need to be updated
to bring them closer to the input vector x. The amount of
change is determined by the distance of the neuron from the
active neuron. The updating rule is:
m
i
(t +1) = m
i
(t ) + (t ) [x (t ) m
i
(t )] if i N
c
(t )
and
m
i
(t +1) = m
i
(t ) if i / N
c
(t ) ,
where N
c
(t ) is the current neighbourhood, and t is discrete
time (i.e. t = 0, 1, 2, . . .).
The training is considered successful if the neurons form
clusters of similar reference vectors. Each cluster of neurons
would correspond to a different class of the data. This clus-
tering is a product of the design of the network and occurs
without any supervision. Once the network is successfully
trained it can be used to classify new data. The new data
would be codied into a new input vector, which would then
be passed to the network. The neuron that becomes active by
Equation1 determines the class of the input data.
Hardware implementations of the SOM. Over the years,
there have beenmanyhardware implementations of the SOM.
In most cases, the SOM algorithm is implemented on a neu-
ral chip designed to support different kinds of networks. An
example of this is the work by Ienne et al. [32] where they
used a chip with a Single Instruction stream Multiple Data
stream(SIMD) architecture called MANTRAI [61] to imple-
ment the SOM algorithm. The authors suggested a few dif-
ferences in the algorithm to improve the parallelisation and
its implementation in hardware.
More recent examples of SOM implementation on neural
hardware can be found in [6, 21] and [48]. The last work by
Porrmann et al. presents a dynamically recongurable hard-
ware acceleration based on FPGA technology for the simu-
lation of SOM. The system, equipped with 5 FPGAmodules,
achieved a respectable maximum performance of more than
50 GCPS (giga connections per second) during recall and 5
GCUPS (giga connection updates per second) during learn-
ing. Other publications from the same research group giving
more details of the system and results can be found in [49]
and [52].
Another fully digital hardware implementation of the
SOM algorithm is the Modular Map [40, 41]. It is composed
of a neural array and a module controller that provides the
interface between the host and the array which is a SIMD
array of processors congured to provide a highly parallel
processing system. Each neuron of the array is implemented
separately as a simple RISCprocessor, and they interact with
each other creating a network of 16 16 neurons with the
topology equivalent to that of a SOM. The design adopts a
modular strategy, which permits the neural array to work as
a fully functional self-contained network or as a part of a
bigger network by interconnecting modules. The commer-
cialised version of the hardware implementation of the Mod-
ular Map design is known as the VindAX processor, and is
developed by AXEON Ltd [2]. The manufacturing company
provides a PCI development board, which contains one Vin-
dAX processor and a software package to run on a PC for
the development of applications.
Through the Vindax Development Board, the VindAXpro-
cessor can be accessed as one 16 16 network or as various
partitionings of the neural array. The partitionings can either
be in the dimension of the neural map (e.g. 4 networks of
4 4 neurons with reference vectors of 16 elements) or in
the dimension of the reference vectors (e.g. 2 networks of
1 3
A hybrid system for embedded machine vision using FPGAs and neural networks 383
16 16 neurons with reference vectors of 8 elements). A
Register Transfer Level synthesisable VHDL description of
the VindAX processor is available for Intellectual Property
ware applications [29]. In the case of embedded IP applica-
tions, the Modular Map soft core could be included as part
of the hybrid system designed in this project, and optimised
for the use of each particular application.
In the VindAX processor implementation of the SOM
algorithm, covers the range (0, 255) and (0 < n 16)
in the multidimensional Euclidean input space
n
, which
means that the vectors have 16 elements with values up to
255. Regarding Eq. 1, a variety of distance metrics can be
used as a measure of similarity. Since the equation is imple-
mented in hardware, the Manhattan distance metric has been
found to be a valid alternative to the more widely used Euclid-
ean distance [40, 51]. The Manhattan distance is less expen-
sive in terms of computational resources.
The hardware implementation of the SOM used in the
present work is the VindAX. Some of the advantages that are
directly relevant are: the availability as an Intellectual Prop-
erty core, the ability to partition the neural array into various
sub-maps, and the user-friendly development system.
2.3 Combination of the pre-processing and post-processing
Because of the requirements for increasingly complex vision
processing, there is a clear motivation towards creating sys-
tems that allowthe mergingof the twoaspects of image analy-
sis presented in the previous sections: the pre-processing and
the post-processing. The pre-processing of the images com-
prises all IP tasks which deal with the image at a very low-
level, whereas the post-processing tasks include the
high-level analysis of information with AI techniques, such
as ANN.
There appears to have been relatively little research inves-
ted in the development of systems combining traditional IP
techniques with ANN. One company that seems active in this
line of research is General Vision [25]. Their main product
is an image recognition engine named CogniSight. Cogni-
Sightincludes a Xilinx Virtex FPGAof 50Kgates together
with a parallel neural network based on the Zero Instruction
Set Chip (ZISC
, the complete
dedication of the processor time to the systemcan not be
assured due to the background processes always present.
The serial cable, although easy to program, is the slowest
method possible to connect the RC200 with the PC.
These points need to be taken into careful consideration when
the architecture for the embedded solution is chosen.
5 The road sign recognition application
As a proof of concept, a real-time machine vision application
has been developed and tested using the HPS. Road sign rec-
ognition is a part of driver support systems. Its main aim is
the increase of trafc safety by calling the drivers attention
to the presence of key trafc signs such as: stop signs, yield
signs, speed limits, etc. Additionally, a vision-based system
able to detect and classify trafc signs from road images in
real-time would also be useful as a support tool for guidance
and navigation of intelligent vehicles.
The problem of RS detection and classication might
seemsimple and well-dened since the colour, shape, dimen-
sions and placement of the RS is tightly regulated. In reality,
the problem is very complex due to several reasons:
Since the images are acquired from a moving vehicle,
they suffer from: vibrations, blurred scenes, and varying
illumination of the captured scenes.
The signs can be found damaged, partly occluded, clus-
tered, and other kind of situations which make their
detection more difcult.
There are variations in the width of the sign borders and
actual pictograms onthe signs, inspite of the regulations.
This section presents a new method using self-organising
maps for the detection and recognition of road signs. The
RSR application is implemented and tested using the HPS
presented in Sect. 4, and is used as a demonstrator of the
hybrid system.
The detection of trafc signs from outdoor images is the
most complex step in a RSRsystem. There are many ways in
which the characteristics of road signs (RS) (e.g. well estab-
lished shapes and colours of the signs) could be exploited.
However, the necessity to analyse the images in real-time is a
limiting factor as to how much information available within
the image should be extracted and analysed. Most approaches
to the problem of RS detection use either colour information
or shape information. A review of various approaches can be
found in [33] and [37].
In most applications, once a region of interest (ROI) from
the image has been detected, i.e. an area of the image that
might contain a trafc sign, this ROI is passed to the rec-
ognition module in charge of identifying the sign. Most of
the RSs contain a pictogram, a string of characters, or both.
The recognition of RSs is therefore very often implemented
with ANN [15, 22, 36], because of their pattern recognition
abilities. Nevertheless, other approaches such as template
matching [30, 47] or Laplace kernel classier [45, 46] have
also been successfully used in the recognition step.
The approachchosenfor the RSRapplicationdevelopedin
this paper uses SOM in both the detection and recognition.
In each task, a vector characterizing the ROI is extracted.
The classication of the vector by the SOM will determine
whether it corresponds to a potential RS or not, at detection
level, and what kind of RS it is, at classication level. More
details of the implementation of this application can be found
in [50].
5.1 The HPS in the RSR application
The RSR application has been chosen as a demonstrator of
the HPS presented in Sect. 4. RSR is an ideal problem for
testing the use of the HPS in the development of embedded
1 3
A hybrid system for embedded machine vision using FPGAs and neural networks 391
machine vision applications. The real-time recognition of
RSs requires a complex interaction between the FPGA and
the ANN while working under strict speed constraints.
In exploring the design space for this application, it is
necessary to take into account the requirements of both the
preprocessing and the learning/classication. In this case, by
step 7 of the life-cycle, it is clear that the preprocessing will
take up a considerable FPGA area. It is the object detec-
tion and extraction, and construction of the feature vector,
which is the processing bottleneck, and therefore requires
as much parallel hardware as possible. This rules out imple-
menting both the preprocessor and the SOM classier in the
FPGA (Sects. 3.1 and 3.2). Moreover, the application also
requires dynamic adaptation and on-line learning, and the
reconguration time of the FPGA (O(10
2
) ms) would ren-
der the method of Sect. 3.2 infeasible.
The other consideration is the nature of the connection
between the FPGA and the ANN. In the RSR application,
the preprocessing produces a feature vector for each ROI in
the image [50]. The VindAX processor requires a feature
vector of 16 bytes, so a typical source image would result in
only O(10
2
) bits to be sent. There is even less data (O(10)
bits) in the reverse direction (fromANNto FPGA). The com-
munication time would be dominated by the latency of the
RS232 or ethernet (O(1), O(10
1
) ms respectively), and in
either case would be insignificant compared with the FPGA
preprocessingtime. As mentionedinSect. 4.3, anRS232con-
nection was chosen for pragmatic reasons. Figure5 shows the
HPS supporting the development of the RSR application, in
the nal test of the system corresponding to step 8 of the life
cycle of an embedded application for the HPS proposed in
Sect. 4.3.
In Fig. 5, the image to be analysed corresponds to a frame
of a road scene captured by a camera connected to the RC200.
From this image, the RC200 analyses the different ROIs that
could potentially be a RS and generates vectors of character-
istics describing them. Each vector is sent to the PC, and then
gets transferred to the VDSconnected to a PCI port of the PC.
The VindAX processor analyses each vector and returns the
class that it belongs to. The PC transfers back the informa-
tion to the RC200, and if the analysis seems to indicate that
the object is a potential RS, then the RC200 extracts more
information and sends it to the VindAX processor again, in
order to determine exactly which kind of RS it is.
Given the large number of RSs in some classes, the train-
ing of an ANN with all the RSs of a class at the same time is
nearly impossible. Atypical solution is to group the RSs into
subclasses, according to the similarity between their picto-
grams, and to assign a neural map to each of the subclasses.
In this way, the process of transferring vectors to the VindAX
processor and receiving their class back is done a few times,
as the RS is further analysed. Once the exact RS has been
identied, the original frame captured by the camera is dis-
played in the Video Output, with the RSthat has been detected
marked appropriately.
The Partitioning of the SOM. One of the features of the Vind-
AX processor is its capacity to partition the neural net into
various independent maps working in parallel. This feature
can be used to great advantage in the RSR application.
Dividing the VindAX processor into four 8 8 networks
gives it the capacity to analyse each vector by four sub-
networks at the same time. Each of the sub-networks could
be trained with four or ve different RSs, giving the pro-
cessor the capacity to examine up to 20 individual RSs for
classication at the same time.
Dynamic adaptation. Another very important feature of the
hybrid system is its ability to adapt the neuron values of
the SOM map while processing vectors for classication. In
order to best exploit this feature, the RSR system has been
implemented in such a way that it is capable of adapting its
SOM maps without having to stop its normal execution.
By using the dynamic adaptation of the neural array at
specic times, the system is able to adapt itself to small
changes in the RSs being analysed. Clearly, the objective here
is to keep the system learning from experience and evolving
through time so as to always obtain the most accurate clas-
sication that is possible, even when changes in the appear-
ance of RSs take place. These changes could be observed,
for example, when the frontier between two countries is
crossed. For this reason, this very same example is recreated
in the experiments, where the system is trained with images
fromSpain and Czech Republic and tested with images from
United Kingdom.
5.2 Results
This section presents the results of the experiments. These
experiments study the performance of the RSR application
running on the HPS. The two most important points being
tested by the experiments are: the accuracy to detect and
classify RSs, and the processing speed of the system.
5.2.1 Detection and classication accuracy
With appropriate lighting, the detection algorithmhas shown
a great reliability to detect RSs of the four main classes (stop,
give way, warning and prohibition signs). In general, the RSs
were perfectly detected and identied under variations of
scale (as small as 25 pixels wide), rotation (up to 20
) and
occlusion (up to 1520% of vertical and 510% of horizon-
tal occlusion, depending on the area being occluded) of the
signs.
It has to be said that all those results were obtained after
the network was adapted to work with the British standards
1 3
392 M. S. Prieto, A. R. Allen
of RSs. As was mentioned in the previous section, the RSs
used in the training of the SOM and the experiments are
slightly different. Using the initial training of the maps, all
RSs except cross-roads and trafc-light ahead were perfectly
recognised by the system. In the case of these two, the SOM
seemed to confuse them when the RSs were shown with the
slightest rotation of the pictograms.
To help the system to differentiate the signs, the sys-
tem was asked to adapt the neural array values about 20
40 frames for each sign. Through this procedure, the system
perfectly learned the two classes and proceeded to correctly
classify them, as well as the other signs, under the same kind
of rotations tested on them.
5.2.2 Speed of the system
Initially, the main concern of the experiment was that the HPS
might not be able to achieve a high-speed processing time.
After all, the HPS gathers a great amount of statistical data,
communicates through a serial port, and part of the system
runs under Windows