 Embedded GNU/Linux System: A

Smart Camera Project

R. Badlishah Ahmad (PhD)

Head of Embedded Computing,
UniMAP Research Cluster

email: badli@unimap.edu.my
http://publicweb.unimap.edu.my/~badli/

1
Contents - I

Embedded System (ES): Definition

Embedded GNU/Linux: products
Example of processors used for ES
Single Board Computer (SBC)
Choices of SBC
Advantages of using SBC
When & Why to choose SBC
Embedded Operating System (EOS)

2
Contents - II

GNU/Linux OS
GNU/Linux Advantages
SBC in UniMAP
Smart Camera Project (SCP)
SCP: Introduction
SCP: Objectives
SCP: System Overview
SCP: Hardware & Software
Difficulties & Lessons Learned
Conclusion

3
Embedded System (ES)

ES consists of input/output (I/O) & Control

Logic
Classified as a combination of hardware and
software
Different from PC that is multitasking
ES performs specific tasks

4
Embedded System (ES)

 Common ES characteristics:
 Single-functioned/specific task
 Tightly constrained resources
 Reactive and real-time

5
Embedded GNU/Linux

http://linuxdevices.com/

6
Embedded GNU/Linux

7
Embedded GNU/Linux

8
Embedded GNU/Linux

Single Board Computer by

Technologic System use in
UniMAP

9
 Processors used for ES
Products Platform
Vendo V-MAX 720 vending machine Microprocessor: 8-bit Motorola 68HC11

Sonicare Plus tootbrush Microprocessor 8-bit Zilog Z8

Miele dishwashers Microprocessor: 8-bit Motorola 68HC05

NASA’s Mars Sojourner Rover Microprocessor: 8-bit Intel 80C85

Garmin SteetPilot GPS Receiver Microprocessor: 16-bit

Palm Vx handheld Microprocessor: 32-bit Motorola

Dragonball EZ

Motorola i1000plus iDEN Multi-Service Microprocessor: Motorola 32-bit MCORE

Digital Phone

Rio 800 MP3 Player Microprocessor: 32-bit RISC

RCA RC5400 DVD Player Microprocessor: 32-bit RISC

Sony Aibo ERS-110 Robotic Dog Microprocessor: 64-bit MIPS10RISC

Single Board Computer (SBC)

 Characteristics
 Computer like
 Manufactures with important peripherals
 Stand-alone computer
 Ready-made device
 Constraint
 Small RAM (compared to Desktop)
 No Graphical Interface or Keyboard

11
 Choices of SBC

Advantech’s “Half-Biscuit” Ampro’s EnCore TS-7200 ARM SBC

Kontron’s EPIC SBC Kontron’s Standalone SBC TS-5500 PC/104

12
Advantages of using SBC

 Faster speed development

 Choices for various clock speed (price varies
as processor speed)
 Capabilities of using Open Source Software
(OSS)/Free Software (FS) and tools
 Support high level programming

13
When & Why to use SBC

 When
– Lack of resources (hardware designers)
– Lack of manufacturing capabilities
 Why
– Faster Time to Market
– Proven hardware design by vendor -> reliable
design and reduce risk of defect
– More vendor choices (different size, functions and
price options)

14
Embedded OS (EOS)

 EOS is a software program that manages all

other programs in embedded devices
 EOS is a small OS designed to operate with
embedded devices to perform certain tasks
which control effectively all the hardware
components

15
Embedded OS (EOS)

 Real-Time & Non Real-Time (commercial)

 QNX, DOS, Windows CE, LynxOS

 VxWorks

 PSOS

 RTLinux

 Other Choices?? create own? Yes,

modify/customize GNU/Linux OS!!!! it's FREE

16
GNU/Linux OS

 Unix-based OS originally developed for Intel-

compatible PC's
 Modern OS -> include features such as virtual
memory, memory protection and pre-emptive
multitasking
 Source code is freely available!!!

17
GNU/Linux OS

 Embedded Linux is increasing due to the ability to

modify subsystem code to suit application designs
 Support Portable Operation Interface (POSIX) ->
easy to port existing code to different processor
architecture
 GNU/Linux are available for x86, SPARC, ARM,
PowerPC, MIPS etc....

18
GNU/Linux Advantages

 Robust Features – multi-threaded, multi-

functions
 Scalability – can run on flash cards and boards
to PCs and larger system
 Widespread acceptance – communities of
programmers available
 NO ROYALTIES!!!!

19
SBC in UniMAP

 Based from Technologic System Co. (USA)

(http://www.embeddedarm.com/)
 Model bought available TS5500,
TS5500
TS7200/TS7800 and TS3300
 133-200MHz, AMD/I386, USBs, 10/100
Ethernet, Wi-Fi (PCMCIA), COM ports, Digital
I/Os, ADC
 Comes with TS-Linux (EOS)

20
Smart Camera Project

 A sample of Embedded GNU/Linux developed

in UniMAP
 Network Vision based system
 Using GNU/Linux & Free Software tools (GCC)
 Using TS5500 SBC Board
 Development tasks:
 Integrating Web Camera
 Developing software for image capture, image
conversion, image manipulations and socket
programming
21
Smart Camera Project
An example of an Embedded GNU/Linux.

Situation awareness is the key to security and safety.

Traffic surveillance system that exist today consist of CCTV (deployed in
critical areas), a lot of monitors and traffic operators.
Traffic operators are assigned the task of visual monitoring and threat
detection process which is really tedious and error prone for a human
investigator.

Existing Surveillance System

Introduction

Problems in traffic surveillance area :


Inefficient network monitoring
 traffic operators are burden with visual monitoring task,
therefore a threat detection process would be ineffective,
unreliable and prone to error.
 the presence of stationary vehicle on the road, especially at a
critical places could cause a secondary crashes, traffic
congestion and other traffic incident.

Therefore, there is a NEED of a vision system to give

earlier detection and notification of stationary vehicle
to the security personnel or traffic operator on the
events
Objective

To develop, integrate, configure and analyze the performance

of Embedded Vision System for stationary vehicle detection
utilizing GNU/Linux and Single Board Computer
System Overview

 utilizing SBC as a processor and

Logitech Quickcam Pro 4000 web Smart Camera

camera as an image sensor. USB Webcam SBC

 Task : to monitor road conditions and

detect stationary vehicle inside the
monitoring area.
 If the object of interest is detected on Network Supervision Center
the scene, the system will notify the
operators by transmitting the image to Monitor

the supervision center. Alarm

Hardware
SENSING UNIT
LOGITECH QUICKCAM PRO4000
 VGA-sized (640x480 pixel) CCD
sensor
 capture video at maximum 30
frames per second.
 driver used : pwc-8.8 & usb-
pwcx-8.2.2
PROCESSING &
COMMUNICATION UNIT
TS-5500 SINGLE BOARD COMPUTER
 AMD Elan 520 processor (133
Mhz)
 32 MB RAM
 10/100 Base-T Ethernet port
Software Module

Divided into 3 modules:

– Module 1: Image Acquisition
– Module 2: Image Processing and Object
Detection
– Module 3: Data Transmission
Processes

IMAGE CAPTURE

OBJECT
COLOR SPACE EXTRACTION
CONVERSION

FILTERING
FRAME (CONVOLUTION)
DIFFERENCING

DETECTION &
NOISE IMAGE SAVING
REMOVAL &
NOTIFICATION
Hardware Performance

Results
Specification Desktop PC SBC

Processor Type Intel(R) Pentium(R) 4 CPU AMD Elan 520

Processor Speed 2.66 GHz 133 MHz

RAM Size 512 MB 32 MB

Operating System Mandriva 2006 TS-Linux

Smart Camera Operation Overall Processing

Platform (s)

Desktop PC 6.09

TS-5500 SBC 38.82

Difficulties & Lessons Learned

 Understanding system & file structure

 Revising programming skill in C/C++ for socket programming, shell
programming & database
 Ability to do various system configuration using CLI
 Understanding cross compiling process & techniques
 Difficulties to integrate external devices such as Webcam, touchscreen etc..
:Kernel version mismatch!!
 Difficulties to compile on the SBC!! programming are done on a host PC
 Execution problem on SBC board due to library mismatch between host PC
and SBC
 Limited/No Technical support

30
Conclusion
 Applications using SBC can be extended into many more
fields such as automation, robotic, firewall, intrusion
detection system, carried grade telecommunication and
network devices
 The availability of Linux kernel and GNU application
development tools ANY applications or system can be
build!!!!
 Wide oppurtunities for system development

31
Q&A

SMART CAMERA
PROTOTYPE

32