PSZ 19:16 (Pind.

1/071

UNIVERSITI TEKNOLOGI MALAYSIA

DECLARATION OF THESIS / UNDERGRADUATE PROJECT PAPER AND COPYRIGHT

Author's full name :

CHOP CHIAN CHOONG

Date of birth

24-04-1988

PC BASED HOME AUTOMATION USING SPEECH

Title

RECOGNITION AND NETWORK

Academic Session:

2010/2011

I declare that this thesis is classified as :

CONFIDENTIAL

(Contains confidential information under the Official Secret

Act 1972)*

RESTRICTED

(Contains restricted information as specified by the

organisation where research was done)*

OPEN ACCESS

I agree that my thesis to be published as online open access

(full text)

I acknowledged that Universiti Teknologi Malaysia reserves the right as follows:

1. The thesis is the property of Universiti Teknologi Malaysia.
2. The Library of Universiti Teknologi Malaysia has the right to make copies for the purpose
of research only.
3. The Library has the right to make copies of the thesis for academic exchange.
Certified by:

SIGNATURE
880424-08-5609
(NEW 1C NO. /PASSPORT NO.)

Date: MAY 16,2011

NOTES :

SIGNATURE OF SUPERVISOR

DR. MD NASIR IBRAHIM

NAME OF SUPERVISOR

Date: MAY 16,2011

If the thesis is CONFIDENTIAL or RESTRICTED, please attach with the letter from
the organisation with period and reasons for confidentiality or restriction.

"I declare that I have read this thesis and in my opinion this thesis is
sufficient in terms of scope and quality for award a Degree of
Bachelor of Engineering (Electrical - Electronics)."

Signature
Name of Supervisor : DR. MUHAMMAD NASIR IBRAHIM
Date

: 16 MAY 2011

PC BASED HOME AUTOMATION USING

SPEECH RECOGNITION AND NETWORK

CHOO CHIAN CHOONG

Submitted to the Faculty of Electrical Engineering

In partial fulfilment of requirement for the degree of
Bachelor in Electrical Engineering (Electronics)

Faculty of Electrical Engineering

Universiti Teknologi Malaysia

APRIL 2011

I declare that this thesis entitled "PC Based Home Automation Using Speech
Recognition and Network" is the results of my own research except as cited
in the references.

The thesis has not been accepted for any degree and

submitted in candidature of any other degree.

Signature

W.-Jf^

Name

: Choo Chian'Choong

Date

: 16 MAY 2011

iii

Specially dedicated to:

My father, Choo Yoon Phing
My mother, Wong Yee Mee
My Grandmother, Lim Poo Kiu
My Auntie, Wong Lee Lih
My siblings,
Choo Mee Lian
Choo Mee Yoon
Choo Chian Yew
Choo Chian Leong
Thanks for the loves and supports

iv

ACKNOWLEDGEMENT

First and foremost, I would like to express my heartily gratitude to my

supervisor, Dr. Muhammad Nasir Ibrahim for guidance and encouragement given
throughout the progress of this project. Without his guidance and support, this
project will not be completed as well as this thesis.

Moreover, my great appreciation also goes to all my family members and

friends that had been so patient and support me all these years, especially my girl
friend, Sim Ai Jin. Without their encouragement and love, I would not be able to
undergo the pressure due to this project.

During the process of this final year project, I had faced lot of problems
especially come to troubleshooting and idea on how the project can be implemented.
Thanks to my friends, Koay Jian Cong, Lam Kong Shen, Voon Zheng Loong, Pui
Hing Ngain and Voon Wee Sun for the advices and idea on my project, finally I was
able to finish my final year project smoothly.

ABSTRACT

Home automation is widely used to control available electric appliances

including centralized control of lighting, HVAC, appliances, and other systems. The
system is able to provide improved convenience, comfort and energy efficiency for
the users. The objective of this project is to develop a home automation system using
computer interface and users are able to control their home electric appliances
through speech recognition, as well as control or monitor the appliances through
network. Programmed events and macros can be downloaded to the memory on the
microcontroller for stand-alone operation (with the PC turned off). Local area
network connection is used to enable client computer remote control the home
computer, which enables users to remote control their home electric appliance
though network. Since the installation of the wire for the home automation is hardly
to be done, hence RF wireless communication is used in the system, where there is
no extra rewiring work is required to implement this system. In order to build this
system, software and hardware had to be developed to make things work. For the
hardware part, the PIC16F877A and PIC18F4550 microcontroller is selected for this
application. Two microcontrollers are used for the system. One is used to transmit
information data using RF module and communication with the computer. USB CDC
is used to communicate with the computer. Another one is used to receive the data
from the RF signal and control the switch of the electric appliances. For the software
part, Microsoft visual basic and Microsoft speech SDK 5.3 are used to develop a
speech recognition graphic user interface for the system. Users are able to remote
control through network by using Microsoft Live Mesh.

vi

ABSTRAK

Home automation digunakan secara meluas untuk mengawal peralatan

elektrik, termasuk pencahayaan, HVAC dan system lain-lain. Sistem ini mampu
membawa kemudahan, keselesaan dan penjimatan tenaga bagi pengguna. Tujuan
projek ini adalah untuk mereka sistem automasi rumah yang menggunakan komputer
dan pengguna boleh mengawal peralatan elekrik dengan menggunakan suara, serta
mengendalikan sistem melalui internet. LAN digunakan untuk pengguna mengawal
sistem dengan komputer lain. Oleh sebab pemasangan wire untuk automasi rumah
amat susah dipasang, maka RF wayarles komunikasi digunakan dalam sistem ini,
supaya rewiring tambahan tidak perlu dibuat untuk memasang sistem ini. Dalam
bahagian hardware, mikropengawal PIC16F877A dan PIC18F4550 dipilih untuk
aplikasi ini. Dua mikropengawal digunakan untuk sistem ini. Salah satunya adalah
digunakan untuk menghantar data maklumat dengan menggunakan modul RF dan
komunikasi dengan komputer. USB CDC digunakan untuk berkomunikasi dengan
komputer. Satu lagi digunakan untuk menerima data dari isyarat RF dan kawalan suis
dari peralatan elektrik. Dalam bahagian perisian, Microsoft Visual Basic dan
Microsoft visual SDK 5.3 digunakan untuk meraka program untuk system
ini. Pengguna boleh mengawal sistem melalui jaringan dan internet dengan
menggunakan Microsoft Live Mesh.

vii

TABLE OF CONTENT

CHAPTER

TITLE

PAGE

THESIS STATUS INFORMATION

SUPERVISOR CONFIRMATION

TITLE

DECLARATION

ii

DEDICATION

iii

ACKNOWLEDGEMENT

iv

ABSTRACT

ABSTRAK

vi

TABLE OF CONTENT

vii

LIST OF CODES

xi

LIST OF FIGURES

xii

LIST OF ABBREVIATIONS

xiv

LIST OF APPENDICES

xv

INTRODUCTION
1.1 Introduction

1.2 Project Objective

1.3 Problem Statement

1.4 Scope of the Project

1.5 Computer used for the system

viii

LITERATURE REVIEW

2.1 Introduction

2.2 Home Automation

2.3 Microcontroller PIC18F4550 and PIC16F877A

2.4 High-Speed Serial Bootloader for PIC16 and PIC18

Devices
2.5 USB CDC Class on an Embedded Device

2.6 RF transmitter and receiver

10

2.7 X-10 system

11

2.7.1

Zero-Crossing Detector

12

2.7.2

120 kHz Carrier Detector

13

2.7.3

120 kHz Carrier Generator

13

2.7.4

Transformerless Power Supply

14

2.7.5

Load switch

15

2.8 RS232 serial communication of microcontroller

16

2.9 Bootlaoder

17

2.10

Microsoft Visual Basic

18

2.11

Microsoft Speech SDK 5.3

19

2.12

MPLAB 7.62

20

2.13

Related Journal

20

2.14

Related Thesis

23

RESEARCH METHODOLOGY

3.1 Introduction

25

3.2 Hardware Implementation

26

3.2.1

Microcontroller PIC18F4550 (transmitter

26

part)
3.2.2

Microcontroller PIC16F877A (receiver part)

27

3.2.3

Radio Frequency Transmitter and Receiver

27

3.2.4

PIC Microcontroller Programmer

27

ix

3.2.5

Computer (Intel platform)

28

3.2.6

Local Area Network (LAN)

29

3.3 Software Implementation

3.3.1

Graphic User Interface of the system

30

3.3.2

PICBASIC PRO Compiler

30

3.3.3

Window Live Mesh

31

3.4 Flow of project

29

32

FUNCTIONAL DESCRIPTION

4.1 Home Automation System Purpose

33

4.2 Operating Environment

33

4.3 Functional Requirements

34

4.4 Performance requirements

41

DESIGN ARCHITECTURE

5.1 System Design Architecture

42

5.2 Hardware Design

42

5.3 Software Design

44

5.3.1

Graphic User Interface of the System

45

5.3.2

Firmware

58

RESULT AND DISCUSSION

6.1 Hardware Verification

61

6.2 System Verification

63

CONCLUSION

7.1 Conclusion

72

7.2 Recommendation

73

REFERENCES

74

APPENDICES

76

xi

LIST OF CODE

CODE NO.

TITLE

PAGE

5.1

Log in form code

45

5.2

Write text coding

46

5.3

Frmhome_load code

47

5.4

Initialize the com port

47

5.5

Open Com Port Coding

48

5.6

SendCommand subroutine function

49

5.7

Buttons Coding

51

5.8

Listen and Stop Listen Coding

52

5.9

Speech Recognition Function

53

xii

LIST OF FIGURES

No.

TITLE

PAGE

2.1

Cytron transmitter modules

10

2.2

Cytron receiver modules

11

2.3

Zero-crossing detector

12

2.4

120 kHz Carrier Detector

13

2.5

120 kHz Carrier Generator

14

2.6

Transformerless Power Supply

15

2.7

Load switch

16

2.8

Connection between D9 Female serial port, MAX232 and

17

PIC16F877A
2.9

Bootloader schematic

18

3.1

Block diagram of the home automation system

25

3.2

USB ICSP PIC Programmer UIC00A

28

3.3

Architecture of the Live Mesh stack

31

3.4

The flow chart of the project work

32

4.1

Transmitter part of the system

35

4.2

Receiver part of the system

35

4.3

Log in form

36

4.4

Update username and password form

36

4.5

Home Automation Control Form

37

4.6

Speech Recognition Function

37

4.7

Show switch status on the picture

38

4.8

Window Live Mesh Sign In Page

39

4.9

Window Live Mesh Status Tab

39

xiii

4.10

Window Live Mesh Remote Tab

40

5.1

Transmitter Part Schematic Diagram

42

5.2

Receiver Part Schematic Diagram

42

5.3

Modification of the Electric Extension Port

43

5.4

New project window

44

5.5

GUI design interface

45

5.6

MicroCode Studio Compiler

54

6.1

Tera Term Connection

56

6.2

Tera Term Serial Port communications

57

6.3

Hardware verification

57

6.4

System Verification

58

6.5

Response after It Recognize ALL ON command

58

6.6

Response after It Recognize ALL OFF command

59

6.7

Response after It Recognize ON ONE command

59

6.8

Receiver place 7m away from the transmitter

60

6.9

Remote Functions in Window Live Mesh (system)

60

6.10

Remote Functions in Window Live Mesh (client)

61

6.11

Connecting Process to the System

61

6.12

Connected to the System

62

6.13

Desktop of the System

62

6.14

Initialize the Home Automation

63

6.15

Start Control the System through Online

63

6.16

Start Control the System through Online

64

6.17

Network Connections Through Window Live Mesh

64

xiv

LIST OF ABBREVIATIONS

LAN

Local Area Network

SMS

Short Message Service

RF

Radio Frequency

PIC

Programmable Interface Controller

USB

Universal Serial Bus

USB CDC

USB communications device class

GUI

Graphic User Interface

LED

Light Emitting Diode

xv

LIST OF APPENDICES

APPENDIX

TITLE

Program in PIC18F4550 for the Transmitter Part of the

PAGE

76

System
B

Program in PIC16F887A for the Receiver Part of the

System

81

CHAPTER 1

INTRODUCTION

1.1

Project Overview

In recent years, home automation is becoming more and more popular due to
the technologies that are available nowadays. Automation had transferred from
industrial sector to home. Home automation allow user to control, monitor various
electric appliances through a single system. Currently there are many technologies
and standard available for home automation, including X10, CEBus, LonWorks,
EIB/KNX [1]. These systems give advantage for the consumers have a more
convenience way of living style, and make the home a safer place to live, save
energy while taking control all the home appliances.

By applying home automation, users are able to perform their daily task
schedule automatically, such as lighting in the home will switch on or off following
the daily schedule. Some use this advantage to make the house looks like occupied
when owner of the house is out for a long vacation. Home automation system for
monitoring and controlling various home appliances can be achieved using variety of
methods, such as LAN technologies, SMS, GSM, and so on. By all these method,
consumers are able to monitor or control their home appliances even they are away
from home [2]. For example, when you are on the way back from the office and the
weather is unbearable, you can remote you air-conditional in your home from the

office to switch on before you get in the house. It brings a lot of convenient for the
consumer.

Many different kind of communication protocol are available when talking

about the communication between the home appliances, including very expensive
wired solution, power line transmission, such as X10 or robust wireless protocols,
such as Zigbee and Wi-fi. To make the system affordable for low income family, the
author had decided to use radio frequency, which provided a wireless solution and it
is the cheapest among the all.

To add alternative control mechanisms, speech recognition can be

implemented in the system. This method allows user to remote control their home
appliances without getting closer to the switch; they can switch on or off of their
home appliances by speaking to the system.

1.2

Project Objective

The objective of this project is to develop and design a PC based home

automation control with speech recognition and network. The design will include the
following feature:

This system enable user to control their home appliances using a computer or
a graphic user interface (GUI).

User is able to control the system through speech recognition technology,

where it is different from voice recognition. Different user is able to control
the system to make it more convenient when any of family members are
allow controlling the home appliances.

Internet connection is used to enable client computer remote control the home
computer, which enables users to remote control their home electric appliance
though internet

Since the installation of the wire for the home automation is hardly to be done,
hence RF wireless communication protocol is used in the system, where there
is no extra rewiring work is required to implement this system.

1.3

Problem Statement

Home automation system is very expensive and yet not popular in Malaysia.
In order to make it affordable for consumer, RF is able to give a cheap solution and
yet affective way to implement home automation.

Home automation usually comes only with limited control method. Hence, to
make the home automation system more convenient, few control method is
implement in the project, including speech recognition. User just has to give
command to the system to control their home appliances, without walking near to the
switch. This brings a lot of benefit to disabled or handicap people to take control
their home appliances.

Most of the people are unable to monitor or control when outside the home.
Many house break-in cases happen in Malaysia, and most of this cases happened
when the owner is not available. The system is able to schedule the on and off time
of the lighting, or control though internet to on the lighting to make it look like the
house is occupied. Camera can be installed to monitor the house.

1.4

Scope of Project

This project can be divided into two parts, hardware and software approach
implementation. The following are the scopes of the project, including:

1. The system is only able to control to switch on or off for lamp and fan.
Dimmer function or speed control are excluded in the project.
2. Speech recognition from the Microsoft SDK 5.3 is used in the GUI.
Only English can be recognized by the system.
3. LED is used to implement the status of the home appliance.

1.5

Computer used for the system

This project required a computer to implement it, the task of the

computer is to provide a GUI for the user to monitor and control the home
appliances. The program has to be designed with speech recognition function
and can be controlled using network.

Intel platform is chosen for the system is because of its powerful

functionality and low cost solution, which is very suitable for a small system.
The Intel desktop board is able to work as a computer for this project, and
LAN support is also available. Hence, the Intel desktop board D510MO is
used and work as a host computer for the system.

CHAPTER 2

LITERATURE REVIEW

2.1

Introduction

This chapter includes the background study regarding home automation

concept, user manuals and several related previous projects, journals and thesis. It
also discusses on the components that are used in the project.

2.2

Home Automation

Home automation is a very promising area. Its main benefits range from
increased comfort and greater safety and security, to a more rational use of energy
and other resources, allowing for significant savings. It also offers powerful means
for helping and supporting the special needs of people with disabilities and, in
particular, the elderly. This application domain is very important and will steadily
increase in the future [3].

According to Khusvinder Gill, Shuang-Hua Yang, Fang Yao, and Xin Lu, in
recent years the introduction of network enabled devices into the home environment
has proceeded at an unprecedented rate [4]. Moreover, with the rapid expansion of
the Internet, there is the potential for the remote control and monitoring of such
network enabled devices. However, the new and exciting opportunities to increase
the connectivity of devices within the home for the purpose of home automation
remain largely unexploited.
The totally automated and super-sophisticated home of the future is inching
closer to reality [5]. By the technology development nowadays, the full automated
home automation can be realized and it will be the trend of the future home.

2.3

Microcontroller PIC18F4550 and PIC16F877A (microchip datasheet,

2009)

For this project, both kinds of microcontrollers are used in the project.
PIC18F4550 is used for the main system which communicates with the computer
through USB and X10 transmitter, and PIC16F877A is used in the X10 receiver to
taking control on the home appliances.

For PIC18F4550, this family of devices offers the advantages of all PIC18
microcontrollers namely, high computational performance at an economical price
with the addition of high-endurance, Enhanced Flash program memory. In addition
to these features, the PIC18F2455/2550/4455/4550 family introduces design
enhancements that make these microcontrollers a logical choice for many highperformances, power sensitive applications. [6]

It comes with new core features such as nanoWatt technology, universal

serial bus and multiple oscillator options and features.

For PIC16F877A, it provides a basic feature and cheap solution, where it

enough to work as a receiver. [7]

Microcontroller core features include:

High performance RISC CPU
Operating speed: DC - 20 MHz clock input DC - 200 ns instruction
cycle
Up to 8K x 14 words of FLASH Program Memory, Up to 368 x 8
bytes of Data Memory (RAM), Up to 256 x 8 bytes of EEPROM Data
Memory
Pinout compatible to the PIC16C73B/74B/76/77
Interrupt capability (up to 14 sources)
Eight level deep hardware stack
Direct, indirect and relative addressing modes
Power-on Reset (POR)
Power-up Timer (PWRT) and Oscillator Start-up Timer (OST)
Watchdog Timer (WDT) with its own on-chip RC oscillator for
reliable operation
Programmable code protection
Power saving SLEEP mode
Selectable oscillator options
Low power, high speed CMOS FLASH/EEPROM technology
Fully static design
In-Circuit Serial Programming (ICSP) via two pins
Single 5V In-Circuit Serial Programming capability
In-Circuit Debugging via two pins
Processor read/write access to program memory
Wide operating voltage range: 2.0V to 5.5V
High Sink/Source Current: 25 mA
Commercial, Industrial and Extended temperature ranges
Low-power consumption:
- < 0.6 mA typical @ 3V, 4 MHz
- 20 A typical @ 3V, 32 kHz
- < 1 A typical standby current

2.4

High-Speed Serial Bootloader for PIC16 and PIC18 Devices (microchip,

2010) [8]

Microchips enhanced Flash microcontrollers enable firmware to program

itself. This is done by a bootloader providing a firmware kernel, residing in the
microcontroller. The kernel uses a small portion of program memory not normally
used by the firmwares main application.

When the bootloader firmware is activated, a host PC can use a serial

protocol to read, write and verify updates to the microcontroller's application
firmware. Once the application firmware is programmed, the bootloader cedes
control, allowing normal application execution until the bootloader is called.
The key features of the AN1310, High-Speed Serial Bootloader for PIC16
and PIC18 Devices include:

Small firmware code size (less than 450 instruction words on most devices)

Automatic baud rate synchronization to the host

Baud rate flexibility, from 1,200 bps to 3 Mbps for extremely fast
programming

A 16-bit CRC packet and Flash memory verification for quick verification of
successful programming, even at low baud rates

An advanced write planner that eliminates unnecessary erase/write

transactions

Support for a wide variety of PIC16 and PIC18 devices through an essential
device characteristics database

Optional application remapping that does not require linker script

modifications or remapping of interrupt service routines

A forced bootloader re-entry mechanism requiring minimal start-up delay and

no additional I/O pins or application firmware code to re-enter the bootloader

Optional MCLR Reset control, allowing the host PC application to

automatically reset the device for robust bootloader re-entry

PC software rewritten in C/C++ for the crossplatform, QtSM SDK, enabling

Linux host support by recompiling the PC software source code

A simple, Serial Terminal Application mode, provided by the PC software,

that eliminates time wasted by switching between separate bootloader host
and serial terminal applications

2.5

USB CDC Class on an Embedded Device (Microchip, 2008)

The Universal Serial Bus (USB) has made it very simple for end users to
attach peripheral devices to a personal computer, all but eliminating the myriad of
different interconnects that used to be necessary. One such interconnect that is
becoming increasingly more rare is the RS-232 serial COM port. In fact, many
modern laptop computers no longer have one. This can pose a challenge, for the
developer needs a serial communication channel from a peripheral to a host PC. [9]

Fortunately, a device can use the USB Communication Device Class (CDC)
and allow the user to take advantage of the simplicity of the USB while providing the
functionality of a COM port. The CDC is how communication devices interface to
the USB. Intended for devices such as MODEMS and network interfaces, a subset of
the CDC features can be used to emulate a serial port providing a virtual UART.

The overall flexibility and power of the USB requires managing protocols for
device identification, configuration, control and data transfer. The Microchip PIC32
CDC serial driver handles the USB so the developer designing a device doesnt have
to.

The CDC serial driver provides a simple UART-like firmware interface for
transmitting and receiving data to and from the host, hiding most of the USB details

10

away from the application. The sample code provided is easily customizable,
reducing the amount of effort and learning that might otherwise be necessary when
adding a USB interface to a device. [10][11]

2.6

RF transmitter and receiver (Cytron, 2008)

The RF transmitter and receiver modules are very small in dimension. The
low cost RF Transmitter can be used to transmit signal up to 100 meters (the antenna
design, working environment and supply voltage will seriously impact the effective
distance). It is good for short distance, battery power device development. Cytron
Technologies provides 2 types of RF Transmitter and Receiver Modules at either
315MHz or 433MHz for user. The following figure showed the Cytron transmitter
and receiver. [12][13]

Figure 2.1: Cytron transmitter modules

11

Figure 2.2: Cytron receiver modules

2.7

X-10 system (Microchip, 2010) [14]

According to microchip, X-10 is a communication protocol designed for

sending signals over 120 VAC wiring. X-10 uses 120 kHz bursts timed with the
power line zero-crossings to represent digital information. Plug-in modules available
from various vendors enable users to create home automation systems by using the
AC wiring already installed within a home.

Microchip had provided a full guideline to build the X-10 system using
PIC16F877A. The PIC16F877A was selected for this application because of its
versatility as a general purpose microcontroller, its FLASH program memory (for
ease of development), data EEPROM, and ample I/O. The application note AN236 is
attached in the appendix for reference.

The hardware functionality of X-10 circuitry can be divided into four

functional blocks:

12

Zero-crossing detector

120 kHz carrier detector

120 kHz signal generator

Transformerless power supply

User interface functions are accomplished with an LCD display and five push
buttons. A real-time clock is created using Timer1 and an external 32 kHz oscillator.
Users modified control data, such as unit on and off times are stored in the PIC
MCUs built-in EEPROM. Load switch is used in this application. The schematic of
the whole system can be referring from the appendix.

2.7.1

Zero-Crossing Detector

In X-10, information is timed with the zero-crossings of the AC power. A

zero-crossing detector is easily created by using the external interrupt on the RB0 pin
and just one external component, a resistor, to limit the current into the PIC MCU,
which shown in figure 2.3.

Figure 2.3: Zero-crossing detector

13

2.7.2 120 kHz Carrier Detector

To receive X-10 signals, it is necessary to detect the presence of the 120 kHz
signal on the AC power line. This is accomplished with a decoupling capacitor, a
high-pass filter, a tuned amplifier, and an envelope detector. The components of the
carrier detector are illustrated in Figure 2.4.

Figure 2.4: 120 kHz Carrier Detector

2.7.3

120 kHz Carrier Generator

X-10 uses 120 kHz modulation to transmit information over 60 Hz power

lines. It is possible to generate the 120 kHz carrier with an external oscillator circuit.
A single I/O pin would be used to enable or disable the oscillator circuit output.
However, an external oscillator circuit can be avoided by using one of the PIC
MCUs CCP modules. The CCP1 module is used in PWM mode to produce a 120
kHz square-wave with a duty cycle of 50%. Because X-10 specifies the carrier
frequency at 120 kHz (+/- 2 kHz), the system oscillator is chosen to be 7.680 MHz,
in order for the CCP to generate precisely 120 kHz. Calculations for setting the
PWM period and duty cycle are shown in the code listing comments for the function
InitPWM. After initialization, CCP1 is continuously enabled, and the TRISC bit for
the pin is used to gate the PWM output. When the TRISC bit is set, the pin is an

14

input and the 120 kHz signal is not presented to the pin. When the TRISC bit is clear,
the pin becomes an output and the 120 kHz signal is coupled to the AC power line
through a transistor amplifier and capacitor, as depicted in Figure 2.5.

Figure 2.5: 120 kHz Carrier Generator

2.7.4 Transformerless Power Supply

The PIC16F877A and other board circuits require a 5V supply. In this

application, the X-10 controller must also transmit and receive its data over the AC
line. Since X-10 components are intended to be plugged into a wall outlet and have a
small form factor, a transformerless power supply is used.

Figure 3.5 illustrates the transformerless power supply used in this

application. To protect the circuit from spikes on the AC power line, a 250V VDR
(voltage dependent resistor) is connected between Line and Neutral. The 47 resistor
limits current into the circuit, and the 1 M resistor provides a discharge path for the
voltage left on the capacitor when the circuit is unplugged from the wall. Two diodes
rectify the voltage across the 1000 F capacitor and 5.1V Zener diode to produce a
5V supply.

15

Figure 2.6: Transformerless Power Supply

2.7.5 Load switch

A load switch is included on the home controller so that it may act as a lamp
module, with its own house and unit address. A Triac was selected as the load switch,
because its medium power switching capacity and rapid switching capability make it
well-suited for lamp control and dimming.

A Triac is an inexpensive, three-terminal device that basically acts as a highspeed, bidirectional AC switch. Two terminals, MT1 and MT2, are wired in series
with the load. A small trigger current between the gate and MT1 allow conduction to
occur between MT1 and MT2. Current continues to flow after the gate current is
removed, as long as the load current exceeds the latching value. Because of this, the
Triac will automatically switch off near each zero-crossing as the AC voltage falls
below the latching voltage.

A Teccor L4008L6 Triac was selected because it has a sensitive gate that
can be directly controlled from the logic level output of the PIC MCU I/O pin. The
sensitive gate Triac can control AC current in both directions through the device,
even though the PIC MCU can provide only positive voltages to the gate.

16

A variable dimmer is created by including a delay between the time of each

zero-crossing and the time that the trigger current is provided to the Triac from the
PIC MCU.

Figure 2.7: Load switch

2.8

RS232 serial communication of microcontroller

SCI is an abbreviation for Serial Communication Interface and, as a special

subsystem of microcontroller PIC16F877A. It provides RS232 serial communication
with PC easily.

As with hardware communication, we use standard NRZ (Non Return to Zero)

format also known as 8 (9)-N-1, or 8 or 9 data bits, without parity bit and with one
stop bit. Free line is defined as the status of logic one. Start of transmission - Start Bit,
has the status of logic zero. The data bits follow the start bit (the first bit is the low
significant bit), and after the bits we place the Stop Bit of logic one. The duration of
the stop bit 'T' depends on the transmission rate and is adjusted according to the
needs of the transmission. For the transmission speed of 9600 baud, T is 104s.

In order to connect a microcontroller to a serial port on a computer, we need

to adjust the level of the signals so communicating can take place. The signal level

17

on a PC is -10V for logic zero, and +10V for logic one. Since the signal level on the
microcontroller is +5V for logic one and 0V for logic zero, we need an intermediary
stage that will convert the levels. One chip specially designed for this task is
MAX232. This chip receives signals from -10 to +10V and converts them into 0 and
5V. The circuit for this interface is shown in the Figure 3.7

Figure 2.8: Connection between D9 Female serial port, MAX232 and PIC16F877A

2.9

Bootlaoder

The Design Devices Bootloader enabled PIC16F877A microcontroller

contains a special program that resides in the upper 255 instructions of program
memory. This special program executes when the microcontroller is powered on or
reset. If B5 input pin is high, it begins to read a MPLAB hex file program and writes
it to user program memory. The hex file data is uploaded using the microcontrollers
SSP serial interface with the help of a special DOS upload program called loader.exe.
The Bootloader program will transfer execution to the current user program in flash

18

memory if B5 input pin is low. Loader.exe is a program used to communicate to a

Design Devices Bootloader enabled PC microcontroller which can be run in a DOS
window under Windows.

Figure 2.9: Bootloader schematic

2.10

Microsoft Visual Basic

The traditional languages are considered as procedural language where the

program specifies the exact sequence of all operations. Program logic determines the
next instruction to execute in response to response to conditions and user request.

19

Microsoft Visual Basic uses a different approach, which is object oriented

programming. It is an event driven programming language where programs are no
longer procedural or does not follow a sequential logic. This software allows the
designed programs run under the windows without complexity associated with
windows programming. Standard windows buttons can be hold on design screen
such as command buttons, text boxes and so on. Each of these windows objects
produces a standard user interface that makes the program becomes user friendly.
A program will be designed using Microsoft Visual Basic to communicate
with the PIC and control the home appliances. This program is designed to handle
speech recognition control and network control.

2.11

Microsoft Speech SDK 5.3

To implement the speech recognition into the system, Microsoft Speech SDK
5.3 is used. Microsoft Speech SDK 5.3 is also known as SAPI 5.3. The Speech
Application Programming Interface or SAPI is an API developed by Microsoft to
allow the use of speech recognition and speech synthesis within Windows
applications. To date, a number of versions of the API have been released, which
have shipped either as part of a Speech SDK, or as part of the Windows OS itself.
Applications that use SAPI include Microsoft Office, Microsoft Agent and Microsoft
Speech Server.

In general all versions of the API have been designed such that a software
developer can write an application to perform speech recognition and synthesis by
using a standard set of interfaces, accessible from a variety of programming
languages. In addition, it is possible for a 3rd-party company to produce their own
Speech Recognition and Text-To-Speech engines or adapt existing engines to work
with SAPI. In principle, as long as these engines conform to the defined interfaces
they can be used instead of the Microsoft-supplied engines.

20

2.12

MPLAB 7.62

MPLAB 7.62 is software program that runs on a computer to develop

applications for Microchip microcontroller. It helps to compile, assemble and link the
software using the assembler or compiler and linker to convert the code into ones
and zeros, which is machine code (*.hex file) for the PIC micro MCUs. The Editor
is aware of the assembler and compiler programming constructs and automatically
color-key the source code to help ensure it is syntactically correct. If the language
tools run into errors when building the application, the offending line is shown and
can be double-clicked to go to the corresponding source file for immediate editing.
Press the build button to try again after editing. Usually a complex program does
not work exactly the way we imagined, therefore MPLAB 7.62 includes components
called debuggers and free software simulators for all PIC microcontroller MCUs to
help test the code in order to get proper results. Lastly a programmer, usually a
bootloader is required to burn the generated machine codes into the devices.

2.13

Related Journal

There are many home automation projects had been done successfully
nowadays. Related journal included:

i.

A GSM, Internet and Speech Controlled Wireless Interactive Home

Automation System [2]

They present the design and implementation of a home automation

system where communication technologies GSM (Global System for Mobile
Communication), Internet, and speech recognition have been used. All these
techniques are successfully merged in a single wireless home automation

21

system. This system offers a complete, low cost, powerful and user friendly
way of real-time monitoring and remote control of a house.

ii.

Remote-controlled Home Automation System via Bluetooth Home Network

[15]

Recently the idea of a home automation has been an important issue

in many publications and home appliances companies. Home automation is a
house or living environment that contains the technology to allow devices and
systems to be controlled automatically. Remote and local control are useful to
keep home comfortable and to support the elderly and the disabled people. In
this project, they discuss possible developments of Bluetooth wireless
technologies and describe the hardware for devices and software for the
considerations of a home automation system.

iii.

An Internet Application for Home Automation [3]

This paper describes an Internet application that allows local and

remote monitoring and control of a home. The application adopts an objectoriented philosophy, in which every home automation device is represented
by an object. This leads to an integrated view of the home, with a common
look and feel across all the devices, while supporting various methods of
actuation and sensing.

The application runs in a PC with a web server. The user interface is

done through a standard browser using common HTML pages or Java applets.

The interaction with the home control system is done through specific
hardware or PC standard interfaces such as the serial port or USB, or by LAN.
The application offers a powerful way of controlling and programming the
behavior of the home, using scripts. For the common user, a simpler version
of a script - a scenario may be used. A scenario consists of a set of devices
in a particular state and can be activated directly by the user, by a time
stimulus or by any event in the home.

22

iv.

Networked Home Automation Projects [16]

This paper outlines several senior capstone projects in the area of

computer engineering. The projects utilize knowledge of several areas
including but not limited to embedded systems, interfacing sensors, accessing
computer peripherals, network programming and graphical user interface
development. Depending on the skills of the student team some elements may
be elaborated and some skipped. The home appliances as the target category
of controlled devices makes this kind of project relatively simple and yet
attractive to students. It also provides hands on experience in the field of
home automation.

v.

A ZigBee-Based Home Automation System [4]

This paper identifies the reasons for this slow adoption and evaluates
the potential of ZigBee for addressing these problems through the design and
implementation of a flexible home automation architecture. A ZigBee based
home automation system and Wi-Fi network are integrated through a
common

home

gateway.

The

home

gateway

provides

network

interoperability, a simple and flexible user interface, and remote access to the
system. A dedicated virtual home is implemented to cater for the systems
security and safety needs. To demonstrate the feasibility and effectiveness of
the proposed system, four devices, a light switch, radiator valve, safety sensor
and ZigBee remote control have been developed and evaluated with the home
automation system

23

2.14

Related Thesis

The following are some of the related thesis by previous UTM student.

i.

Voice Recognition Home Automation System [17]

This project involve a simple switching system that used the transistor
along with relay to do all the connecting of the power to the devices, a voice
recognition system that consists of voice recognition chip HM2007, and the
PIC18F8722 microcontroller to build up the system. The HM2007 serves as
the ear that will listen and interpret the command by the given while the
PIC18F8722 serve as the brain of the system that will coordinate the correct
output with the input command given. This project able to recognition the
command trained by the user and successfully to execute the correct output.
This project is a small scale design which consists of 8 commands that will
used to control three different switches. The command is able to individually
switch on and switch off each of the switch. Besides that, the command
able to switch on all and off all switches at the same time.

ii.

Home Automation Control Using Voice Recognition And Phone Sms [18]

In this project, a graphic user interface is designed so that lights and

fan can be controlled simultaneously through their home personal computer
using voice recognition or phone SMS as their remote control.

iii.

Home Automation Lighting System [19]

This project is using the automation concept to control the lights in

our house in order to optimize energy consumption and further providing
better comfort to residents. A motion detection circuit was designed and set
up to turn on or off the lights when someone is passing by at that certain areas.
Besides, a graphic user interface is designed so that lights can be controlled
simultaneously through their home personal computer. In the way to increase

24

the mobility of this project, local area network connection concept is

implemented here to interconnect the client side personal computer which
located somewhere else within the house, with the home personal computer.
This bring convenience to the family members since anyone who able to pass
the authorization can take full control of the lighting system at anywhere such
as their room as long as their computer is locally connected to home personal
computer.

iv.

Power Line Communication For Home Automation System [20]

This project is to develop a home automation system using the AC

(alternating current) power line and to establish a network between main
controlling unit and client units with an own designed power line
communication protocol. This project is implemented in two phases, namely
software and hardware. Software phase is mainly the programming of the
microcontroller which is the brain of the home automation system. For the
hardware phase of this project, microcontroller, FSK modem, coupling
network and power supply unit are used to develop a working prototype of
the home automation system. The designed system would consist of one main
controlling unit and several client units. Each client unit would be able to
control several electrical devices.
A network protocol would be designed to have error-handling ability, 2-way
communication, and others. Main controller will receive the input from users
and the corresponding client units will respond to the command which is
either switch on, switch off.

25

CHAPTER 3

METHODOLOGY

3.1

Introduction

In this project, microcontroller PIC 18F4550, PIC16F877A and computer will

be used as the controller for the system. Block diagram of the system is shown in
Figure 3.1.

Figure 3.1: Block diagram of the home automation system

The home automation system is consist of a personal computer which is used

for the speech recognition control and networking control. USB CDC protocol is
used to communicate between personal computer and microcontroller PIC18F4550.

26

Two microcontrollers is used for the system, one is for transmitter of the radio
frequency wireless communication, and one is for the receiver on the port of the
home appliances switch.

The system is divides into two parts that are software and hardware
implementation.

3.2

Hardware Implementation

This section will discuss about components that had been used in the system,
included RF system, USB CDC communication of microcontroller and computer.

3.2.1

Microcontroller PIC18F4550 (transmitter part)

Microcontroller PIC18F4550 is used in the system for the transmitter part.

PIC18F4550 has the USB interface which able to control through USB, rather using
the complicated serial port interface. The RF transmitter is controlled by this PIC
with serial out function. 4 LED will be used to indicate the status of the home
appliance.

To communicate with the computer, USB Communication Device Class

(CDC) protocol will be used, where it create a virtual COM port in computer to make
the communication easier.

27

3.2.2

Microcontroller PIC16F877A (receiver part)

Microcontroller PIC16F877A is used in receiver part for the system.

PIC16F877A is responsible to receive the command from the transmitter and control
the switch using relay.

3.2.3

Radio Frequency Transmitter and Receiver

The radio frequency is used to communicate between the computer and

electric port switch. Radio frequency is suitable for short range control and not
bounded by the wall and obstacle. Hence it is very suitable for this system.

3.2.4

PIC Microcontroller Programmer

A product by Cytron technology (USB ICSP PIC Programmer UIC00A) is

used to program the PIC. UIC00A offers low cost yet reliable and user friendly PIC
USB programmer solutions for developer, hobbyist and students. It is designed to
program popular Flash PIC MCU which includes PIC12F, PIC16F and PIC18F
family. It can also program 16bit PIC MCU. On board ICSPTM (In Circuit Serial
Programming) connector offers flexible method to load program. It supports on
board programming which eliminate the frustration of plug-in and plug-out of PIC
MCU. This also allow user to quickly program and debug the source code while the
target PIC is on the development board. Since USB port is commonly available and
widely used on Laptop and Desktop PC, UIC00A is designed to be plug and play
with USB connection. This programmer obtained it power directly from USB

28

connection, thus no external power supply is required, making it a truly portable

programmer. This programmer is ideal for field and general usage. UIC00A offers
reliable, high speed programming and free windows interface software.

Figure 3.2: USB ICSP PIC Programmer UIC00A

3.2.5

Computer (Intel platform)

Computer that had a minimum requirement of window vista, LAN card and
RS-232 serial port available is used in the system. The intel platform is used for the
computer that used in the system. This platform provides a cheap solution to
implement the system. The platform fit the minimum requirement of the system and
it is able to handle the network control and the speech recognition control of the
system.

29

3.2.6

Local Area Network (LAN)

A small geographical area, likes house, office, or building is covered by a

computing network called local area network. Characteristics of LAN including
higher data transfer rate, small graphical range, and lack of a need for leased
telecommunication lines.

Switched Ethernet is now the most common data link layer protocol and IP as
a network layer protocol, but many options have been used and continue popular in
niche areas. LANs may have connections with other LANs via leased lines, leased
services, or by 'tunneling' across the Internet using VPN technologies. They may
become a Metropolitan Area Network (MAN), a Wide Area Network (WAN), or a
part of the internet which depend on how the connections are made, secured, and the
distance involved.

3.3

Software Implementation

The software implementation including the software used to design the GUI
of the system in the computer, the speech recognition software, and the
microcontroller compiler and programmer.

30

3.3.1

Graphic User Interface of the system

In order for the user to control the home appliance by using computer, a
graphic user interface had been design for the user easier to control their home
appliance.

The graphic user interface of the system was build using Visual Basic 2008.
The program code used in this project is visual basic .net. It had provided a lot of
useful API for the user to code their program, including COM port interface, speech
recognize (SAPI) and so on. The code is more simplify compare to C++ and C#.

3.3.2

PICBASIC PRO Compiler

Instead of using MPlab for the PIC compiler, this project use PICBASIC
PRO compiler for the programming in PIC.
The PICBASIC PRO Compiler (or PBP) makes it even quicker and easier
for you to program Microchip Technologys powerful PIC microcontrollers (MCUs).
The English-like BASIC language is much easier to read and write than assembly
language. The PICBASIC PRO Compiler is BASIC Stamp II like and has most of
the libraries and functions of both the BASIC Stamp I and II. Being a true compiler,
programs execute much faster and may be longer than their Stamp equivalents.

31

3.3.3

Window Live Mesh

Windows Live Mesh (formerly known as Live Mesh, Windows Live

Sync and Windows

Live

FolderShare) is

free-to-use

Internet-based

file

synchronization application by Microsoft that is designed to allow files and folders

between two or more computers be in sync with each other on Windows (Vista and
later) and Mac OS X computers or the Web via Windows Live SkyDrive. Windows
Live Mesh also enables remote desktop access via the internet. Windows Live Mesh
is part of the free Windows Live Essentials suite of software.

The system is used this free-to-use software to remote desktop through

internet. By using this software, user is able to control the home automation software
through internet.

Figure 3.3: Architecture of the Live Mesh stack

32

3.4

Flow of project

This project is summarized to the flow chart below. The project is done step
by step and Figure 3.3 shown the flow chart of the project work.

Design the stand alone RF circuit with user interface.

hardware

LED is used to implement the status of home appliances.

testing

bootloader

Software

Network

Testing

Design the bootloader to make connection between computer and the system.
Test the functionality of the bootloader with MPLAB.

Design the GUI for the system in the computer.

The GUI included the speech recognition function for the system.

Establish a remote connection with the window remote desktop connection function.
The computer of the system (host) can be taking control by the other's computer (client).

Test the functionality of the whole system.

Improvement to the hardware and software implementation .

Figure 3.4: The flow chart of the project work

33

CHAPTER 4

FUNCTIONAL DESCRIPTION

4.1

Home Automation System Purpose

This home automation system purpose is to use the computer to control the
electric home appliance. The features that come with the system included:
1. Speech recognition using SAPI 5.3
2. Network controlling using Windows Live Mesh
3. Wireless RF home appliance controlling
This system is directed to residential user, where the maximum ampere the
system can handle is 7A. This is not suitable to use in the industrial field.

4.2

Operating Environment

The system software has to be work under a computer that has the following
minimum requirements:
1. Operating system window vista or window 7
2. Minimum one LAN and USB port

34

3. Sound card
4. Internet connection
5. Install the window live mesh program

For the hardware part, this system is suitable for residential user, where the
maximum ampere the system can handle is 7A for 240VAC power line. For the
home appliances, make sure the power of home appliance used is not more than
1680watt.

4.3

Functional Requirements

There are three methods for a user to control the home appliance, shown as
following:
1. By clicking the desired output on the GUI
2. By speech recognition function
3. By using network control using window mesh
Before the user can control the home appliances, the transmitter have to
connect to the computer via USB and the receiver have to connect to the power
supply.

35

Figure 4.1: Transmitter part of the system

Figure 4.2: Receiver part of the system

In order for the user to control the home appliance by using computer, a
graphic user interface had been design for the user easier to control their home
appliance.

36

First, the user has to log in before they can control to the home appliance. A
login form shown as below had been designed to provide authorize to the user.

Figure 4.3: Log in form

User can either choose log in to the home automation or update username and
password after they key in their password. For the update option, a form will be
shown, figure 4.4, in order for the user to update his detail.

37

Figure 4.4: update username and password form

In the home automation form (figure 4.5), the user is able to control which
switch to be ON/OFF, by clicking the desire button. For the ALL ON and ALL OFF
button, it will on or off all the switches. The user has to choose the right comport in
comport list box before he can interface to the device.

Figure 4.5: Home Automation Control Form

38

For the speech recognition function, user can activate it by clicking the listen
button. A window speech recognition dialog will be pop out and user can start give
command. To start the speech recognition, user is required to speak start listening
to wake it up.

Figure 4.6: Speech Recognition Function

To on the certain switch, user has to say computer first, follow by the
switch you want to on or off, example: if you want to on switch 1, you may say
computer, on one.

The computer will respond back to you which switch is off or on after you
give the command. This will beneficial to blind people to control their home switch
by just speaking. The picture also will update the status of the switch, shown in
figure 4.7.

39

Figure 4.7: Show switch status on the picture

The next section we will discussed about the network controlling using
window live mesh. First, the system has to sign in with window live mesh.

Figure 4.8: Window Live Mesh Sign In Page

40

After sign in for your system, then you can leave it alone and you can control
it by sign in the window live mesh in other computer and click for the remote tab.
This allow user to control their system online.

Figure 4.9: Window Live Mesh Status Tab

Figure 4.10: Window Live Mesh Remote Tab

41

4.4

Performance requirements

The system has to work in a few conditions in order to working get the desire
output, these included:

1. Silent environment to make sure the speech recognition function is working

well.
2. Mic should be place 10cm to users mouth.
3. The user has to do the speech recognition training for more sustainable result.
4. The maximum range between the transmitter and receiver should less than
100m.
5. The system required to online for the network controlling using windows live
mesh. User has to create an online account for the window live before it can
be used.

42

CHAPTER 5

DESIGN ARCHITECTURE

5.1

System Design Architecture

This chapter will be discussed about the design architecture of the system;
there are two sections, hardware design and software design. Hardware design
included the transmitter part, receiver part and the electric appliance port part.
Software design included the programming coding for GUI, window live mesh,
speech recognition and hardware firmware.

5.2

Hardware Design

For the transmitter part of the system, it requires to handle USB

communication, and then give the output to the receiver using RF transmitter. For the
actual look of the circuit, you can refer to Figure 4.1.

43

Figure 5.1: Transmitter Part Schematic Diagram

For the receiver part of the system, it required to receive the signal from the
transmitter and activate the certain relay to ON/OFF the switch. For the actual look
of the circuit, you can refer to Figure 4.2.

Figure 5.2: Receiver Part Schematic Diagram

44

Figure 5.3: Modification of the Electric Extension Port

5.3

Software Design

This can divide into two parts, one is the GUI programming code, and
another one is the firmware of the PIC microcontroller. For the GUI programming
code, the author had used Microsoft Visual Basic to code my program, using
BASIC .net coding. For the firmware, PICBASIC is used as the assembler of the
program.

45

5.3.1

Graphic User Interface of the System

In order for the user to control the home appliance by using computer, a
graphic user interface had been design for the user easier to control their home
appliance.

To start coding the GUI, you have to follow the following step:

1. Go to file menu and choose new project.

2. A window shown in figure 5.4 will pop out, and choose windows forms
application to build a new GUI project.
3. For speech recognition function, Microsoft speech object library should be
added. Go project tab, click add reference, and the library can be found in the
COM.
4. A new project will be built after that, and user can start design their project
graphic user interface by putting the button, text box, label and so on.
5. Double click the form or the button to start your programming coding there.
6. Compile it after finish.

Figure 5.4: New project window

46

Figure 5.5: GUI design interface

First, the user has to log in before they can control to the home appliance. A
login form shown as Figure 4.3 had been designed to provide authorize to the user.

Code 5.1: Log in form code

47

There are two options after user have key in the username and password,
either the user want to login or update his username and password. The username
and password are saved in the user.txt and password.txt file. It will compare the
username and password that user type in the text box, with the username and
password in the txt file, to make decision that the username and password type by the
user is right or not. If the user want to login to the home automation system,
frmhome will show up and hide the login form. If the user wants to update his
username and password, form2 will show up and hide the login form.

Form2 (figure 4.4) is the update username and password form, as shown in
figure. It require user to key in the new username and password, and it will update
the user.txt and password.txt using the following code.

My.Computer.FileSystem.WriteAllText(user.txt, CStr(txtnewusername.Text),
False)
My.Computer.FileSystem.WriteAllText(pass.txt, CStr(txtnewpassword1.Text),
False)
Code 5.2: Write text coding

The home automation form is the control form for the whole system. Figure
4.5 had shown the design of the home automation form. To implement this project, 4
light is use in the form, and two buttons beside the light is to control the light
ON/OFF. The four lights will update when user open or close the light, these picture
will show the status of the light. The listen button is to start the speech recognition
function and the stop listen is the stop the function.
Before we start control the light, we have to load the com port initialization
and the speech recognition initialization first. The following is the load code for the
frmhome.

48

Private Sub frmhome_Load_1(ByVal sender As System.Object, ByVal e As

System.EventArgs) Handles MyBase.Load
Label8.Text = Say computer first, follow by your command. Ex: computer,
on one
initialize com port
InitializeForm()
picture
Picoff1.Show()
Picon1.Hide()
Picoff2.Show()
Picon2.Hide()
Picoff3.Show()
Picon3.Hide()
Picoff4.Show()
Picon4.Hide()
open com port
OpenComPort()

SetState(False)
initialize voice object
Voice = New SpeechLib.SpVoice
computer speak welcome to home automation
Voice.Speak(welcome

to

home

SpeechLib.SpeechVoiceSpeakFlags.SVSFlagsAsync)
End Sub
Code 5.3: frmhome_load code

Sub InitializeForm()
Dim nameArray() As String
' Find the COM ports on the system.
nameArray = SerialPort.GetPortNames
Array.Sort(nameArray)
' Fill a combo box with the port names.

automation,

49

cmbPorts.DataSource = nameArray
cmbPorts.DropDownStyle = ComboBoxStyle.DropDownList
' Select a default port.
End Sub
Code 5.4: Initialize the com port

Sub OpenComPort()
Try
' Get the selected COM port's name from the combo box.
If Not mycomport.IsOpen Then
mycomport.PortName = cmbPorts.SelectedItem.ToString
' Set other port parameters.
mycomport.Parity = Parity.None
mycomport.DataBits = 8
mycomport.StopBits = StopBits.One
mycomport.Handshake = Handshake.None
mycomport.ReadTimeout = 3000
mycomport.WriteTimeout = 5000
' Open the port.
mycomport.Open()
End If
Catch ex As InvalidOperationException
MessageBox.Show(ex.Message)
Catch ex As UnauthorizedAccessException
MessageBox.Show(ex.Message)
Catch ex As System.IO.IOException
MessageBox.Show(ex.Message)

End Try
End Sub
Code 5.5: Open Com Port Coding

50

Most of the coding is explained in the comment; refer to the code for further
understanding.

In order to sent a string and receive response string through COM port, a
sendcommand function is used, shown in code 3.6.

Private Sub SendCommand(ByVal command As String)

Dim response As String
Try
'sent the string to the device via COM port
mycomport.WriteLine(command)
'receiver string from the device via COM port and store in response
response = mycomport.ReadLine

Select Case response

Case "0"
Picoff1.Show()
Picon1.Hide()

Case "1"
Picon1.Show()
Picoff1.Hide()

Case "2"
Picoff2.Show()
Picon2.Hide()

Case "3"
Picon2.Show()
Picoff2.Hide()

Case "4"

51

Picoff3.Show()
Picon3.Hide()

Case "5"
Picon3.Show()
Picoff3.Hide()

Case "6"
Picoff4.Show()
Picon4.Hide()

Case "7"
Picon4.Show()
Picoff4.Hide()
Case Else
End Select

Catch ex As TimeoutException
MessageBox.Show(ex.Message)

Catch ex As InvalidOperationException
MessageBox.Show(ex.Message)

Catch ex As UnauthorizedAccessException
MessageBox.Show(ex.Message)

End Try
End Sub
Code 5.6: SendCommand subroutine function

For each button click, a string will sent to the device through COM port using
the sendcommand subroutine function. To on switch 1, 2, 3 and 4, it should sent

52

string command 1, 3, 5 and 7. To off switch 1, 2, 3 and 4, it should sent

sring command 0, 2, 4, 6.

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As

System.EventArgs) Handles Button1.Click
SendCommand("1")
SendCommand("3")
SendCommand("5")
SendCommand("7")
Voice.Speak("All

component

had

on",

SpeechLib.SpeechVoiceSpeakFlags.SVSFlagsAsync)
End Sub

Private Sub Button2_Click(ByVal sender As System.Object, ByVal e As

System.EventArgs) Handles Button2.Click
SendCommand("0")
SendCommand("2")
SendCommand("4")
SendCommand("6")
Voice.Speak("All

component

had

off",

SpeechLib.SpeechVoiceSpeakFlags.SVSFlagsAsync)
End Sub

Private Sub btnon1_Click(ByVal sender As System.Object, ByVal e As

System.EventArgs) Handles btnon1.Click
SendCommand("1")
Voice.Speak("Light

had

on",

SpeechLib.SpeechVoiceSpeakFlags.SVSFlagsAsync)
End Sub

Private Sub btnon2_Click(ByVal sender As System.Object, ByVal e As

System.EventArgs) Handles btnon2.Click
SendCommand("3")

53

Voice.Speak("Light

had

on",

SpeechLib.SpeechVoiceSpeakFlags.SVSFlagsAsync)
End Sub

Private Sub btnon3_Click(ByVal sender As System.Object, ByVal e As

System.EventArgs) Handles btnon3.Click
SendCommand("5")
Voice.Speak("Light

had

on",

SpeechLib.SpeechVoiceSpeakFlags.SVSFlagsAsync)
End Sub

Private Sub btnon4_Click(ByVal sender As System.Object, ByVal e As

System.EventArgs) Handles btnon4.Click
SendCommand("7")
Voice.Speak("Light

had

on",

SpeechLib.SpeechVoiceSpeakFlags.SVSFlagsAsync)
End Sub

Private Sub btnoff1_Click(ByVal sender As System.Object, ByVal e As

System.EventArgs) Handles btnoff1.Click
SendCommand("0")
Voice.Speak("Light

had

off",

SpeechLib.SpeechVoiceSpeakFlags.SVSFlagsAsync)
End Sub

Private Sub btnoff2_Click(ByVal sender As System.Object, ByVal e As

System.EventArgs) Handles btnoff2.Click
SendCommand("2")
Voice.Speak("Light

had

off",

SpeechLib.SpeechVoiceSpeakFlags.SVSFlagsAsync)
End Sub

Private Sub btnoff3_Click(ByVal sender As System.Object, ByVal e As

System.EventArgs) Handles btnoff3.Click

54

SendCommand("4")
Voice.Speak("Light

had

off",

SpeechLib.SpeechVoiceSpeakFlags.SVSFlagsAsync)
End Sub

Private Sub btnoff4_Click(ByVal sender As System.Object, ByVal e As

System.EventArgs) Handles btnoff4.Click
SendCommand("6")
Voice.Speak("Light

had

off",

SpeechLib.SpeechVoiceSpeakFlags.SVSFlagsAsync)
End Sub
Friend WithEvents btnexit As System.Windows.Forms.Button

Private Sub btnexit_Click(ByVal sender As System.Object, ByVal e As

System.EventArgs) Handles btnexit.Click
CloseComPort()
End
End Sub
Code 5.7: Buttons Coding

The following section is the speech recognition function for the GUI. When
the listen button is click, the following code will be run. The following code is to
initialize the speech recognition and add in the SAPI into the program. The
initialization included the specific grammar we will used (the initial state and the
transition state).

Private Sub btnstart_Click(ByVal sender As System.Object, ByVal e As

System.EventArgs) Handles btnstart.Click

Dim AfterCmdState As SpeechLib.ISpeechGrammarRuleState

System.Diagnostics.Debug.Assert(Not m_bRecoRunning, "")

55

Voice.Speak("what

can

help

you?",

SpeechLib.SpeechVoiceSpeakFlags.SVSFlagsAsync)
' Initialize recognition context object and grammar object, then
' start dictation
If (RecoContext Is Nothing) Then
Debug.Print("Initializing SAPI reco context object...")
RecoContext = New SpeechLib.SpSharedRecoContext

'Adding grammar to speech recognition

Grammar = RecoContext.CreateGrammar(m_GrammarId)

' Add two rules. The top level rule will reference the items rule.
TopRule

Grammar.Rules.Add("TopLevelRule",

SpeechLib.SpeechRuleAttributes.SRATopLevel

Or

SpeechLib.SpeechRuleAttributes.SRADynamic, 1)
ListItemsRule

Grammar.Rules.Add("ListItemsRule",

SpeechLib.SpeechRuleAttributes.SRADynamic, 2)

AfterCmdState = TopRule.AddState

TopRule.InitialState.AddWordTransition(Nothing, "computer", " ", , "", 0, 0)

AfterCmdState.AddRuleTransition(Nothing, ListItemsRule, "", 1, 1)

' Now add existing list items to the ListItemsRule

ListItemsRule.InitialState.AddWordTransition(Nothing, "on one", " ", , "on
one", 0, 0)
ListItemsRule.InitialState.AddWordTransition(Nothing, "on two", " ", , "on
two", 1, 1)
ListItemsRule.InitialState.AddWordTransition(Nothing, "on three", " ", , "on
three", 2, 2)
ListItemsRule.InitialState.AddWordTransition(Nothing, "on four", " ", , "on
four", 3, 3)
ListItemsRule.InitialState.AddWordTransition(Nothing, "off one", " ", , "off
one", 4, 4)

56

ListItemsRule.InitialState.AddWordTransition(Nothing, "off two", " ", , "off

two", 5, 5)
ListItemsRule.InitialState.AddWordTransition(Nothing, "off three", " ", , "off
three", 6, 6)
ListItemsRule.InitialState.AddWordTransition(Nothing, "off four", " ", , "off
four", 7, 7)
ListItemsRule.InitialState.AddWordTransition(Nothing, "all on", " ", , "all
on", 8, 8)
ListItemsRule.InitialState.AddWordTransition(Nothing, "all off", " ", , "all
off", 9, 9)

Grammar.Rules.Commit()
Grammar.CmdSetRuleState("TopLevelRule",
SpeechLib.SpeechRuleState.SGDSActive)
End If
SetState(True)
End Sub

Private Sub btnStop_Click(ByVal sender As System.Object, ByVal e As

System.EventArgs) Handles btnStop.Click
System.Diagnostics.Debug.Assert(m_bRecoRunning, "")
Grammar.DictationSetState(SpeechLib.SpeechRuleState.SGDSInactive)
SetState(False)
End Sub

Private Sub SetState(ByVal bNewState As Boolean)

m_bRecoRunning = bNewState
btnstart.Enabled = Not m_bRecoRunning
btnStop.Enabled = m_bRecoRunning
End Sub
Code 5.8: Listen and Stop Listen Coding

Private Sub RecoContext_Recognition(ByVal StreamNumber As Integer, ByVal

StreamPosition

As

Object,

ByVal

RecognitionType

As

57

SpeechLib.SpeechRecognitionType, ByVal Result As SpeechLib.ISpeechRecoResult)

Handles RecoContext.Recognition

Dim index As Short

Dim oItem As SpeechLib.ISpeechPhraseProperty

oItem = Result.PhraseInfo.Properties.Item(1).Children.Item(0)
index = oItem.Id

If Result.PhraseInfo.GrammarId = m_GrammarId Then

If oItem.Name = "on one" Then

btnon1.PerformClick()
End If

If oItem.Name = "on two" Then

btnon2.PerformClick()
End If

If oItem.Name = "on three" Then

btnon3.PerformClick()
End If

If oItem.Name = "on four" Then

btnon4.PerformClick()
End If

If oItem.Name = "off one" Then

btnoff1.PerformClick()
End If

If oItem.Name = "off two" Then

btnoff2.PerformClick()
End If

58

If oItem.Name = "off three" Then

btnoff3.PerformClick()
End If

If oItem.Name = "off four" Then

btnoff4.PerformClick()
End If

If oItem.Name = "all on" Then

Button1.PerformClick()
End If

If oItem.Name = "all off" Then

Button2.PerformClick()
End If

End If
End Sub
Code 5.9: Speech Recognition Function

5.3.2

Firmware

To make the device working, firmware has to be boot in the PIC

microcontroller to make it function. For the complier of the firmware, MicroCode
Studio is used.

59

Figure 5.6: MicroCode Studio Compiler

For the transmitter part, the PIC has to handle USB CDC protocol and the RF
transmits. The code for the transmitter part PIC18F4550 is shown in the appendix 1.
Before coding for the PIC18F4550, it should include two libraries, one is HID
descriptors for the USB CDC stack (cdc_desc.bas) and one is the serial modes
(modedefs.bas) for the serial communication for transmitter. The firmware will
continuously check the data in from the USB and give the corresponding respond to
the command, by using USBIN function.

When it receives the command (0 to 7), it will show respond on the led and
then sent the command through RF transmitter 10 times to the terminal port, using
SEROUT function. This is to make sure the string is successfully sent out. The
command sent format is set to be, AABB, _., where it sent (0-7) string after
sending AABB string. This is to make sure the receiver can react accurately after
receive the command. After that, it will send a response ending back to the computer
using USBOUT function. To make sure the USB connection does not drop,
USBSERVICE should be done regularly.

For the receiver part, the PIC has to handle the RF receiver and give the
output to the relay to on/off the relay. The code for the receiver part PIC16F877A is
shown in the appendix 2. Serial mode library should be included in the firmware. To
receive the RF signal, SERIN function is used. It responds only after it detects string

60

AABB, and it will save the string in B0. By using SELECT CASE B0 function, it
will give respond on the corresponding result, and switch ON/OFF the switch.

61

CHAPTER 6

RESULT AND DISCUSSION

6.1

Hardware Verification

To verify the hardware functional, TeraTerm program is used. For window

vista onward, there is no COM port hiperterminal, hence TeraTerm is to replace it.
First, make the connection to the COM port after connecting the device to the USB
using TeraTerm, figure 6.1. Choose the right serial port after the device connect to
USB.

Figure 6.1: Tera Term Connection

62

After that, you can send string by typing on the TeraTerm window. The
device will give back echo response via USB. TeraTerm will shown the response
after receive the response from PIC18F4550, shown in figure 6.2.

Figure 6.2: Tera Term Serial Port communications

It can show the result in the fastest way before the software developed. When
7 is sent to the PIC18F4550, it will show the respond by ON the fourth led, then it
will sent the rf signal to the terminal part and ON the switch in the receiver part,
figure 6.3

Figure 6.3: Hardware verification

63

6.2

System Verification

We start our system verification using speech recognition function in our

program. First, connect the terminal part and headphone to the atom board; connect
the receiver part to the electric supply. Then, click listen button to start the speech
recognition function.

Figure 6.4: System Verification

To start the speech recognition, say start listening to wake up the computer.
Then you may give the command by saying computer follow with the command.
the author started saying with computer, all on and the system responded by ON all
the switch, shown in figure 6.5.

64

Figure 6.5: Response after It Recognize ALL ON command

After that the author tested on the, computer, all off command. As the
result, it had OFF all the switch, shown in figure 6.6.

Figure 6.6: Response after It Recognize ALL OFF command

Lastly the author tried to test on the specific switch by using, computer, on 1
command. As the result, it had ON the switch 1, as shown in figure 6.7.

65

Figure 6.7: Response after It Recognize ON ONE command

As a result, the speech recognition function is perfectly workable for the

system. But in the noisy environment, it may give the wrong result. If the user gives
the command not clearly, the program will confirm one more time for the user to
choose which button should be click.

For the next test, the author tried on the reliability of the RF transmits. The
author placed the receiver part around 7m apart from the transmitter part. As a result,
it still working perfectly without any error occurred.

66

7m

Figure 6.8: Receiver place 7m away from the transmitter

The last section is to test the network controlling using the window live mesh
program. First we connect the system to the window live mesh online, please refer to
figure 4.8 for the log in part.

Figure 6.9: Remote Functions in Window Live Mesh (system)

67

After log in the Window Live Mesh on the system, then you may connect the
client computer to Window Live Mesh as well using the same account. On the client
computer, clicked connect to the computer to connect to the system.

Click to connect
to the system

Figure 6.10: Remote Functions in Window Live Mesh (client)

As a result, it pop out the window below, which is the process connecting
with your system, as shown in Figure 6.11. You may require waiting until the
connection is established.

Figure 6.11: Connecting Process to the System

68

After connected successfully, a window will pop out, showing the screen of
the system computer. You have to key in the password before you can take control to
the system computer, as shown in figure below.

Figure 6.12: Connected to the System

After log in, you may start you home automation program by clicking the
home automation shortcut. Log in to it and start control the system through internet,
which shown in figure 6.13 to figure 6.16.

69

Figure 6.13: Desktop of the System

Figure 6.14: Initialize the Home Automation

70

Figure 6.15: Start Control the System through Online

Figure 6.16: Start Control the System through Online

71

Figure 6.17: Network Connections Through Window Live Mesh

For the network connection through Window Live Mesh, the result is
satisfied, but slow respond may occur if the internet service is not good enough.

72

CHAPTER 7

CONCLUSION

7.1

Conclusion

This project presents the design and the implementation of a home

automation system with the computer based speech recognition and networking
control feature. The proposed project is fully implemented and developed.

The speech recognition made the convenient to the motion disabled. It can
easily control the switch by just using speech. With SAPI 5.3, the result of the
recognition is reliable with lowest error recognition occurred.

The networking controlling through Window Live Mesh can enable the user
to control the system through online. The whole system is secured through a login
password and username based authentication.

The design is wireless with RF communication, which makes the system easy
to install, instead of using wired communication. It provided a low-cost, robust and
easily operable system. You can connect any electric appliance to the system, as long
as it does not existing 7A current usage.

73

7.2

Recommendation

There are few recommendations for further research on this project:

1. This project can be further research on controlling not only the ON/OFF of
the home appliance.
2. Sensor may add to the system to off the light automatically if there is no one
in the room.
3. ON/OFF schedule should be added to enable user to set the ON/OFF time for
the home appliance.

74

REFERENCES

1. Renato Jorge Caleira Nunes. Decentralized Supervision for Home

Automation. IEEE MELECON. 2006, May 16-19. Benalmadena (Malaga),
Spain: IEEE. 785-788.
2. Baris Yuksekkaya, A. Alper Kayalar, M. Bilgehan Tosun, M. Kaan Ozcan,
and Ali Ziya Alkar. A GSM, Internet and Speech Controlled Wireless
Interactive Home Automation System. 2006. 52(3): 837-843.
3. Renato J. C. Nunes, Member, IEEE, and Jose C. M. Delgado, Member, IEEE.
An Internet Application for Home Automation. 10th Mediterranean
Electrotechnical Conference. 2000. USA: IEEE. 2000 298-301.
4. Khusvinder Gill, Shuang-Hua Yang, Fang Yao, and Xin Lu. A ZigBee-Based
Home Automation System. 2009. 55(3): 422-430.
5. Renato Nunes, Jose Delgado, "An Architecture for a Home Automation
System", ICECS '98 - International Conference on Electronics, Circuits and
Systems, September 1998, pp.259-262.
6. Microchip Technology Inc. PIC18F2455/2550/4455/4550 Data Sheet. United
States of America: Microchip Technology Inc. 2009.
7. Microchip Technology Inc. PIC16F877A Data Sheet. United States of
America: Microchip Technology Inc. 2009.
8. E. Schlunder. High-Speed Serial Bootloader for PIC16 and PIC18 Devices.
United States of America: Microchip Technology Inc. 2010.
9. Jan Axelson. USB Complete: The Developers Guide. 4th. ed. United States of
America: Lakeview Research LLC. 2009.
10. Microchip Technology Inc. PICDEM FS USB Demonstration Board Users
Guide. United States of America: Microchip Technology Inc. 2004.
11. Microchip Technology Inc. MCHPFSUSB Firmware Users Guide. United
States of America: Microchip Technology Inc. 2004.

75

12. Cytron Technologies Sdn. Bhd. RF-TX-433 RF Transmitter Module Users

Manual V1.1. Malaysia: Cytron Technologies Sdn. Bhd. 2008.
13. Cytron Technologies Sdn. Bhd. RF-TX-433 RF Receiver Module Users
Manual V1.1. Malaysia: Cytron Technologies Sdn. Bhd. 2008.
14. Jon Burroughs. X-10 Home Automation Using the PIC16F877A. United
States of America: Microchip Technology Inc. 2010.
15. Kwang Yeol Lee and Jae Weon Choi. Remote-Controlled Home Automation
System via Bluetooth Home Network. SICE Annual Conference in Fukui.
August 4-6,2003 Korea: IEEE. 2003. 2824-2829.
16. Aleksander Malinowski. Networked Home Automation Projects. The 27th
Annual Conference of the IEEE Industrial Electronics Society. 2001. USA:
IEEE. 2001. 1758-1762.
17. Choy Meng Onn. Voice Recognition Home Automation System. Degree
Thesis. Universiti Teknologi Malaysia; 2010.
18. Adam bin Ibrahim. Home Automation Control Using Voice Recognition and
Phone Sms. Degree Thesis. Universiti Teknologi Malaysia; 2010.
19. Goh Lin Chian. Home Automation Lighting System. Degree Thesis. Universiti
Teknologi Malaysia; 2010.
20. Muhammad Hafidzuddin bin Mazlan. Power Line Communication for Home
Automation. Degree Thesis. Universiti Teknologi Malaysia; 2010.

76

APPENDIX A

Program in PIC18F4550 for the Transmitter Part of the System

Include"cdc_desc.bas"
Include "modedefs.bas"
SO

VAR PORTC.6

led1

var PORTD.3

led2

var PORTD.2

led3

var PORTD.0

led4

var PORTD.1

buffer Var
cnt
i

Byte[16]

Var Byte
var byte

Define

OSC

'led blinking

for i=1 to 2
high led1
pause 50
high led2
pause 50
high led3
pause 50
high led4
pause 50

low led1
low led2

48

' Include the HID descriptors

' Include serial modes
' Define pinout as PORTC.6
'Define LED port

'Define variable

77

low led3
low led4

high led4
pause 50
high led3
pause 50
high led2
pause 50
high led1
pause 50

low led1
low led2
low led3
low led4
next i

USBInit

' Get USB going

USBService

' Wait for USB input

idleloop:
USBService

' Must service USB regularly

cnt = 16

' Specify input buffer size

buffer[0]=0
buffer[1]=0
USBIn 3, buffer, cnt, idleloop

select case buffer[0]

case "7"
high led4

' Turn on the LED.

'sent rf signal
FOR I=1 To 10
Serout SO,N2400,["AABB","7"]

78

next i
USBService
USBOut 3, buffer, 2, outloop

' Send a response ending

case "6"
low led4

'Turn off the LED.

'sent rf signal
FOR I=1 To 10
Serout SO,N2400,["AABB","6"]
next i
USBService
USBOut 3, buffer, 2, outloop

' Send a response ending

case "5"
high led3

'Turn on the LED.

'sent rf signal
FOR I=1 To 10
Serout SO,N2400,["AABB","5"]
next i
USBService
USBOut 3, buffer, 2, outloop

' Send a response ending

case "4"
low led3

'Turn off the LED.

'sent rf signal
FOR I=1 To 10
Serout SO,N2400,["AABB","4"]
next i
USBService
USBOut 3, buffer, 2, outloop

case "3"
high led2
'sent rf signal

'Turn on the LED.

' Send a response ending

79

FOR I=1 To 10
Serout SO,N2400,["AABB","3"]
next i
USBService
USBOut 3, buffer, 2, outloop

' Send a response ending

case "2"
low led2

'Turn off the LED.

'sent rf signal
FOR I=1 To 10
Serout SO,N2400,["AABB","2"]
next i
USBService
USBOut 3, buffer, 2, outloop

' Send a response ending

case "1"
high led1

'Turn on the LED.

'sent rf signal
FOR I=1 To 10
Serout SO,N2400,["AABB","1"]
next i
USBService
USBOut 3, buffer, 2, outloop

' Send a response ending

case "0"
low led1

'Turn off the LED.

'sent rf signal
FOR I=1 To 10
Serout SO,N2400,["AABB","0"]
next i
USBService
USBOut 3, buffer, 2, outloop
case else
GOTO outloop

' Send a response ending

80

end select

outloop:
' Must service USB regularly
USBService
Goto idleloop

' Wait for next buffer

81

APPENDIX B

Program in PIC16F877A for the Receiver Part of the System

Include "modedefs.bas"
SI

VAR

PORTC.7

B0

Var Byte

LED1

VAR PORTD.0

LED2

VAR PORTD.1

LED3

VAR PORTD.2

LED4

var PORTD.3

var byte

Define OSC

'led blinning

for i=1 to 2
high led1
pause 50
high led2
pause 50
high led3
pause 50
high led4
pause 50

low led1
low led2
low led3

20

' Define pinin as PORTC.7

82

low led4

high led4
pause 50
high led3
pause 50
high led2
pause 50
high led1
pause 50

low led1
low led2
low led3
low led4
next i

START:
B0=0
Serin SI,N2400,["AABB"],B0
select case B0
CASE "7"
HIGH LED4
CASE "6"
LOW LED4
CASE "5"
HIGH LED3
CASE "4"
LOW LED3
CASE "3"
HIGH LED2
CASE "2"
LOW LED2
CASE "1"

83

HIGH LED1
CASE "0"
LOW LED1
END SELECT

GOTO START