Data Acquisition & Transfer for Remote Video Surveillance

Project in Lieu of Thesis For Masters Degree The University of Tennessee, Knoxville Sarath K. Dandala December 2003

Abstract
There is an increased awareness on security among all the nations of the world after the tragic incident of 9/11. There is also an increased threat from terrorists who might use weapons of mass destruction (WMD) at any area of high national security. The objective of this project is to detect these threats using the latest and innovative available technologies. This project is an attempt to extend the ever-increasing applications of image processing to under bridge inspection. The main objective of the project is to collect range and video data of under bridge traffic such as vessels, container ships, etc. by placing different sensors on the bridge and then transmit this data over a wireless network to a remote location for post processing. This data is then used to generate a textured 3D model for interactive inspection. The focus of the project is to monitor traffic such as large cargo ships, barges and other vessels under one of the main bridges in the port of Port Arthur, TX. The use of wireless internet technologies for security and surveillance applications is increasing day by day. The surveillance applications vary from monitoring super markets to large coast line harbors. This project is an example of one such application called harbor security and surveillance. Video surveillance can be a potent tool for law enforcement. A very basic video surveillance system consists of a PTZ camera, a video server, and transmission media to connect the camera to other components in the system. The PTZ camera, placed at ideal locations for surveillance, captures live video of the traffic under the bridge. The video is then fed as input to the video server. The video server is special hardware which digitizes and compresses the video data and then uploads the video data on to an existing TCP/IP network. However, many applications involve situations where it would be convenient to have the camera placed at some location where it's not practical to run a cable from the camera. It is here that the need for a reliable wireless network that can transmit video data over long distances arises. This project concentrates on designing different network layouts with available system components and implements each one of the network layouts. A basic network skeleton is then framed which forms the foundation for developing more robust designs.

-2-

Contents
Table of Figures..............................................................................................4 1. Introduction....5
1.1 Motivation.......................................................................................................5 1.2 Proposed approach.......6

2. Theory and Methods..8

2.1 Project hardware....8 2.1.1 Data acquisition hardware..8 2.1.2 Wireless hardware...12 2.2 Data acquisition techniques...15 2.3 Wireless LAN Set up....20 2.3.1 System design and implementation.20 2.3.2 Wireless LAN set up on the Henley St. Bridge......30

3. Experimental Results..35
3.1 Data acquisition results .35 3.2 Wireless design results ..38 3.3 Testing on Henley St Bridge-results....39 3.4 Wireless system comparisons43

4. Conclusions..44 References46

-3-

List of Figures
Fig 1.1 Satellite image of the port of Port Arthur, TX Fig 1.2 Martin Luther King Jr. (MLK) bridge in Port Arthur, TX Fig 1.3 General network lay out for the harbor surveillance system Fig 2.1 The RIEGL 3D LMZ range scanner Fig 2.2 Measurement principle of a pulsed range finder Fig 2.3 Two- axis scanning mechanism of RIEGL scanner Fig 2.4 PTZ camera model- Panasonic WV CS854A Fig 2.5 Axis wireless components; (a) Access point, (b) device point Fig 2.6 Tsunami Quick bridge 11 wireless kit. Fig 2.7 (a) Barge traffic on the Tennessee River (b) Tugboat driving the barge Fig 2.8 Henley St Bridge on the Tennessee River, Knoxville, TN Fig.2.9 Data source menu Fig.2.10 Set scanner parameters window Fig 2.11 3D RiSCAN control interface window Fig 2.12 Scan plan for scanning Henley St Bridge Fig 2.13 The general network layout for the W-A-D design. Fig 2.14 The network layout for the W-A-D design in real time Fig 2.15 The general network layout for the W-A-R-D design Fig 2.16 The network layout for the W-A-R-D design in real time Fig 2.17 Generalized network layout for the W-A-N-D design Fig 2.18 Network layout for the W-A-N-D design in real time Fig 2.19 The general network layout for the L-A-D design Fig 2.20 Network lay out for the L-A-D layout in real time. Fig 2.21 Generalized network layout for the W-R-A-B design Fig 2.22 Heavy duty tripod with the system components mounted. Fig 2.23 Tripod stand with the receiving equipment mounted on it. Fig 2.24 Network layout for the real time W-R-A-B design Fig 2.25 Network layout for the confidence test Fig 2.26 Initial prototype designs; (a) Transmitting end tripod stand, (b) receiving end tripod stand Fig 2.27 Weather hardened enclosures; (a) Video server enclosure, (b) base station enclosure Fig 2.28 Initial prototype of the power sub system; (a) Weather hardened power sub system, (b) inner view of the power sub system Fig 2.29 Receiving equipment placed on the roof top of Ferris hall Fig 2.30 Wireless LAN set up on the Henley St. Bridge, Knoxville, TN Fig 3.1 Range image of a tug boat as viewed in RAPIDFORM Fig 3.2 Range images of tug boat and barge as viewed in RAPIDFORM Fig 3.3 Range image of Henley St. Bridge from left bottom Fig 3.4 Range image of Henley St. Bridge from left up Fig 3.5 Range image of Henley St. Bridge from left up (opposite bank) Fig.3.6 Range image of Henley St. Bridge from right bottom Fig 3.7 GUI provided by the Net IQ Check software to measure throughput. Fig 3.8 Throughput variation with file size Fig 3.9 Throughput variation with the angle of deviation from the LOS Fig 4.1 Flow diagram of the 3D Scene reconstruction algorithm -4-

1. INTRODUCTION
In this section, we discuss the main motivation behind the project and also the proposed approach that we intend to deploy for this project.

1.1 Motivation
The main objective of this project in a broader sense is harbor surveillance and security. The harbor that is being considered for this project is the port of Port Arthur, TX. The main focus of this project is to set up a number of stand alone sensor and communication systems on a selected bridge in this area and monitor the traffic that goes under the bridge. As clearly seen from the Fig 1.1, the vegetation around the Port Arthur area does not facilitate the set up of a wired data (mainly video) transmission system. Hence, this calls for a dedicated and separate wireless network that can transmit video data over a long range. The bridge selected for inspection is the Martin Luther King Jr. (MLK) Bridge situated

Fig 1.1 Satellite image of the port of Port Arthur, TX on the Sabine Neches Canal in Port Arthur, TX. Port Arthur is located on the northwest shore of Lake Sabine, which is just nine miles from the Gulf of Mexico. Fig 1.1 above shows an

-5-

aerial image of the MLK Bridge in Port Arthur. A picture of the MLK Bridge with traffic is shown in Fig 1.2.

Fig 1.2 Martin Luther King Jr. (MLK) bridge in Port Arthur, TX The horizontal clearance of the bridge is 60 ft and the vertical clearance is 136 ft [Map].The channel is 40 ft deep and is suitable for heavy vessel traffic like huge container and cargo ships. These ships mainly carry oil to the large number of ports situated beside this channel.

1.2 Proposed Approach

For data acquisition, the proposed approach consists of acquiring real time data such as range scans and video data of real ships that travel under the Henley St. Bridge on the Tennessee River. For scanning purposes, we use a 3D-laser mirror scanner called the RIEGL scanner. These scans are then later processed using special algorithms to generate a textured 3D model. For wireless transmission, the proposed approach consists of designing an integrated system that satisfies the following system requirements Wireless transmission of video to a remote base station Independent and continuous power supply for the system components Weather hardened system World wide accessibility (access over the internet) Remote operability of the system components (GUI applications)

Keeping the system requirements in view, we propose a basic network layout that satisfies the above mentioned system requirements as shown in Fig 1.3. -6-

Server Sub system

PTZ Camera Video Server Point-toPoint

Panasonic, Pelco Axis Server Axis, Tsunami 11 Sierra Wireless 555 Direct TV, Star Link Dell Systems

Power Sub system

Wireless Sub system

Cellular Service Satellite DSL Desktop PC

Client Sub system

Laptop Pocket PC HP IPAQ

Fig 1.3 General network lay out for the harbor surveillance system As clearly evident from Fig 1.3, we have divided the entire system into different subsystems. After determining the functionalities of each sub system, we evaluate different components available in the market and perform a best fitting process on these components to make them fit into the appropriate sub system. Once the components are decided, we evaluate the performance of each component and determine its existence in the final network layout. A specific protocol is laid out based on which components of the system are evaluated. A given component continues to exist in the final lay out as long as it meets the project requirements. After a thorough evaluation of the performance of each sub system, we integrate all the subsystems into a single, multifunctional, rugged and portable surveillance system.

-7-

2. Theory and Methods

This section discusses the project hardware, background theory and the implementation methods used for the project. The project hardware consists of data acquisition hardware for acquiring data and a wireless hardware system for data transmission to a remote location. This section also discusses about the various data acquisition methods that we have deployed to acquire range and video data of under bridge traffic and the wireless transmission techniques deployed to transmit the acquired data to a remote location for post-processing.

2.1 Project hardware

In this section, we present the various hardware such as the data acquisition hardware, wireless hardware, etc. that we have used for the entire project. The project hardware comprises of data acquisition software like range scanners, PTZ cameras and wireless hardware like Axis device & access points, wireless routers & bridges and long range, high gain directional antennas. 2.1.1 Data Acquisition Hardware The data acquisition hardware consists principally of the RIEGL 3D LMS range scanner for obtaining range data of the under bridge traffic, a Panasonic WV CS854A PTZ camera for obtaining video data of the under bridge traffic, and finally a NIKON CoolPix 5700 digital still camera for obtaining still images of the under bridge traffic. Note that the range data is necessary for obtaining 3D information of the traffic and the video data is required to obtain extra shape and texture information of the under bridge traffic. The RIEGL 3D-LMS Range Scanner The 3D-laser mirror scanner LMS-Z210 (Fig 2.1) is a surface imaging system based upon accurate distance measurement by means of electro-optical range measurement and a twoaxis beam scanning mechanism. The 3D images are gained by performing a number of independent laser range measurements in different, but well-defined angular directions. These range data together with the associated angles form the basis of the 3D images.

-8-

Fig 2.1 The RIEGL 3D LMZ range scanner The system configuration of the RIEGL scanner is described in the following paragraph: Rangefinder Module: The rangefinder system is based upon the principle of time-of-flight measurement of short laser pulses in the infrared wavelength region. The basic measurement principle of a pulsed range finder is shown in the Fig 2.2.
Transmitter lens Data Display Micro Computer Time Measurement Unit Photodiode Receiver Diode laser Target

Data Communication

Receiver lens

Fig 2.2 Measurement principle of a pulsed range finder An electrical pulse generator periodically drives a semiconductor laser diode sending out infrared light pulses, which are collimated by the transmitter lens. Via the receiver lens, part

-9-

of the echo signal reflected by the target hits a photodiode which generates an electrical receiver signal. The time interval between transmitted and receiver pulses is counted by means of a quartz-stabilized clock frequency. The calculated range value is fed into the internal microcomputer which processes the measured data and prepares it for data output. Scanner Mechanism: The task of the scanner mechanism is directing the laser beam for range measurement in a precisely defined position. The 3D images are composed of a number of lines, and each line is composed of a number of pixels. The scanning mechanism is designed to provide fast line scans (typically 20 lines/second), and several hundred pixels per line. The frame scan is significantly slower than the line scan and typically contains hundreds of lines. The 2-axis beam scanning mechanism is shown in the Fig 2.3.

Fig 2.3 Two- axis scanning mechanism of RIEGL scanner The vertical deflection ("line scan") of the laser beam (2) is realized by a polygon (3) with a number of reflective surfaces. For high scanning rates and/or a vertical scan angle up to 80, the polygonal mirror rotates continuously at adjustable speed. For slow scanning rates and/or small scanning angles, it is oscillating linearly up and down. The horizontal scan ("frame scan") is provided by rotating the complete optical head (4) up to 360.The gained

- 10 -

information: range, angle, signal amplitude, and target color is provided via a parallel data output which can be connected directly to the ECP compatible LPT printer port (5) of a laptop (6) running the RIEGL RiSCAN software (7). PTZ Cameras The main objective of this project is to place a sensor system on a bridge that captures live video and transmits it to a remote location. Image capture is accomplished using PTZ cameras. An upgraded version of the current system may include a fixed camera performing hand over to the PTZ camera that locks on to the suspicious entity and tracks it. This forms a part of an entirely challenging research topic called video tracking. PTZ stands for Pan/Tilt/Zoom camera and it provides the functionality of panning, tilting or zooming the camera using the RS 232/422/485 communication protocol. This gives PTZ cameras a cutting edge over fixed cameras and makes them ubiquitous in tracking applications. This feature forms one of the guidelines in selecting a PTZ camera for this project. The PTZ camera (See Fig 2.4) selected for this project is the Panasonic CS WV854A.

Fig 2.4 PTZ camera model- Panasonic WV CS854A This camera has been used in a tracking project that is being developed in the IRIS Lab. Hence these cameras were selected for the harbor surveillance project because monitoring and tracking the under bridge traffic is the main motivation behind the project. The Panasonic PTZ is an all-in-one unit with a type color CCD camera, 22x zoom lens, and rotating chassis packed inside a 110mm (4.3-inch) diameter housing that fits discretely in almost any location. With its full 360 horizontal rotation, 180 vertical pan, and 300/second speed, surveillance of any subject is constant and continuous. Some of the key features of the Panasonic CS WV854A are [Panasonic854]- 11 -

Compact 110mm (4.3") diameter dome-sized all-in-one color unit. Built-in Super Dynamic II function has 64 times wider dynamic range when compared to a conventional camera. 570-line at B/W and 480-line at color imaging horizontal resolution. Built-in Digital-FLIP by memory read out allows automatic 180-degree turn. Can automatically or manually select color or B/W capturing by removal of the IR Filter; 0.06 lx (0.006 fc) at B/W or 1 lx (0.1 fc) at color imaging (sensitivity 2x up). Privacy zone masking function. Linear 32x electronic sensitivity enhancement function. Built-in motion detector. Patrol Learn function. Auto panning function with 64 preset positions. Panning speed of max. 300 degree/s at preset mode. Image hold function during panning (NEW). 22x optical zoom lens (3.79 - 83.4mm at F1.6) plus 10x electronic digital zoom for total zoom capacity of 220 x. Auto focus lens. Multiplex-coaxial or RS485 data communication. Four (4) alarm inputs and two (2) outputs terminals. 24V AC, 60Hz power source.

2.1.2 Wireless hardware In this section, we discuss the various wireless components acquired for the wireless LAN set up. The selection of these components was based on the range of the wireless network we intend to set up. We have acquired two wireless components for the system an Axis wireless device and access point for low range applications and a Tsunami Quick Bridge 11 wireless kit for long range applications [Tsunami11]. Initially, we have obtained the wireless access and device points to develop various network designs and implemented them in the IRIS Lab. Later, we obtained the Tsunami Quick Bridge 11 for outdoor wireless LAN set up. The hardware specifications for each of these wireless components are given in the next subsection.

- 12 -

Axis Wireless Device and Access Points The 802.11b wireless access point is an easy to use access point that complements the Axis network video server in wireless applications. A complete wireless networking solution consists of any Axis network video product, an 802.11b wireless device point from Axis and the 802.11b wireless access point. The 802.11b wireless device point from Axis is used for bridging Ethernet devices to the wireless LAN or to an access point. The key features of the wireless access point are listed below [AxisAP] Offers wireless communication over distances of up to 300ft/100m Supports 128-bit Wireless Encryption Protocol (WEP) encryption for reduced risk of unauthorized data access Compatible with all 802.11b compliant devices Dual antenna design ensures optimal RF performance Offers web based management TCP/IP protocol RJ 45 Ethernet port Power in take 5V/2A

Axis Wireless Device Point The 802.11b wireless device point from Axis is a bridge between cabled Ethernet and wireless Ethernet (IEEE 802.11b standard). Connected to the Ethernet port of the Axis video server, it provides a transparent connection to an existing WLAN infrastructure. The key features of the wireless device point are listed below [AxisDP] Plugs directly in to Axis network cameras Supports 128-bit WEP encryption for reduced risk of unauthorized video access Offers wireless communication over distances of up to 300ft/100m Compatible with all 802.11b compliant devices Platform independent with no special driver software required Dual antenna design ensures optimal RF performance Web based management

The wireless access and device points that we have acquired for the project are shown in Fig 2.5.

- 13 -

(a) Fig 2.5 Axis wireless components; (a) Access point, (b) device point Tsunami Quick Bridge 11

(b)

The Tsunami Quick Bridge 11 kit is a highly reliable wireless networking solution for setting up point-to-point networks. Two pre-configured bridges enable users to easily, quickly, and economically install a wireless LAN extension between two locations eliminating the need for a costly leased line or cable alternatives. It utilizes new Wireless Outdoor Router Protocol (WORP) technology that not only offers all the benefits of an 802.11b 11 Mbps solution but also delivers unbeatable performance, range and throughput in an outdoor environment. Created specifically for outdoor applications, WORP ensures Quick Bridge 11 provides a more reliable building-to-building wireless LAN extension than standard Wi-Fi based 802.11b point-to-point solutions. The Quick Bridge 11 kit also offers a growth path to a multipoint network. Expanding connectivity to multiple buildings is easy by simply adding additional Tsunami MP.11 subscriber units. The kit contains everything we need to establish a point-to-point connection, including one Tsunami MP.11 Base Station Unit, one Tsunami MP.11 Subscriber Unit, surge arrestors, antennas and Ethernet cables. The Tsunami Quick bridge 11 kit is shown in Fig 2.6. The key features of the Tsunami Quick Bridge 11 unit are listed below [Tsunami11]: New Wireless Outdoor Router Protocol (WORP) enables superior performance and scalability Pre-configured solution for easy and quick deployment Migration path to high-speed multipoint solution secures investment

- 14 -

Fig 2.6 Tsunami Quick bridge 11 wireless kit. 2.4GHz unlicensed frequency band, FCC, ETSI & MKK Compliant 128 bit WEP plus encryption with Weak Key Avoidance 802.3af Power over Ethernet remote management and configuration via SNMP, Telnet CLI and HTTP reduce maintenance costs. offers line of sight range of 3-4 miles. maximum data rate of 11 Mbps web based management LED indications for power, Ethernet and wireless connectivity

2.2 Data acquisition techniques

The most important part of this project is the real time data acquisition of the under bridge traffic. The main focus of discussion in this section is in obtaining real time range and video data of water traffic such as container ships, barges etc. For this purpose, a location having some environmental resemblance with Port Arthur, must be located. One such location is the Henley St. Bridge on the Tennessee River. Scanning of Water Traffic on the Tennessee River Once the test area was located, the next step was to collect the required data of the under bridge traffic. The location shown here is different in comparison with the Port of Port Arthur, TX. Unlike Port Arthur, this part of the Tennessee River is suitable only for barge traffic, ferrys etc, but cannot handle huge cargo ships. There is no regular schedule for the

- 15 -

barge traffic or the ferrys and hence we approached the Tennessee Valley Authority (TVA).The TVA monitors the traffic on the river and they helped us by informing beforehand whenever there would be barge traffic on the river. Consequently, we were able to set up the range scanner on the bridge beforehand. The tugboat and barge that we had scanned is shown in the Fig 2.7.

Fig 2.7 (a) Barge traffic on the Tennessee River (b) Tugboat driving the barge

Henley St. Bridge Scanning

The next step in the project was to obtain range and video data of the Henley St. Bridge located on the Tennessee River, Knoxville, TN. Again, the RIEGL 3D-LMS scanner was used for this purpose. The set up required for scanning the bridge is as follows. The RIEGL 3D-LMS Range scanner was used to obtain the range scans of the bridge. Suitable locations were used to place the scanner and take the scans. The scanner is controlled by software called 3D RiSCAN. A picture of Henley St Bridge is shown in Fig 2.8

- 16 -

Fig 2.8 Henley St Bridge on the Tennessee River, Knoxville, TN

The first step to acquire range scans using the scanner is to configure the scanner port. The scanner has to be configured to send/receive data via the RiPort. Selecting the Data Source option in the Configure menu does this. This window is shown in Fig 2.9.

Fig. 2.9 Data source menu

- 17 -

The next step is to set the scanner parameters. This is done by clicking the define/scanner parameters. The window shown in Fig 2.10 then opens-

Fig. 2.10 Set scanner parameters window Then we click the start data acquisition button. When the scanner acquires data, it is displayed as shown in Fig 2.11. The display window consists of the following sub windows A window showing the acquired range image in color coded format. There is a color bar to show the color variation in the image. A window displaying the range data in gray scale. There is also a gray bar to show the intensity variation in the image. A window showing the true color image. A data readout panel depicting various parameter values such as range, azimuth angle, polar angle etc. The obtained range image can be saved as a .3DD file or exported as a VRML file or an inventory file depending on the application. A Mocha PC was used to control and communicate with the RIEGL scanner. The control software 3D RiSCAN was installed in this device beforehand for this purpose. A laser rangefinder binocular was used to find the range of the target to be scanned. It is very

- 18 -

important to always know the range of the target because the RIEGL scanner doesnt capture any reliable data that is at a distance greater than 350 meters.

Fig 2.11 3D RiSCAN control interface window

- 19 -

A stand-alone battery pack was used to supply power to the RIEGL scanner and the Mocha PC. The RIEGL scanner requires 12V DC to operate. In order to generate a textured 3D model of the bridge, scans of the bridge from different perspectives should be taken and integrated. For this purpose, we planned to take eight scans from different locations as shown in Fig 2.12.
Opposite Bank Left Top & Left Bottom Opposite Bank Right Top & Right bottom

Henley St. Bridge top view Left Top & Left Bottom

Right Top-No scan & Right Bottom

Fig 2.12 Scan plan for scanning Henley St Bridge There were, however, some place constraints in scanning the bridge. There was no sufficient room to set up the scanner in some places especially on the left top & bottom sides, right top side of the bridge and right top and bottom sides on the opposite bank of the bridge. Hence, out of the total eight scans planned, only four scans were taken. These results are presented in the results section.

2.3 Wireless LAN set up

In this section, we discuss the various designs and layouts that we can frame with the available wireless hardware. We also discuss about the implementation techniques that we have adopted for deploying the various system designs. 2.3.1 System design and Implementation Initially, we developed different wireless designs and layouts with the available system components in hand. The various system components include a PTZ camera, a video server, wireless components such as access & device points, wireless routers, wireless bridges and high gain directional antennas. Based on the components used in the network layout, we have developed the following designs: - 20 -

W-A-D

Design

W-A-R-D Design W-A-N-D Design L-A-D W-R-A-B Design Design

The terminologies used for naming these designs are not standard notations and are abbreviated for convenience. It should be noted that the Panasonic PTZ camera and the Axis video server are a part of all the network designs and the wireless components vary depending upon the range and bandwidth requirements. For example, if we want to cover a range of 300 ft or so, we use the wireless access and device points and if the range is around 3 to 4 miles, we opt for the high gain directional antennas. A description of the above terminologies follows. W-A-D Design

The first and foremost design developed and implemented is the W-A-D design. The system components for this design consist of a Panasonic PTZ camera [Panasonic854], Axis video server [AxisVS], a Dell Laptop and Wireless Access & Device points [AxisP]. The basic network layout of the W-A-D design is shown in Fig 2.13.

PTZ Cam
Video Server
Wireless transmitter
Wireless

receiver

Laptop

Fig 2.13 The general network layout for the W-A-D design. As can be seen from Fig 2.13, W-A-D is a very simple, easy to set up, low range, point-topoint wireless network. The range of this design is around 300 ft line of sight (LOS). The real time network layout of the W-A-D design is shown in Fig 2.14. The analog video from the PTZ camera is fed as input to the video server. The video server digitizes and compresses the video data so it will fit in the available bandwidth. The

- 21 -

wireless device point uploads the digitized video stream on to the wireless network. The wireless access point on the client side downloads this video stream and makes it available for the client to view on a laptop. The network is very simple and takes about 10 minutes to set up. We have implemented the W-A-D design both in the IRIS Lab and outside. The system set up inside the IRIS Lab is simple and easy. The outdoor set up is a bit tedious and requires locations having a clear LOS for the wireless device and access points to see each other.

Fig 2.14 The network layout for the W-A-D design in real time We surveyed various locations and fixed the roof top of Ferris Hall as the location for placing the PTZ cam, video server and wireless device point (transmission equipment). We placed the wireless access point and the client near the Tennessee Grill (opposite Ferris Hall) which has a clear LOS with the roof top of Ferris hall. The W-A-D design was implemented as mentioned above and the implementation results are discussed in the results section. [Section 3] W-A-R-D Design

The W-A-D design allows us to implement a physical and separate wireless network but it does not overcome the hurdles of range and bandwidth. However, the W-A-R-D design overcomes the constraints of range and bandwidth but not satisfactorily. The system components for the W-A-R-D design consist of the Panasonic PTZ camera, Axis video server and Wireless Access, Repeater & Device points. The generalized network layout for the W-A-R-D design is shown in Fig 2.15.

- 22 -

~300 ft IP 10.0. 0.4

~300 ft

~300 ft

10.0.0.5

10.0.0.6

10.0.0.7

10.0.0.N

10.0.0.N+1

PTZ camera + Video Server IP 10.0.0.n+2

PC

AP+Repeater AP+Repeater Channel x Channel x

AP+Repeater Channel x

AP+Repeater Device Point Channel x

Fig 2.15 The general network layout for the W-A-R-D design The W-A-R-D design can be considered as an extension of the W-A-D design. It has all the components of the W-A-D design and some additional devices called repeaters. The repeater is a device that receives a particular signal from a transmitter and then re-transmits the same signal to a remote receiver. The repeater thus increases the range of a particular design. The network layout for the W-A-R-D design is shown in Fig 2.16.

~ 300 ft ~ 300 ft

10.0.0.1 10.0.0.N+1 10.0.0.2 10.0.0.3 . 10.0.0.N-1 10.0.0.N

Fig 2.16 The network layout for the W-A-R-D design in real time All the components in this design are set up in a local network. This is done by manually assigning the IP addresses to the individual components of the system in such a way that they

- 23 -

form a part of the network. For example, if we assign an arbitrary IP address, say 10.0.0.1 to the first component in the design and the subnet mask as 255.0.0.0, the network can have 255 sub components in the entire network. Hence, we assign the IP addresses for the remaining components of the system in the order of their placement in the network. In this manner, we set up a separate and physical local network. The use of repeaters every 300 ft. definitely increases the radio coverage of the design but at the same time it increases the network complexity. As the range increases, more and more repeaters have to be included in the network. Such an arrangement of repeaters also increases network instability. If there is a failure of wireless link between any two repeaters in the network, the entire system connection will be lost. Each repeater requires an independent power supply and thus the use of more and more repeaters places a constraint on the available power supply. The W-A-R-D design was also implemented in the IRIS Lab and outdoors too. The implementation results and comparisons with other network designs are documented in the results section. [Section 3] W-A-N-D Design

The W-A-R-D design addresses the problem of range and bandwidth to a certain extent but not completely. Increase in network complexity, and the limited and independent power supply still pose problems for the network implementation. In addition, only the clients that form a part of the network layout can access the data sent over the wireless network. This design does not address one of the main objectives of the project i.e. worldwide accessibility. In order to resolve this issue, we need to upload the wireless data to an existing server that has a connection to the World Wide Web. Once the data is uploaded to a server, this data can be accessed over the web from anywhere in the world. To serve this purpose, we propose the W-A-N-D design. The generalized network layout of W-A-N-D design is shown in Fig 2.17. The system components for the W-A-N-D design consist of the Panasonic PTZ camera, Axis video server, the University of Tennessee (UT) wireless network NOMAD and the wireless access and device points. The network layout for the W-A-N-D design in real time is shown in Fig 2.18.

- 24 -

UT Wireless Network NOMAD

PTZ Cam

Ethernet LAN

Video Server

Wireless Transmitter

Wireless Receiver

Laptop

Fig 2.17 Generalized network layout for the W-A-N-D design

UT Ethernet LAN

UT Wireless Gateway

Fig 2.18 Network layout for the W-A-N-D design in real time In the W-A-N-D design, the network flow up to the wireless device point node is similar to the W-A-R-D design. After this node, the wireless data is uploaded to an existing server instead of transmitting to a remote receiver. We have uploaded the wireless data on to the existing server at our university called NOMAD. The wireless device and access points are configured so that they can talk to NOMAD. The client can access the wireless data from the NOMAD in two ways. The client either connects to the NOMAD using a Wi-Fi connection or by connecting to the UT gateway via the Ethernet LAN connection. The W-A-N-D design was implemented at the IRIS Lab and the implementation results are documented in the results section. [Section 3]

- 25 -

L-A-D Design

The network layout for the L-A-D design is very much similar to the W-A-N-D design except for the placement of the wireless components in the network. The generalized network layout for the L-A-D design is shown in Fig 2.19.

PTZ Cam

Wireless Access point

Video Server

Wireless Device point

UT Ethernet LAN

Laptop

Fig 2.19 The general network layout for the L-A-D design In this design, a local network is set up between the wireless access and device points. The local network is then bridged on to an existing LAN thereby uploading the data on the local network to a server having connectivity to World Wide Web. This serves the purpose of world wide accessibility.

UT Ethernet LAN Fig 2.20 Network lay out for the L-A-D layout in real time. Here, the access and device points should be properly configured so as to see the existing UT LAN. Once again, we tried to bridge the local network on to the UT LAN gateway. The - 26 -

L-A-D design was implemented at the IRIS Lab (Fig 2.20) and the implementation results are documented in the results section. W-R-A-B Design

The final design in this section is the W-R-A-B design. All the designs discussed so far have constraints such as range, network complexity or dependency on power outlet. To overcome some of these constraints, we propose a robust, reliable and portable design namely the W-R-A-B design. For the generalized layout of the W-R-A-B design, see Fig 2.21. Unlike the other designs, the W-R-A-B design deploys different wireless components to overcome the range problems. This design makes use of the Tsunami Quick Bridge 11 instead of the wireless access and device points.

TX Antenna

PTZ Cam Independ ent Power supply

Rx Antenna

Video Server

Wireless

Client

Bridge

Wireless router Weather Hardening System

Lightning arrestor

Fig 2.21 Generalized network layout for the W-R-A-B design

- 27 -

To mount the system components during experimentation, we have acquired a heavy duty tripod stand and stacked the components on to the tripod. The tripod stand consisting of the transmitting equipment is shown in the Fig 2.22.

Fig 2.22 Heavy duty tripod with the system components mounted. The transmitting equipment consists of a Panasonic PTZ camera, Axis video server, Tsunami wireless base station and a 14 dB high gain, directional transmitting antenna. The system components have been enclosed in proper weather proofing enclosures to protect the components from hostile weather conditions during experimentation. Also, we have acquired a similar tripod stand to mount the receiving equipment. The receiving equipment consists of a 14 dB high gain, directional receiving antenna, wireless satellite station and a client PC (HP Laptop). The tripod stand with the receiving antenna mounted on it is shown in Fig 2.23. The network layout after mounting the system components on both tripods is as shown in Fig 2.24. - 28 -

Fig 2.23 Tripod stand with the receiving equipment mounted on it.

Client Independent power supply TX equipment (PTZ, server, router) Rx equipment

Weather hardening system

Lightning arrestor

Fig 2.24 Network layout for the real time W-R-A-B design

- 29 -

Before we tested the equipments performance outdoors, we completed the Confidence test indoors. The confidence test verifies the equipments operation and configuration, as well as the network connection with the main LAN prior to the actual installation. The network layout for the confidence test is shown in Fig.2.25.
Ethernet Port Up to 3 ft

Laptop

BSU

SU

Power

Power

BSU Base station unit SU Subscriber unit

Fig 2.25 Network layout for the confidence test To perform the confidence test, we connect the laptop directly to the network Ethernet port (with the straight-through cable) and ensure that the connection is working. Then, we disconnect the laptop and set up the equipment as shown in Fig 2.25. A straight-through cable must be used to connect to the Ethernet port and a cross-over cable must be used to connect the satellite unit to the laptop. When we power up both the MP.11 units, after no more than 90 seconds, the wireless LED should briefly flicker green and turn solid green when the wireless link is established between the two units. A successful confidence test verifies the proper functionality of the equipment. The next step is to test the equipment outdoors, preferably on the Henley St. Bridge. But before we proceed at this location, we tested the performance for a range of 10 ft initially and then for a range of 100 ft LOS to ensure proper functionality of the equipment outdoors. The equipment is very reliable and the wireless link was set up for these ranges. The implementation of the final design on the Henley St. Bridge is discussed in the next section. 2.3.2 Wireless LAN set up on the Henley St. Bridge We have almost reached the final milestone on the road we had planned to go at the beginning of this project. The Henley St. Bridge on the Tennessee River in Knoxville, TN forms a perfect platform for the outdoor implementation of the wireless network. This is - 30 -

because implementing a wireless LAN on the Henley St Bridge is similar, at least in geographical terms, to implementing it on the MLK Bridge in the port of Port Arthur, TX. Also, the Henley St. Bridge is the nearest location that has clear line of sight with the roof top of Ferris Hall. Both these reasons motivated us to select the Henley St. Bridge as the test site for the wireless LAN set up. We have developed an initial prototype design with heavy duty tripod stands for the wireless LAN set up. The initial prototype consists of two heavy duty tripods to mount the transmission and receiving equipment. The Tsunami high gain transmitting antenna, Panasonic PTZ camera, Axis video server and the Tsunami MP 11 base station are mounted on the transmitting end tripod stand as shown in Fig 2.26 (a).The Tsunami high gain receiving antenna, Tsunami MP 11 satellite unit are mounted on the receiving end tripod stand as shown in Fig 2.26 (b).

(a)

(b)

Fig 2.26 Initial prototype designs; (a) Transmitting end tripod stand, (b) receiving end tripod stand The components that are used in different sub systems are placed in proper weather hardened enclosures to protect them from rain and heavy winds. We have developed a weather hardening sub system [see Fig 2.27] and power sub system [see Fig 2.28] in parallel

- 31 -

with the wireless system under the supervision of Hari Kishan Iddamsetty (graduate student, IRIS Lab) and Tak Motoyama (hardware specialist, IRISLAB).

Fig 2.27 Weather hardened enclosures; (a) Video server enclosure, (b) base station enclosure The power sub system that we have developed has the following features Completely independent system that delivers both AC and DC power supply to the components of the different sub systems. Delivers a continuous power supply to the system components. Weather hardened against hostile weather conditions.

(a)

(b)

Fig 2.28 Initial prototype of the power sub system; (a) Weather hardened power sub system, (b) inner view of the power sub system

- 32 -

We have thus incorporated the features of ruggedness and an independent power supply for all the components in the system. The next step is to set up the wireless LAN on the Henley St. Bridge. We have selected the rooftop of Ferris Hall to set up the receiving equipment as shown in Fig 2.29. The wireless LAN set up and the corresponding transmitting equipment for the set up is shown in Fig 2.30.

Fig 2.29 Receiving equipment placed on the roof top of Ferris hall

Fig 2.30 Wireless LAN set up on the Henley St. Bridge, Knoxville, TN - 33 -

To set up the wireless network, we have to take several steps. First , we have to set up the receiving antenna on the roof of Ferris Hall. The installation procedures for setting up the receiving antenna are stated below Open the tripod stand and spread the three legs of the tripod stand so that it is balanced and stable. Connect the antenna cable coming from the receiving antenna to the pigtail connector coming from the satellite station unit. Connect the power supply cord to the wireless router and the client. Connect a cross-over cable from the client to the satellite station Ethernet port. Switch on the power supply and wait for the power LED on the satellite station to turn solid green. The next step is to set up the equipment on the transmitting end. The installation procedures for setting up the transmitting equipment are stated below Open the tripod stand and spread the three legs of the tripod stand so that it is balanced and stable. Connect the antenna cable coming from the receiving antenna to the pigtail connector coming from the base station unit. Connect a cross over cable from the video server to the base station unit. Connect a video cable and data cable from the PTZ camera to the video server. Connect the power supplies to the PTZ camera, video server and base station. Switch on the power supply and wait for 90 seconds. The power and wireless LEDs should blink solid green. If the wireless LED is not blinking solid green, then take a pointed object and press the reload button on the base station for 10 seconds. After 90 seconds, the wireless LED will blink green indicating that a wireless link has been established. For testing purposes, we transmitted images of resolution 704x576 with a compression ratio of 50 over the wireless network and measured the corresponding range, bandwidth and frame rates. The implementation results are discussed in detail in the results section. After these steps, the power and wireless LEDs on both the stations should blink green. The Ethernet LEDs on both stations should blink amber. If not, disconnect the cross over cable and connect them once again. To test the wireless network, we have to open the command prompt and ping the IP address of the video server. If we get a reply from the - 34 -

video server, then a wireless link has been established between the server and the client successfully. Now we can view the video from the PTZ camera on a web interface and also be able to control the PTZ camera with a GUI application that comes with the video server.

3. Experimental Results
In this section, we discuss the various results that we have obtained from data acquisition and wireless transmission. The data acquisition results consist of range scans of the traffic under Henley St. Bridge and also range scans of the Henley St. Bridge. The wireless transmission results consist of the bit rates, available bandwidth obtained and other parameters of the transmission channel.

3.1 Data acquisition results

The results in this section consist of range scans of the barge traffic on the Tennessee River and the range images of the Henley St Bridge. The range scans of the barge traffic are obtained using the RIEGL scanner. These scans are later processed using the software RAPIDFORM 2002 to remove noise and integrate the different scans obtained. These scans are shown in Fig. 3.1 & 3.2. The range images of the Henley St Bridge are shown in the Figs 3.3-3.6.

Fig 3.1 Range image of a tug boat as viewed in RAPIDFORM

- 35 -

Fig 3.2 Range images of tug boat and barge as viewed in RAPIDFORM

Fig 3.3 Range image of Henley St. Bridge from left bottom

- 36 -

Fig 3.4 Range image of Henley St. Bridge from left up

Fig 3.5 Range image of Henley St. Bridge from left up (opposite bank)

- 37 -

Fig. 3.6 Range image of Henley St. Bridge from right bottom One item of note here is that all targets and objects that are out of the range of the scanner will appear in blue color. This can be clearly seen from the range images as the sky appears blue in color. All these images are grayscale.

3.2 Wireless design results

As part of the wireless testing, we have implemented all the five designs discussed in section 2.2. Some of the designs were implemented at IRIS Lab and also outdoors. However, we have implemented the W-R-A-B design on the Henley St. Bridge. The implementation results of the designs are presented in this section. W-A-D design Implementation results

The wireless components for the W-A-D design mainly consist of wireless access and device points. The W-A-D design is a simple, low range, point-to-point wireless network. We transferred images of resolution 704x576 with compression ratio of 50 over the wireless network and measured the corresponding range, bandwidth and frame rates. The results for the W-A-D design are listed below1. Range approximately 300 ft.(LOS) 2. Bandwidth 2 Mbps (average) 3. Frame rate 5.12 fps (average) [BandCalc]

- 38 -

W-A-R-D design Implementation results

The system components for this design include Wireless Access point, Repeaters and Device point. The W-A-R-D design is an extension of the W-A-D design. The low range of the W-A-D design is extended by the use of repeaters at ideal locations (every 200-300 ft. LOS) in between the access & device points. We transferred images of resolution 704x576 with a compression ratio of 50 over the wireless network and measured the corresponding range, bandwidth and frame rates. We have used only one repeater for this implementation. The results for the W-A-D design are listed below1. Range approximately 500 ft [ideal- 300 ft + (number of repeaters) x 300 ft.] 2. Bandwidth 1.0 Mbps (average) 3. Frame rate 2.56 fps (average) W-A-N-D design Implementation results

The system components for the WAND design consists of a Wireless Access point, the UT wireless network Nomad and the Device point. In this design, the video stream is uploaded to the UT wireless network using the device point. This data is then accessed using the access point or the Ethernet LAN. Both the access & device points should be configured to see the Nomad. The W-A-N-D design is a failure. The wireless access & device points we acquired for the project are not designed to see or communicate with existing outside wireless networks. As a result, we were not able to set up the wireless network. L-A-D design Implementation results

The system components for the LAD design consists of an Ethernet LAN, wireless Access and Device points. In this design, the video stream is transmitted wirelessly to a remote access point which then uploads this data on to the UT Ethernet network. The L-AD also failed because the wireless device and access points cannot see outside wireless networks.

3.3 Testing on Henley St. Bridge-results

For testing on the Henley St. Bridge, we have deployed the W-R-A-B design. The system components for the WRAB design consist of a Wireless Router, Antennas and wireless Bridge. In this design, the video stream is transmitted wirelessly to a remote location using high gain antennas and is then accessed by the client via a wireless bridge. In order to measure the throughput of the wireless network, we have used two laptops. One laptop on the - 39 -

transmitting end plays the role of a server that uploads applications and data on to the wireless network. The other laptop on the receiving end plays the role of a client requesting the data from the server over the wireless network. We have installed server-client software on both the laptops. The software is called Net IQ Check. With this software, we can transmit TCP/IP files of sizes varying from 1-1000 Kbytes and measure the throughput of the transmission medium. The GUI application for the Net IQ Check software is shown in Fig 3.7.

Fig 3.7 GUI provided by the Net IQ Check software to measure throughput. Using this software, we have transmitted files of data size varying from 10 to 1000 KB over a range of 0.3 miles (LOS distance between Ferris Hall roof top and Henley St. Bridge) and measured the throughput variation of the wireless network. The different throughput values we have obtained for different file sizes with a constant range of 520 meters are shown in Table 1.

- 40 -

Table 1 Throughput values for different file sizes

File Size (KB) Throughput (Mbps)

10

50

100 4.73

200 5.18

300 5.30

400 5.36

500 5.44

600 5.44

700 5.47

800 5.48

900 5.48

1000 5.49

2.38 4.12

Based on the values from Table 1, we plotted a graph showing the variation of throughput of the wireless network with the file size transmitted over the network. (Fig 3.8)

Throughput variation (Range-520 mts L-O-S)

6
Range 0.3 miles (from roof top of Ferris hall to Henley St Bridge-LOS)

4. Bandwidth 5 Mbps (average) 5. Frame rate 12.8 fps (average)

T h ro u g h p u t (M b p s )

0 10 50 100 200 300 400 500 600 700 800 900 File size (KB)
Fig 3.8 Throughput variation with file size

- 41 -

At the same time, we have tabulated the different throughput values obtained by varying the angle of deviation of the transmitting antenna from the LOS with the receiving antenna. See Table 2. Note that the range is again fixed at 520 meters and the file size is fixed at a value of 1 MB. Table 2 Throughput values for different angles of deviation from LOS
Angle of deviation from LOS (deg) Throughput (Mbps)

0 5.6

10 5.54

17 5.5

22 5.18

28 4.7

34 3.7

40 2.6

45 1.9

48 1.1

62 0.28

68 0.11

70 0.0

Based on the values from Table 1, we have plotted a graph showing the variation of throughput of the wireless network with the file size transmitted over the network. (Fig 3.9)

Throughput variation (File size of 1 MB and range=520 ms)

6 5 4 3 2 1 0 0 10 17 22 28 34 40 45 48 62 68

Fig 3.9 Throughput variation with the angle of deviation from the LOS

- 42 -

3.4 Wireless system comparisons

The different wireless system designs that we have developed, the various system components, the implementation methods and results are compared in Table 3. Table 3 Wireless system comparisons

Design

Wireless Components

Experimented Frame Recommendation in IRISLAB rate fps (avg)

Reasons

WAD Wireless Access & Device points Yes 5.12 Not Satisfactory Low range, bandwidth and not and security recommended

WARD Wireless Access, Repeater & Device points Yes 2.56 Satisfactory but Network complexity, not recommended power supply, low bandwidth

WAND Wireless Access point, UT wireless Nomad & Device Point Yes 0 Failed The access & device points cannot see outside networks.

LAD UT Ethernet LAN, wireless Access & Device point Yes 0 Failed The access & device points cannot see outside networks

WRAB (peer- Wireless Router, to-peer Antennas & Bridge and bridged)

Yes

13.8

Very good and recommended

Increase in range (up to 4 miles L-O-S), simple network configuration

- 43 -

4. Conclusions
As stated previously, the main objective of this project is under bridge inspection. To meet this objective, we collected real time range & video data of water traffic such as barges and also the Henley St Bridge. But the road to project completion doesnt end here. The real usefulness of the collected data lies in the fact that one should be able to generate a reliable textured 3D model of the original scanned model. There is some extensive research being done in this area under the title 3D Scene Reconstruction from Multiple Range Maps and Color Images by the PhD student Mr. Faysal Boughorbel, under the supervision of Dr. Mongi A. Abidi, IRIS Lab, The University of Tennessee. The collected data serves as the input for this algorithm whose output would be textured 3D models of the original models. At this juncture, I would like to convey my regards to Mr. Faysal Boughorbel for his help rendered in analyzing & post processing of acquired data to generate textures 3D models of the scanned data. A brief overview and flow diagram of this algorithm is presented Fig 4.1 to have a better understanding of the entire procedure.

Fig 4.1 Flow diagram of the 3D Scene reconstruction algorithm

- 44 -

Image reconstruction consists of the following steps generic model of the object is first reconstructed from the range image. landmark points are selected in this model and tracked in the input image sequence. shape from motion algorithms are used to reconstruct the image matches. final model is obtained by 3D warping of the generic model using the set of stereorange correspondences. Since the discussion of this algorithm is out of the scope of the project, it is not discussed in detail. If the communication system is considered, it is clearly evident that the system we have built now is fully functional and is able to transmit live video to a remote location over the wireless network that we set up. We are also able to control the camera from a remote location over the wireless network and these issues really open doors for innovative applications ranging from home security to coastline video surveillance. Do these accomplishments mean that we have reached the end of the road in our pursuit for a robust, rugged, portable and wireless video surveillance system? No, these accomplishments mean that we have just started and there are many more milestones to attain before we reach the end of the road. Now is the time to take a step back, look at the system and question its robustness . The system that we have developed definitely gives functionality but not the required robustness and portability. Hence, we have to propose new designs that will address the problems of bandwidth, range, security and mobility. Also, we have to freeze the current working system design and duplicate it. The working system will now be used for testing and experimentation and the duplicated version will be used to incorporate more robust features in to the system. The video surveillance system we have built so far is just another milestone that we achieved in our path and there are still many more to achieve down the road. Future work for this project is to attribute the R-R-R concept to the system design. The R-R-R concept stands for Relay networking The motivation behind the project, as stated earlier, is to set up a remote video surveillance system on the MLK Bridge in the Port Arthur, TX. But there are similar bridges in the Port Arthur, TX on the same channel. The future work is to set up similar systems on all the bridges and then have a relay network between

- 45 -

all the systems on the bridges so that they can provide superior tracking. The relay networking allows a PTZ camera to handover the tracking to the next camera once the object of interest is out of its field of view. The concept of relay networking is totally another research field which mainly involves the issues of sensor placement optimization and camera handover. Redundant router back up Since we are implementing a wireless network, there is a possibility of a wireless link failure. We are dealing with security issues in this project and hence we should not have even the slightest margin of error that in turn gives a huge opportunity for terrorists to conduct the attack. So, in the event of a wireless link failure, there should be back up wireless service available. The back up service need not be as robust as the main system and could be a low bandwidth wireless network like satellite DSL service. Future work consists of setting up a back up wireless network so as to provide a continuous wireless link even if the main system breaks down. Remote operability- We have already seen that each of these systems set up on the bridges is powered by individual power systems. Hence, minimizing power consumption plays a vital role in the system design for continuous operation of the system components. Suggested future work consists of incorporating remote operability to the system. This feature allows the client to switch on/off the surveillance system from a remote location over the wireless network which in turn ensures the intelligent use of the available power supply.

References
[Map] http://www.businessintexas.com/sabinepilots/maps.htm

[Panasonic854] - http://imaging.utk.edu/~sarath/ece599/web_links.htm/WV-CS854A.pdf [Axis VS] [BandCalc][AxisAP] [AxisDP] [Tsunami11]http://imaging.utk.edu/~sarath/ece599/web_links.htm/axis2401p.pdf http://axis.com/techsup/cam_servers/cam_2400plus/calculator.htm http://imaging.utk.edu/~sarath/ece599/web_links.htm/axisap.pdf http://imaging.utk.edu/~sarath/ece599/web_links.htm/axisdp.pdf http://imaging.utk.edu/~sarath/ece599/web_links.htm/tsunami11.pdf

- 46 -