TRACKING CONTROL OF A SPOT BEAD WELDING MOBILE ROBOT USING CAMERA SENSOR

Tan Tien Nguyen, Trong Hieu Bui*, Tan Lam Chung* and Sang Bong Kim* Dept. of Mechanical Eng., Hochiminh City University of Technology 268 Ly Thuong Kiet Str., Dist.10, Hochiminh City, Vietnam * Dept. of Mechanical Eng., Pukyong National University, Pusan 608-739, Korea

ABSTRACT In shipbuilding industry, it is needed to make uniform welding beads on the floor of ship. In this paper, a line/curve tracking controller based on Lyapunov function is designed and applied to a twowheeled spot bead welding mobile robot for making it possible to track a white straight line on the ship floor. The mobile robot moves a long a reference straight line with a constant speed while performing welding process with desired setting parameters. The control system uses a camera as sensor to detect the deviation between the line and the center of the mobile robot which is used to drive the mobile robot to go straight and the spot welding process is performed as desired. The welding experiments are carried out to verify the designed controller.

1. INTRODUCTION Welding automation has been increasingly used since they are of much benefit to welding applications in terms of weld quality, increased productivity, reduced welding costs, and increased weld consistency. Specifically, in shipbuilding industry, it is needed to make uniform welding beads on the floor of ship such as engine room, the platform on board, and so on. This work used to be done by hand with an array caliber in crater mode of welding for the spot bead quality. It takes time and effort for the worker, so this is the case of applying mobile robot. Tracking is a popular and useful technique for a mobile robot navigation in both known and unknown environment. Several control strategies have been proposed in the literature for the control of nonholonomic systems; also, there are several ways for a mobile robot to sense its surrounding. J. M. Yang and J. H. Kim [5] proposed a robust control law for trajectory tracking of nonholonomic wheeled mobile robots using sliding mode control; T. L. Chung et al. [7] proposed a control law for wall-following problem of a two-wheeled mobile robot of which modeling, velocity and steering control method have some similarities to those proposed

in this paper. Y. Kanayama et al. [4] proposed a stable control rule using Lyapunov function and the experiment was done on the autonomous mobile robot Yamabico-11. The objective of this paper is to design a mobile robot with a simple controller and low cost hardware. This paper introduces a simple tracking controller for a two-wheeled welding mobile robot. To design such the tracking controller, a configuration of errors is defined, and the controller is designed to drive the error to zero as fast as desired by adopting Lyapunov stability. A camera sensor on the mobile robot is used to detect the deviation between the line and the center of the mobile robot to make it possible tracking a white line in chalk made by a simple tool on platform of ship. To generate the controller, the distance and the orientation of the mobile robot need to be measured using the camera sensor. The sensor utilizes a low cost CMOS color camera module, and all image data is processed by a high speed low cost microcontroller (Anthony Rowe et al.). The mobile robot has four torch holders in the rear for making four spot beads simultaneously. For welding process, there are five adjustable parameters: spot distance, mobile robot velocity, welding time, crater time, and spot bead number. The effectiveness of the designed controller has

- 408 -

been verified by means of simulation and experiments. This welding mobile robot has been successfully applied to practical field in shipyard company in Korea.

Here,

2. SYSTEM MODELING In this section, a dynamic configuration of the mobile robot is derived with a geometrical motion as shown in Fig.1, and the problem is stated under the following assumptions: (i) The radius of curve is sufficiently larger than turning radius of robot. (ii) Robot has two rotating wheels for body motion control and those are positioned on an axis passed through the robot geometric center. (iii)Two passive wheels are installed at the bottom of front and rear for balance of mobile platform (iv) A seam tracking sensor is attached at the robot geometric center. (v)The velocity component at the point contacted with the ground in the plane of the wheel is zero and the robot turns around one point. The reference line is straight line for this application, and also smooth curve can be tracked in accordance with application request.
y

( x, y ) : fixed coordinates of mobile robot, ( X , Y ) : moving coordinates of mobile robot, r : wheels diameter, b : distance from mobile robots center to driving wheel, R : a reference point on the line, : curved radius at point R , t t : reference line, R : mobile robots orientation at point R , : head angle of mobile robot, d : distance from mobile robots center to tt . The kinematic equation under the nonholonomic constraints of pure rolling and non-slipping is given by [5] & q = S ( q )v (1)

& where q(t ) and q (t ) R 3 are defined as q = [x

& & => q = x

y ]T
T & & y

Left wheel

t Reference line
Y R

where x(t ) and y (t ) denote the position of the center of mass of the wheeled mobile robot, and (t ) R1 represents the orientation of the wheeled mobile robot. The center of mass and the center of rotation are assumed to coincide. The matrix S (q ) R 2 x3 is defined as follows

yR

e2 = R

X y b
e1 = d

Camera frame

cos S (q ) = sin 0

0 0 1

(2)

The velocity vector v(t ) R 2 is defined as v = [v1 v 2 ]T = [v ]T

2r

(3)

Right wheel

xR

Fig. 1. Scheme for deriving the kinematic equations.

where v and are the straight and angular velocities of mobile robot at its center point. From (1)-(3), the ordinary form of a mobile robot with two actuated wheels can be derived as follows

- 409 -

& x cos 0 y = sin 0 v & & 0 1

3. CONTROLLER DESIGN

(4)

3.1 Controller design Using Eqs. (4), (6) and (8), we can rearrange as follows:

The relationship between v, and the angular velocities of the two driving wheels, r , l , is given by
r 1 / r = 1 / r l b / r v b / r (5)

& d = v sin( R ) + d R tan( R ) The error dynamics can be represented as follows e1 = v sin e2 + e1 R tan e2 & e2 = R &

(10)

The point R( x R , y R ) satisfies the dynamic equation

x R = v R cos R & & y R = v R sin R & R = R

To obtain the above tracking controller, the Lyapunov function candidate is chosen as
(6)

V=

1 2 1 cos e2 e1 + 0 k2 2

(11)

Its derivative yields

where v R is the constant velocity of point R , and R is the time rate of the change in t t direction. The equation of the reference line t t through point R ( x R , y R ) is given as the following

1 & 1 V = sin e2 [ v e1k 2 + e1 R e1k 2 k2 cos e2 + ( R )]

If we choose

( x x R ) sin R ( y y R ) cos R = 0 The distance from the MRs center C ( x, y ) to reference line is as follows

(7)

d = ( x x R ) sin R ( y y R ) cos R

(8)

v = vr = k 2 [ vr e1 R / cos e2 ] e1 k1 sin e2 + R (12)

& then V 0 and the mobile robot achieves the control problem. In this application, the tracking line is straight line, that is, R = 0 ; So, Eq. (12) can be rewritten as

For a straight and infinite reference line described in Cartesian plane as shown in Fig. 1, determine a feedback control law such that the mobile robot (4) moves at a constant velocity v R along the line. To solve the above problem, let the errors be

e1 = d e 2 = R

(9)

v = vr = k 2 vr e1 k1 sin e 2

(13)

The above problem becomes to design a controller which achieves e1 0 and e2 0 as t .

Controller (13) is rather simple for this application to implement totally. The two errors have to be measured, and they are achieved by the camera sensor shown in the next section.
3.2 Measurement of the errors

- 410 -

In this paper, the controller is derived from measurement of the tracking errors. Fig. 2 describes the measurement scheme for the errors e1 and e2 in Eq. (13). Several information can be extracted from the camera sensor. But the return packet M is the key one to get the error information; that is to say, as sending a request command for tracking an object with a specified color to camera sensor, the return packet M is achieved as follows [1]:

M mx m y x1 y1 x2 y2 pixels confidenc\ r
The errors as shown in Fig. 2 can be expressed by

e1 = (71 m y ) cos e2 y y1 e2 = arctg ( 2 ) 80

t (x1,y1) 1 1 my 71 Reference line Bounding box Camera frame 143 Y

3.3 Welding process design While tracking the reference line, the mobile robot performs welding process as the pattern shown in Fig. 3. The line is 1mm width on grey, yellow, black or red background and welding spot beads is 10mm in diameter and the distance between them is 100mm. The operation sequence of the spot bead welding mobile robot is as follows: the mobile robot starts at a specified velocity (parameter 2); when the distance reaches the spot distance (parameter 1), the MR stops for welding, and the spot bead is made through two stages: first, welding operation is done for welding time (parameter 3); second, the crater operation, for crater time (parameter 4). The process is repeated for spot bead number (parameter 5). The mobile robot stops when it detects the end of the line. To achieve the specified spot bead pattern tracking, five parameters are loaded the mobile robots operation as follows: (1) Spot distance, (2) Velocity, (3) Welding time, (4) Crater time, (5) Spot bead number.

C (40,71) mx 40 e2
d e 1=

4. HARDWARE OF THE SYSTEM

80 X t (x2,y2)

Fig. 2. Errors measurement scheme.

100 100 Welding spot bead

10

MR move toward

Reference Line

Fig. 3 Spot bead pattern.

The hardware of system is composed of two parts: the camera module and the control system. The camera module used in this application has been developed by Carnegie Mellon University, Pittsburgh, PA, USA (Photo. 1). The cameras capabilities are as the following: Track user defined color blobs at 17 frames per second. Find the center of the blob. Gather mean color and variance data. Transfer a real-time binary bitmap of the tracked pixels in an image. Arbitrary image windowing. Adjust the cameras image properties. Dump a raw image. 80x143 resolution. 115,200/ 38,400/ 19,200/ 9600 baud serial communication. Slave parallel image processing mode of a single camera bus. Automatically detect a color. The vision system connects to the control system via RS232 communication.

100

100

100

- 411 -

The control system is very simple due to its functionality offered by Microchip PIC16F877. The configuration of the system is shown in Fig. 4. In the diagram, the two PIC16F877 microprocessors are integrated: one is main CPU, which receives information from camera sensor, sending the information to the motor controller and performing welding process; the other is motor controller, which controls the velocity of the two wheels. The two microprocessors are linked via I2C. Motors are fed and controlled via a LMD18200 dual fullbridge driver. Pulses coming from the encoders are used for velocity control and for distance counting in the welding process.

PM 1: Poll Mode - only one packet is returned when an image processing function is called. SM 0: Switching Mode - enable of color tracking which return its normal C or M color packet. SW 34 1 46 141: Sets the Window specify size of the camera. The following tracking color command is sent to the camera sensor at the sampling time of 100ms: TC 135 145 148 150 155 160 With the tracking color above, the mobile robot can track a white line on black, grey, yellow and red background.

Photo. 1. CMU cam for measuring the errors.

Encoder

Right Wheel Motor

Photo. 2. Experimental mobile robot MSB-02.

Camera Sensor Motor Controller PIC16F877 Main CPU PIC16F877 Camera CPU 75MIPS Ubicom SX28 Camera Unit

Encoder

Left Wheel Motor

Welding Process

Fig. 4. The configuration of the control system.

5. SIMULATION AND EXPERIMENTAL RESULTS

To verify the effectiveness of the proposed controllers, simulations have been done and applied to the experimental mobile robot. The parameters of controller is chosen as b = 105 mm, r = 25 mm , v = 75 mm / s . The camera setting parameters are as the following:

Photo. 3. Spot welding beads. The controller (13) is used. The positive constants in controller are chosen as k1 = 1.25 and k 2 = 250 . The initial values are d (0) = 10 mm , R (0) = 20 o , and (0) = 15o .

- 412 -

The simulation for straight line tracking is shown in Fig. 5 to Fig. 8. Trajectory of mobile robot and the line are given in Fig. 5. The two errors, distance and angular errors, are given in Fig. 6. The head angle of the mobile robot is shown in Fig. 7. The speeds of left and right wheels as control input are given in Fig. 8. From the simulation we can conclude as follows: The proposed controllers can be used for a mobile robot to follow any smooth curved line; as the result, it can be applied to this application for straight line totally. The errors approach zero after about 180mm as fast as desired, and, in fact, it makes the robot avoid swing which causes the distance error of spot welding beads. To track any smooth curved line, the parameter w = d w / dt must be known. Without this information, w must be estimated using an observer. This is our next study.
0.48 0.46

0.30 0.25

Head angle (rad)

0.20 0.15 0.10 0.05 0 -0.05 -0.1 0 2 4 6 8 10 12 14 16 18 20

Time (s)

Fig. 7. MRs head angle (rad).

45 40

right wheel

Linear velocity (rpm)

35 30 25 20 15 10 5 0

left wheel

10

12

14

16

18

20

Time (s)

Fig. 8. Control inputs: MRs wheels velocities.

reference line

y-coordinate (m)

0.44 0.42 0.40 0.38 0.36 0.34 0.00

6. CONCLUSION
camera frame mobile robot trajectory

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16

0.18

x-coordinate (m)

Fig. 5. Reference line and MRs trajectories.

12 10 8

error e1 (mm) error e2 (rad)

Tracking error ei

6 4 2 0 -2 -4 -6 0 2 4 6

This paper introduced the line tracking controller design method based on Lyapunov function for a two-wheeled spot bead welding mobile robot. The mobile robot moves along a reference straight line with a constant speed while performing welding process with the desired setting parameters. To generate the controller, the distance and the orientation of mobile robot are measured using camera sensors. The controllers have been verified by means of the simulation and applied to the real mobile robot MSB-02. This welding mobile robot has been successfully applied to practical field in shipyard company in Korea.

10

12

14

16

18

20

Time (s)

Fig. 6. Tracking errors e1 and e .

REFERENCES Anthony Rowe, Charles Rosenberg, Illah Nourbakhsh, Technical Sketch Session of CVPR, (2001). 2. A. Bemporad, M.D. Marco and A. Tesi, Proceedings of the 36th Conference on

1.

- 413 -

3.

4.

5.

6. 7.

Decision & Control, USA, (1997), pp. 30633068. W.E. Dixon, D.M. Dawson, E. Zergeroglu, F. Zhang, International Journal of Robust and Nonlinear Control, (2000), pp. 199-216. Y. Kanayama, Y. Kimura, F. Miyazaki, T. Noguchi, Proceedings of the IEEE International Conference on Robotics and Automation, (1990), pp. 384-389. J.M. Yang, J.H. Kim, Transactions on Robotics and Automation, (1999), pp. 578587. J.J.E. Slotine, W. Li, Prentice Hall, Inc., (1991). T.L. Chung, T.H. Bui, T.T. Nguyen, and S.B. Kim, Proceedings of the 8th International Conference on Science and Technology, Hochiminh City, Vietnam, (2002), pp. 51-55.

- 414 -