Real-time Image Processing System Based on FPGA for Electronic Endoscope

Jie Jiang Daoyin Yu Zheng Sun

School of Precision Instrument and @to-electronic Engineering, Tianjin University, hn Tianjin, 300072, C i a K Lab of Optoelectronic Information Science and Technology, MOE,Tianjin, China E-mail: jiangiie@eyou.com processing system for Electronic Endoscope in Abstract: Medical Electronic Endoscope is the outcome of combining traditional endoscope great number. Because of the limit of its space Electronic technology with modern computer technology Endoscope adopts a kind of compact full frame f and micro-electronic technolow. Because o the CCD whose output image pattem is different limit of its space endoscope adopts a kind of from the pattem of PAL monitor. And to improve compact full pame CCD whose output image image quality and displaying effect the system pattem is diflerent f i m the pattern of PAL implements real-time image processing with monitor And to improve image quality and
displaying eflect the system implements real-time image processing with field programmable gate array

(FPGA).

The

image

pattern

transformation ensures the correct display of the

field programmable gate array (FPGA). FPGA is adopted for its short development cycle, insystem programming, cost-effectiveness, highdensity logic integration and high performance.

CCD output image. With the function of image interpolation zooming, image enhancement and
digital video technolow, image quality and the accuracy rate of diagnosis are improved.

I1 Performance and Configuration of

the Real-time Image Processing System
The function of CCD image acquisition is implemented. But the signal from CCD is image packets shown as the f i g 1 which is different from the image pattem of PAL monitor shown as the fig.2.. .
i

I Introduction
Electronic Endoscope -a kind of medical instrument is used widely. Through endoscope doctors can duectly visualize internal areas of the human body, diagnose quickly and do some miniature injury surgery. Doctors and patients recognize the advantages of Electronic Endoscope. Electronic Endoscope is the outcome of combining traditional endoscope technology with modern computer technology and microelectronic technology. From 198O's, with the development of Electronic Endoscope, the image processing for Electronic Endoscope develops fast. J.F.Rey captured the video signal from Electronic Endoscope into computer for image processing and analyzing. S.Guadagni implemented an image processing and analyzing system with an Electronic Endoscope and a 386 computer. Japan develops and produces Electronic Endoscope as well as image

Hori-Antal syn

-... : f

Image signal

Fig. 1 Image signal from CCD CCD output signal can not be displayed correctly on PAL monitor because of the different between their horizontal frequency, frame frequency and synchronization signal.

+ 9".
682

Limited by digestive tract, endoscope is about CCD is built in the distal end of endo-

0-7803-6253-5/00/$10.00 IEEE. 02000

+
..,*I.

......... ,, .> .......... .................>...,...... .

.I..-

Image Processing System

3.1 Image Pattern Transformation

. , * . . a . . .

..)....!...: Image si

\; .........;* ...
Light integral

Fig. 2 Video signal of PAL scope, so the size of CCD is also limited, only 190X 190 pixels. The picture is so small that doctors can watch clearly. So it need realize a real-time digital interpolator. Because of reflection of bead in digestive tract and different luminance, image is blurry. So the image enhancement is necessary. To improve the displaying effect, some digital video technology is implemented: mult-picture effect and image freezing. The shape and location of pictures are control too for beauty. As described above, the real-time image processing system performs the function of image pattern transformation, image interpolation zooming, image enhancement and digital video technology etc. The figure above illustrates that the real-time image processing system is implemented with first-in-first-out (FIFO) buffer and video-doubleport memory (VRAM) cooperating with FPGA. The implementation of the real-time image processing system is described in detail as follows.

-.

.
.... ...J

Fig. 3 The diagram of real-time image processing system

mThe Design and Implementation with FPGA of Real-time

683

There is still a long distance between the CCD output image pattern and the requirement of patterns which PAL monitor can display mainly because of the difference between their horizontal frequency, frame frequency and synchronization signal. CCD s horizontal frequency, approximately 30 kHz, is twice as much as the line scan frequency (15.625 W Z ) required by PAL. To resolve the discrepancy between two horizontal frequencies the system setups a FIFO buffer to buffer the CCD output signal. Because of using full frame CCD which outputs signals line by line and outputs one frame signal in one field time, frame frequency, equal with field frequency, is 50 Hz and PALS frame scan frequency is 25 Hz adopting interleaved scan. The system removes the inconsistency of frame frequency by partitioning VRAM. After such man-made separation of odd and even field, a line-by-line CCD image can be &splayed on the monitor correctly. Both read & write control and address generation of FIFO and VARM are implemented with FPGA. CCD output image signals havent any synchronization while PAL image correct displaying needs synchronization as its time norm. The system generates synchronization based on principle of phase-lock loop (PLL) and adds synchronization signal to CCD output image by video encoder to ensure the image displaying on PAL monitor correctly. 3.2 Image Interpolation Zooming The Walsh method is adopted in image zooming from many interpolation algorithms for it is simple, intuitionistic. It can easily be implemented with hardware too. Interpolation zooming algorithmic is based on superimposure filter which is put forward by Wang Zhaohua. Because it overcomes edge effect, the image quality is good. In order to simplify the hardware, the zooming template is decomposed and

-+,

implemented with simple arithmetic of FPGA logic circuit. Observing the image [M,] = - 2 4 2 for interpolation, for 4 1 2 1 example of 4 fold zooming, we find out that in convolution zeros do not take part in compute, so we decompose the template of 4 fold zooming(3 X 3 matrix) which is showed as above into four simple templateswhich is shown as follow:

which accomplishes enhancing and zooming simultaneously .The new template shown as fig.5, after being decomposed, can be easily
0
-114 -112 -114 0 -114 114 1 114 -114 -112 1

-114
1/4

0 -114 -112 -114

0

3
1

1 114
-114

-112

Fig. 5 Walsh enhancement zoomina temdate

1

From above, we know that the original pixel is remained, the other three pixels are interpolated with four neighboring pixels. The speed of the algorittim is 4 times faster than convolution. And the algorithm can be implemented with simple addition or subtraction operation. All of these make the implementation in bardware is not only simple but also fast. Through observation, all zooming templates can be decomposed, implemented with simple operation in this way. The interpolated image has good quality and hasnt jagged lines. Fig. 4 is the block diagram of 4 fold zooming processing with FPGA.

implemented with hardware. Walsh enhancement zooming template reduces computing procedures and time and simplifies ~. hardware And it can also be
~

decomposed and implemented with simple arithmetic, For such reasons it satisfies our real-time hardware implementation very much. On the basis of interpolation zooming realtime image enhancement zooming is impfemented with FPGA.

3.4 Digital Video Technology Multi-picture effect The system displays two pictures: the small

one is CCD output image and the large one is obtained by zooming 4 times CCD output image. Fig 6 shows the effect of double-picture.

,
.. 1

__

_.

Fig.6 Double-picture effect

Fig. 4 Block diagram of 4 fold zooming processing with FPGA.

3.3 Image Enhancement

For the purpose of simpllfying system as well

as being easy to the hardware implementation

the system still adopts Walsh method to enhance

From the figure we can find that the two pictures shapes are controlled by cutting each ones four edges for the purpose of being aesthetic and every pictures position is carehlly l chosen to reach the best displaying effect. Al of this is implemented with FPGA image interpolation zooming and VRAM read & write controlled by P G A . * lmage fmezing The image that contains probable focus is

images. During experiment after combining the

template of zooming and enhancement, a new enhancement zooming template is obtained

684

frozen for doctors to diagnose further. Of the two images one is real-time displaying and the other is frozen. Showing states of them can be transformed: real-time one can be changed to be frozen and vice versa. Two switches are designed to perform real-time control of the transformation of pictures.

image is better. With enhancement zooming, the definition of images is improved greatly which can be helpful to doctors diagnose.

V Conclusion
The performance and configuration of realtime image processing system for Electronic Endoscope are presented above. The design and implementation with FPGA of image pattem transformation, image interpolation zooming, image enhancement and digital video technology are described in detail, too. The real-time image processing has been used in Electronic Endoscope successfully. The image quality and the accuracy rate of diagnosis are improved. FPGA is adopted as the core of the system for its changeable and reconfigumble feature, so the system can be easily modified, moreover the performance of the system can be improved in the future, without the change of the hardware. With a large amount of gates in a single device, FPGA provides the system with high integration and excellent performance.

Experiment Results

The experiment results of interpolation zooming and enhancement zooming is shown below:

(a) Original image

References S. M. Greengrass & M. Cunningham,

Endoscopy, Measurement+Control, Vol. 26, 1993: 109-114

(b) 4 Times zooming image

(c) 4 Times enhancement zooming image

Fig.7 Experiment Results The figures illustrate that after interpolation zooming the image without the edge indention and block effect is obtained. The quality of

M. Classen, etc. al., Electronic Endoscopy -The Latest Technology, Endoscopy, Vol. 19,1987: 118-123 S. Guadagm, etc. al., Imaging in Dgestive Video Endoscopy, SPIE Vol. 1420, Optic Fibers in Medicine, 1991: 178- 182 Z.Wang, Theoretical analysis and experimental investigation of the twolmensionai spectrum of a PAL composite video signal, ntzArchir Bd.5 1983 H.5 Nixos Herodotou and Anastasios N.Venetsanopoulos, COLOUR IMAGE INTERPOLATION FOR HIGH RESOLUTION AXQUISITION AND DISPLAY DEVICEJEEE Tran. on Consumer Electronics, Vol.Jl,N0.4 , 1995,1111118-1125

685