http://www.paper.edu.

cn

USB Image Acquisition System Based on CMOS Image

Sensor
Wang KejunZhang Xiaolei
Pattern Recognition LaboratoryHarbin Engineering UniversityHarbin150001
E-mailzhangxiaolei@hrbeu.edu.cn

Abstract
Using the OmniVision OV7620 CMOS image sensor as photoelectric imaging device,
glueless connecting was achieved between the USB2.0 controller CY7C68013A and
the sensor by general programmable interface (GPIF). The simple high efficiency and
low-cost hardware which be apt for high speed image capture was designed and the
software system was also compiled based on the multi-thread technology. The image
processing system with acquisition, transmission and display was designed based on
the new type CMOS image sensor and USB2.0 controller.
KeywordsCMOS image sensorUSBCY7C68013AGPIF

1 Introduction
Because of the rapid development of the semiconductor technics, Complementary Metal Oxide
Semiconductor (CMOS) [1-3] image sensor makes progress quickly in recent years because of its low cost,
small volume, light weight, low power consumption and single chip integration compared with traditional
CCD [4-5] sensor. It has become the key facility of image processing technology and has been applied to a
great many fields such as video camera, digital camera, mobile communication, safeguard, and pattern
recognize. In illustration of USB Image acquisition system, OmniVision single chip OV7620 is described
in this paper, including its features, operate mode and usages.
The Universal Serial Bus (USB) has gained wide acceptance as the connection method of choice for PC
peripherals. Equally successful in the Windows and Macintosh worlds, USB has delivered on its promises
of easy attachment, an end to configuration hassles, and true plug-and-play operation. USB brings the
ultimate in implicity when connecting peripheral devices. The longer needs to determine the right
connection for the mouse versus the keyboard. There isnt a different cable for the monitor, printer, or high
storage capacity external disk drive. The user never has to determine if the connection is a parallel or serial
port. In the world of USB, there are no longer dip switches, jumpers, IRQ conflicts, and DMA conflicts. In
fact, users will always be able to connect or disconnect a new device when the PC is up and running. Now
thats true Plug and Play! Because of this consistency and simplicity of connection, USB will make it easy
to add peripheral image acquisition system that was previously thought to be too difficult for the average
user.
To sum up, it has been very important and necessary to integrate all the advantages of CMOS image sensor
and USB interface into the image acquisition system. In this paper, we use the OmniVision OV7620 CMOS
[6] image sensor and Cypress FX2LP high-speed USB2.0 CY7C68013A [7-8]as the main chips to achieve
our design.

2 Hardware Design
2.1 CMOS Image Sensor
OmniVision OV7620 is a highly integrated high resolution (640x480) Interlaced / Progressive Scan
CMOS digital color /black&white video camera chip. The digital video port supports 60Hz YCrCb
-1-

http://www.paper.edu.cn

4:2:2 16Bit / 8 Bit format [9], ZV Port output format, RGB raw data 16Bit/8Bit output format and
CCIR601/CCIR656[10-11] format. The builtin SCCB interface provides an easy way of controlling
the builtin camera functions.

2.2 USB2.0 Controller

Cypress Semiconductor Corporations EZ-USB FX2LP (CY7C68013A) is a low-power version of the
EZ-USB FX2 (CY7C68013), which is a highly integrated, low-power USB 2.0 microcontroller. By
integrating the USB 2.0 transceiver, serial interface engine (SIE), enhanced 8051 microcontroller, and a
programmable peripheral interface in a single chip, CY7C68013A has created a very cost-effective solution
that provides superior time-to-market advantages with low power to enable bus powered applications. The
ingenious architecture of CY7C68013A results in data transfer rates of over 53 Mbytes per second in theory,
the maximum-allowable USB 2.0 bandwidth, while still using a low-cost 8051 microcontroller in a small
package. Because it incorporates the USB 2.0 transceiver, the CY7C68013A is more economical, providing
a smaller footprint solution than USB 2.0 SIE or external transceiver implementations. The Smart SIE
handles most of the USB 1.1 and 2.0 protocols in hardware, freeing the embedded microcontroller for
application-specific functions and decreasing development time to ensure USB compatibility. The General
Programmable Interface (GPIF) [12]and Master/Slave Endpoint FIFO (8- or 16-bit data bus) provides an
easy and glueless interface to popular interfaces such as ATA, UTOPIA, EPP, PCMCIA, and most DSP
processors.

2.3 Principle of USB Image Acquisition System

There are three parts of the USB image acquisition system.

Referring to Fig 1.

Figure 1: USB Acquisition System Block Diagram

2.3.1 Acquisition Part:

The light from outside scenery is converged to CMOS pixel array through lens. Each compact active pixel
element has high photo-sensitivity and converts photon energy to analog voltage signal. The sensor
OV7620 has on-chip Analog to Digital Converter and comparators to digitize the pixel output. The output
rate of CMOS image sensor is relatively high. It is required that USB controler's rate match the sensor's.
There are several signals to be considered in system design: 8 bit image data, PCLK , VSYNC , HREF, and
I2C bus etc. The majority of signal processing in OV7620 is performed in the analog processing block,
which does color separation, matrixing, AGC, gamma correction, color correction, color balance, black
level calibration, knee smooth, aperture correction, controls for the luminance and chrominance picture and
anti-alias filtering. The analog video signals are based on the formula:
-2-

http://www.paper.edu.cn

Y = 0.59G + 0.31R + 0.11B;

U = R-Y;
V = B-Y; [13]
In our design, It is not necessary to capture the color image.The Y channel is enough to satisfy the system
requirement, So there are only image data of Y channel can be transmit to the USB controller with the 8 bit
mode on the data bus.
PCLK , VSYNC , and HREF signals are the main hand-shake signals of control interfaces. Standard
video timing signals include VSYNC, HREF, PCLK, FODD and CHSYNC. Here VSYNC is asserted high
during several scan lines in the vertical sync period, after that will be the HREF, while image data clocking
out at the pace of PCLK, HREF will assert high till all the data in a frame are transmitted out. By default,
data is updated at the falling edge of PCLK and is stable at its rising edge. PCLK runs at the pixel rate
(normally at 13.5M when crystal clock frequency is 27MHz). Data bus including 8 data lines, the
luminance (Y channal) data is clocked out of this bus at the rate of one byte per pixel.
I2C bus (inter integrate SCCB bus) is a communication technology adopting two lines (SDAT and SCLK).
I2C bus transmitting rate is up to 400kbps. It supports CMOS process, thereby CMOS image sensor
contains I2C bus interface and need not design it.

2.3.2 Transmission Part:

To achieve the maximum sustained throughput in USB 2.0 High Speed designs, the physical interconnect
should never be the primary bottleneck in the system. CY7C68013A 's GPIF (General Programmable
InterFace) provides a highly configurable and flexible glueless peripheral interface that allows the highest
possible bandwidth to be achieved over the physical layer.
In our design, the OV7620s PCLK pin will connect to GPIFs IFCLK pin which can be either an input or
output signal and is determined by what the system requirements are; The OV7620s VSYNC and HREF
will connect to GPIFs RDY0 and RDY1 respectively. To achieve the maximum bandwith of the
transmission, it will configure the GPIFs CTL0 signal as the master strobe of FLOWSTATES.

2.3.3 Display Part:

The development environment of software is Borland C++ Builder 6.0 under Windows. The application
software capture the image data from the USB cable and display on the screen.

3. Software Design
3.1 Firmware
In order to simplify the design and make the engineers concentrate their attention on the function not
to waste their time on the complex protocol, it is necessary and convenient to use the integrate
framework. The framework is helpful to the research worker to realize the most parts of the USB
protocol. According to peripheral device and requirement, we just need to write different functions to
finish our design. To be aimed at the high-speed data and the USB 2.0 protocol, there are several
aspects in our firmware software design:
3.1.1 Initial The USB Port and Fifo Register:
Set the 8051 CPU clock to 48MHz. Set the endpoint2 IN, bulk, size 512, 4x buffered. Set NAKALL
bit to NAK all transfers from host. Reset EP2 FIFO and clear NAKALL bit to resume normal
operation. Set EP2FIFOCFG register auto in mode, disable PKTEND zero length send, word ops.
-3-

http://www.paper.edu.cn

Initialize GPIF registers For EP2IN, GPIF uses FF flag. Initialize global FLOWSATES register. Parts
of the program are as follows.
CPUCS = ((CPUCS & ~bmCLKSPD) | bmCLKSPD1);
SYNCDELAY;
EP2CFG = 0xA0;
SYNCDELAY;
FIFORESET = 0x80;
SYNCDELAY;
FIFORESET = 0x02;
SYNCDELAY;
FIFORESET = 0x00;
SYNCDELAY;
EP2FIFOCFG = 0x10;
GpifInit ();
EP2GPIFFLGSEL = 0x01;
SYNCDELAY;
EP6GPIFFLGSEL = 0x02;
FLOWLOGIC = FlowStates[19];
FLOWSTB = FlowStates[22];
GPIFHOLDAMOUNT = FlowStates[26];
FLOWSTBEDGE = FlowStates[24];
FLOWSTBHPERIOD = FlowStates [25];
SYNCDELAY;

3.1.2 Setup The GPIF Interface Engine:

Except the initial code in the function Gpifinit(), the function code void TD_Poll(void) must be
written in the firmware program. It is the kernel code to ensure the GPIF engine operates smoothly. In
the TD_Poll (void) function, we define the condition of when the GPIF engine start and how many
times it operate. Parts of the program are as follows.
void TD_Poll(void)
{
if( GPIFTRIG & 0x80 )
// if GPIF interface IDLE
{
if(in_enable)
// if IN transfers are enabled
{
if ( GPIFTRIG & 0x80 )
// if GPIF interface IDLE
{
if ( EXTFIFONOTEMPTY )
// if external FIFO is not empty
{
if ( !( EP24FIFOFLGS & 0x01 ) )
{
if(enum_high_speed)
-4-

http://www.paper.edu.cn

{
GPIFTCB1 = 0x01;
SYNCDELAY;
GPIFTCB0 = 0x00;
SYNCDELAY;
}
Setup_FLOWSTATE_Read();
SYNCDELAY;
GPIFTRIG = GPIFTRIGRD | GPIF_EP2;
SYNCDELAY;
while( !( GPIFTRIG & 0x80 ) )
{
;
}
SYNCDELAY;

There is another software GPIF TOOL use to generate the C code file which was compiled in the
firmware project. Referring to Fig 2.

Figure 2: GPIF TOOL Software Interface

GPIF is an internal master to the CY7C68013As endpoint FIFOs. It replaces the external glue logic
which might otherwise be required to build an interface between the CY7C68013A and the outside
world. At the GPIFs core is a programmable state machine which generates up to six control and
nine address outputs, and accepts six external and two internal ready inputs. Four user-defined
Waveform Descriptors control the state machine; generally (but not necessarily), one is written for
FIFO reads, one for FIFO writes, one for single-byte/word reads, and one for single-byte/word
writes.Only the FIFO reads waveform have been used in our design. Referring to Fig 3.

-5-

http://www.paper.edu.cn

Figure 3: FIFO Reads Waveform Interface

In order to achieve the maximum USB 2.0 High-speed bandwidth, the CPU (in this case an enhanced
8051) should never be directly involved in moving the payload data from the external slave device to
the USB host and vice versa. The CPU would clearly be the largest bottleneck in a High-speed design.
So instead we adopt an Auto-Transfer mode, whereby the payload data is auto-committed from the
external slave device to the USB host and likewise in the other direction. The GPIF engine uses this
Auto-Transfer mode to move data to and from the external slave device.
As mentioned above, the GPIF engine generates the GPIF waveform descriptors that implement the
physical bus timing. The GPIF engine can implement multiple waveform behaviors, which can then be
assigned to four waveform types. The assignment is made through the GPIFWFSELECT register
setting. Each descriptor is 32 bytes long and resides in a special GPIF waveform descriptor area in
on-chip memory space, once loaded by the CPU. When we finish the GPIF design, these descriptors
will be add to firmware project and compiled soon.

3.2 Application Program:

When first plugged into USB, the CY7C68013A enumerates automatically and downloads firmware
and USB descriptor tables over the USB cable. Next, the CY7C68013A enumerates again, this time as
a device defined by the downloaded information. This patented two-step process, called
ReNumeration, happens instantly when the device is plugged in, with no hint that the initial
download step has occurred. At the same time, pc get the PID/VID code which use to identify the
USB device through CYUSB.INF [14] and CYUSB.SYS [15].
The CYUSB.INF file can be modified to accomplish several different objectives.To make the driver
match to a specific device, the device's vendor ID and product ID need to be added to the CYUSB.INF
file. For instance,
:%VID_VVVV&PID_PPPP.DeviceDesc%=CyUSB,USB\VID_ VVVV&PID_PPPP.
Change the VVVV to contain the hexadecimal value of the VendorID for the device.
Change the PPPP to contain the hexadecimal value of the ProductID for the device.
CYUSB.SYS is a USB device driver. The API library is only able to communicate with USB devices
that are served by this driver. we must force Windows to use the CYUSB.SYS driver with the
device.If Windows prompts for a driver or indicates that it needs a driver, direct the PC to use the
CYUSB.SYS driver by steering it to the CYUSB.INF file in the Driver directory.If Windows does not
-6-

http://www.paper.edu.cn

prompt for a driver, it has already matched the device to a driver itself. In this case, manually instruct
Windows to use that driver.
The application software was compiled by Borland C++ Builder 6.0. CYAPI.H [16] and CYAPI.LIB
was cited in the software for opening the device and transmiting data.
Take accout of the high-speed data rates, double buffer technology and scanline method were adopt in
the multi-thread software. Parts of the program are as follows.

Graphics::TBitmap*tmpBitmap=newGraphics::TBitmap();
tmpBitmap->Assign((TPersistent*)Image1->Picture->Graphic);
tmpBitmap->PixelFormat=pf24bit;

CCyUSBDevice *USBDevice = new CCyUSBDevice (Handle) ;

//Get the devive handle and open the device.
USBDevice->~CCyUSBDevice();
//Close the USB device.

USBDevice->BulkOutEndPt->XferData(buf, length);
//Asynchronous transfer mode.
BeginDataXfer(buffer, length, &outOvLap);

BeginDataXfer is an advanced method for performing asynchronous IO. This method sets-up all the
parameters for a data transfer, initiates the transfer, and immediately returns, not waiting for the
transfer to complete. BeginDataXfer allocates a complex data structure and returns a pointer to that
structure. FinishDataXfer de-allocates the structure. Therefore, it is imperative that each
BeginDataXfer call have exactly one matching FinishDataXfer call.
There are two threads in the application software. Capturing the image data occupy one thread
specially. With this method, we can get the image data smoothly and avoid no respond to system when
transmit the USB data. The application software display interface. Referring to Fig.4

Figure 4: Application Software Interface

-7-

http://www.paper.edu.cn

4. Summary
This USB image acquisition system adopt the new type CMOS image sensor OV7620 and USB 2.0
highspeed controller CY7C68013A to realize the design which has created a very cost-effective,
simple high efficiency and low-cost solution compare with the traditional method. It can be applied to
a great many fields such as video camera, automatic device, robot, mobile communication, safeguard,
and pattern recognize conveniently and flexibly.

References
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]

[13]
[14]
[15]

Abbas El, Gamal Helmy Eltoukhy, CMOS image sensors, IEEE Circuits & Devices, pp. 6-20, May/June 2005.
B. Fowler, A. El Gamal, and D.X.D. Yang, A CMOS area image sensor with pixel-level A/D conversion, IEEE
Int. Solid-State Circuits Tech, Dig., pp. 226227, 1994.
N. Stevanovic, M. Hillegrand, B.J. Hostica, and A. Teuner, A CMOS image sensor for high speed imaging, in
ISSCC Tech. Dig., vol. 43, pp. 104105, 2000.
N. S. Saks, A technique for suppressing dark current generated by interface states in buried channel CCD
imagers, IEEE Electron Device Lett, vol. 1, pp. 131133, July 1980.
OV7620 Product Specifications-Rev. 1.3 2000.http://www.ovt.com.
CY7C68013 EZ-USB FX2 USB Microcontroller Data Sheet Rev. *B. Cypress Semiconductor Corporation, June
2002.
The EZ-USB FX2 Technical Reference Manual Version 2.1. Cypress Semiconductor Corporation, 2001.
Cliff Reiter, "With J: image processing 2: color spaces,"
ACM SIGAPL APL Quote Quad, Vol.34, Issue 3,
pp3-12, June 2004.
James M. Kasson, Wil Plouffe, "An analysis of selected computer interchange color spaces," ACM Transactions
on Graphics (TOG), Vol.11, Issue 4, pp373-405, October 1992.
Didier Le Gall, "MPEG: a video compression standard for multimedia applications,"
Communications of the
ACM, Vol.34, Issue 4, pp47-58, April 1991.
EZ-USB FX2 GPIF Primer. Cypress Semiconductor Corporation, March 2003.
B. Hill, Th. Roger, F. W. Vorhagen, "Comparative analysis of the quantization of color spaces on the basis of the
CIELAB color-difference formula,"ACM Transactions on Graphics (TOG), Vol.16, Issue 2, pp109-154, April
1997.
Cypress CyUsb.sys Programmer's Reference.2003
Cypress USB Console Users' Guide.2003.
Cypress CyAPI Programmer's Reference.2003.

-8-