TI Embedded Processors and Video Technology

TI Embedded Processors and Video Technology
Agenda
Video standards and concepts
Brief overview of video signal formats
OMAP35xx /DM355 / DM646x /DM643x / DM644x Video-Specific Core IP

VPFE and video input capture
Image capture, image pipe, scaling, histogram
Video Processing
Resizing, Color conversion, Range mapping, Edge Padding Video compression
VPBE and video display or output

Overlaying, OSD, output methods (analog DAC, DLCD, etc.)
OMAP
TEXAS INSTRUMENTS TECHNOLOGY
Accelerating Your Success
Video standards and concepts

Brief overview of video signal formats
Resolution
Perceived resolution is a function of: Actual resolution Distance Size of display (i.e. size of pixels)
OMAP
Various Standard Resolutions

Format
D1 SIF
Application(s)
Full Analog Television Resolution Resolution VHS VCR is capable of Digital Television
NTSC
720 x 480 352 x 240
PAL
720 x 576 352 x 288
18 different resolutions/rates
(three most common are shown)
ATSC
Standard Definition (SDTV) High Definition (HDTV)
720 x 480 1280 x 720 1920 x 1080 704 x 576 352 x 288 176 x 144
5
4CIF CIF QCIF
Often used in Video Conferencing or for small screen applications

(specified for various codecs, e.g. H.261)
OMAP
NTSC, PAL, and CIF Formats

NTSC
D1 SIF
Resolution
720 x 480 352 x 240
Frame Rate
59.94 fields/sec
I/P
I
Data Rate
10M pix/sec 2.5M pix/sec
PAL
D1 SIF
Resolution
720 x 576 352 x 288
Frame Rate
50 fields/sec
I/P
I
Data Rate
10M pix/sec 2.5M pix/sec
CIF
4CIF CIF QCIF OMAP
Resolution
704 x 576 352 x 288 176 x 144
Frame Rate
30 frames/sec 30 frames/sec 30 frames/sec
I/P
P P P
Data Rate
12M pix/sec 3M pix/sec 760K pix/sec
6
ATSC Formats
ATSC
SDTV
Name
480i 480p 720p 1080i 1080p
Resolution
720 x 480 720 x 480 1280 x 720 1920 x 1080 1920 x 1080
Frame Rate
(per second) 60 fields 60 frames 60 frames 60 fields 60 frames
I/P
I P P I P
Data Rate
(pixels/sec) 10 M 20 M 55 M 62 M 124 M
HDTV
Key
SDTV = Standard Definition Television HDTV = High Definition Television I = Interlaced P = Progressive
Standard supports both NTSC rates and integer rates: i.e. 60.00, 59.94, 30.00, 29.97, 24.00, and 23.98
7
OMAP
Summary: Pixel's in 4D
width
Dimensions: 1. Height 2. Width

h
3. Frame rate determines how long the pixel exists, 4. Color Depth of the pixel
How many bits are used to represent the color of each pixel? i.e. how it moves
1 bit
OMAP
8 bits
Red: 8 bits Green: 8 bits Blue: 8 bits
or YCbCr 4:4:4
RGB Color
8-bits Red
8-bits
Green
8-bits
Blue
255
0 All color can be composed by adding specific amounts of R, G, & B 8-bits (28) specifies the amount of each color This is the scheme used by most electronic displays to generate color; e.g. we often call our computer monitors, "RGB displays"
OMAP
Color Reduction
Human eye is not as sensitive to color as it is to Luminance
dark vs light To this end, to save costs the various standards decided to: Maintain luminance information in our images, but Reduce color information
Using RBG, though, how do we easily reduce color information without removing luminance? For this, and other technical reasons, a separate color space was chosen by most video standards
10
OMAP
What is YCbCr?
Video Source Y Cb Cr
Y Cb Cr R G B
Even though most displays actually use RGB to create the image, YCbCr is used most often in consumer electronics for transmission of the image Historically, B/W televisions transmitted only luminance (Y) The color signals were added later
OMAP
11
Video Input /Capture
Digital Video Overview BT.656

BT.656 Stream
SAV-Start of Active Video EAV-End of Active Video
Transmitter
Processor or Video Decoder SAV
Synchronization Codes
EAV EAV SAV Line of Data Blanking Line of Data Etc
Receiver
Processor or Video Encoder
Clock (27Mhz Typ)
8 bit data bus

Clock (27Mhz Typ)
13
OMAP
Digital Video Overview - RAW Uses external synchronization signals Primarily for video, but also useful for general data More complex to implement as it can use separate hardware synchronization signals
14
OMAP
Digital Video Overview - RAW
Transmitter
Processor or Video Decoder
Raw Stream There can also be a Vertical Sync Edge if at the start of a 8-16 bit data bus field/frame
Clock (up to ~80Mhz Typ) Horizontal Sync Vertical Sync Horizontal Sync Edge
Receiver
Processor or Video Encoder
Line of Data OMAP

Blanking
Line of Data
Etc
15
Video Port/Transport Stream Modules - DM6467

VPIF Capture
16b BT.1120 or 8b BT.656 8b BT.656 or 12b RAW
Benefits Provides a direct digital connection to Video Encoders, Decoders, ADCs, DACs, and ATSC Tuners Features 16-bit Digital Interface to capture/display High-Definition Video Content from/to Video ADCs/Decoders/DACs/Encoders Dual 8-bit BT.656 interfaces for Multi-Channel Standard Definition Video Capture/Display 8/10/12-bit RAW Capture Interface to CMOS Sensors MPEG2 Transport Stream Interface
Synchronous/Asynchronous Modes Stream Parser, PID Filtering, ATS Detection/Correction Bypass Mode allows RAW Data, not just Transport Streams
Display
16b BT.1120 or 8b BT.656 8b BT.656
TSIF
Serial/Parallel I/O Serial I/O
Documentation VPIF Users Guide http://focus.ti.com/lit/ug/spruer9/spruer9.pdf TSIF Users Guide http://focus.ti.com/lit/ug/sprueq2a/sprueq2a.pdf
16
OMAP
VPSS Front End CCDC DM644x, DM643x, DM3xx, OMAP35x

Interfaces to analog CCD imagers, CMOS imagers, and BT.656 standard (8 or 16 bit modes) Devices 10-bit to 8-bit A-law compression Downsampling (via programmable culling patterns) Fault pixel correction Interlaced or progressive sensors
A/D
CCD (Analog Out)
Timing Gen
8/16 bit BT.656 CCD or CMOS Sensors (Digital Out)
CCDC
Previewer, H3A, External Memory
17
OMAP
Camera Interface OMAP35x

Image Sensor Support
RAW/RGB, YUV(4:2:2) Electronic Shutter
Image Sensor Interface (Parallel)

Sync (RAW/RGB) ITU-R BT 656 (YUV422 data over an 8-Bit or 10-Bit interface)
Video processing/Image Processing

CCDC
Physical Interface for camera/image sensors Data formatter, output formatter, black-level compensation, LUT faulty pixel correction
Image-Pipe:
Preview Engine Resizer Module
18
OMAP
Camera Interface Sync Mode OMAP35x
Start and End of a horizontal line
SYNC Mode : Supports 8, 10 and 12-bit data Provides horizontal and vertical synchronization signal (to identify the active pixel)
OMAP3530
Cam_hs
Camera ISP module
Cam_vs Cam_pclk Cam_d[11:0]
Generic camera sensor
Start and End of a frame
19
OMAP
Camera Interface BT656 OMAP 35x

ITU Mode : Supports data in ITU-R BT.656 format - Transfers YUV 4:2:2 data over 8 or 10-bit interface - Multiplexes the data in the following order: Cb0 Y0 Cr0 Y1 Cb2 - Provides timing reference signals embedded in the video stream (to reduce the number of wires)
Horizontal Blanking Vertical Blanking
OMAP3530
Camera ISP module
Cam_pclk
Cam_d[9:0]
BT656 camera sensor
20
OMAP
Video Processing
Video Data Conversion Engine DM6467

Benefits
Offloads DSP and ARM for common processing algorithms required by video codecs and video applications
VDCE
Chroma SubSampler Down Scalar Anti-Alias Filter 2-bit Menu Overlay Edge Padder
Features
Pre- and Post-Processing Engine for Video Compression and Capture/Display Downscale from 1x (1/8)x; 256/n, where 256 < n < 2048 YUV420) for Video Chroma Conversion (YUV422 Processing algorithms
2-bit Menu Overlay for simple menus and text overlays

Documentation VDCE Users Guide
http://focus.ti.com/lit/ug/sprueq9/sprueq9.pdf
22
OMAP
VPSS Front End Previewer (Image Pipe) DM644x, DM643x, DM3xx, OMAP35x
Converts raw Bayer pattern output of image sensors to format suitable for image processing or display Other features:
Can capture and subtract dark frames for better image quality Lens shading compensation A-law Decompression (8-bit A-law to 10-bit Linear) 3x3 Noise Filter Digital gain and white balance Gamma Correction
G R G R B G B G
Raw Bayer Pattern From CCDC or External Memory
Previewer (IPIPE)
Y0 Cb0 Y1 Cr0 Yn Cbn Yn+1 Crn

YCbCr 4:2:2 Gamma Corrected External Memory
23
OMAP
VPSS Front End Resizer DM644x, DM643x, DM3xx, OMAP35x

Accepts data from the Previewer or from External Memory, Output is to external memory (format is YCrCb) Resizes (Up or Down) from x to 4x using a polyphase resampling filter Independent resizing factor for the horizontal and vertical directions
Downscaling
Resizer
Horizontal 1/4X to 4X Vertical 1/4X to 4X
Upscaling
Resizer
Horizontal 1/4X to 4X Vertical 1/4X to 4X
24
OMAP
VPSS Front End H3A/Histrogram- DM644x, DM643x, DM3xx, OMAP35x

Aids control loops for Auto focus, Auto White Balance, and Auto Exposure Works with Raw Bayer (RGB) data only, direct from CCDC Image broken down into Paxels
Pixel 2-D block of Pixels Pixel size is user definable

Statistics collected across Paxel
H3A
Auto Focus Auto White Balance Auto Exposure
25
OMAP
Range Mapping - DM646x

Image of Parsing Control Flag for Range Mapping of VC-1:
Equation for Range Mapping of VC-1:
26
OMAP
Edge Padding (Interlaced) - DM646x

Top field 0 0 0 0 a p 5 i 0 0 0 0 a p 5 i 1 1 1 1 b q 6 j 2 2 2 2 c r 7 k Interlace Frame (field) Image 0 0 0 0 0 a a p p 5 5 i i 0 0 0 0 0 a a p p 5 5 i i 0 0 0 0 0 a a p p 5 5 i i 0 0 0 0 0 a a p p 5 5 i i 1 1 1 1 1 b b q q 6 6 j j 2 2 2 2 2 c c r r 7 7 k k 3 3 3 3 3 d d s s 8 8 l l 4 4 4 4 4 e e t t 9 9 m m 0 0 0 0 a p 5 i 0 0 0 0 a p 5 i 3 3 3 3 d s 8 l 4 4 4 4 e t 9 m
v v v v
v v v v
v v v v
v v v v
w w w w
x x x x
y y y y
z z z z
v v v v v
v v v v v
v v v v v
v v v v v
w w w w w
x x x x x
y y y y y
z z z z z
Bottom field 0 0 0 0 a p 5 i 0 0 0 0 a p 5 i 0 0 0 0 a p 5 i 0 0 0 0 a p 5 i 1 1 1 1 b q 6 j 2 2 2 2 c r 7 k 3 3 3 3 d s 8 l 4 4 4 4 e t 9 m
v v v v
v v v v
v v v v
v v v v
w w w w
x x x x
y y y y
z z z z
27
OMAP
VPSS Front End Flow Diagram DM644x, DM643x
28
OMAP
VPSS Front End Flow Diagram DM3xx
29
OMAP
OMAP35xx Front End Flow Diagram

To DDR SDRAM For YUV/JPEG CM or Ext. ISP
Bypass Mux
Resizer
Back-end Pipeline Frontend Pipeline
Central Resource
Imager Input
3A
Histogram Statistics Collection

30
OMAP
Transport Stream Interface (TSIF) DM646x

Input TS stream Parallel or Serial I/F Receive I/F TS header Parser AF Parser Payload Parser Write Ring Buffer (x8) PID filter table DMA I/F ping-pong buffer
Consequential I/F Input TS Stream (PCI or HPI) Output TS stream Parallel or Serial I/F Transmit I/F
PCR (output to CRGEN) ATS generator
Format Conv.
Read Ring Buffer (x1) ping-pong buffer
ATS checker
DDR2
31
OMAP
Defect Pixel Correction DM3xx

General Defect Correction (up to 1024 pixels)
Defect pixels can be substituted by using left, right or average of left and right pixels in the same color position.
Vertical Line Defect Correction (up to 16 lines)

Mode0: Defect level subtraction (fed through if pixel is saturated) Mode1: Defect level subtraction (horizontal interpolation if pixel is saturated) Mode2: Horizontal interpolation Defect Correction above of defect pixels can be skipped. Different defect level can be set to Above, On and Below defect pixels.
32
OMAP
Lens Shading Correction DM3xx

Corrected pixel = Original pixel x Gain Factor Gain Factor is calculated based on look-up table and the address of the table is determined by Optical center 1. Horizontal distance between Equi-gain line the optical center and the pixel
2. Vertical distance between the optical center and the pixel 3. Horizontal gain factor One of two values is chosen depending on the horizontal location of the pixel compared to the center pixel. 4. Vertical gain factor One of two values is chosen depending on the vertical location of the pixel compared to the center pixel.
3 different mode to determine Gain Factor from the Look-up table, and 3 tables for R, G and B.
33
OMAP
Why compression?
Video Capture Device Driver Compression Store Transmit Decompression Video Display Device Driver
Without it
Format
30 frames/s, 4:2:0
Storage (90 min.) 83.7 GBytes 23.3 GBytes
Transmission ~15.5 Mbytes/s (124.4 Mbits/s) ~4.5 Mbytes/s (36.5 Mbits/s)
D1 (720x480) CIF (352x288)
A movie wont fit on a CD (800 MBytes) or a DVD (4.7 GBytes) and it cant be streamed over ADSL (384 Kbits/s 1.5Mbits/s) or common ethernet (10-100 Mbits/s)
34
OMAP
Image Compression
JPEG (Joint Photographic Experts Group)
Best known standard is IS 10918-1 (ITU-T T.81) JPEG committee reported to 3 international standards organizations (ISO/ITU/IEC)
JPEG used in variety of applications

Printing Digital Cameras Video editing systems Security Medical Imaging
Typically 10:1 compression Still Image coding technique to remove spatial redundancy Block-based DCT, Huffman Coding, Perceptual Quantization Extensions for lossless, progressive coding Initially aimed at Monochrome
Separate compression of components in color image
Baseline, Progressive, Hierarchical, Lossless modes

35
OMAP
Motion JPEG
M-JPEG or Motion JPEG
Commonly used term for application of JPEG to motion video sequences Not really covered by JPEG time standard, but useful tool for compressing motion sequences (prior to arrival of MPEG)
Each Frame coded as a JPEG image

Usually with an adaptive encoding to fix no. bits/frame
Frames Transmitted / Stored Sequentially Used when each individual frame needs to be independently decoded
Security Applications Basis of most Non-Linear Editors
Non-Standardized
MPEG can be used in JPEG style in controlled environment Interchange etc.
36
OMAP
MPEG (Moving Picture Experts Group)

Motion JPEG
No advantage of temporal redundancy (i.e. between frames)
MPEG uses Motion Prediction/Compensation to exploit temporal redundancy Frame Types

Intra-Frame similar to JPEG or MJPEG frames Inter-Frame re-use redundant information between frames
37
OMAP
Temporal Redundancy
x y
time
High correlation between regions A, B and C produces high degree of temporal redundancy
38
OMAP
MPEG-2
The standard for Digital TV (HDTV, DVD) Most successful of the MPEG standards Typical output rate 2-6 Mbps (40:1 compression) To MPEG-1 it adds support for: BT.601 format (720 x 480), YUV, 30 frames/sec Interlaced video Multi-Channel Audio Motion Film (Pan/Scan, 3:2 Pulldown) Encoder is more complex than the decoder
39
OMAP
MPEG-4 Video Compression

Based on MPEG-2 Includes Bit Rates Below 64kbps Standardized Scene Description Parametric Descriptions of Human Face and Body Resynchronization/Data Recovery Tools Media Objects to represent aural, visual, or audiovisual content
Organized in Hierarchical Fashion Random Access to Objects Objects have Spatial and Temporal Scalability
40
OMAP
H.264: Baseline & High Profiles

Baseline Common Features
Arbitrary Slice Ordering Flexible Macroblock Ordering Redundant Slices Motion Prediction: 7 block sizes, sample accuracy, multiple reference frames Intra Prediction: 17 modes Reversible Transform & Non-uniform Quantization Universal & Context Adaptive VLC (UVLC/CAVLC) Loop (de-blocking) Filter
High
B pictures: several prediction modes Context Adaptive Binary Arithmetic Coding (CABAC) Weighted Prediction Adaptive Frame/Field Coding
Extended: adds support for streaming

41
OMAP
Compression Ratio Rules of Thumb

Compression ratios to maintain excellent quality:
10:1 for general images using JPEG 30:1 for general video using H.263 and MPEG-2 50:1 for general video using H.264 / MPEG-4 AVC
Emerging standards offer further improvements (e.g., JPEG2000 at 40:1 generally looks much better than JPEG at 40:1, and H.264 / MPEG-4 AVC produces excellent quality at 60:1 for some types of video content) Application-specific conditions can enable much higher compression ratios (e.g., 1000s:1 when nothing is moving) Event-based methods that selectively identify certain important images to keep while discarding others
42
OMAP
ARM Cortex A8 Subsystem

Up to 600 MHz ARMv7 Architecture
Thumb-2 MMU Enhancements

In-Order, Dual-Issue, Superscalar Microprocessor Core NEON Multimedia Architecture
Over 2x Performance of ARMv6 SIMD Supports Both Integer and Floating Point SIMD
Jazelle RCT Execution Environment Architecture Dynamic Branch Prediction with Branch Target Address Cache (95% accurate across industry benchmarks), Global History Buffer, and 8-Entry Return Stack Embedded Trace Macrocell (ETM) Support for Non-Invasive Debug ARM Cortex-A8 Memory Architecture:
16K-Byte Instruction Cache (4-WaySetAssociative) 16K-Byte Data Cache (4-Way Set-Associative) 256K-Byte L2 Cache
Big Endian
43
OMAP
Embedded Processor Feature Comparison Chart

OMAP 3530/3525 TI DSP C64x+ (IVA) 430 MHz Cortex-A8 600 MHz Yes, Flexible,IVA DM355 N/A DM6467 C64x+ 600 MHz ARM926 300 MHz Yes, Flexible HD, HDVICP DM644x C64x+ 600 MHz ARM926 300 MHz Yes, Flexible, VICP DM643x C64x+ 700 MHz N/A DM648/ DM647 C64x+ 900 MHz N/A
ARM
ARM926 270 MHz MPEG-4 encode or decode JPEG
Video Accelerator
N/A
N/A
Graphics Accelerator
3530: SGX graphics engine 3525: N/A
N/A
N/A
N/A
N/A
Hardware Accelerators, Video Interface
VPSS( VPFE, VPBE)
VPSS (VPFE, VPBE)
VPIF, VDCE
VPSS (VPFE, VPBE)
VPSS (VPFE, VPBE)
N/A
44
OMAP

TI Embedded Processors and Video Technology

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

TI Embedded Processors and Video Technology

Загружено:

Авторское право:

Доступные форматы

TI Embedded Processors and Video Technology

OMAP35xx /DM355 / DM646x /DM643x / DM644x Video-Specific Core IP

VPBE and video display or output

Accelerating Your Success

Video standards and concepts

Accelerating Your Success

Various Standard Resolutions

Standard Definition (SDTV) High Definition (HDTV)

4CIF CIF QCIF

Often used in Video Conferencing or for small screen applications

Accelerating Your Success

NTSC, PAL, and CIF Formats

Accelerating Your Success

TEXAS INSTRUMENTS TECHNOLOGY

Accelerating Your Success

Dimensions: 1. Height 2. Width

Red: 8 bits Green: 8 bits Blue: 8 bits

Accelerating Your Success

Accelerating Your Success

Accelerating Your Success

Accelerating Your Success

Video Input /Capture

Digital Video Overview BT.656

Clock (27Mhz Typ)

8 bit data bus

Accelerating Your Success

Accelerating Your Success

Digital Video Overview - RAW

Line of Data OMAP

Accelerating Your Success

Video Port/Transport Stream Modules - DM6467

Documentation VPIF Users Guide http://focus.ti.com/lit/ug/spruer9/spruer9.pdf TSIF Users Guide http://focus.ti.com/lit/ug/sprueq2a/sprueq2a.pdf

Accelerating Your Success

VPSS Front End CCDC DM644x, DM643x, DM3xx, OMAP35x

Previewer, H3A, External Memory

Accelerating Your Success

Camera Interface OMAP35x

Image Sensor Interface (Parallel)

Video processing/Image Processing

Accelerating Your Success

Camera Interface Sync Mode OMAP35x

Start and End of a horizontal line

Camera ISP module

Cam_vs Cam_pclk Cam_d[11:0]

Generic camera sensor

Start and End of a frame

Accelerating Your Success

Camera Interface BT656 OMAP 35x

Camera ISP module

BT656 camera sensor

Accelerating Your Success

Video Data Conversion Engine DM6467

2-bit Menu Overlay for simple menus and text overlays

Accelerating Your Success

Y0 Cb0 Y1 Cr0 Yn Cbn Yn+1 Crn

Accelerating Your Success

VPSS Front End Resizer DM644x, DM643x, DM3xx, OMAP35x

Accelerating Your Success

VPSS Front End H3A/Histrogram- DM644x, DM643x, DM3xx, OMAP35x

Pixel 2-D block of Pixels Pixel size is user definable

Accelerating Your Success

Range Mapping - DM646x

Equation for Range Mapping of VC-1: