1

Bitmapped Images
Created : Rawesak Tanawongsuwan itrtw@mahidol.ac.th Modified : Damras Wongsawang itdws@mahidol.ac.th

Bitmapped Images
Also known as raster graphics Record a value for every pixel in the image Often created from an external source
Scanner, digital camera,

Painting programs allow direct creation of images with analogues of natural media, brushes,

Resolution
A measure of how finely a device approximates continuous images using finite pixels (density of dots or pixels)
299

755

Device Resolution
Printers, scanners: specify as dots per unit length, often dots per inch (dpi)
Desktop printer 600 dpi, typesetter 1270 dpi, scanner 300 3600 dpi,

Video, monitors: specify as pixel dimensions (pixel per inch - ppi)

PAL TV 768x576 px, NTSC TV 640x480 px, 17" CRT monitor 1024x768 px, dpi depends on physical size of screen

Image Resolution

Array of pixels has pixel dimensions, but no physical dimensions By default, displayed size depends on resolution (dpi) of output device
physical dimension = pixel dimension image resolution

Can store image resolution (ppi) in image file to maintain image's original size
Scale by displaying device resolution original device resolution

Resolution Calculation
128-pixel line displayed 6 x 4 inches image scanned at 600 dpi at 72 dpi 45 mm 3600 x 2400 pixels at 115 dpi 28 mm Displayed at 72 dpi at 600 dpi 5 mm 50 x 33.3 To make it appear at 6 x 4 inches must be scaled by 72/600 = 0.12

Changing Resolution
Changing Resolution can be done by resampling Reducing the pixel dimensions is called downsampling ; increasing them is called upsampling . Both can lead to a loss of quality.

10

Image Scaling
Scaling can be done either by applying a transformation to each original pixel or by applying the inverse transformation to each pixel in the scaled image. Interpolation is needed because of the finite size of pixels.

11

Trouble in scaling-up

12

Forward mapping: x = s*x, y = s*y Reverse mapping : x = x /s, y = y /s

Interpolation
Nearest neighbour

13

Use value of pixel whose centre is closest in the original image to real coordinates of ideal interpolated pixel Bilinear interpolation Use value of all four adjacent pixels, weighted by intersection with target pixel Bicubic interpolation Use values of all 16 adjacent pixels, weighted using cubic splines Nearest-neighbour interpolation is quickest but produces poor-quality results; bicubic is slowest but produces very good results; bilinear is in between.

14

15

16

bilinear Nearest Neighbor

bicubic

17

Other Ideas

Cropping Super-resolution Big Image (Gigapixel Image) Content Aware Image Resizing (Seam Carving)

18

Compression
Image files may be too big for network transmission, even at low resolutions Use more sophisticated data representation or discard information to reduce data size Effectiveness of compression will depend on actual image data For any compression scheme, there will always be some data for which 'compressed' version is actually bigger than the original

19

Lossless Compression
Always possible to decompress compressed data and obtain an exact copy of the original uncompressed data
Data is just more efficiently arranged, none is discarded

Run-length encoding (RLE) Huffman coding Dictionary-based schemes LZ77, LZ78, LZW (LZW used in GIF, licence fee charged)

Run-length encoding (RLE)

20

RLE is an easy compression algorithm to understand It replaces sequences of the same data values within a file by a count number and a single value. Suppose the following string of data (17 bytes) has to be compressed:

ABBBBBBBBBCDEEEEF
Using RLE compression, the compressed file takes up 10 bytes and could look like this:

A *8B C D *4E F

21

RLE

Huffman code
Another use of binary trees Binary trees are not only for searching alone (binary search trees) Speedup the sending process over the Internet or other communication medium Efficient in compressing text or program files Images are sometimes better handled by other compression algorithms

22

Character codes
ASCII code

23

Characters are represented by 8 bits (1 byte) There are 256 possible values Every character requires the same number of bits (8 bits) Example:
A B C 65 66 67 01000000 01000001 01000010

Compressing text
SUSIE SAYS IT IS EASY
Character A E I S T U Y Space Linefeed frequency 2 2 3 6 1 1 2 4 1

24

If using the same number of bits for each character, then the number of bits required to represent N-character texts are constant Goal: Try to reduce the number of bits used to represent text to reduce the amount of data A method: represent the most used characters with the fewest bits as possible

Create the Huffman code

Make a Node object for each character used in the message Make a tree object for each of these nodes The node becomes the root of the tree Insert these trees in a priority queue. (order by frequency)
1 LF 1 U 1 T 2 Y 2 E 2 A 3 I 4 SP 6 S

25

Create the Huffman code

Remove two trees from the priority queue and make them into children of a new node The new node has a frequency which is the sum of the children s frequencies
1 LF 2 1 LF 1 U 1 U 1 T 1 T 2 Y 2 Y 2 E 2 E 2 A 2 A 3 I 3 I 4 SP 4 SP 6 S 6 S

26

27

Create the Huffman code

Insert this new tree back into the priority queue Keep repeating the steps above until there is only one tree left in the queue
1 T 1 LF 2 Y 2 E 2 1 U 2 A 1 T 1 LF 3 2 1 U 3 I 4 SP 6 S 2 Y 2 E 2 A 3 I 4 SP 6 S

28

Create the Huffman code

22

9 4 SP 2 A 1 T 1 LF 5

13 6 S 3 2 1 U 3 I 2 Y 7 4

2 E

Character A E I S T U Y Space Linefeed

frequency 2 2 3 6 1 1 2 4 1

Code 010 1111 110 10 0110 01111 1110 00 01110

29

S U S I E Sp S A Y S 10 01111 10 110 1111 00 10 010 1110 10

30

Decoding the Huffman code

10 01111 10 110 1111 00 10 010 1110 10 S U S I E Sp S A Y S
0 1
SP S

0
A T LF U I E

31

Dictionary-based Coding
LZW uses fixed-length codewords to represent variable-length strings of symbols/characters that commonly occur together, e.g., words in English text The LZW encoder and decoder build up the same dictionary dynamically while receiving the data LZW places longer and longer repeated entries into a dictionary, and then emits the code for an element, rather than the string itself, if the element has already been placed in the dictionary

ALGORITHM LZW Compression

32

LZW compression for string ABABBABCABABBA

Let s start with a very simple dictionary (also referred to as a string table ), initially containing only 3 characters, with codes as follows: Now if the input string is ABABBABCABABBA , the LZW compression algorithm works as follows: The output codes are 1 2 4 5 2 3 4 6 1. Instead of sending 14 characters, only 9 codes need to be sent (compression ratio = 14/9 = 1.56)

33

LZW Decompression (simple version)

34

Input codes to the decoder are 1 2 4 5 2 3 4 6 1 LZW decompression output string ABABBABCABABBA The initial string table is identical to what is used by the encoder.

LZW Decompression (simple version) 35

Apparently, the output string is ABABBABCABABBA , a truly lossless result

JPEG (Join Photographic Experts Group)

36

A photo of a flower compressed with successively higher compression ratios from left to right. http://en.wikipedia.org/wiki/Jpeg

Materials on the JPEG topic are obtained from fundamentals of Multimedia by Ze-Nian Li and Mark S. Drew http://en.wikipedia.org/wiki/Jpeg Digital multimedia text book

37

JPEG Compression
Lossy technique, well suited to photographs, images with fine detail and continuous tones Consider image as a spatially varying signal that can be analyzed in the frequency domain Experimental fact: people do not perceive the effect of high frequencies in images very accurately Hence, high frequency information can be discarded without perceptible loss of quality

38

Overview of JPEG compression algorithm

http://stargate.ecn.purdue.edu/~ips/tutorials/jpeg/jpegtut1.html

JPEG encoding process

YUV and subsample color

39

Transform RGB to YIQ or DCT on image blocks Quantization Zig-zag ordering and
run-length encoding

Entropy coding

40

Color Space Transformation

The image is converted from RGB into a different color space called YUV The Y component represents the brightness of a pixel, and the U and V components together represent the hue and saturation This part is useful because the human eye can see more detail in the Y component than in the others

Discrete Cosine Transform (DCT)

41

Similar to Fourier Transform, analyses a signal into its frequency components Takes array of pixel values, produces an array of coefficients of frequency components in the image Computationally expensive process time proportional to square of number of pixels
Apply to 8x8 blocks of pixels Using blocks, however, has the effect of isolating each block from its neighboring context. This is why JPEG images look choppy blocky when a high compression ratio is specified by the user

42

Definition of DCT
Given an input function f(i; j) over two integer variables i and j (a piece of an image), the 2D DCT transforms it into a new function F(u; v), with integer u and v running over the same range as i and j. The general definition of the transform is:

2C(u)C(v) M 1 N 1 (2i 1) uT (2 j 1) vT F (u, v) ! cos 2M cos 2N f (i, j) MN i!0 j !0

where i; u = 0;1; : : : ; M 1; j; v = 0;1; : : : ; N 1; and the constants C(u) and C(v) are determined by
2 C (\ ) ! 2 1 if \ ! 0 otherwise

43

2D DCT & IDCT

C(u)C(v) 7 7 (2i 1) uT (2 j 1) vT F (u, v) ! cos 16 cos 16 f (i, j) 4 i !0 j !0

7 7 ~ C(u)C(v) (2i 1) uT (2 j 1) vT f (i, j) ! cos cos F(u, v) 4 16 16 u!0 v!0

DCT (example)
If one such 8x8 8-bit subimage is:

44

and then taking the DCT results in

45

Quantization
The human eye is fairly good at seeing small differences in brightness over a relatively large area but not so good at distinguishing the exact strength of a high frequency brightness variation So we can get away with greatly reducing the amount of information in the high frequency components. Divide each component in the frequency domain by a constant for that component, and then round to the nearest integer (u , v )
(u , v ) ! round Q (u , v )

This is the main lossy operation in the whole process.

46

JPEG Quantization
Applying DCT does not reduce data size
Array of coefficients is same size as array of pixels

Allows information about high frequency components to be identified and discarded Use fewer bits (distinguish fewer different values) for higher frequency components Number of levels for each frequency coefficient may be specified separately in a quantization matrix

 F (u , v ) ~ F (u , v ) ! round (u , v ) The entries of Q(u,v) tend to have larger values towards the lower right corner. This aims to introduce more loss at the higher spatial frequencies The tables below show the default Q(u,v) values obtained from psychophysical studies with the goal of maximizing the compression ratio while minimizing perceptual losses in JPEG images.
New result after quantization

Quantization

47

Example detail (1)

48

DCT

Example detail (1)

Q DQ

49

IDCT

Diff

Example detail (2)

50

Example detail (2)

51

JPEG Encoding

52

After quantization, there will be many zero coefficients

Use RLE on zig-zag sequence (maximizes runs) Use Huffman coding of other coefficients (best use of available bits)

53

JPEG Decompression
Expand runs of zeros and decompress Huffmanencoded coefficients to reconstruct array of frequency coefficients Use Inverse Discrete Cosine Transform (IDCT) to take data back from frequency to spatial domain Data discarded in quantization step of compression procedure cannot be recovered Reconstructed image is an approximation (usually very good) to the original image

Decoding process

54

Multiply by quantization matrix

Apply IDCT

55

Compression Artifacts
If use low quality setting (i.e. coarser quantization), boundaries between 8x8 blocks become visible If image has sharp edges, these become blurred
Rarely a problem with photographic images, but especially bad with text Better to use good lossless method with text or computer-generated images

Example

56

JPEG2000

57

JPEG2000 improves on JPEG in many areas, including image quality at high compression ratios. It can be used losslessly as well as lossily. For JPEG2000 compression the image is divided into tiles, but these can be any size, up to the entire image. Take advantage of a Discrete Wavelet Transform (DWT) Encode by using Arithmetic Coding

58

File Formats
GIF (Graphics Interchange Format)

59

GIF files use LZW compression ( lossless) Restricted to 256 colours (indexed colour) One colour may be used to designate transparency. They are most suitable for simple images with areas of flat colour (cartoon-style drawing, animation)

60

61

PNG (Portable Network Graphics)

PNG was developed to supersede GIF because at that time LZW required licence fee It uses deflate compression (LZ77 + Hoffman Coding, lossless) Not restricted to 256 colours Supports alpha channels for partial transparency and special effects

62

JPEG Format
JPEG data can be stored in several different formats. JFIF and SPIFF are compatible formats for JPEG images and are widely used on the Web. Exif can hold either JPEG or TIFF data, together with extensive metadata. JP2 and JPX formats are defined for storing JPEG2000 data.

63

TIFF (Tag Image File Format)

TIFF is an extensible format Store full-colour bitmaps Uses several different compressions, LZW+JPEG, or may be no compression Often used for storing uncompressed digital photographs, and for interchange of images.

64

BMP (MS Windows Bitmap)

BMP is a simple bitmapped image format that is native to Windows, but widely supported. BMP files are often uncompressed Platform-dependent

65

PDF (Portable Document Format)

PDF documents can include bitmapped image data, that may be compressed using JPEG, JPEG2000, LZW, deflate, and others.

66

Raw Formats
Camera raw data is used when complete control over image processing is required There is no standard format for camera raw data. Adobe s DNG (Digital Negative) format is a standard, based on the TIFF format, intended for archiving camera raw images.

67

A Raw File
A record of the data captured by the camera sensor Unprocessed sensor data

68

Raw Image Files

File extension:
.raf(Fuji) .crw, .cr2 (Canon) .kdc, .dcr(Kodak) .mrw(Minolta) .nef(Nikon) .orf(Olympus) .dng(Adobe) .ptx, .pef(Pentax) .arw, .srf(Sony) .x3f (Sigma) .erf(Epson)

69

Digital Camera
Most digital cameras sample an image with red, green and blue sensors arranged in a photoreceptor array

70

Digital Camera
The purpose of the microlens array is to focus the light onto each pixel The color filter array changes the color responsivity of each pixel Finally, the sensor array captures the light and generates the electrical signal

Mosaic Sensor / Color Filter Array (CFA) Camera

A 2D array to collect the photons that are recorded in the image Made up of rows and columns of photosensitive detectors (CCD or CMOS) to form an image Each element of the array contributes on pixel to the final image

71

72

Color Filter Array (CFA)

The sensors count photons, produce a charge that s directly proportional to the amount of light that strikes them The raw files from color filter array cameras are grayscale Grayscale to Color
Each element in the array is covered by a color filter, so that each element captures only red, green, or blue light

73

A Bayer filter pattern

A mosaic of red, blue and green filters in alternating rows of RG and GB Twice as many green filters are used as red or blue because our eyes are most sensitive to green light

74

Filter patterns
RGB or CMY or 4-color-mix are possible Each element in the sensor captures only one color The red-filtered elements produce a grayscale value proportional to the amount of red light reaching the sensor Same as the green-filtered and blue filtered elements

75

Info. in the raw files

The image pixels themselves The image metadata ( data about data )
Records shooting data such as the camera model, serial number, shutter speed, aperture, focal length, flash Additional information that might be needed to convert into an RGB image

76

Raw Converter

77

Demosaicing
To display the image, we must create an image that has a red, green and blue pixel at each location Interpolate the missing sensor values

78

Demosaicing Illustration

This is the original image, made with Adobe Illustrator

79

Demosaicing Illustration

A simulated sampling taken by a Bayer filtered sensor array Each pixel only has a value of either R or G or B

80

Demosaicing Illustration
A zoom-in version

81

Demosaicing Illustration

An example reconstruction

82

Demosaicing Illustration
A zoom-in version

original

reconstruct

Demosaicing Algorithms
Nearest Neighbor Replication Simple interpolation
Bilinear, Bicubic, Spline, Laplacian interpolation

83

simply copies an adjacent pixel of the correct color component

Synthetic field based interpolation

Compute an alternate representation Hue interpolation, Log hue interpolation

Adaptive
Adapt their methods of estimation depending on features of the area surrounding the pixel of interest

Proprietary
Commercial products

84

Raw Conversion
In addition to demosaicing
White balance Colorimetric interpretation assigns the correct, specific color meanings to the red , green , and blue pixels, usually in a colorimetrically defined color space such as CIE XYZ, which is based directly on human color perception Gamma correction is done to redistribute the tonal information so that it corresponds more closely to the way our eyes see light and shade Noise reduction, anti-aliasing, and sharpening

Benefits
Finer control is easier for the settings

85

For example, the white point can be set to any value, not just discrete values like "daylight" or "incandescent"

The settings can be previewed and tweaked to obtain the best quality image or desired effect Camera raw files have 12 or 14 bits of brightness information JPEG loses fine details and is ill-suited for major color or brightness changes The working color space can be set to whatever is desired Different demosaicing algorithms can be used, not just the one coded into the camera

Drawbacks

86

Camera raw files are typically 2-6 times larger than JPEG Fewer images can fit on a given memory card It also takes longer for the camera to write raw images to the card
fewer pictures can be taken in quick succession (a sports sequence)

No single widely-accepted standard raw format

Adobe's DNG format has been put forward as a standard, but is not adopted by major camera companies

Specific software may be required to open raw files on some systems, as opposed to JPEG or TIFF Time taken in the image workflow

Software Supports
Dcraw Adobe Photoshop / Adobe Photoshop Lightroom Microsoft's Digital Image Mac OS X

87

added raw support directly to the system which adds raw support automatically to the majority of Mac OS X applications (such as Preview, Mac OS X's PDF and image viewing application)

Helicon Filter Bibble Pro Picasa

a free image editing and cataloging program from Google, but only limited tools for RAW processing

UFRaw is free software based on dcraw

88

References
Adobe.com Wikipedia.org http://www.imageval.com/public/Produ cts/ISET/ISET_Manual/Demosaicing.htm http://photo.net/learn/raw/ http://www.cambridgeincolour.com/tut orials/RAW-file-format.htm

89

Image Manipulation
Image manipulation software provides high level operations for systematically altering pixels Most operations are described by analogy with traditional photographic techniques, such as the use of masks and filters. Bitmapped images are manipulated to correct technical deficiencies, alter the content (retouch) or create artificial compositions.

90

Image Editing Software

Photoshop is the de facto industry standard; The Gimp is an Open Source alternative. Image Magick can be used for command-line processing.

91

Compositing
The process of assembling multiple images to make a final image. Images are often organized into layers, which are like overlaid sheets that may have transparent areas. Layers are used for compositing or experimenting with different versions of an image.

92

93

Selection
Areas may be selected by drawing with
marquee : geometrical shape lasso tools or Bzier pen : free hand magic wand : selected on the basis of colour similarity magnetic lasso : selected on the basis of edges

94

(Selection based on color)

95

(Selection based on edge detection)

96

Mask & Alpha Channel

Any selection defines a mask the area that is not selected. Masked areas of the image are protected from changes (all or nothing). A greyscale mask, which is partially transparent, is an alpha channel. An alpha channel can be associated with a layer as a layer mask, and used for effects such as knock-outs and vignettes.

97

(Knock-out)

98

p = Ep1 + (1-E)p2

99

Pixel Point Processing

In pixel point processing, each pixel s new value depends only on its old value

p = f(p)
Brightness, contrast and levels are relatively crude pixel point adjustments.

Brightness (increase B&W)

100

(Original image)

(increase difference between B&W)

101

102

Levels adjustment
(Adjust B&W independently)

103

Curve Adjustments
Curves adjustments provide full control over the relationship between original and new values. A sigmoid curve is often used to enhance contrast.

104

Increase white

fixed
Increase black

105

106

Pixel Group Processing

Pixel group processing uses the values of neighbouring pixels as well. The convolution operation in the frequency domain can be implemented as a weighted average in the spatial domain: for each pixel in the filtered image, the pixels of a convolution kernel are combined using a convolution mask.

107

108

Gaussian Filter
Simple blurring uses a 3 x 3 mask with equal values but produces crude results. Gaussian blur is preferred, as it produces more natural results.

109

110

Sharpening
Sharpening with a 3x3 mask is crude. Unsharp masking combining an image with a Gaussian blurred copy of itself produces better-looking results. Over-sharpening should always be avoided.

111

112

113

Image Editing and Manipulation

Image Inpainting / Image Completion Texture Synthesis Shape manipulation . . .