Two Image-Template Operations for Binary Image Processing

Hongchi Shi Department of Computer Engineering and Computer Science Engineering Building West, Room 331 University of Missouri - Columbia Columbia, MO 65211 Phone: (573)882-3088 Fax: (573)882-0397 Email: shi@ece.missouri.edu

Two Image-Template Operations for Binary Image Processing

HONGCHI SHI Department of Computer Engineering and Computer Science, University of Missouri - Columbia, Columbia, MO 65211

Abstract This paper presents two new image algebra image-template operations match and mismatch derived from the general image-template product. These image algebra operations extend the binary morphological erosion and dilation operations and can be used to express elegantly most of binary image processing algorithms in a more natural way than binary morphological operations from the image processing viewpoint. In addition, the match and mismatch operations are easy to implement e ciently on SIMD bit-serial parallel computers.

Keywords image algebra, mathematical morphology, binary image processing

1. Introduction
Binary image processing involves manipulating binary images consisting of only 1-pixels and 0-pixels. Binary image processing techniques are employed by binary machine vision systems in the detection and recognition of objects or object defects 3]. Binary morphology is an algebraic system concerning with analysis of shapes. It plays an important role in machine vision, since shape is a prime carrier of information in machine vision 3]. Binary morphology can be used to enhance binary images by removing noise, decompose objects, extract object features, and identify objects in binary images 1, 10, 4]. Binary mathematical morphology is based on the set theory. The set of all the 1-pixels in a binary image gives a complete description of the binary image. The primary morphological operations are dilation and erosion. All other morphological operations such as opening, closing, and hit-and-miss operations are based on these two operations. Since binary mathematical morphology primarily deals with sets, one has to consider binary images as sets, perform operations on sets, and convert sets back to images, when applying mor2

phological operations on images. Although this process is not complicated, it involves three steps conceptually. Besides, it can not consider both 1-pixels and 0-pixels at the same time. For example, the hit-and-miss operation needs two structuring elements, one for the 1-pixels and the other for the 0-pixels of the object. Another mathematical theory concerning image processing and analysis, image algebra, de nes images as its primary operands and deals with images directly 9, 6]. Image algebra extends mathematical morphology and it can manipulate image transformations between di erent domains and transformations between di erent value sets which are ignored in morphology-based image algebras. This paper introduces two image algebra operations and demonstrates how they are used to express binary image processing algorithms in a more natural way for binary image processing.

2. Match and Mismatch Operations

Image algebra is a heterogeneous algebra concerned with image processing and analysis. In image algebra, basic objects in image processing such as images, point sets, and templates are formally de ned as its operands. Most of image processing and analysis algorithms can be concisely expressed using image algebra 8]. A complete description of image algebra can be found in Ritter's book 7]. Images in image algebra are de ned in terms of two other types of elementary operands: value sets and point sets. Given a point set X and a value set F , an F -valued image a on X is a function a : X ! F , which is usually expressed in terms of the graph of a by

a = f(x; a(x)) : x 2 Xg;

where a(x) 2 F . An element (x; a(x)) of a is a pixel. The set of all F -valued images on X is denoted by F X. In image algebra, the de nition of a template uni es and generalizes the usual concepts of templates, masks, windows, structuring elements, and neighborhood functions into one general mathematical entity. Templates are special types of images. An F -valued template from Y to X is an element of (F X)Y . If t 2 (F X )Y , then for notational convenience we de ne ty t(y) in order to denote the image ty for each y 2 Y. The pixel values ty (x) of the image ty = f(x; ty(x)) : x 2 Xg 3

are the weights of the template t at the target point y. Let X Rn . A template t 2 (F X)X is a translation invariant template if and only if for x; y 2 X with x + z; y + z 2 X, where z 2 Rn , we have ty (x) = ty+z (x + z). Translation invariant templates can be de ned pictorially. Figure 1 shows a translation invariant template. Image algebra de nes many operations between images and between images and templates. In terms of image processing, image-template operations that combine images and templates are the most powerful tools of image algebra. They combine images and templates by using appropriate binary operations. They may be used for transformations between di erent domains and transformations between di erent value sets. Suppose that F1, F2, and F are value sets. The operation : F1 F2 ! F is a binary operation and the operation : F F ! F is an associative and X X commutative binary operation on F . If a 2 F1 and t 2 (F2 )Y , then the generalized product of a with t is the binary operation : F1X (F2X)Y ! F Y de ned by

a t = f(y; b(y)) : b(y) = ?x2Xa(x) ty (x); y 2 Yg:

We may derive di erent image-template operations by substituting appropriate operations for and in the de nition of the generalized image-template product. Ritter has de ned the most commonly used image-template operations in image processing such as + , 2 , 2 , _ , and ^ 7]. _ ^ Here, we derive two image-template operations mismatch and match for binary image processing. ~ ~ ~ First, we de ne two logic operations + and +0 . The operation + is de ned as the exclusive-or ~ ~ operation, i.e., a+b = 1 if and only if a and b are di erent. The operation +0 is de ned as the ~ ~ complement of the exclusive-or operation, i.e., a+0b = a+b. ~ ~ The mismatch operation 2 is obtained by substituting the logic or _ and the operation + _ for and . Speci cally, ~ a 2 t = f(y; b(y)) : b(y) = _

x2X

_ a(x)+t (x); y 2 Yg: ~

y

~ ~ The match operation 2 is obtained by substituting the logic and ^ and the operation +0 for ^ and . Speci cally, ~ a 2 t = f(y; b(y)) : b(y) = ^ 4

x2X

^ a(x)+0t (x); y 2 Yg: ~

y

Most of binary image processing algorithms require only translation invariant templates. Thus, a template for binary image processing usually can be de ned by a con guration of the neighborhood of each target pixel. For example, the template t in Figure 1 speci es the con guration of the 3 3 neighborhood of each target pixel.

Figure 1 goes here.

For the mismatch operation, each target pixel in the result image obtains 1 if and only if the con guration of its neighborhood does not match the con guration given by the template. For the match operation, each target pixel gets 1 if and only if the con guration of its neighborhood matches the con guration given by the template. The match and mismatch operations are bit-serial operations. They usually use information from a small neighborhood of each pixel to check if it matches/mismatches the con guration speci ed by the template. They are perfectly suited for implementation on bit-serial SIMD meshconnected computers such as Lockheed-Martin's CISP computer 11].

3. Binary Image Processing Using Match and Mismatch

Haralick and Shapiro present many applications of binary morphological operations such as erosion, dilation, and hit-and-miss to binary image processing 3]. The two basic binary morphological operations erosion and dilation can be expressed using match and mismatch, respectively. The operations given by match and mismatch are more natural to image processing researchers. The useful morphological operation hit-and-miss can be expressed using match with a single template specifying the foreground (1-pixels) and background (0-pixels) of the object to be matched.

3.1. Erosion and Dilation

In morphology, the erosion of a set A E N by another set B E N , denoted by A B , is de ned by A B = fy 2 E N : y + xB 2 A; 8xB 2 Bg: 5

This can also be expressed as follows:

A B = fy 2 E N : By Ag;
where By is the set of elements of B translated by y. The dilation of a set A E N by another set B E N , denoted by A B , is de ned by

A B = fz 2 E N : y = xA + xB ; 9xA 2 A and 9xB 2 Bg:

This can also be represented as follows:

A B = fy 2 E N : xA 2 By ; 9xA 2 Ag;
where B = fx : ?x 2 B g. For binary image processing, the set A corresponds to a source binary image a with a(x) = 1 if and only if x 2 A, while the set B corresponds to a template t. To achieve the erosion e ect, the template is de ned by ty (x) = 1 if x 2 By . To achieve the dilation e ect, the template t is de ned by ty (x) = 0 if x 2 By . For example, if the structuring element B is as shown in Figure 2(a), the corresponding erosion template is t1 shown in Figure 2(b) and the dilation template is t2 shown in Figure 2(c). Figure 2 goes here. ~ The match operation implements exactly the morphological erosion operation. Let c = a 2 t. ^ V a(x)+0ty (x), which means c(y) = 1 if and ~ By the de nition of the match operation, c(y) = x2By only if all the elements in By are in A. The mismatch operation implements exactly the morphological dilation operation. Let c = ~ ~ a 2 t. By the de nition of the mismatch operation, c(y) = Wx2By a(x)+ty (x), which means _ c(y) = 1 if and only if some element of A is in By . When we use the match and mismatch operations for binary image processing, we work on images directly and design templates specifying what kind of con guration of the neighborhood to match or to avoid, which is more natural. 6

3.2. Hit-and-Miss
The structuring elements in morphological operations can only specify one kind of pixels either 1-pixels or 0-pixels. The match and mismatch operations extend the morphological erosion and dilation operations by allowing the neighborhood to contain 1-pixels and 0-pixels, making some image processing algorithms more natural and concise. The hit-and-miss transformation is a useful operation for selecting pixels that satisfy certain geometric properties 3]. It is represented in morphology using two structuring elements, one for the object to be hit and the other for the object to be missed. Let J and K be two structuring T elements that satisfy J K = ;. The hit-and-miss transformation of set A by (J; K ), denoted by A (J; K ), is de ned by \ A (J; K ) = (A J ) (Ac K ); where Ac is the complement of A. When applied to binary image processing, the hit-and-miss operation gives a pixel value 1 if and only if the pixel's neighborhood of 1's speci ed by J matches J and its neighborhood of 0's speci ed by K matches K . Let a be the binary image corresponding to set A. De ne a template t by ty (x) = 1 if x 2 Jy and ty (x) = 0 if x 2 Ky . The match ~ operation a 2 t performs the hit-and-miss operation with one template specifying the con guration ^ of the neighborhood to be matched. We give three examples to show the match operation gives more natural expression of some binary image processing algorithms.

Finding 8-isolated pixels

An 8-isolated pixel is a 1-pixel with its eight 8-neighbors all 0 as shown in Figure 3.

Figure 3 goes here.

To identify all 8-isolated pixels of an image using the hit-and-miss transform, we need to design two structuring elements J = f(0; 0)g shown in Figure 4(a) and K = f(0; 1); (0; ?1); (1; 0); (?1; 0)g shown in Figure 4(b). Using the match operation, we just need to de ne a template as shown in Figure 4(c). The con guration given by the template directly re ects the de nition of 8-isolated 7

pixels.

Figure 4 goes here.

Identifying upper right-hand corner pixels

An upper right-hand corner pixel can be de ned as a 1-pixel whose south and west neighbors are 1-pixels and whose north, northeast, and east neighbors are 0-pixels as shown in Figure 5.

Figure 5 goes here.

To identify upper right-hand corner pixels using hit-and-miss, we have to de ne two structuring elements J and K as shown in Figure 6. The structuring element J speci es the object to be hit and K speci es the object that has to be missed. Using the match operation, we need to derive only one template directly from the de nition of upper right-hand corner pixels. The template is de ned as shown in Figure 6(c).

Figure 6 goes here.

Pattern matching
In binary image processing, a pattern to be matched in a binary image can be de ned as a smaller binary image. Figure 7 gives a pattern of a diamond with side length 3. A pixel at x in the source image is said to match the pattern exactly if the pattern image translated by x matches the subimage at x in the source image.

Figure 7 goes here.

Using the morphological hit-and-miss operation for pattern matching, we have to design two structuring elements J and K , one for the 1-pixels and the other for the 0-pixels of the pattern. To perform pattern matching using the match operation, we only need a template t which can be derived directly from the pattern to be matched. The structuring elements and the template for matching the pattern de ned in Figure 7 are shown in Figure 8.

Figure 8 goes here.

Pattern matching with tolerance can be done by leaving some 1's out in the template con guration. For example, to match diamond-like patterns with side lengths from 2 to 3 using the match operation, we can de ne a template as shown in Figure 9.

Figure 9 goes here.

3.3. Computing Binary Image Topological Properties

We further demonstrate the applicability of the match and mismatch operations by describing two more binary image processing algorithms using these operations. The algorithms given here compute some topological properties of binary images. We only consider 8-connected components. That is, we use 8-connectivity for 1-pixels and 4-connectivity for 0-pixels.

Shrinking and Counting Components

Levialdi gives an algorithm that counts the components in a binary image by shrinking the components into isolated pixels 5]. The basic idea is to shrink each component to an isolated pixel 9

at one corner of its bounding rectangle and then count the isolated pixels. The bounding rectangle of a component is the smallest upright rectangle containing all the 1-pixels of the component. The shrinking process preserves the component connectivity. Let a 2 f0; 1gX denote the source image, where X is an n n point set. The shrink operator ' that shrinks components toward the upper left corners of their bounding rectangles can be de ned in terms of the con gurations of the 2 2 neighborhood as shown in Figure 10.

Figure 10 goes here.

The operator ' assigns a value to each pixel as follows: If the pixel's neighborhood has the con guration shown in Figure 11(a), it is assigned 0; If its neighborhood has the con guration shown in Figure 11(b), it is assigned 1; Otherwise, it keeps its old value.

Figure 11 goes here.

De ning two templates t1 and t2 shown in Figure 12, the operator ' can be expressed using the match and mismatch operations as follows. Let a = '(a), the image resulted from applying ' to a. Then, ~ ~ a = a ^ (a 2 t1) _ (a 2 t2): ^ _ To count components in a binary image, we apply the shrink operator repeatedly. After each iteration, we use a match operation with the template t de ned in Figure 4. Let a0 be the source P ^ ~ binary image and c0 = a0 2 t, the number of isolated pixels in the source image. De ne ~ ~ ak+1 = ak ^ (ak 2 t1) _ (ak 2 t2) _ ^ 10

and

ck+1 = ck +

Xa

~ ^ k+1 2 t:

The number of components in a0 is given by cK , where K is the smallest integer for which aK = 0.

Figure 12 goes here.

Computing Binary Image Euler Number

Euler number is a topological descriptor for binary images. It is useful in image processing because the topological counts in an image are of intrinsic interest in characterizing and recognizing the image. The Euler number of a binary image is de ned to be the number of connected components minus the number of holes inside the connected components. Gray studied computation of Euler numbers using local information 2]. Given a direction , let X denote a -facing convexity and V denote a -facing concavity as shown in Figure 13. Gray proved that

E = #(X ) + #(V );
where #(X ) is the number of -facing convexities and #(V ) is the number of -facing concavities. Choosing to point to the northwest direction, we have an X only if we have the con guration shown in Figure 14(a). A V can only arise from the con guration shown in Figure 14(b).

Figure 13 goes here.

Figure 14 goes here.

11

Thus, we can design two templates t1 and t2 as shown in Figure 15 and compute the Euler number of a binary image a as follows:

E=

Xa ^ t ? Xa ^ t : ~ ~ 2 2 2 1

Figure 15 goes here.

4. Conclusion
We have derived two image algebra image-template operations match and mismatch. They extend the binary morphological erosion and dilation operations. We have demonstrated how to design templates directly from binary image processing problems. When applied to binary image processing, the match and mismatch operations give more natural algorithms.

Acknowledgements
The author wishes to thank Dr. Gerhard Ritter and Dr. Joseph Wilson for their advice on this work. The author also wishes to thank Dr. Patrick Co eld of Wright Laboratory, Eglin AFB, for his continued support of this research.

References
1] T. R. Crimmins and W. R. Brown. Image algebra and automatic shape recognition. IEEE Transactions on Aerospace and Electronic Systems, 21:60{69, 1985. 2] S. B. Gray. Local properties of binary images in two dimensions. IEEE Transactions on Computers, 20(5):551{561, May 1971. 12

3] R. M. Haralick and L. G. Shapiro. Computer and Robot Vision, volume 1. Addison-Wesley, Reading, MA, 1992. 4] R. M. Haralick, S. R. Sternberg, and X. Zhuang. Image analysis using mathematical morphology. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9:523{550, 1987. 5] S. Levialdi. On shrinking binary picture patterns. Communications of the ACM, 15:7{10, 1972. 6] G. X. Ritter. Recent developments in image algebra. In P. Hawkes, editor, Advances in Electronics and Electron Physics, 80, pages 243{308. Academic Press, New York, NY, 1991. 7] G. X. Ritter. Image algebra, 1996. in preparation. 8] G. X. Ritter and J. N. Wilson. Handbook of Computer Vision Algorithms in Image Algebra. CRC Press, 1996. 9] G. X. Ritter, J. N. Wilson, and J. L. Davidson. Image algebra: An overview. Computer Vision, Graphics, and Image Processing, 49(3):297{331, March 1990. 10] J. Serra. An introduction to mathematical morphology. Computer Vision, Graphics, and Image Processing, 35:283{305, 1986. 11] M. S. Tomassi and R. D. Jackson. An evolving SIMD architecture approach for a changing image processing environment. DSP & Multimedia Technology, pages 1{7, October 1994.

13

1 t = 1 0

1 0

1 0 0

Figure 1: Template t specifying a 3 3 neighborhood con guration

14

1 t 1= 1 1 (a)

1 1 1 (b)

1 1 1

0 t 2= 0 0

0 0 0 (c)

0 0 0

Figure 2: A structuring element and its corresponding templates

15

0 0 0

0 1 0

0 0 0

Figure 3: An 8-isolated pixel in the center

16

K 0 t = 0 0 0 1 0 (c) 0 0 0

(a)

(b)

Figure 4: Structuring elements and template for nding 8-isolated pixels

17

0 1 1 1

0 0

Figure 5: An upper right-hand corner pixel in the center

18

K 0 t = 1 1 1 0 0

(a)

(b)

(c)

Figure 6: Structuring elements and template for identifying upper right-hand corner pixels

19

0 0 0 0 0 0 0

0 0 0 1 0 0 0

0 0 1 1 1 0 0

0 1 1 1 1 1 0

0 0 1 1 1 0 0

0 0 0 1 0 0 0

0 0 0 0 0 0 0

Figure 7: A pattern image

20

(a) 0 0 0 t = 0 0 0 0 0 0 0 1 0 0 0 0 0 1 1 1 0 0 0 1 1 1 1 1 0 (c) 0 0 1 1 1 0 0 0 0 0 1 0 0 0

(b) 0 0 0 0 0 0 0

Figure 8: Structuring elements and template for pattern matching

21

0 0 0 t = 0 0 0 0

0 0 0

0 0

0 1

0 0

0 0 0

0 0 0 0 0 0 0

1 0 0 0

1 1 0

1 0 0 0 0

Figure 9: Template for pattern matching with tolerance

22

Figure 10: Neighborhood for the Levialdi shrinking operator '

23

0 0 0 (a) 1

1 (b)

Figure 11: Neighborhood con gurations for the Levialdi shrinking operator '

24

t 1= 0

0 0

t 2= 1

Figure 12: Templates for the Levialdi shrinking operator '

25

V X V

Figure 13: -facing convexities and concavities

26

1 0

0 0 (a)

* 1

1 0 (b)

Figure 14: Con gurations for convexity and concavity, where is 0 or 1

27

t 1= 1 0

0 0

t 2= 1

1 0

Figure 15: Templates derived from the con gurations for convexity and concavity

28