Unit IV Machine vision (Contd.

)
Image acquisition: The business of a machine vision system can be divided in to two somewhat sequential steps: 1.The gathering of the image into a machine readable form, and 2.The manipulation and analysis of that image to interpret it and accomplish the goals of the user. Image scanning: The first step is to gather input data from the original image or object. A camera lens arrays an image on a glass faceplate that consists of a very fine grid of photo sensitive material. Each tiny spot on the grid is electrically charged in proportion to the light that impinges on it. These tiny spots are the source of the most basic picture element, called pixels. The electrical charge for each pixel is gathered by the system, usually with a scanning electron gun. A typical system uses a raster scan, similar to that employed by a black and white television

camera that gathers pixels to reproduce an entire picture on a CRT screen as shown below.

The quality of the image reproduced is dependent upon the ability of the scanning system to distinguish between two closely separated points, a characteristic called resolution similar to resolution of industrials robots. the horizontal resolution of the scan depicted in the previous slide is the width of the screen divided by 640. The vertical resolution is the height of the screen divided by 480. Another determinant of image quality is contrast. This is the ability of the scanning system to detect shades of difference from pixel to pixel. The lowest quality machine vision system uses a simple two state sensor that registers each pixel as being black or white. Despite its low quality, surprisingly good images have been achieved by systems that are capable of producing only an array of black dots. Moving up the quality scale from simple bi-state contrast system are systems that are able to detect and reproduce four, six or eight shades of gray from black to white for each pixel.

Lighting: Lighting of the subject should be considered an important element of the image acquisition system, and it is mentioned separately because it plays an important part in the success of the application. Machine system can be made more effective by cleverly taking advantage of unusual lighting effects. Lighting system should be dedicated one rather than depending upon ambient light. Dedicated light may consist of point source that enhances sharp features or takes advantage of sharp shadows. For other applications the object may be surrounded with multiple source to eliminate the presence of shadows that may give rise to dark pixels that can be falsely interpreted. Sometimes a silhouette is desired so that edges can be detected, in this case back lighting may be desirable. But if the object has an important feature to be detected on its face, such as hole or a groove, back lighting will be counter productive. Another lighting problem is glare. Black shiny surfaces can be sensed as white owing to the reflection of the light source.

The human eye more easily distinguishes between glare and white background than does a machine vision sensor. Although glare from a black surface is brilliantly white, the human eye(and brain) can recognize the context of the scene and ignore glare much as the human ear (and brain) can interpret speech from context. Because humans take this capability for granted, they sometimes fail to adequately deal with the problem of glare when designing a machine vision system. Digitization: The sensor that gathers the image for a machine vision system is typically an analog device. However the computer vision of the image must be digital. a central problem of automated manufacturing is the gathering of analog signals and converting them to digital approximations for storage and analysis by computer. This calls for the use of A to D converters.

A to D converter application: We return to the idea of gray scale when we refer to digitization. The process of digitization converts an analog signal, which can take on an infinitely variable value, to an integer from 1 to N, where N represents the degree of gray scale recognizable by the system. Invariably N is power of 2 because machine vision systems are designed to store digitized image in binary registers. Typical values are 2,4,8,64, 256, which represents binary registers of lengths 1,2,6,and 8 respectively. The first level (N=2) contains of no gray scale capability at all, because its two levels represent absolute black and absolute white. Still the N=2 system remains popular because of its low cost, its low storage requirements for the digitized image,and its appropriateness for industrial applications of machine vision system. A technology called thresholding helps simplifying an image by reducing it back to N=2 image. Fig in next slide compares the digitization of a 32 pixel scan using N=2,4, and 8, as might be encountered in in raster scan of a machine vision image.

Note that more detail is lost in in the N=2 version, but this version consumes only 33 % as much computer storage space as does N=8 version. One strategy is to retain the gray-level contrast data gathered in the image acquisition phase in order to permit experimentation with various thresholds, or transitions between black and white, in the image analysis phase. Image analysis techniques: Windowing: windowing is a means of concentrating vision system analysis on a small field of view, thereby conserving computer resources of run time and storage. Windowing is an essential first step for virtually every robotic vision system analysis, though we do not think about it, our human vision systems also are constantly performing a windowing process. Although largely taken for granted the human vision system has an amazing facility for looking directly at a small field while maintaining a much larger field within the range of peripheral vision.

The human brain quietly monitors the peripheral field and ignores it unless unexpected motion or something out of place interrupts the attention span and brings the disturbing influence to the full attention of the observer. Machine vision systems are greatly inferior to the human vision system in this regard, but the use of windowing enhanses the capability of the system to emulate the work of a human concentrating upon a small detail. The most practical application of windowing employ fixed windows, that is the window is always set up in the same place within the image. This usually means that some sort of fixturing must be used to identically position every work piece so that consistency of the window subject is maintained. The principles of positioning and orientation are important for machine system effectiveness as well. More sophisticated machine vision system are able to employ adaptive windowing, in which the system is able to select the appropriate window out of the context. In such systems a search of the entire image detects known landmarks that identify the position and orientation of the subject work piece.

The landmarks can then be used by the system to find the window area of interest and proceed as in a fixed window scheme. The advantage is obvious; orientation and positioning become unnecessary, resulting in dramatic savings in production costs. Some robots, equipped with adaptive windowing capability, have experienced success with binpicking, the selection and picking up of work pieces piled randomly in a bin, an easy task for humans, but a very difficult one for a robot. Whether its position is fixed or adaptive, the window can be adjusted in size for the purpose of the application. A window can be compressed to a single pixel if desired to pin point a colour or gray level with a single sample. However because of the possibility of spurious readings due to irregularities in the object or lighting or variations in sensor operation, it is usually more practical to open window to sample a reasonable number of pixels and compute average gray level even for a point measurement. Once a window of interest has been identified, a variety of analyses can enhance the target features or identify and describe them in detail. the moost basic of these analyses is tresholding.

Thresholding: Earlier thresholding has been described as reducing an image to binary black and white pixels. Thresholding is the oldest, simplest and still one of the most effective methods of image analysis. One reason it was so popular in the early days of machine vision is that thresholding is capable of screening out the operational variability of the sensing system. Any binary imaging system is usually designated as thresholding, though the system is not necessarily limited to a single threshold. Two well chosen thresholds could be used to select a band of gray level and thus pick up only a specifically desired object, perhaps of a different colour from the undesired objects or the background. Because of this utility, thresholding systems are usually designed to accept two thresholds. If single level threshold is desired, one of the two thresholds can be adjusted completely to the right or left extreme of the brightness scale. It is difficult to conceive of an industrial application of a thresholding scheme that provides more than two thresholds in a system that produces only a binary image.

finding exactly the right threshold for a binary vision system can be the key to success in automation application. This problem is trickier than it may seem. Selection of a threshold exactly half way between extreme light and dark can result in totally dark picture, if both light and dark are below midrange. The machine vision system needs to some how employ some intelligence in the selection of an appropriate threshold, whether that intelligence is supplied by a human or by reasoning algorithms programmed into the machine. Histogramming: A popular method of selecting a threshold is by histogramming. A frequency histogram is constructed for the pixel counts at each level of gray accommodated by the system. If the image is simply a dark object on a light background, histogram will be bimodal, as in fig in next slide. The shape of the object is irrelevant as is illustrated in slide 15. Histogramming can also be used to select a given shape, if its gray level is different from other shapes and if its pixel count is approximately known relative to the pixel count of other shapes in the image.

Shape identification: The selection of shape is stretching the limits of capability for histogramming and thresholding techniques. There are other ways to identify a desired shape or to perceive its orientation, and a variety of algorithms have these purposes. We use the term algorithms because in this area of image analysis, mathematical or computer procedures, often iterative in nature, are used to process the data to arrive at some conclusion about an objects shape or orientation in the image. One method of finding the orientation of a known shape is to scan a series of straight lines across a binary image looking for characteristic run lengths of black or white pixels. Suppose that is desired to properly orient gaskets as shown in slide 18. The task is to align the minor axis of symmetry (y-y in the top diagram) with the direction of travel of the conveyor.

A simple approach to this problem takes advantage of the computational speed of the system computer to quickly strike a variety of scan lines across the image of various, perhaps random angles. this type of scanning should not be confused with the original raster scan of the entire image during the image capture. Rather this scan is an imaginary one in which the control computer is examining the existing pixel matrix and searching for runs of either black or white pixels or a specific combination of both. For the gasket of the in fig next slide the system is attempting to find the major axis (x-x) by the following series of runs. White-black- White-black- White-black- White-black- White The two white runs at the series extremes represent the background. Besides these beginning and ending white runs, there are three white runs one for each of the two small holes and one longer run for the larger center hole. If gross orientation is all that is desired, an industrial robot equipped with a vacuum gripper can immediately proceed to pick up and reorient the gasket as soon as the first conforming series is detected.

Finer alignment can be made by repeated scans in near proximity to the first successful conforming scan, in an attempt to maximize the pixel length of the interior white run. the maximum run length would indicate that the diameter, rather than the chord has been identified for the inner hole. Perhaps even simpler than looking for the series having correct number of black and white run transitions would be a search for the longest straight line pixel run of the following pattern: black-white - black-white- black-white-black Here the interior transitions are ignored and the objective of the search is to find the first and the last black pixel and maximize their separation. Once this maximum is found , the major axis of the gasket would have been identified. A variation of this procedure would work even if the gasket had no holes.

Template matching: A binary pixel image in the machine vision system memory enables comparisons, pixel-by-pixel, with a template matrix that has the desired pattern. The method will be illustrated with an example.

consider the 171 pixel matrix in the figure in previous slide that contains 6 familiar patterns six letters of the alphabet. The task is to locate the letter F using 35 pixel template for comparison. In nested iterations throughout the matrix each 35 set of pixels can be examined for degree of match with template. As suspected, the letter E is a close match, attaining a score of 13 out of a possible 15. The letter F of course rates a perfect 15. In real applications the search can assume a large amount of computer time as the number of trial areas expands exponentially. Even in the small example illustrated, consider how many comparisons would have to be made to check all the feasible positions in all of the feasible orientations for the 35 template superimposed upon the 171 pixel image screen. Then multiply this task by all of the target templates within the template library. The enormous computation task of an exhaustive search for a template match gives a person new appreciation for pattern recognition of humans, which though taken for granted, is phenomenal compared to machine speeds and accuracy.

A strategy for cutting down the search space by employing a data driven scheme that attempts to make sensible, closed line drawings from the given data. working with the without the benefit of a template, the researchers have succeeded in constructing polygons that bear some semblance of the objects shape, as can be seen in the following figure.

These polygonal approximations are then identified with template classes to drive the system towards a more efficient search for an appropriate template to conform the match. This approach is particularly appropriate when the contents of the image are not known and the objective is to find a match with any of a variety of templates stored within the machine vision systems data base. In template matching it is sometimes better to take advantage of one peculiar feature feature instead of trying to find a match for the entire object. Consider the design of a metal stamping in the following figure, in which the stamping has an unusual and easily recognizable feature in the crosses positioned at each of the four corners.

The crosses are ideal for identification and also serve to ascertain the objects orientation because their axes are parallel with the sides of the stamping. Even if the axes were not so aligned, if they were of a fixed known angle, with the sides, the machine vision system can be programed to figure out the orientation of the work piece. In such an application there is no reason that the angles would not be known, as they would become fixed in the original design of the stamping die set. This is called local feature focus method. Not unusual to keep this point in view while designing the stamping. Edge detection: Edge detection is a procedure that uses binary logic to guide a search through an image, one pixel at a time, as it finds an edge and follows it completely around the object until it repeats itself. At the point of repeat conclusion is reached that the entire object has been circumscribed. An example edge detection procedure is described in the figure in next slide, displaying a binary edge in which the object is dark(binary 1) and the background is light (binary 0). A similar procedure ( with opposite logic) works as well for a light object on a dark background.

The system begins on a straight path across the background examining each pixel it encounters determining whether it is light or dark. As long as it encounters light pixels it continues on a straight path. As soon as the system encounters a dark pixel, it knows that it crossed the boundary into the region within the object, so it turns in an attempt to maintain the edge. From that point forward the system keeps turning so that by continually crossing the boundary it maintains contact with it. Note that the edge detection system is able to detect both interior corners and exterior corners and in crude fashion can even follow a curve, although the computer image will be squared off into a series of tiny square corners. The construction of practical procedure for edge detection is in truth a design problem, because there are many different ways the job can be done. Design case study(edge detection): Design a simple logic flow diagram suitable for developing a computer programme for executing an edge dtection algorithm as sketched in the previous slide. one design for this edge detection procedure appears in the next slide.

One problem with edge detection procedure is that if it ever makes a wrong turn, it can get lost in a tight loop, constantly turning left if it is lost inside the object, and constantly turning right if it is outside. An algorithm can watchdog this situation, and if it is ever discovered that pixels are being repeated in a tight lop, the system can go back to the pixel repeated most and know that this is where the circle pattern began. At that point the system can reverse the turn taken in that pixel and be back on track. Such an algorithm is especially useful when negotiating curves,for which a clear decision regarding a single pixel may not be obvious. The logic flow in the previous slide is unable to deal with possible wrong turns that can result in the system being caught in a tight loop either within or outside the edge of the object. Try adding this sophistication to the logic flow on slide 27 ( previous slide)

Roberts Cross Operator: an algorithmic method of edge detection makes use of the Roberts Cross Operator. The algorithm computes the square root of the sum of the squares of the adjacent diagonal differences between pixel gray scale values. This process is illustrated in the following fig.

The idea is to identify a transition (edge) by finding points at which the diagonal differences are greatest. The Roberts Cross Operator is calculated for all pixels in the image for which the adjacent diagonals to the right and down exists. This is to say that the operator in undefined for the bottom row of pixels and the right column of pixels. Example: Calculate the Roberts Cross Operator for the following pixel matrix.

Solution :

3 1 2 2 3 3 0 2 3 1 5 3 1

Using Roberts Cross Operator Matrix to find an edge: Example:

Let us use a gray scale range of 26 so that ideally all white pixels will be gray scale 63 and all black pixels gray scale 0. By carefully placing the triangle in our ideal image, all pixels will either be wholly within or wholly outside the triangle except for those pixels on the diagonal sides of the triangle. since these pixels are exactly half in half out in our idealized model, we will assign each of them a gay scale of 31. Pixel matrix is shown in next slide.

Now the Roberts Cross Operator for each feasible pixel is calculated. Result is the following matrix.