bw = imread('box.

jpg');
bw=im2bw(bw);
imshow(bw)
L = bwlabel(bw);
s = regionprops(L, 'Area', 'BoundingBox');
s(1);
area_values = [s.Area];
idx = find((100 <= area_values) & (area_values <= 1000)); % list of all the objects
%whose area is between 100 and 1000
bw2 = ismember(L, idx); %construct a binary image containing all the objects whose
%area is between 100 and 1000 by passing L and idx to ismember.
imshow(bw2)
I treid this after the above code. But its not a good way of doing this: imshow(bw2) rectangle('Position',
[ s(2).Centroid+35, 70, 100], 'EdgeColor','r')
--------------------------------------------------------------------------

Pseduo

Pick largest y, largest x, smallest x, smallest y with in the blob. That is, points on the blob. These are
your coordinates that you can use to build the bounding box.
assuming top left of image as (0,0)
(smallestX,smallestY)-----------------(largestX,smallestY)
|
|
|
|
|
|
|
|
(smallestX,largestY)------------------(largestX,largestY)
And for finding minimum/maximum values and indices.
[r,c]=find(img==min(min(img)))
[r,c]=find(img==max(max(img)))
r,c represent row and column in the img matrix.

I have marked the points on your image that you can use to create the bounding box.
Zoomed Image to get a better view.

I think you can try to use bwboundries

boundaries = bwboundaries(blob);
numberOfBoundaries = size(boundaries);
for k = 1 : numberOfBoundaries
thisBoundary = boundaries{k};
plot(thisBoundary(:,2), thisBoundary(:,1), 'g', 'LineWidth', 2);
end

Find the corners of a polygon represented by a region mask

BW = poly2mask(x, y, m, n) computes a binary region of interest (ROI) mask, BW, from an ROI polygon,
represented by the vectors x and y. The size of BW is m-by-n.
poly2mask sets pixels in BW that are inside the polygon (X,Y) to 1 and sets pixels outside the polygon to 0.
Problem: Given such a binary mask BW of a convex quadrilateral, what would be the most efficient way to
determine the four corners?
E.g.,

Best Solution so far: Use edge to find the bounding lines, the Hough transform to find the 4 lines in the edge
image and then find the intersection points of those 4 lines or use a corner detector on the edge image. Seems
complicated, and I can't help feeling there's a simpler solution out there.
Btw, convhull doesn't always return 4 points (maybe someone can suggest qhull options to prevent that) : it
returns a few points along the edges as well.
EDIT: Amro's answer seems quite elegant and efficient. But there could be multiple "corners" at each real
corner since the peaks aren't unique. I could cluster them based on and average the "corners" around a real
corner but the main problem is the use of order(1:10).
Is 10 enough to account for all the corners or will this exclude a "corner" at a real corner?
This is somewhat similar to what @AndyL suggested. However I'm using the boundary signature in polar
coordinates instead of the tangent.
Note that I start by extracting the edges, getting the boundary, then converting it to signature. Finally we find the
points on the boundary that are furthest from the centroid, those points constitute the corners found.
(Alternatively we can also detect peaks in the signature for corners).
The following is a complete implementation:
I = imread('oxyjj.png');
if ndims(I)==3
I = rgb2gray(I);
end
subplot(221), imshow(I), title('org')

%%# Process Image

%# edge detection
BW = edge(I, 'sobel');
subplot(222), imshow(BW), title('edge')
%# dilation-erosion
se = strel('disk', 2);
BW = imdilate(BW,se);
BW = imerode(BW,se);
subplot(223), imshow(BW), title('dilation-erosion')
%# fill holes
BW = imfill(BW, 'holes');
subplot(224), imshow(BW), title('fill')
%# get boundary
B = bwboundaries(BW, 8, 'noholes');
B = B{1};
%%# boudary signature
%# convert boundary from cartesian to ploar coordinates
objB = bsxfun(@minus, B, mean(B));
[theta, rho] = cart2pol(objB(:,2), objB(:,1));
%# find corners
%#corners = find( diff(diff(rho)>0) < 0 );
[~,order] = sort(rho, 'descend');
corners = order(1:10);

%# find peaks

%# plot boundary signature + corners

figure, plot(theta, rho, '.'), hold on
plot(theta(corners), rho(corners), 'ro'), hold off
xlim([-pi pi]), title('Boundary Signature'), xlabel('\theta'), ylabel('\rho')
%# plot image + corners
figure, imshow(BW), hold on
plot(B(corners,2), B(corners,1), 's', 'MarkerSize',10, 'MarkerFaceColor','r')
hold off, title('Corners')

-------------------------

EDIT: In response to Jacob's comment, I should explain that I first tried to find the peaks in the signature using
first/second derivatives, but ended up taking the furthest N-points. 10 was just an ad-hoc value, and would be
difficult to generalize (I tried taking 4 same as number of corners, but it didn't cover all of them). I think the idea
of clustering them to remove duplicates is worth looking into.
As far as I see it, the problem with the 1st approach was that if you plot rho without taking into account, you
will get a different shape (not the same peaks), since the speed by which we trace the boundary is different and
depends on the curvature. If we could figure out how to normalize that effect, we can get more accurate results
using derivatives.

If you have the Image Processing Toolbox, there is a function called CORNERMETRIC which can implement a
Harris corner detector or Shi and Tomasi's minimum eigenvalue method. This function has been present
since version 6.2 of the Image Processing Toolbox (MATLAB version R2008b).
Using this function, I came up with a slightly different approach from the other answers. The solution below is
based on the idea that a circular area centered at each "true" corner point will overlap the polygon by a smaller
amount than a circular area centered over an erroneous corner point that is actually on the edge. This solution
can also handle cases where multiple points are detected at the same corner...
The first step is to load the data:
rawImage = imread('oxyjj.png');
rawImage = rgb2gray(rawImage(7:473,9:688,:)); %# Remove the gray border
subplot(2,2,1);
imshow(rawImage);
title('Raw image');
Next, compute the corner metric using CORNERMETRIC. Note that I am masking the corner metric by the
original polygon, so that we are looking for corner points that are inside the polygon (i.e. trying to find the corner
pixels of the polygon). IMREGIONALMAX is then used to find the local maxima. Since you can have clusters of
greater than 1 pixel with the same corner metric, I then add noise to the maxima and recompute so that I only
get 1 pixel in each maximal region. Each maximal region is then labeled using BWLABEL:
cornerImage = cornermetric(rawImage).*(rawImage > 0);
maxImage = imregionalmax(cornerImage);
noise = rand(nnz(maxImage),1);
cornerImage(maxImage) = cornerImage(maxImage)+noise;
maxImage = imregionalmax(cornerImage);
labeledImage = bwlabel(maxImage);
The labeled regions are then dilated (using IMDILATE) with a disk-shaped structuring element (created
usingSTREL):
diskSize = 5;
dilatedImage = imdilate(labeledImage,strel('disk',diskSize));
subplot(2,2,2);
imshow(dilatedImage);
title('Dilated corner points');
Now that the labeled corner regions have been dilated, they will partially overlap the original polygon. Regions
on an edge of the polygon will have about 50% overlap, while regions that are on a corner will have about 25%
overlap. The function REGIONPROPS can be used to find the areas of overlap for each labeled region, and the
4 regions that have the least amount of overlap can thus be considered as the true corners:
maskImage = dilatedImage.*(rawImage > 0);
%# Overlap with the polygon
stats = regionprops(maskImage,'Area');
%# Compute the areas
[sortedValues,index] = sort([stats.Area]);
%# Sort in ascending order
cornerLabels = index(1:4);
%# The 4 smallest region labels
maskImage = ismember(maskImage,cornerLabels); %# Mask of the 4 smallest regions
subplot(2,2,3);
imshow(maskImage);
title('Regions of minimal overlap');
And we can now get the pixel coordinates of the corners using FIND and ISMEMBER:
[r,c] = find(ismember(labeledImage,cornerLabels));
subplot(2,2,4);
imshow(rawImage);
hold on;
plot(c,r,'r+','MarkerSize',16,'LineWidth',2);

title('Corner points');

And here's a test with a diamond shaped region:

I like to solve this problem by working with a boundary, because it reduces this from a 2D problem to a 1D
problem.
Use bwtraceboundary() from the image processing toolkit to extract a list of points on the boundary. Then
convert the boundary into a series of tangent vectors (there are a number of ways to do this, one way would be
to subrtact the ith point along the boundary from the i+deltath point.) Once you have a list of vectors, take the
dot product of adjacent vectors. The four points with the smallest dot products are your corners!
If you want your algorithm to work on polygons with an abritrary number of vertices, then simply search for dot
products that are a certain number of standard deviations below the median dot product.
I decided to use a Harris corner detector (here's a more formal description) to obtain the corners. This can be
implemented as follows:
%% Constants
Window = 3;
Sigma = 2;
K = 0.05;
nCorners = 4;
%% Derivative masks
dx = [-1 0 1; -1 0 1; -1 0 1];
dy = dx'; %SO code color fix '
%% Find the image gradient
% Mask is the binary image of the quadrilateral

Ix = conv2(double(Mask),dx,'same');
Iy = conv2(double(Mask),dy,'same');
%% Use a gaussian windowing function and compute the rest
Gaussian = fspecial('gaussian',Window,Sigma);
Ix2 = conv2(Ix.^2, Gaussian, 'same');
Iy2 = conv2(Iy.^2, Gaussian, 'same');
Ixy = conv2(Ix.*Iy, Gaussian, 'same');
%% Find the corners
CornerStrength = (Ix2.*Iy2 - Ixy.^2) - K*(Ix2 + Iy2).^2;
[val ind] = sort(CornerStrength(:),'descend');
[Ci Cj] = ind2sub(size(CornerStrength),ind(1:nCorners));
%% Display
imshow(Mask,[]);
hold on;
plot(Cj,Ci,'r*');
Here, the problem with multiple corners thanks to Gaussian windowing function which smooths the intensity
change. Below, is a zoomed version of a corner with the hot colormap.

Table 1

Nope. Listen I was testing your approach, and it seems it's not perfect either (which is always the case
in Computer Vision!). Try the following mask with your code above: x = [16 282 276 30 16]; y = [14
29 200 225 14]; BW = poly2mask(x,y, 246,300); you will get some duplicate corners (even when
tried tuning the other params), and will have to increase nCorners too.. Amro Nov 12 '09 at 21:48
You're completely right - looks like
clustering is required for some
polygons. Jacob Nov 12 '09 at 22:58
I don't want to push my idea too strongly,
but I think that the signature method could
be perfected for most quadrilateral if we
locate the exact 4 theta peaks, which are
after all clearly visible in the plot. Picking
the highest N-points may not have been
the best way, but maybe someone can
improve that part

---------------------------------------------------Here's an example using Ruby and HornetsEye. Basically the program creates a histogram of the quantised
Sobel gradient orientation to find dominant orientations. If four dominant orientations are found, lines are fitted
and the intersections between neighbouring lines are assumed to be the corners of the projected rectangle.
#!/usr/bin/env ruby
require 'hornetseye'
include Hornetseye
Q = 36
img = MultiArray.load_ubyte 'http://imgur.com/oxyjj.png'
dx, dy = 8, 6
box = [ dx ... 688, dy ... 473 ]
crop = img[ *box ]
crop.show
s0, s1 = crop.sobel( 0 ), crop.sobel( 1 )
mag = Math.sqrt s0 ** 2 + s1 ** 2
mag.normalise.show
arg = Math.atan2 s1, s0
msk = mag >= 500
arg_q = ( ( arg.mask( msk ) / Math::PI + 1 ) * Q / 2 ).to_int % Q
hist = arg_q.hist_weighted Q, mag.mask( msk )
segments = ( hist >= hist.max / 4 ).components
lines = arg_q.map segments
lines.unmask( msk ).normalise.show
if segments.max == 4
pos = MultiArray.scomplex *crop.shape
pos.real = MultiArray.int( *crop.shape ).indgen! % crop.shape[0]
pos.imag = MultiArray.int( *crop.shape ).indgen! / crop.shape[0]
weights = lines.hist( 5 ).major 1.0
centre = lines.hist_weighted( 5, pos.mask( msk ) ) / weights

vector = pos.mask( msk ) - lines.map( centre )

orientation = lines.hist_weighted( 5, vector ** 2 ) ** 0.5
corner = Sequence[ *( 0 ... 4 ).collect do |i|
i1, i2 = i + 1, ( i + 1 ) % 4 + 1
l1, a1, l2, a2 = centre[i1], orientation[i1], centre[i2], orientation[i2]
( l1 * a1.conj * a2 - l2 * a1 * a2.conj l1.conj * a1 * a2 + l2.conj * a1 * a2 ) /
( a1.conj * a2 - a1 * a2.conj )
end ]
result = MultiArray.ubytergb( *img.shape ).fill! 128
result[ *box ] = crop
corner.to_a.each do |c|
result[ c.real.to_i + dx - 1 .. c.real.to_i + dx + 1,
c.imag.to_i + dy - 1 .. c.imag.to_i + dy + 1 ] = RGB 255, 0, 0
end
result.show
end

----------------------------------------------------------------------------x=[ 39 63 71 7 53 39 64 23 29 65 ];
y=[ 20 11 22 78 61 71 9 20 78 94 ];
AA=[x ; y]
% Find point in AA closest to p1.
p1 = [39,20]
squaredDistance = sum((AA-repmat(p1', [1, size(AA, 2)])).^2, 1)
[maxSqDist1, indexOfMax1] = max(squaredDistance)
% Find point in AA closest to p2.
p2 =[63,11]

squaredDistance = sum((AA-repmat(p2', [1, size(AA, 2)])).^2, 1)

[maxSqDist2, indexOfMax2] = max(squaredDistance)

----------------------------------------------------------------------------By following this post Using ismember with the output of regionprops I am able to selectively isolate the
connected component I want. For example using my code below:
img = rgb2gray(imread('W1\Writer1_01_02.jpg'));
bw_normal2 = im2bw(img, graythresh(img));
bw22 = imcomplement(bw_normal2);
bw3 = bwmorph(bw22, 'dilate');
[label2,n2] = bwlabel(bw3);
stats2 = regionprops(label2, {'Area', 'BoundingBox'});
area2 = [stats2.Area];
idx = find((28 <= area2) & (area2 <= 40));
BW2 = ismember(label2,idx);
figure, imshow(BW2)
I can easily display the output that contains ONLY the connected component whose area is between 28 and 40.
As so

But instead can I make a bounding box around this connected component in the original image. I mean if this is

the original image:

Can I make a bounding box around my desired component on the original image? I know that this is the code
for making a bounding box around all of my connected components
imshow(img);

for j=1:n2
hold on
rectangle('Position',
[stats(j).BoundingBox(1),stats(j).BoundingBox(2),stats(j).BoundingBox(3),stats(j).BoundingBox(4)],
...
'EdgeColor','r','LineWidth',2 );
end
But how do I only make a bounding box only around the element having area between 28 and 40? Rather than
producing a completely different image as shown above.

Table 2

might not have understood the question properly. But it seems like that you have the following piece of
codefor j=1:n2 which would mean that you are traversing each of the component on the image. Find
the number which represents the object of interest and draw the bounding box. Lokesh A. R. Feb 24
at 16:24
That would be quite difficult and time
consuming. Say I only want the object having
an area of 40. I would then open
the stats2.Area in my workspace, find it and
then apply it to my code. I just added that
second code to show that I know how to draw a
bounding box around all components but how
to draw on the original image for the element
selected by ismember? Faraz Khan Feb 24
at 16:54

Er, what's difficult

about stats2(idx).BoundingBox? If you're not
clear how the code works, try stepping through
in the debugger to see what the values are, and
how they change, line-by-line (use "step over"
rather than "step into" to avoid the guts of all the
functions)

Keep a if condition in your second half of the code...

imshow(img);
for j=1:n2
hold on
area2 = stats2(j).Area;
if((28 <= area2) & (area2 <= 40))
rectangle('Position',
[stats2(j).BoundingBox(1),stats2(j).BoundingBox(2),stats2(j).BoundingBox(3),stats2(j).BoundingBo
x(4)],...'EdgeColor','r','LineWidth',2 );
end
end

-------------------------------------------------------------------------------------

How to automatically identify text lines from projection plot?

Asked by Faraz on 18 Jan 2014
Latest activity Answered by fawzi about 14 hours ago
Accepted Answer by Image Analyst

I have been reading about automatic text line recognition in Matlab and although there are many advanced methods to do
this every paper mentions that the simplest way of detecting text lines is via horizontal projections. So I decided to try this
method for myself.
I am very new to this and have hit a brick wall, I have reached a level beyond which I do not know how to proceed. This is
what I have achieved so far:
I'm trying for a system that is language independent and only interested in text lines, so I chose Arabic text:

I used the function ``radon`` to get the projections.

img = rgb2gray(imread('arabic.jpg'));
[R, xp] = radon(bw_closed, [0 90]);
figure; plot(xp,R(:,2)); title('at angle 90');
This is the plot(projection)

So clearly the 5 peaks represent the 5 lines detected but how do I go from here to segmenting the original document?
Can anyone help me beyond this point? All the papers I read make no mention of how to proceed from this step, they just
say that from the projections we have our detected lines.

What I'm asking is how, from the plot data can I tell matlab what is the line of text and what is the gab between lines?
I would just find where the black is in the profile
darkPixels = R < 20; % Threshold
% label
labeledRegions = bwlabel(darkPixels);
% Find centroids
measurements = regionprops(labeledRegions, 'Centroid');
% Get them into an array
allCentroids = [measurements.Centroid];
Now you can just crop out some line of text you're interested in, into a separate image:
thisLine = yourImage(allCentroids(k):allCentroids(k+1), :);

-------------------------------------------------------------------------------------------------------------------

Image projection in Matlab with Hough transform

I am using Matlab to input a 4x4 grid of coloured squares, and output a list of colours. My program works fine
for squares, but I am having trouble adapting it to rotated and projected images. I have been advised to use
Hough transforms, and I am able to use this to access the lines in the image with the following code:
[H, theta, rho] = hough(image,'RhoResolution',0.1,'Theta',-90:0.5:89.5);
peaks = houghpeaks(H,4);
lines = houghlines(dilated, theta, rho, peaks, 'MinLength', 40)
figure, imshow(dilated), hold on;
max_len = 0;
for k = 1:length(lines)
xy = [lines(k).point1; lines(k).point2];
plot(xy(:,1),xy(:,2),'LineWidth',2,'Color','green');
% Plot beginnings and ends of lines
plot(xy(1,1),xy(1,2),'x','LineWidth',2,'Color','yellow');
plot(xy(2,1),xy(2,2),'x','LineWidth',2,'Color','red');
angle = atand(((xy(1,1)-xy(2,1))/(xy(1,2)-xy(2,2))));
end
So now I would like to use these lines, and straighten my image accordingly, so that image edges are parallel
with these lines. However, I don't know how to go about this. I would appreciate advice on how to do this.
Thanks.

Detecting intersecting lines after a hough transform

Using Hough transform in Matlab,detected some lines. Using the end points of these lines I have plotted them. I
cant understand how I can find intersecting lines when I have all the variables.
Line7
Point1 [50,66]
Point2 [11,106]
theta,rho [45,81]
Line9
Point1 [19,83]
Point2 [53,79]
theta,rho [82,84]
Since the parametric equations are as follows
rho = xCos(theta) + ySin(theta)
I am unsure how to solve this. With all this information there must be a quick way of finding if the lines intersect,
if so, the points as well.
Any guidance much appreciated.
function FindHoughLines(I,filename)
[H,T,R] = hough(I);
rotI = imrotate(I,0,'crop');
imshow(H,[],'XData',T,'YData',R,...
'InitialMagnification','fit');
xlabel('\theta'), ylabel('\rho');
axis on, axis normal, hold on;
P = houghpeaks(H,10,'threshold',ceil(0.1*max(H(:))));
x = T(P(:,2)); y = R(P(:,1));
plot(x,y,'s','color','white');
% Find lines and plot them
lines = houghlines(I,T,R,P,'FillGap',5,'MinLength',7);
figure, imshow(rotI), hold on
max_len = 0;
for k = 1:length(lines)
if(isField(lines,'point1') ~= 0)
xy = [lines(k).point1; lines(k).point2];
plot(xy(:,1),xy(:,2),'LineWidth',2,'Color','green');
% Plot beginnings and ends of lines
plot(xy(1,1),xy(1,2),'x','LineWidth',2,'Color','yellow');
plot(xy(2,1),xy(2,2),'x','LineWidth',2,'Color','red');
text(xy(1,1),xy(1,2),[ num2str(k)],'HorizontalAlignment','center','BackgroundColor',[.7 .9 .7]);
% Determine the endpoints of the longest line segment
len = norm(lines(k).point1 - lines(k).point2);
if ( len > max_len)
max_len = len;

xy_long = xy;
end
end
end

Are you wanting intersections of lines or line segments? For instance, suppose one of your line segments has
endpoints (1,0) and (2,0) and the other has endpoints (0,1) and (0,2); do you want the intersection at (0,0)
counted or not?
Do you need to cope efficiently with the case where there are many many lines and find all their intersections,
or is it OK to consider all pairs of lines and check them one by one?
The easiest case is if it's OK just to consider pairs of lines, and if line (as opposed to line segment) intersections
are good enough. Then for each pair of lines you're just solving two simultaneous linear equations: aX+bY=c,
dX+eY=f. This is a very standard thing to do; it amounts to inverting a 2x2 matrix.
If you need to notice when an intersection doesn't actually lie within the given line segments, here are a couple
of approaches. (1) First first the intersection as above, then check that it lies within each given segment. You
can do that by picking one coordinate (say, the x-coordinate) and seeing whether it lies between the xcoordinates of the two endpoints. Of course you can't use the x-coordinate for vertical lines and shouldn't use it
for nearly-vertical lines; better to pick whichever coordinate has the larger coefficient. (2) Write the lines
parametrically as (x,y)=(x1,y1)+t(dx1,dx1) and (x,y)=(x2,y2)+u(dx2,dy2); now instead of simultaneous equations
for x,y you have simultaneous equations for t,u; solve those and check that 0 <= t,u <= 1.
If you need to cope efficiently when you have many lines, there are algorithms for this; google "line sweep" to
find a standard one that's pretty easy to understand.