Content Based Image

Retrieval and its Application

in Remote Clincal Diagnosis
and Healthcare
Suyog Dutt Jain
Dept. Of Computer Science & Engineering
MIT, Manipal
Before Starting..
 Center for Soft Computing Research, Indian Statistical Unit, Kolkata

 Department of Computer Science & Technology, MIT Manipal

 Dr. Niranjan U C , Adjunct Faculty, Dept. Of ECE, MIT Manipal

 All my Teachers

 Friends
"To Dream About Something Is the First
Step Towards Achieving It"
Background
 Content Based Image Retrieval [CBIR] : A topic of research from decades

 Its time to develop some domain specific applications using CBIR

 Application of CBIR in Healthcare Domain

 Global Availability and Ease of access : Internet Based System

Introduction
 What is Content Based Image Retrieval [CBIR] ?

 What is the need for CBIR?

 How it Works?

Feature Extraction

Feature Matching
Objectives of the Work
 Identification of potential users of the application.

 Study of the existing techniques for Content Based Image Retrieval.

 Maintenance of a database of actual medical cases and images.

 Deciding upon Image Features to be extracted and Similarity Metric.

 Make the system web deployable.

 Authorized access to the system.

 Web Based efficient Graphical User interface

Overview of the sotwares used
 MATLAB R2006B

 MATLAB Builder for Java

 Java Servlet API

 MySQL Database Mangagement System

System Architecture
DATABASE

SITE
DATABASE
SERVER
COMMUNICATION SERVER
OVER IP SIDE

REMOTE
DISTRIBUTED
DIAGNOSIS
SERVER
MACHINE
COMMUNICATION SERVER
OVER IP SIDE
CLIENT
SIDE
System Architecture
System Architecture
CBIR MODULE
JAVA SERVLET

USER AUTHETICATION

USER MANAGEMENT

QUERY HANDLING

MATLAB FEATURE COMPARISON

FEATURE EXTRACTION RESPONSE HANDLING

DATABASE HANDLING
Proposed Method: Intensity Based Retrieval
The Distribution Function cdf (i) upto the gray level ‘i’ is given by

Where,

h (j) is the normalized histogram at gray level j

nj is the number of pixels with gray level j

M.N is the size of the image

The CDF contains the same information as the histogram but in redundant way
Proposed Method: Intensity Based Retrieval
 CDFs at different grayscale intervals is modeled with lines.

 Each of these models has two parameters viz. slope 'a' and intercept 'b‚

(a)

 Estimation of the slope ‘a’ and the intercept ‘b’ by the least squares method

(b)
Proposed Method: Intensity Based Retrieval
Solution of equation (b) is:

Where,
Proposed Method: Intensity Based Retrieval
 Similarity Metric

Eintensity = adiff + bdiff

adiff = (|a1 (q)-a1 (d)| + (|a2 (q)-a2 (d)| +……… (|ap (q)-ap (d)|

bdiff = (|b1 (q)-b1 (d)| + (|b2 (q)-b2 (d)| +……… (|bp (q)-bp (d)|

Where,

a1(q), a2(q)...ap(q) and a1(d), a2(d)...ap(d) represent slopes

b1(q), b2(q)...bp(q) and b1(d), b2(d)...bp(d) represent intercepts

Intensity Based Retrieval: Results

CDF fitted with

p=4
Intensity Based Retrieval: Results

CDF fitted with CDF fitted with

p=8 p=16
Intensity Based Retrieval: Results

CDF fitted with

p= 4
Intensity Based Retrieval: Results

CDF fitted with CDF fitted with

p=16 p=16
Proposed Method: Shape Based Retrieval
 Edge Detection
 Edges: Areas with strong intensity contrast i.e. a jump in intensity from one pixel to the
next
 Reduces the amount of data and filters out useless information, while preserving the
structural properties of the image.

 Gradient method
 Detects the edges by looking for the maximum and minimum in the first derivative of the
image.

 Laplacian method
 Searches for zero crossings in the second derivative of the image to find edges.
Proposed Method: Shape Based Retrieval

Signal showing edge in an

image

Signal showing the First Derivative of the Signal showing the Second Derivative of the
Original Signal Original Signal
Proposed Method: Shape Based Retrieval
 Sobel Edge Detector
 Performs a 2-D spatial gradient measurement on an image.

 Uses a pair of 3x3 convolution masks, one estimating the gradient in the x-direction
(columns) and the other estimating the gradient in the y-direction (rows).

 Sobel Masks

 Applied separately to obtain gradient components in each orientation

Proposed Method: Shape Based Retrieval
 Gradient Magnitude

 Approximate Faster Magnitude

|G|=|Gx| + |Gy|
 Gradient Image

 Edge Histogram

 Eshape
Shape Based Retrieval: Results

Original Gradient
Image Image
Shape Based Retrieval: Results

Edge
Histogram
Shape Based Retrieval: Results
Shape Based Retrieval: Results
Shape Based Retrieval: Results
Shape Based Retrieval: Results
Proposed Method: Texture Based Retrieval
 Grey Level Co-occurrence Matrix (GLCM)
 Tabulation of how often different combinations of pixel brightness values (grey levels)
occur in an image

0 0 1 1
       

0 0 1 1
       

0 2 2 2
       

2 2 3 3
       
Proposed Method: Texture Based Retrieval
 Grey Level Co-occurrence Matrix (GLCM)

0 0 1 1 0 1 2 3

0 2 2 1 0
0 0 1 1

1 0 2 0 0
0 2 2 2
2 0 0 3 1

2 2 3 3
3 0 0 0 1

Image Intensity GLCM

Distribution
Proposed Method: Texture Based Retrieval
 Normalization of GLCM

Where,

Vi,j is the (i , j) element of the GLCM

Pi,j is the (i , j) element of the Normalized GLCM

Proposed Method: Texture Based Retrieval
 Calculation of Texture Features from GLCM
Contrast [ T1 ]

Dissimilarity [ T2 ]

Homogeneity [ T3 ]

Angular Second Moment [ T4 ]

Entropy [ T5 ]

 Feature Vector F = [ T1,T2,T3,T4,T5]

 Similarity Metric Etexture = | Fd – Fq |

Where.

Fd = Feature Vector of Database Image

Fq = Feature Vector of Query Image

Proposed Method: Similarity Metric
 Overall Similarity Metric

Efinal = (Etexture + Eintenstiy + Eshape)/3

 Efinal is used to sort similar images

 Similar past cases are retrieved

Experimental Results & Discussion

Query
Image1
Experimental Results & Discussion

Query
Image2
Experimental Results & Discussion

User
Authenticatio
n Interface
Experimental Results & Discussion

Query
Submissi
on
Interface
Experimental Results & Discussion

Matching
Cases
Interface

Fig. 3 User Interface for viewing similar cases retrieved by the diagnosis server
Response Time Analysis

Average Response Time = 29.28 seconds.

Conclusion
 CBIR & its application in healthcare domain

 Web based architecture for CBIR.

 Global availibility and ease of access.

 Easily expandable to include modern CBIR algorithms

 Can act as a prototype for other domain specific applications

Publication from the work

Suyog Dutt Jain, Dr Niranjan U C, “Distributed Framework for Remote Clinical Diagnosis with
Visual Query Support”, 5th IEEE International Conference on Information Technology and
Application in Biomedicine, China, May 2008.
Bibliograph & References
[1] Andriole KP. “Addressing the Coming Radiology Crisis” The Society for Computer
Applications in Radiology Transforming the Radiological Interpretation Process
(TRIP) Initiative. White Paper (http://www.siimweb.org). November 2005.

[2] Antani S, Long LR, Thoma G. “Content-based image retrieval for large biomedical
image archives”. Proc. 11 World Cong. on Med Info (MEDINFO) 2004, pp. 829-33.

[3] http://www.mypacs.net/

[4] Flickner, M et. al. “Query by image and video content: the QBIC system” IEEE
Computer 28(9), 23-92, 2001

[5] Kato, T. “Database architecture for content-based image retrieval”, Image Storage
and Retrieval Systems. Proc SPIE 1662, 112–123, 1992.

[6] Swain, M. J., and Ballard, D. H., “Color indexing for Content Based Image Retrieval”
International Journal of Computer Vision 7(1):11–32, 1991.
Bibliograph & References
[7] Gonzalez, R. C., and Woods, R. E., Digital image processing, 2004 2nd Edition, pp
94–103.

[8] Stricker, M., and Dimai, A., “Color indexing with weak spatial constraints”. In: Storage
and Retrieval for Image and Video Databases IV. Proc SPIE 2670, 29–40, 1996.

[9] Manjunath K N, Niranjan U C, “Linear Models of Cumulative Distribution Function for

Content Based Medical Image Retrieval”, Appears in Proc. Of 2005 IEEE
Engineering in Medicine and Biology 27th Annual Conference, Shanghai, China, 2005
pp.6472-6475

[10]Robert M. Haralick, K. Shanmugam, “Textual Features for Image Classification”,

IEEE Transactions on Systems, Man, and Cybernetics, VOL. SMC-3, No.6,
November 1973 pp.610-621