IntentSearch: Capturing User Intention for One-Click Internet Image Search

Guided by Presented by

NISHA T M

DILSHA V V IT09106008 MESCE

Objective
Search engine which helps to interpret users search

intention by using ONE-CLICK query image

Contents

Introduction Existing System Proposed System 4 steps used Search techniques Visual feature design Adaptive Weight Schema Features for query categorization Image Clustering Advantages Future enhancement Conclusion References

Introduction
Novel

Internet image search

engine which helps to interpret users search intention by using ONE-CLICK query image
Search

Uses
Text

4 steps for image searching

based information of query word and visual content of query image to expand the image pool

Introduction(cont..)
User search intention only by query keywords is difficult because text based image search suffers from..

Ambiguity of query keywords

User doesn't have enough knowledge

Hard for users to describe the visual content of target images

Easier search by using both textual and visual content of query

Existing System
Web-scale image search engines mostly rely on surrounding text features. Users search intention by only by query keywords

Proposed system
Image

search on the basis of both textual and visual

content of images
Image

pool is re-ranked based on textual and visual

features

Fig. 1: Top-ranked images returned from Bing using

apple as query

4 steps used
Key contribution is to capture the users search intention from

this one-click query image in four steps

Adaptive similarity

Keyword expansion
Visual query expansion

Image pool expansion

Adaptive similarity

User always has specific intention when submitting a query image

Categorized into one of the predefined adaptive weight categories, such as portrait and scenery.

Correspondence between the query image and its proper similarity measurement reflects the user intention.

Keyword expansion

Query keywords are expanded to capture users search intention inferred from the visual content of query images

A word w is suggested as an expansion of the query

Image pool expansion

Retrieved by text-based search accommodates images with a large variety of semantic meanings

More accurate query by keywords is needed to narrow the intention and retrieve more relevant images.

The user to click on one of the suggested keywords

Both visual and textual information captured are

automatically added into the text query and enlarge the image pool

Visual query expansion

One query image is not diverse enough to capture the

users intention.

To learn visual and textual similarity metrics, which are more robust and more specific to the query, for image reranking.

Search techniques
The user first submits query keywords q.

A pool of image is retrieved by text-based search User is asked to select the query image from image pool The query image is classified as one of the predefined

adaptive weight categories

Images in the pool are re-ranked based on their visual similarities to the query image

Similarities are computed using the weight schema

Visual feature design

Existing features : Gist , SIFT, Daubechies Wavelet , Histogram of Gradient (HoG)

New features : Attention guided Color Signature, Color spatialet (CSpa) , Multilayer Rotation Invarient ( EOH), Facial Feauter

Adaptive Weight Schema

Weight Lets

schema is used for similarity calculations

take image i and j Adaptive similarity between i & j

Sq(i , j) = fm=1 mq sm(i , j) where sm(i , j) similarity between i and j on feature m f is the visual feature mq is the express the importance of feature m for measuring similarity

Features for query categorization

Existence of

faces, the number of faces in the image

the image frame taken up by the face region face center relative to the centre of image

Percentage of Coordinate of

Directionality Color Spatial Homogeneousness (variance of values in

different blocks of Color Spatialet)

Total Edge

energy of edge map obtained from Canny Operator

Spatial Distribution

Image Clustering
Image Each

is divided into clusters

word wi has ti clusters C(wi)= { ci,1 ,.............,ci,ti }

Visual

distance between the query image and a cluster c is

calculated as the mean of the distances between the query image and the images in c.
The

cluster Ci,j with the minimal distance is chosen as

visual query expansion and its corresponding word wi . q = wi + q

Advantages
User friendly
Easy search for a particular image(on the internet) Can find the image is real or not

Disadvantages
Duplicate images

Future enhancement
Query log data, which provides valuable co-occurrence information of keywords , for keyword expansion Can be improved by including duplicate detection in the future work

Conclusion
Internet image search approach which only requires one-click user feedback Intention specific weight schema

Without additional human feedback

Possible for industrial scale image search by both text and visual content

References

[1] Y. Zhang, Z. Jia, and T. Chen, Image Retrieval with Geometry-Preserving Visual Phrases, Proc. IEEE Intl Conf. Computer Vision and Pattern Recognition, 2011. [2] J. Cui, F. Wen, and X. Tang, IntentSearch: Interactive On-Line Image Search Re-Ranking, Proc. 16th ACM Intl Conf. multimedia,2008. [3] Bing Image Search, http://www.bing.com/images, 2012. [4] J. Deng, A.C. Berg, and L. Fei-Fei, Hierarchical Semantic Indexing for Large Scale Image Retrieval, Proc. IEEE Intl Conf.Computer Vision and Pattern Recognition, 2011.

References(cont)

[5] Y. Cao, C. Wang, Z. Li, L. Zhang, and L. Zhang, Spatial-

Bag-of-Features, Proc. IEEE Intl Conf. Computer Vision

and Pattern Recognition, 2010.

[6] J. Deng, A.C. Berg, and L. Fei-Fei, Hierarchical Semantic Indexing for Large Scale Image Retrieval, Proc. IEEE Intl Conf.Computer Vision and Pattern Recognition, 2011. [7] Y. Huang, Q. Liu, S. Zhang, and D.N. Metaxas, Image Retrieval via Probabilistic Hypergraph Ranking, Proc. IEEE Intl Conf.Computer Vision and Pattern Recognition, 2011

THANK YOU