Scribd
Upload
16 views

Content-Based Image Retrieval

This document describes content-based image retrieval (CBIR) and two proposed research areas to improve CBIR. CBIR uses automatically derived image features like color, texture, and shape to search image collections, but struggles with the gap between low-level features and high-level semantics. The first proposed area is using machine learning algorithms like SVMs with relevance feedback, where users label examples to iteratively improve results. The second area is a memory learning framework that learns semantics from previous user feedback logs rather than image contents alone.

Uploaded by

Sudheer Gangisetty
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
16 views

Content-Based Image Retrieval

This document describes content-based image retrieval (CBIR) and two proposed research areas to improve CBIR. CBIR uses automatically derived image features like color, texture, and shape to search image collections, but struggles with the gap between low-level features and high-level semantics. The first proposed area is using machine learning algorithms like SVMs with relevance feedback, where users label examples to iteratively improve results. The second area is a memory learning framework that learns semantics from previous user feedback logs rather than image contents alone.

Uploaded by

Sudheer Gangisetty
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Content-based Image Retrieval

Project Description:
· Problem Statement
Image search engines become indispensable tools for users who look for images from a large-
scale image collection and World-Wide Web. Its key technique is content-based image retrieval
(CBIR) having the ability of searching images via automatically derived image features, such as
color, texture or shape. The major difficulty of CBIR lies in the big gap between low-level image
features and high-level image semantics. Fig. 1 shows an image. Human beings may identify its
semantics (a balloon is flying in the sky) effortlessly. However, computers only know it contains
white, black, and blue color. Hence, if one user selects Fig. 1 as the query example, the CBIR
system retrieves images with similar colors to Fig. 1. The poor results are displayed by Fig. 2.
　

           
         Fig. 1 An image of balloon                               Fig. 2 Query results based on color
features
　

· Proposed Research
In order to address the above problem, we focus on two researches: 

(1) Relevance Feedback using Machine Learning Algorithms

Relevance feedback (RF) is an interactive process to incorporate human perception subjectivity


into the query process and provide users with the opportunity to evaluate the retrieval results.
Fig. 3 shows the basic architecture of a  RF based CBIR system.

   

Fig. 3 The basic architecture of a general RF based CBIR system


　

We consider RF from the perspective of supervised learning. Given a query, the system first
retrieves a list of ranked images using a similarity metric. Then, the user selects a set of positive
and negative examples from the returned results. The systems learn from labeled examples to
train a classifier. Many classical machine learning schemes may be applied to train the classifier,
such as Bayesian learning, Support Vector Machines (SVM), Boosting, and so on. The retrieval
performance is gradually improved after several feedback iterations.

Fig. 4 illustrates a RF iteration. (a) shows the results of the first round of retrieval. The top left
image is the query example. The images marked by "√" are the positive example, which are
relevant to the query image. On the contrary, images marked by "×" are the negative example,
which are irrelevant to the query image.  (b) provides the results after this feedback iteration.
　
 

     (a) The results of the first round retrieval                                   (b) The results after one
feedback iteration

Fig. 4 An example of the RF process

(2) Memory Learning by Accumulating User Feedback Log

To further improve the performance of CBIR systems, we propose a memory learning (ML)
framework. Its basic idea is to learn semantics from previous users’ feedback knowledge instead
of image contents. The architecture of ML framework is shown in Fig. 5.

　
Fig. 5 The architecture of ML framework
　

Comparing with general RF, our work has four contributions:

o A feedback knowledge memory model is presented to gather the users’ feedback


information;

o A learning strategy based on the memorized information is proposed. It can estimate the
hidden semantic relationships among images.
o A seamless combination of normal RF (low-level features based) and the memory learning
(semantics based) is proposed to improve the retrieval performance.
o A semantics based annotation propagation scheme is proposed using learned semantics.
　

Project Investigators:
Prof. Ngan, King Ngi,    Dr. Han, Jun Wei  

Email: {knngan, jwhan}@ee.cuhk.edu.hk                       


                                                                                                         

Homepage: http://www.ee.cuhk.edu.hk/~knngan

Address:  Department of Electronic Engineering                 

                The Chinese University of Hong Kong                                                                  


                                  

                Shatin, New Territorie, HONG KONG  


Tel: +852 26098255
　

Current Project Achievements


· Experimental Results
· Publications
J. Han, K.N. Ngan, M. Li and H. Zhang, “A memory learning framework for effective image
retrieval,” To appear in IEEE Transactions on Image Processing, U.S.A.

J. Han, K.N. Ngan, M. Li and H. Zhang, "Learning semantic concepts from user feedback log for
image retrieval," To appear in IEEE International Conference on Multimedia and Expo 2004, Taipei,
Taiwan.

   
　

You might also like