Final Year Project Report
Final Year Project Report
INTRODUCTION
1.1. MOTIVATION:
In many areas of commerce, government, academia, and hospitals, large
collections of digital images are being created. Many of these collections are the
product of digitizing existing collections of analogue photographs, diagrams,
drawings, paintings, and prints. Usually, the only way of searching these collections
was by keyword indexing, or simply by browsing. Digital images databases however,
open the way to content-based searching.
Since the 1970s Image Retrieval has become a very active research topic, with
two major research communities, database management and computer vision. One is
text-based and another is visual-based. Text-based image retrieval has become very
popular since 1970s, which involves annotating the image with keywords, and use
text-based database management systems (DBMS) to retrieve the images. In text-
based image retrieval system, keywords of semantic information are attached to the
images.
They can be typed manually or by extracting the captions of the images. It is
very efficient for simple and small image databases, since the whole database can be
described by just few hundreds of keywords. But in the 1990s, several large leaps in
development of processor, memory and storage made the size of image databases
grow dramatically. As the image database and image size grow, there will be more
images having different contents and the images having rich contents cannot be
described by only several Semantic keywords. The demand of labor on annotating the
images also rises dramatically.
Retrieval image providing effective and efficient tool querying large image
database. Information retrieval provides the textual representation of images. It
requires the text descriptions to the respective images.
Recent technology development in various fields has made large digital image
databases practical. Well organized database and efficient browsing, storing, and
1
retrieval algorithms are very important in such systems. Image retrieval techniques
were developed to aid these components.
Image Retrieval was originated from Information Retrieval, which has been
very active research topic since 1940s. “We have huge amounts of information to
which accurate and speedy access is becoming ever more difficult.” In principle,
Information Retrieval is simple. It can be illustrated by a scene of a store of
documents and a person (user of the store). He formulates a question to which the
answer is a set of documents satisfying his question. He can obtain the set by reading
all the documents in the store, retaining the relevant documents and discarding all the
others. In this scene, it is a ’perfect’ retrieval. But in practice, we need to model the
“read” process in both syntactic and semantic to extract useful information. The target
of Information Retrieval is not only “how to extract useful information”, but also
“how to measure relevance among documents”. These challenges also exist in Image
Retrieval.
Also the keywords are very dependent on the observer’s interest and they are
subjective. Captions are not always precisely describing the picture.
Indexing and searching a large image database via keywords are time-
consuming and inefficient. Content Based Image Retrieval (CBIR) researches attempt
to automate such complex process of retrieving images that are similar to the
reference image or descriptions given.
Libraries have traditionally used manual image annotation for indexing and
then later retrieving their image collections. However, manual image annotation is an
expensive and labor intensive procedure and hence there has been great interest in
coming up with automatic ways to retrieve images based on content. Here, we
propose an automatic approach to annotating and retrieving images based on a
training set of images.
We assume that regions in an image can be described using a small vocabulary
of blobs. Blobs are generated from image features using clustering. Given a training
set of images with annotations, we show that probabilistic models allow us to predict
the probability of generating a word given the blobs in an image
This may be used to automatically annotate and retrieve images given a word
as a query. We show that relevance models. Allow us to derive these probabilities in a
natural way. Experiments show that the annotation performance of this cross-media
relevance model is almost six times as good (in terms of mean precision) than a model
based on word-blob co-occurrence model and twice as good as a state of the art model
derived from machine translation. Our approach shows the usefulness of using formal
information retrieval models for the task of image annotation and retrieval.
Query expansion methods have been studied for a long time - with debatable
success in many instances. In this project we present a probabilistic query expansion
model based on a similarity thesaurus which was constructed automatically. A
similarity thesaurus reflects domain knowledge about the particular collection from
which it is constructed. We address the two important issues with query expansion:
the selection and the weighting of additional search terms. In contrast to earlier
methods, our queries are expanded by adding those terms that are most similar to the
concept of the query, rather than selecting terms that are similar to the query terms.
Our experiments show that this kind of query expansion results in a notable
improvement in the retrieval effectiveness when measured using both recall-precision
and usefulness.
Query System Bridging the Semantic Gap for Large Image Databases
We propose a novel system called HISA for organizing very large image
databases. HISA implements the first known data structure to capture both the
ontological knowledge and visual features for effective and efficient retrieval of
images by either keywords, image examples, or both. HISA employs automatic image
annotation technique, ontology analysis and statistical analysis of domain knowledge
to precompile the data structure. Using these techniques, HISA is able to bridge the
gap between the image semantics and the visual features, therefore providing more
user-friendly and high performance queries. We demonstrate the novel data structure
employed by HISA, the query algorithms, and the pre-computation process.
2.1.1. DISADVANTAGES:
2.2.1. MODULES:
• INDEXING:
• SEARCHING:
In this module we are going search for the images whatever we selected
for the search.
Query update User’s
Feedbac k
Feature Extractio n
Similarit y measure Find all image s?
Input Retrieval Result
Query
SPECIFICATION: Introduction:
Software Requirements Specification plays an important role in creating
quality software solutions. Specification is basically a representation process.
Requirements are represented in a manner that ultimately leads to successful software
implementation. The software requirements specification is produced at the
culmination of the analysis task. The function and performance allocated to the
software as a part of system engineering are refined by establishing a complete
information description, a detailed functional and behavioral description, and
indication of performance requirements and design constraints, appropriate validation
criteria and other data pertinent to requirements.
The incremental model combines elements of the linear sequential model with
the iterative philosophy of prototyping. Each linear sequence produces a deliverable
“increment” of the software. When an incremental model is used, the first increment
is often a core product. That is, basic requirements are addressed, but many
supplementary features (some known, others unknown) remain undelivered. The core
product is used by the customer (or undergoes detailed review). As a result of use
and/or evaluation, a plan is developed for the next increment. The plan addresses the
modification of the core product to better meet the needs of the customer and the
delivery of additional features and functionality. This process is repeated following
the delivery of each increment, until the complete product is produced.
The model that is basically being followed is the WATER FALL MODEL,
which states that the phases are organized in a linear order. First of all the feasibility
study is done. Once that part is over the requirement analysis and project planning
begins. If system exists one and modification and addition of new module is needed,
analysis of present system can be used as basic model.
The design starts after the requirement analysis is complete and the coding
begins after the design is complete. Once the programming is completed, the testing is
done. In this model the sequence of activities performed in a software development
project are: -
WATER FALL MODEL was being chosen because all requirements were
known beforehand and the objective of our software development is the
computerization/automation of an already existing manual working system.
Figure2: WATER FALL MODEL
3.11. Software Requirements:
4.1. INTRODUCTION:
Software design sits at the technical kernel of the software engineering process
and is applied regardless of the development paradigm and area of application. Design
is the first step in the development phase for any engineered product or system. The
designer’s goal is to produce a model or representation of an entity that will later be
built. Beginning, once system requirement have been specified and analyzed, system
design is the first of the three technical activities -design, code and test that is required
to build and verify software.
The importance can be stated with a single word “Quality”. Design is the place
where quality is fostered in software development. Design provides us with
representations of software that can assess for quality. Design is the only way that we
can accurately translate a customer’s view into a finished software product or system.
Software design serves as a foundation for all the software engineering steps that
follow. Without a strong design we risk building an unstable system – one that will be
difficult to test, one whose quality cannot be assessed until the last stage.
During design, progressive refinement of data structure, program structure,
and procedural details are developed reviewed and documented. System design can be
viewed from either technical or project management perspective. From the technical
point of view, design is comprised of four activities – architectural design, data
structure design, interface design and procedural design.
UML DIAGRAMS:
Every complex system is best approached through a small set of nearly
independent views of a model; no single viewer is sufficient. Every model may
be expressed at different levels of fidelity. The best models are connected to reality.
The UML defines nine graphical diagrams.
1. Class diagram
2. Object diagram
3. Use-case diagram
4. Sequence diagram
5. Collaboration diagram
6. Activity diagram
7. State Chart diagram
8. Component diagram
9. Deployment diagram
CLASS DIAGRAM:
Class diagrams are the most common diagrams found in modeling object-
oriented systems. A class diagram shows a set of classes, interfaces, and
collaborations and their relationships. Graphically, a class diagram is a collection of
vertices and arcs.
SEQUENCE DIAGRAM:
A sequence diagram is an interaction diagram that emphasizes the time
ordering of the messages. Graphically, a sequence diagram is a table that shows
objects arranged along the X-axis and messages, ordered in increasing time, along the
Y-axis.
COLABORATION DIAGRAM:
Collaboration diagrams render how behavior is realized by components
with in a system. These diagrams contain classes, associations, and their message
exchanges with in a collaboration to accomplish a purpose. In generic form,
these diagrams describe a set of classes and associations involved in message
exchange sequences. In instance form(scenarios), these diagrams describe a set of
objects of those classes links confirming to the associations, and one actual
message exchange sequence that inconsistent with the generic form and uses
those objects and links
ACTIVITY DIAGRAM:
An Activity Diagram is essentially a flow chart showing flow of control from
activity to activity. They are used to model the dynamic aspects of as system. They
can also be used to model the flow of an object as it moves from state to state at
different points in the flow of control.
COMPONENT DIAGRAM:
A component diagram shows a set of components and their relationships. We
use the component diagram to illustrate the static implementation view of a system.
Component diagrams are related to class diagrams in that a component typically maps
to one or more classes, interfaces, or collaborations.
DEPLOYMENT DIAGRAM:
A deployment diagram shows a set of nodes and their relationships. We use
deployment diagrams to illustrate the static deployment view of architecture.
Deployment diagrams are related to component diagrams in that a node typically
encloses one or more components.
UML DIAGRAMS
Writing in the Java programming language is the primary way to produce code
that will be deployed as Java bytecode. There are, however, bytecodecompilers
available for other languages such as Ada, JavaScript, Python, and Ruby. Several
new languages have been designed to run natively on theJava Virtual Machine (JVM),
such as Scala, Clojure and Groovy. Java syntax borrows heavily from C and C++,
but object-oriented features are modeled after Smalltalk and Objective-C.[6]
Java eliminates certain low-level constructs such as pointers and has a very simple
memory model where every object is allocated on the heap and all variables of object
types are references. Memory management is handled through integrated automatic
garbage collection performed by the JVM
An edition of the Java platform is the name for a bundle of related programs
from Sun that allow for developing and running programs written in the Java
programming language. The platform is not specific to any one processor or operating
system, but rather an execution engine (called a virtual machine) and a compiler with
a set of libraries that are implemented for various hardware and operating systems so
that Java programs can run identically on all of them.
The essential components in the platform are the Java language compiler, the
libraries, and the runtime environment in which Java intermediate byte code
"executes" according to the rules laid out in the virtual machine specification.
The heart of the Java platform is the concept of a "virtual machine" that
executes Java byte code programs. This byte code is the same no matter what
hardware or operating system the program is running under. There is a JIT(Just In
Time) compiler within the Java Virtual Machine, or JVM. The JIT compiler translates
the Java bytecode into native processor instructions at run-time and caches the native
code in memory during execution.
For instance, Swing library paints the user interface and handles the events
itself, eliminating many subtle differences between how different platforms handle
even similar components.
The Java class libraries serve three purposes within the Java platform. First,
like other standard code libraries, the Java libraries provide the programmer a well-
known set of functions to perform common tasks, such as maintaining lists of items or
performing complex string parsing. Second, the class libraries provide an abstract
interface to tasks that would normally depend heavily on the hardware and operating
system. Tasks such as network access and file access are often heavily intertwined
with distinctive implementations of each platform. The java.net andjava.io libraries
implement an abstraction layer in native OS code, then provide a standard interface
for the Java applications to perform those tasks. Finally, when some underlying
platform does not support all of the features a Java application expects, the class
libraries work to gracefully handle the absent components, either by emulation to
provide a substitute, or at least by providing a consistent way to check for the
presence of a specific feature
Implementation is the stage of the project when the theoretical design is turned
out into a working system. Thus it can be considered to be the most critical stage in
achieving a successful new system and in giving the user, confidence that the new
system will work and be effective.
The implementation stage involves careful planning, investigation of the
existing system and it’s constraints on implementation, designing of methods to
achieve changeover and evaluation of changeover methods.
6. OUTPUT SCREENS
6.1. EXPERIMENT:
The results obtained after training the neural network on the dataset are illustrated in
table below. The neural network was trained on the dataset for each label. It can be
observed in the table that the validation error rate and testing error rate for each label
was quite low. Buildings and trees were two of the worst classified labels in the
network with both their error rates in testing and validation being over 40%. In cases
like snow the validation and error rate was initially 100% but after some iterations
the error rates came down tremendously to 14.6% and 11.4% respectively as shown
in table This shows that the network learned this class label very well. High error
rates can be attributed to the fact that images were downsized to 28x28 pixels which
led to loss of information in them.
We can see that in all the classes the best test and validation error rates are
achieved on iteration 3, 6 or 9. Training runs for 500 iterations just to validate that
there is no change in the error rate after every 100 iterations and once it reaches 500
with a constant rate it stops and returns the best error rates. The training runs for a
long time as can be seen in the table. Average time for it to finish training a label is
almost 35 minutes.
This can be improved by using a GPU and a better processor. It can be
concluded from the results shown in table that the network did a good 27 job in
learning the labels Sky, Snow and Ground. Water, Car and Mountain labels showed
average classification rate. The network did not train that well for labels Building
and Tree. We can say this is due to the fact that loss of information due to
downsizing made it difficult for network to classify between trees and buildings
because they both overlap each other in many cases. This results in the error rate to
go high for these labels
6.2. Evaluation:
6.3. Output Screen
INTRODUCTION:
Unit testing:
Unit testing involves the design of test cases that validate that the internal
program logic is functioning properly, and that program inputs produce valid outputs.
All decision branches and internal code flow should be validated. It is the testing of
individual software units of the application .it is done after the completion of an
individual unit before integration. This is a structural testing, that relies on knowledge
of its construction and is invasive. Unit tests perform basic tests at component level
and test a specific business process, application, and/or system configuration. Unit
tests ensure that each unique path of a business process performs accurately to the
documented specifications and contains clearly defined inputs and expected results.
Integration testing:
Functional testing:
System Testing:
System testing ensures that the entire integrated software system meets
requirements. It tests a configuration to ensure known and predictable results. An
example of system testing is the configuration oriented system integration test.
System testing is based on process descriptions and flows, emphasizing pre-driven
process links and integration points.
VALIDATION:
]
8. CONCLUSION
8.1. CONCLUSION
Among the objectives of this paper performed to design, implement and test a
sketch-based image retrieval system. Two main aspects were taken into account. The
retrieval process has to be unconventional and highly interactive. The robustness of
the method is essential in some degree of noise, which might also be in case of simple
images. The drawn image without modification cannot be compared with color image,
or its edge representation. Alternatively a distance transform step was introduced. The
simple smoothing and edge detection based method was improved, which had a
similar importance as the previous step.
At the tests the effectiveness of EHD and the dynamically parameterized HOG
implementation was compared. It was examined with more databases. In our
experience the HOG in more cases was much better than the EHD based retrieval.
However, the situation is not so simple. The edge histogram descriptor can mainly
look better for information poor sketches, while in other case better results can be
achieved for more detailed. This is due to the sliding window solution of HOG.
9. REFERENCES
[1] Multimedia Content Description Interface – Part 3: Visual, ISO/IEC
JTC1/SC29/WG11/N4062, 2008.
[2] J. Wang and G. Wiederhold, “SIMPLIcity: Semantics-Sensitive Integrated
Matching for Picture Libraries,” IEEE Transactions On Pattern Analysis And
Machine Intelligence, vol. 23, no. 8, pp. 1-17, September 2008.
[3] M. Swain and D. Ballard, “Color indexing,” International Journal of Computer
Vision, vol.. 7, no.1, pp. 11-32, 2008.
[4] K. Hirata and T. Kato, "Query by visual example, content based image retrieval,"
Advances in Database Technology-EDBT'92, vol. 580, pp. 56-71, A. Pirotte,
C.Delobel, and G. Gottlob, Eds., 2006, Springer-Verlag.
[5] J. Jeon, V. Lavrenko, and R. Manmatha, “Automatic Image Annotation and
Retrieval Using Cross-Media Relevance Models,” Proc. 26th Ann. Int’l ACM SIGIR
Conf. Research and Development in Information Retrieval (SIGIR ’03), pp. 119-126,
2003.
[6] J. Kalervo, K. Jaana, and N. Timo, “ExpansionTool: Concept-Based
Query Expansion and Construction,” Information Retrieval, vol. 4, no. 3, pp. 231-
255, 2001.
[7] G. Chen, X. Li, L. Shou, J. Dong, and C. Chen, “HISA: A Query
System Bridging the Semantic Gap for Large Image Databases (Demo),” Proc. 32nd
Int’l Conf. Very Large Data Bases (VLDB ’06), pp. 1187-1190, 2006.
[8] X.Y. Li, L.D. Shou, G. Chen, and K.-L. Tan, “An Image-Semantic
Ontological Framework for Large Image Databases (Poster),” Proc. 12th Int’l Conf.
Database Systems for Advanced Applications (DASFAA ’07), pp. 1050-1053, 2007.
[9] J.R. Paek and S. Smith, “Detecting Image Purpose in World-Wide Web
Documents,” Proc. IS&T/SPIE Symp. Electronic Imaging: Science and Technology—
Document Recognition, vol. 3305, pp. 151- 158, Jan. 1998.
[10] A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, “Content-
Based Image Retrieval at the End of the Early Years,” IEEE Trans. Pattern Analysis
and Machine Intelligence, vol. 22, no. 12, pp. 1349-1380, Dec. 2000.
APPENDIX
10.1. SAMPLE CODE:
public class MainFrame extends JFrame
{
public MainFrame() {
try {
UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName(
));
} catch (Exception e) {
}
initComponents();
try {
Image icon = ImageIO.read(getClass().getResource("/resources/viewmag16.png"));
if (icon!=null) setIconImage(icon);
} catch (IOException ex)
{ Logger.getLogger(MainFrame.class.getName()).log(Level.SEVERE, null,
ex);
}
selectboxDocumentBuilder.setSelectedIndex(5);
buttonSwitchIndex.setBackground(highlightSelectColor);
DropTarget t = new DropTarget(searchPanel, new DropTargetListener() {
textfieldIndexDir.setEditable(false);
textfieldIndexDir.addActionListener(new java.awt.event.ActionListener() {
public void actionPerformed(java.awt.event.ActionEvent evt) {
textfieldIndexDirActionPerformed(evt);
}
});
buttonOpenDir.setText(bundle.getString("button.open.indexdirectory")); //
NOI18N
buttonOpenDir.addActionListener(new java.awt.event.ActionListener() {
public void actionPerformed(java.awt.event.ActionEvent evt) {
buttonOpenDirActionPerformed(evt);
}
});
buttonStartIndexing.setText(bundle.getString("button.start.indexing")); //
NOI18N
buttonStartIndexing.addActionListener(new java.awt.event.ActionListener() {
public void actionPerformed(java.awt.event.ActionEvent evt) {
buttonStartIndexingActionPerformed(evt);
}
});
progressBarIndexing.setFocusable(false);
progressBarIndexing.setName(bundle.getString("progressbar.indexing.name")); /
/ NOI18N
progressBarIndexing.setString(bundle.getString("progressbar.indexing.name"));
// NOI18N
progressBarIndexing.setStringPainted(true);
textfieldSearchImage.setEditable(false);
buttonOpenImage.setText(bundle.getString("button.open.searchimage")); //
NOI18N
buttonOpenImage.setActionCommand(bundle.getString("openImageButton")); //
NOI18N
buttonOpenImage.addActionListener(new java.awt.event.ActionListener() {
public void actionPerformed(java.awt.event.ActionEvent evt) {
buttonOpenImageActionPerformed(evt);
}
});
buttonStartSearch.setText(bundle.getString("button.start.search")); // NOI18N
buttonStartSearch.addActionListener(new java.awt.event.ActionListener() {
public void actionPerformed(java.awt.event.ActionEvent evt) {
buttonStartSearchActionPerformed(evt);
}
});
progressMosaic.setToolTipText(bundle.getString("mosaic.progress.tooltip")); //
NOI18N
progressMosaic.setString(bundle.getString("progressbar.mosaic.name")); //
NOI18N
progressMosaic.setStringPainted(true);
jLabel12.setText("Number of tiles:");
jLabel12.setVerticalAlignment(javax.swing.SwingConstants.BOTTOM);
buttonMosaicSave.setText("Save result.....");
buttonMosaicSave.setToolTipText(bundle.getString("mosaic.saveButton.tooltip"
));
private void vieMenuStartPageActionPerformed(java.awt.event.ActionEvent evt)
{//GEN-FIRST:event_vieMenuStartPageActionPerformed
((CardLayout) topPane.getLayout()).first(topPane);
}
public static void main(String args[])
{ java.awt.EventQueue.invokeLater(new Runnable()
{
public void run() {
new MainFrame().setVisible(true);
}
});
}