ACTIVE LEARNING METHODS FOR INTERACTIVE IMAGE RETRIEVAL

Abstract

The project titled ACTIVE LEARNING METHODS FOR

INTERACTIVE IMAGE RETRIEVAL is designed using.NET

with Microsoft Visual Studio.Net 2005 as front end which works in .Net framework version 2.0. The coding language used is C# .Net. Active learning methods have been considered with increased interest in the statistical learning community. Initially developed within a classification framework, a lot of extensions are now being proposed to handle multimedia applications. This paper provides algorithms within a statistical framework to extend active learning for online content-based image retrieval (CBIR). The classification framework is presented with experiments to compare several powerful classification techniques in this information retrieval context. Focusing on interactive methods, active learning strategy is then described. The limitations of this approach for CBIR are emphasized before presenting our new active selection process RETIN. First, as any active method is sensitive to the boundary estimation
1

between classes, the RETIN strategy carries out a boundary correction to make the retrieval process more robust. Second, the criterion of generalization error to optimize the active learning selection is modified to better represent the CBIR objective of database ranking. Third, a batch processing of images is proposed. Our strategy leads to a fast and efficient active learning scheme to retrieve sets of online images (query concept). Experiments on large databases show that the RETIN method performs well in comparison to several other active strategies. 1. INTRODUCTION Human interactive systems have attracted a lot of research interest in recent years, especially for content- based image retrieval systems. Contrary to the early systems, which focused on fully automatic strategies, recent approaches have introduced human-computer interaction. In this paper, we focus on the retrieval of concepts within a large image collection. We assume that a user is looking for a set of images, the query concept, within a database. The aim is to build a fast and efficient strategy to retrieve the query Concept. In content-based image retrieval (CBIR), the search may be initiated using a query as an example. The top rank similar images are then presented to the user. Then, the interactive process allows the user to refine his request as much as necessary in

a relevance feedback loop. Many kinds of interaction between the user and the system have been proposed, but most of the time, user information consists of binary labels indicating whether or not the image belongs to the desired concept. 1.1 PROJECT DESCRIPTION The project entitled as Active Learning Methods for Interactive Image Retrieval developed using .NET using C#. Modules display as follows.

RGB Projection Image Utility Comparable Image Similarity Images Result

MODULES DESCRIPTION:

1) RGB Projections: The RGB color model is an additive color model in which red, green, and blue light are added together in various ways to reproduce a broad array of colors. The name of the model comes from the initials of the three additive primary colors, red, green, and blue. The main purpose of the RGB color model is for the sensing, representation, and display of images in electronic systems, such as conventional photography. In this module the RGB Projections is used to find the size of the image vertically and horizontally. 2) Image Utility: Whenever minimizing the error of classification is interesting for CBIR, this criterion does not completely reflect the user satisfaction. Other utility

criteria Closer to this, such as precision, should provide more efficient selections.

3) Comparable Image: In this module a reselection technique to speed up the selection process, which leads to a computational complexity negligible compared to the size of the database for the whole active learning process. All these components are integrated in our retrieval system, called RETIN and the user gives new labels for images, and they are compared to the current classification. If the user mostly gives relevant labels, the system should propose new images for labeling around a higher rank to get more irrelevant labels. 4) Similarity measure: The results in terms of mean average precision according to the training set size (we omit the KFD which gives results very close to inductive SVMs) for both ANN and Corel databases. One can see that the classification-based methods give the best results, showing the power of statistical methods over geometrical approaches, like the one reported here (similarity refinement method).
5

5) Result: Finally, the image will take the relevant image what the user search. One can see that we have selected concepts of different levels of complexities. The performances go from few percentages of Mean average precision to 89%. The concepts that are the most difficult to retrieve are very small and/or have a much diversified visual content. The method which aims at minimizing the error of generalization is the less efficient active learning method. The most efficient method is the precision- oriented method.

Graph: This module is used to determine relationships between the two Images. The precision and recall values are measured by simulating retrieval scenario. For each simulation, an image category is randomly chosen. Next, 100 images are selected using active learning and labeled according to the chosen category. These labeled images are used to train a classifier, which returns a ranking of the database. The average precision is then computed

using the ranking. These simulations are repeated 1000 times, and all values are averaged to get the Mean average precision. Next, we repeat ten times these simulations to get the mean and the standard deviation of the MAP

Input/Output:

The image will take the relevant image what the user search. one can see that we have selected concepts of different levels of complexities. The performances go from few percentages of Mean average precision to 89%. The concepts that are the most difficult to retrieve are very small and/or have a very diversified visual content

2. SYSTEM STUDY

2.1 FEASIBILITY STUDY

The feasibility of the project is analyzed in this phase and business proposal is put forth with a very general plan for the project and some cost estimates. During system analysis the feasibility study of the proposed system is to be carried out. This is to ensure that the proposed system is not a burden to the company. For feasibility analysis, some

understanding of the major requirements for the system is essential.

Three key considerations involved in the feasibility analysis are

ECONOMICAL FEASIBILITY TECHNICAL FEASIBILITY SOCIAL FEASIBILITY

ECONOMICAL FEASIBILITY

This study is carried out to check the economic impact that the system will have on the organization. The amount of fund that the company can pour into the research and development of the system is limited. The expenditures must be justified. Thus the developed system as well within the budget and this was achieved because most of the technologies used are freely available. Only the customized products had to be purchased.

TECHNICAL FEASIBILITY

This study is carried out to check the technical feasibility, that is, the technical requirements of the system. Any system developed must not have a high demand on the available technical resources. This will lead to high demands on the available technical resources. This will lead to high

demands being placed on the client. The developed system must have a modest requirement, as only minimal or null changes are required for implementing this system.

SOCIAL FEASIBILITY

The aspect of study is to check the level of acceptance of the system by the user. This includes the process of training the user to use the system efficiently. The user must not feel threatened by the system, instead must accept it as a necessity. The level of acceptance by the users solely depends on the methods that are employed to educate the user about the system and to make him familiar with it. His level of confidence must be raised so that he is also able to make some constructive criticism, which is welcomed, as he is the final user of the system. 2.2 EXISTING SYSTEM In the existing system the CBIR method faced a lot of disadvantage in case of the image retrival. The following are the main disadvantage faced in case of the medical field - Medical image description is an important problem in content-based medical image retrieval. Hierarchical medical image semantic features description model is proposed according to the main sources to get

10

semantic features currently. Hence we propose the new algorithim to over come the existing system.

In existing system ,Images were first annotated with text and then searched using a text-based approach from traditional database management systems.

2.3 PROPOSED SYSTEM In case of the proposed system we use the following method to improve the efficiency. They are as follows. We implemented our models in a CBIR system for a specific application domain, the retrieval of coats of arms. We implemented altogether 19 features, including a color histogram, symmetry features. Content-based image retrieval, uses the visual contents of an image such as color, shape, texture, and spatial layout to represent and index the image

11

3. SYSTEM SPECIFICATION

3.1 HARDWARE CONFIGURATION

SYSTEM HARD DISK

: Pentium IV 2.4 GHz : 40 GB

FLOPPY DRIVE : 1.44 MB MONITOR MOUSE RAM KEYBOARD : 15 VGA colour : Logitech. : 256 MB : 110 keys enhanced.

3.2 SOFTWARE CONFIGURATION

Operating system :- Windows XP Professional Front End :- Microsoft Visual Studio .Net 2005
12

Coding Language : - C# 2005.

4. LANGAUGE SPECIFICATION

4.1 FEATURES OF. NET

Microsoft .NET is a set of Microsoft software technologies for rapidly building and integrating XML Web services, Microsoft Windowsbased applications, and Web solutions. The .NET Framework is a languageneutral platform for writing programs that can easily and securely interoperate. Theres no language barrier with .NET: there are numerous languages available to the developer including Managed C++, C#, Visual Basic and Java Script. The .NET framework provides the foundation for components to interact seamlessly, whether locally or remotely on different platforms. It standardizes common data types and communications protocols so that components created in different languages can easily interoperate.

.NET is also the collective name given to various software components built upon the .NET platform. These will be both products

13

(Visual Studio.NET and Windows.NET Server, for instance) and services (like Passport, .NET My Services, and so on).

THE .NET FRAMEWORK

The .NET Framework has two main parts:

1. The Common Language Runtime (CLR). 2. A hierarchical set of class libraries.

The CLR is described as the execution engine of .NET. It provides the environment within which programs run. The most important features are

Conversion from a low-level assembler-style language, called Intermediate Language (IL), into code native to the platform being executed on. Memory management, notably including garbage collection. Checking and enforcing security restrictions on the running code.

14

 Loading and executing programs, with version control and other such features. The following features of the .NET framework are also worth description: Managed Code The code that targets .NET, and which contains certain extra Information - metadata - to describe itself. Whilst both managed and unmanaged code can run in the runtime, only managed code contains the information that allows the CLR to guarantee, for instance, safe execution and interoperability.

Managed Data With Managed Code comes Managed Data. CLR provides memory allocation and Deal location facilities, and garbage collection. Some .NET languages use Managed Data by default, such as C#, Visual Basic.NET and JScript.NET, whereas others, namely C++, do not. Targeting CLR can, depending on the language youre using, impose certain constraints on the features available. As with managed and unmanaged code, one can have both managed and unmanaged data in .NET applications - data

15

that doesnt get garbage collected but instead is looked after by unmanaged code.

Common Type System The CLR uses something called the Common Type System (CTS) to strictly enforce type-safety. This ensures that all classes are compatible with each other, by describing types in a common way. CTS define how types work within the runtime, which enables types in one language to interoperate with types in another language, including cross-language exception handling. As well as ensuring that types are only used in appropriate ways, the runtime also ensures that code doesnt attempt to access memory that hasnt been allocated to it.

Common Language Specification The CLR provides built-in support for language interoperability. To ensure that you can develop managed code that can be fully used by developers using any programming language, a set of language features and

16

rules for using them called the Common Language Specification (CLS) has been defined. Components that follow these rules and expose only CLS features are considered CLS-compliant.

THE CLASS LIBRARY .NET provides a single-rooted hierarchy of classes, containing over 7000 types. The root of the namespace is called System; this contains basic types like Byte, Double, Boolean, and String, as well as Object. All objects derive from System. Object. As well as objects, there are value types. Value types can be allocated on the stack, which can provide useful flexibility. There are also efficient means of converting value types to object types if and when necessary.

The set of classes is pretty comprehensive, providing collections, file, screen, and network I/O, threading, and so on, as well as XML and database connectivity.

17

The class library is subdivided into a number of sets (or namespaces), each providing distinct areas of functionality, with dependencies between the namespaces kept to a minimum.

LANGUAGES SUPPORTED BY .NET

The multi-language capability of the .NET Framework and Visual Studio .NET enables developers to use their existing programming skills to build all types of applications and XML Web services. The .NET framework supports new versions of Microsofts old favorites Visual Basic and C++ (as VB.NET and Managed C++), but there are also a number of new additions to the family. Visual Basic .NET has been updated to include many new and improved language features that make it a powerful object-oriented programming language. These features include inheritance, interfaces, and overloading, among others. Visual Basic also now supports structured exception handling, custom attributes and also supports multi-threading.

18

Visual Basic .NET is also CLS compliant, which means that any CLS-compliant language can use the classes, objects, and components you create in Visual Basic .NET. Managed Extensions for C++ and attributed programming are just some of the enhancements made to the C++ language. Managed Extensions simplify the task of migrating existing C++ applications to the new .NET Framework. C# is Microsofts new language. Its a C-style language that is essentially C++ for Rapid Application Development. Unlike other languages, its specification is just the grammar of the language. It has no standard library of its own, and instead has been designed with the intention of using the .NET libraries as its own.

Microsoft Visual J# .NET provides the easiest transition for Java-language developers into the world of XML Web Services and dramatically improves the interoperability of Java-language programs with existing software written in a variety of other programming languages.

19

Active State has created Visual Perl and Visual Python, which enable .NET-aware applications to be built in either Perl or Python. Both products can be integrated into the Visual Studio .NET environment. Visual Perl includes support for Active States Perl Dev Kit.

Other languages for which .NET compilers are available include

FORTRAN COBOL Eiffel

Fig1 .Net Framework

ASP.NET XML SERVICES WEB Forms

Windows

20

Base Class Libraries Common Language Runtime Operating System

C#.NET is also compliant with CLS (Common Language Specification) and supports structured exception handling. CLS is set of rules and constructs that are supported by the CLR (Common Language Runtime). CLR is the runtime environment provided by the .NET Framework; it manages the execution of the code and also makes the development process easier by providing services. C#.NET is a CLS-compliant language. Any objects, classes, or components that created in C#.NET can be used in any other CLScompliant language. In addition, we can use objects, classes, and components created in other CLS-compliant languages in C#.NET .The

21

use of CLS ensures complete interoperability among applications, regardless of the languages used to create the application. CONSTRUCTORS AND DESTRUCTORS: Constructors are used to initialize objects, whereas destructors are used to destroy them. In other words, destructors are used to release the resources allocated to the object. In C#.NET the sub finalize procedure is available. The sub finalize procedure is used to complete the tasks that must be performed when an object is destroyed. The sub finalize procedure is called automatically when an object is destroyed. In addition, the sub finalize procedure can be called only from the class it belongs to or from derived classes. GARBAGE COLLECTION Garbage Collection is another new feature in C#.NET. The .NET Framework monitors allocated resources, such as objects and variables. In addition, the .NET Framework automatically releases memory for reuse by destroying objects that are no longer in use. In C#.NET, the garbage collector checks for the objects that are not currently in use by applications. When the garbage collector comes

22

across an object that is marked for garbage collection, it releases the memory occupied by the object. OVERLOADING Overloading is another feature in C#. Overloading enables us to define multiple procedures with the same name, where each procedure has a different set of arguments. Besides using overloading for procedures, we can use it for constructors and properties in a class.

MULTITHREADING: C#.NET also supports multithreading. An application that supports multithreading can handle multiple tasks simultaneously, we can use multithreading to decrease the time taken by an application to respond to user interaction. STRUCTURED EXCEPTION HANDLING C#.NET supports structured handling, which enables us to detect and remove errors at runtime. In C#.NET, we need to use TryCatchFinally statements to create exception handlers. Using

23

TryCatchFinally statements, we can create robust and effective exception handlers to improve the performance of our application. THE .NET FRAMEWORK The .NET Framework is a new computing platform that simplifies application development in the highly distributed environment of the Internet. OBJECTIVES OF. NET FRAMEWORK 1. To provide a consistent object-oriented programming environment whether object codes is stored and executed locally on Internetdistributed, or executed remotely. 2. To provide a code-execution environment to minimizes software deployment and guarantees safe execution of code. 3. Eliminates the performance problems. There are different types of application, such as Windows-based applications and Web-based applications.

24

5. SYSTEM DESIGN

5.1 MODULE DIAGRAM:

Computing Similarity Measure

Image Utility and compare eg color Average and graph values RGB Projections 25

5.2 UML Diagrams: 5.2.1 Use case Diagram:

26

5.2.2 Class Diagram:

27

5.2.3 Object Diagram:

RGB Projections

Image Utility and select the image

Compare the source and designation images

Draw the graph

Find the average values of the image

28

5.2.4 State Diagram:

29

5.2.5 Activity Diagram:

30

31

5.2.6 Sequence Diagram:

5.2.7 Collaboration Diagram:

32

5.2.8 Component Diagram:

33

5.3 E-R Diagram:

34

Source Remove

Zip

Send the File

process

Processes

Unzip

Server System
User Select File
Source Client

create
process Remove Destn

delete

5.4 Data Flow Diagram:

35

Check Image

No RGB Projections Yes New Image

Compare& Utility Similarity Images

Draw the Graph Exit

5.5 Project Flow Diagram:

Image Utility Compare the Image

RGB Projections

36

5.6 System Architecture:

37

6. SYSTEM TESTING AND MAINTENANCE

Testing is vital to the success of the system. System testing makes a logical assumption that if all parts of the system are correct, the goal will be successfully achieved. In the testing process we test the actual system in an organization and gather errors from the new system operates in full efficiency as stated. System testing is the stage of implementation, which is aimed to ensuring that the system works accurately and efficiently. In the testing process we test the actual system in an organization and gather errors from the new system and take initiatives to correct the same. All the front-end and back-end connectivity are tested to be sure that the new system operates in full efficiency as stated. System testing is the stage of implementation, which is aimed at ensuring that the system works accurately and efficiently. The main objective of testing is to uncover errors from the system. For the uncovering process we have to give proper input data to the system. So we should have more conscious to give input data. It is important to give correct inputs to efficient testing.

38

Testing is done for each module. After testing all the modules, the modules are integrated and testing of the final system is done with the test data, specially designed to show that the system will operate successfully in all its aspects conditions. Thus the system testing is a confirmation that all is correct and an opportunity to show the user that the system works. Inadequate testing or non-testing leads to errors that may appear few months later. This will create two problems Time delay between the cause and appearance of the problem. The effect of the system errors on files and records within the system. The purpose of the system testing is to consider all the likely variations to which it will be suggested and push the system to its limits. The testing process focuses on logical intervals of the software ensuring that all the statements have been tested and on the function intervals (i.e.,) conducting tests to uncover errors and ensure that defined inputs will produce actual results that agree with the required results. Testing has to be done using the two common steps Unit testing and Integration testing. In the project system testing is made as follows:

39

The procedure level testing is made first. By giving improper inputs, the errors occurred are noted and eliminated. This is the final step in system life cycle. Here we implement the tested error-free system into real-life environment and make necessary changes, which runs in an online fashion. Here system maintenance is done every months or year based on company policies, and is checked for errors like runtime errors, long run errors and other maintenances like table verification and reports. 6.1. UNIT TESTING Unit testing verification efforts on the smallest unit of software design, module. This is known as Module Testing. The modules are tested separately. This testing is carried out during programming stage itself. In these testing steps, each module is found to be working satisfactorily as regard to the expected output from the module. 6.2. INTEGRATION TESTING Integration testing is a systematic technique for constructing tests to uncover error associated within the interface. In the project, all the modules are combined and then the entire programmer is tested as a

40

whole. In the integration-testing step, all the error uncovered is corrected for the next testing steps.

7. SYSTEM IMPLEMENTATION

Implementation is the stage of the project when the theoretical design is turned out into a working system. Thus it can be considered to be the most critical stage in achieving a successful new system and in giving the user, confidence that the new system will work and be effective. The implementation stage involves careful planning, investigation of the existing system and its constraints on implementation, designing of methods to achieve changeover and evaluation of changeover methods. Implementation is the process of converting a new system design into operation. It is the phase that focuses on user training, site preparation and file conversion for installing a candidate system. The important factor that should be considered here is that the conversion should not disrupt the functioning of the organization.

41

7.1 SCOPE FOR FUTURE ENHANCEMENTS The project has covered almost all the requirements. Further requirements and improvements can easily be done since the coding is mainly structured or modular in nature. Improvements can be appended by changing the existing modules or adding new modules.

8. CONCLUSION In this paper, the RETIN active learning strategy for interactive learning in content-based image retrieval context is presented. The classification framework for CBIR is studied and powerful classification techniques for information retrieval context are selected. After analyzing the limitation of active learning strategies to the CBIR context, we introduce the general RETIN active learning scheme, and the different components to deal with this particular context. The main contributions concern the boundary correction to make the retrieval process more robust, and secondly, the introduction of a new criterion for image selection that better represents the CBIR objective of database ranking. Other improvements, as batch

42

processing and speed-up process are proposed and discussed. Our strategy leads to a fast and efficient active learning scheme to online retrieve query concepts from a database. Experiments on large databases show that the RETIN method gives very good results in comparison to several other active strategies. The framework introduced in this article may be extended. We are currently working on kernel functions for object classes retrieval, based on bags of features: each image is no more represented by a single global vector, but by a set of vectors. The implementation of such a kernel function is fully compatible with the RETIN active learning scheme described in this article, and the initial results are really encouraging.

43

9. LITERATURE REVIEW

There are alternative ways to avoid the scheduling latency issue described above.

The main options are: 1) Bring the scheduler closer to the adapters; 2) Use provisioning (circuit switching); 3) Use a buffered switch core; 4) Eliminate the scheduler altogether. Although one can attempt to locate the scheduler as close to the adapters as possible, a certain distance determined by the system packaging limitations and requirements will remain. Although the RTT can be minimized, the fundamental problem of non-negligible RTTs remains valid. One can also do without cell-level allocation and rely on provisioning to resolve contention. Of course, this approach has several well-known drawbacks, such as a lack of flexibility, inefficient use of resources, and long set-up times when a new connection is needed, which make this approach unattractive for Parallel computer interconnects. An alternative approach is to provide buffers in the switch core and employ some form of link-level flow control (e.g.,credits) to manage them. As long as an adapter has credits, it can send immediately
44

without having to go through a centralized scheduling process. However, as optical buffering technology is currently neither practically nor

economically feasible and the key objective of OSMOSIS is to demonstrate the use of optics, this is not an option.

The last alternative is the load-balanced Birkhoffvon-Neumann switch, which eliminates the scheduler entirely. It consists of a distribution and a routing stage, with a set of buffers at the inputs of the second stage. Both stages are reconfigured periodically according to a sequence of permutation matrices.The first stage uniformizes the traffic regardless of destination, and the second stage performs the actual switching. Its main advantage is that, despite being crossbar-based, no centralized scheduler is required. Although this architecture has been shown to have 100% throughput under a technical condition on the traffic, it incurs a worst-case latency penalty of time slots: if a cell arrives at an empty VOQ just after the VOQ had a service opportunity, it has to wait for exactly time slots for the next opportunity. The mean latency penalty is time slots plus a minimum transit latency intrinsically added by the second stage. Moreover, missequencing can occur. This approach results in overall lower latency if

45

the total architecture-induced latency penalty can be expected to be less than the control-path latency In a traditional IQ switch. In the OSMOSIS system this is not the case, hence we choose the centrally-scheduled architecture.

SPECULATIVE TRANSMISSION:

Our objective is to eliminate the control-path latency in the absence of contention. To this end, we introduce a speculative transmission (STX) scheme. The principle behind STX is related to that of the original ALOHA and Ethernet protocols: Senders compete for a resource without prior scheduling. If there is a collision, the losing sender(s) must retry their data transmissions in a different time slot. However, the efficiency of ALOHAlike protocols is very poor (18.4% for pure ALOHA and 36.8% for slotted ALOHA) because under heavy load many collisions occur, reducing the effective throughput. Therefore, we propose a novel method to combine scheduled and speculative (non-scheduled) transmissions in a crossbar switch. The objective is to achieve reduced latency at low utilization owing to The speculative mode of operation and achieve high maximum Throughput owing to the scheduled mode of operation.

46

We consider the presence of multiple receivers per output port, allowing up to cells to arrive simultaneously. Although in OSMOSIS , we are interested in the general case with here. We exploit this feature to improve the STX success rate. The first receiver is for either a scheduled or a speculative cell. The extra receivers can accommodate additional speculative cells. Correspondingly, the STX arbitration can acknowledge multiple STX requests per output per time slot. The following rules govern the design of the STX scheme: Upon cell arrival, a request for scheduling (REQ) is issued to the central scheduler. This request is processed by a bipartite graph matching algorithm, and will eventually result in a corresponding scheduled grant (GRT). An adapter is eligible to perform an STX in a given time slot if it has no grant for a scheduled transmission in that time slot. Performing an STX involves selecting a cell, sending it on the data path, and issuing a corresponding speculative request (SRQ) on the control path. When multiple cells collide, cells proceed and the remaining cells are dropped. If the number of colliding cells is smaller than or equal to , all cells proceed. If more than cells collide, a scheduled cell (if present) always proceeds. Moreover, or (if a scheduled cell is present) randomly chosen speculative cells proceed. Every cell may be speculatively transmitted at most once.

47

Every speculative cell remains stored in its input adapter until it is either acknowledged as a successful STX or receives a scheduled grant. The scheduler acknowledges every successful speculative cell to the sending input by returning an acknowledgment (ACK). To this end, every cell, SRQ, and ACK carries a sequence number. However, when a grant arrives before theACK, a cell is transmitted a second time. These are called duplicate cells as opposed to the pure cells, which are transmitted through grants but are not duplicate. The corresponding grants are classified as duplicate and pure accordingly.Every grant is either regular, spurious, or wasted. It is regular if it is used by the cell that initiated it. A grant corresponding to a successfully speculatively transmitted and acknowledged cell is spurious when used by another cell residing in the same VOQ, resulting in a spurious transmission, or wasted if the VOQ is empty.If it is wasted, the slot can be used for a speculative transmission. STX Policy

According to , an adapter performs an STX in a given time slot if it receives no grant at and has an eligible cell. If it receives a grant, it performs the corresponding scheduled transmission. allows the STX scheme to operate in conjunction with regular scheduled transmissions, which take precedence?

48

over the speculative ones. Accordingly, we distinguish between scheduled and speculative cells. When an adapter is eligible to perform an STX, it selects a non-empty VOQ according to a specific STX policy, dequeue its HOL cell and stores it in a retransmission buffer, marks the cell as speculative, and sends it to the crossbar. On the control path, it sends an SRQ indicating that a cell has been sent speculatively to the selected output. Both the cell and the SRQ comprise a unique sequence number to enable reliable, in-order, single-copy delivery. The STX policy defines which VOQ the adapter selects when it is eligible to perform an STX. This policy can employ, e.g., a random (RND), oldest-cell-first (OCF), or youngest-cell-first (YCF) selection. First, we consider the OCF policy. It chooses the cell that has been waiting longest at the input adapter for an STX opportunity.

Collisions: An important consequence of STX is the occurrence of collisions in the switch fabric: As STX cells are sent without prior arbitration, they may collide with either other STX cells or scheduled cells destined to the same output, and as a result they may be dropped. In OSMOSIS, it is possible to always allow up to cells to survive the collision, because the colliding cells do not share a physical medium until they arrive at the crossbar. The

49

scheduler knows about incoming STX cells from the accompanying SRQs on the control path, and it also knows which scheduled cells have been scheduled to arrive in the current time slot. Therefore, it can arbitrate between arriving STX cells if necessary and configure the crossbar to allow up to to pass, while dropping the others. Therefore, transmissions are always successful, even in the case of a collision. This is an important difference to ALOHA or Ethernet, where all colliding cells are lost. When multiple STX cells collide, we can forward up to of them, but when a scheduled cell collides with one or more STX cells, the scheduled cell always takes precedence to ensure that STX does not interfere with the basic operation of the underlying matching algorithm (see and ). Note also that the matching algorithm ensures that collisions between scheduled cells can never occur. The collision arbitration operates as follows. Before resolving contention among SRQs destined to port , a scheduled matching for the time slot under consideration must be ready. For every matched port , a number of SRQs are randomly accepted and the others denied. For every unmatched port , a number of SRQs are randomly accepted and the others denied. Granting SRQs does not affect the operation of the matching algorithm, e.g., in the case of -SLIP, the round-robin pointers are not updated. The scheduler notifies the sender of a successful SRQ by means of an acknowledgment

50

(ACK). Of course, it also issues the regular grants according to the matching. These grants may cause duplicate cell transmissions as described in . The scheduler does not generate explicit negative acknowledgments (NAK) for dropped cells.

Retransmission:

Collisions imply cell losses and out-of-order (OOO) delivery, which in turn imply a need for link-level retransmissions and ACKs, as this loss probability is orders of magnitude higher than that due to transmission errors. Reliability and ordering can be restored by means of a reliable delivery (RD) scheme. Any RD scheme requires that an STX cell remain in the input adapter buffer until successfully transmitted. The ACKs are

generated by the scheduler for every successful STX cell and include the sequence number of the acknowledged cell. specifies that a speculative cell remains stored in the adapter until either of the following two events occurs:

51

 The cell is positively acknowledged, i.e., an ACK arrives corresponding sequence number. The cell is dequeued and dropped.

with the

A grant for this output arrives and the cell is the oldest unacknowledged STX cell. When a grant arrives and there are any unacknowledged STX cells for the granted output, the oldest of these is dequeued and retransmitted. Otherwise, the HOL cell of the VOQ is dequeued and transmitted, as usual. This rule implies that unacknowledged STX cells take precedence over other cells in the VOQ, to expedite their reliable, in-order delivery. According to , unacknowledged STX cells are never eligible for STX, because they have already been transmitted speculatively once. Allowing only one STX attempt per cell reduces the number of STXs, which increases their chance of success. Moreover, if an STX cell fails, the potential gain in latency has been lost in any case, so retrying the same cell serves no purpose. This is also the reason for not using explicit NAKs.

According to and , a non-wasted grant can be classified in two orthogonal ways: It is either pure or duplicate, and it is either regular or spurious depending on whether it is used by the cell that initiated it. There are several methods of achieving reliable, in-order delivery in the presence of STX, e.g., Go-Back-N (GBN) and Selective Retry (SR). First,

52

we consider SR. SR allows a predetermined maximum number of cells per output to be unacknowledged at each input at any given time. STX cells are stored in retransmission (RTX) queues (one RTX queue per VOQ). The output adapter accepts cells in any order and performs resequencing to restore the correct cell order. To this end, it has a resequencing queue (RSQ) per input to store OOO cells until the missing ones arrive. The input adapter accepts ACKs in any order. This implies that only the failed STX cells need to be retransmitted, hence the name Selective Retry, as opposed to

retransmitting the entire RTX queue as is done with GBN. SR requires resequencing logic and buffers at every output adapter. In addition, the RTX queues require a random-out organization, because cells can be dequeued from any point in the queue. However, SR minimizes the number of retransmissions, thus improving performance.

Advantages: It is used to reduce the computational time.

Application: The computation time is also an important criterion for CBIR in generalized applications, since people will not wait several minutes

53

between two feedback steps. Furthermore, a fast selection allows the user to provide more labels in the same time. Thus, it is more interesting to use a less efficient but fast method than a more efficient but highly-computational one. It will reduce the control-path latency incurred between issuance of a request and arrival of the corresponding grant.

10. BIBLIOGRAPHY

1] R. Veltkamp, Content-based image retrieval system: A survey, Tech. Rep., Univ. Utrecht, Utrecht, The Netherlands, 2002. [2] Y. Rui, T. Huang, S. Mehrotra, and M. Ortega, A relevance feedback architecture for content-based multimedia information retrieval systems, in Proc. IEEE Workshop Content-Based Access of Image and Video Libraries, 1997, pp. 9289. [3] E. Chang, B. T. Li, G. Wu, and K. Goh, Statistical learning for effective visual information retrieval, in Proc. IEEE Int. Conf. Image Processing, Barcelona, Spain, Sep. 2003, pp. 609612. [4] S. Aksoy, R. Haralick, F. Cheikh, and M. Gabbouj, A weighted distance

54

approach to relevance feedback, in Proc. IAPR Int. Conf. Pattern Recognition, Barcelona, Spain, Sep. 38, 2000, vol. IV, pp. 812815. [5] J. Peng, B. Bhanu, and S. Qing, Probabilistic feature relevance learning for content-based image retrieval, Comput. Vis. Image Understand., vol. 75, no. 1-2, pp. 150164, Jul.-Aug. 1999. [6] N. Doulamis and A. Doulamis, A recursive optimal relevance feedback scheme for CBIR, presented at the Int.Conf. Image Processing, Thessaloniki, Greece, Oct. 2001. [7] J. Fournier, M. Cord, and S. Philipp-Foliguet, Back-propagation algorithm for relevance feedback in image retrieval, in Proc. Int. Conf. Image Processing, Thessaloniki, Greece, Oct. 2001, vol. 1, pp. 686689. [8] O. Chapelle, P. Haffner, and V. Vapnik, Svms for histogram based image classification, IEEE Trans. Neural Netw., vol. 10, pp. 10551064, 1999. [9] N. Vasconcelos, Bayesian models for visual information retrieval, Ph.D. dissertation, Mass. Inst. Technol., Cambridge, 2000. [10] S.-A. Berrani, L. Amsaleg, and P. Gros, Recherche approximative de plus proches voisins: Application la reconnaissance dimages par descripteurs

55

locaux, Tech. Sci. Inf., vol. 22, no. 9, pp. 12011230, 2003. [11] N. Najjar, J. Cocquerez, and C. Ambroise, Feature selection for semi supervised learning applied to image retrieval, in Proc. IEEE Int. Conf. Image Processing, Barcelona, Spain, Sep. 2003, vol. 2, pp. 559562. [12] X. Zhu, Z. Ghahramani, and J. Lafferty, Semi-supervised learning using gaussian fields and harmonic functions, presented at the Int. Conf. Machine Learning, 2003. GOSSELIN AND CORD: ACTIVE LEARNING METHODS FOR INTERACTIVE IMAGE RETRIEVAL 1211 [13] A. Dong and B. Bhanu, Active concept learning in image databases, IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 35, pp. 450466, 2005. [14] N. Vasconcelos and M. Kunt, Content-based retrieval from image databases: Current solutions and future directions, in Proc. Int. Conf. Image Processing, Thessaloniki, Greece, Oct. 2001, vol. 3, pp. 69. [15] Y. Chen, X. Zhou, and T. Huang, One-class SVM for learning in image retrieval, in Proc. Int. Conf. Image Processing, Thessaloniki, Greece, Oct. 2001, vol. 1, pp. 3437. [16] S.-A. Berrani, L. Amsaleg, and P. Gros, Approximate searches: k-neighbors + precision, in Proc. Inf. Conf. Information and Knowledge

56

APPENDIX

SCREEN SHOTS

57

58

CODING: MAIN FORM:

namespace { using using using using using using using using Retrieval System; System.Collections.Generic; System.ComponentModel; System.Diagnostics; System.Drawing; System.IO; System.Windows.Forms; Processing;

internal partial class Main : Form { private bool exit = false; public Main() { this.InitializeComponent(); } #region "GUI delegates" private delegate void ProcessImagesDelegate(FileInfo[] files); private delegate void SetMaximumDelegate(ProgressBar progressBar, int value);

59

private delegate void UpdateOperationStatusDelegate(string format, System.Windows.Forms.Label label, ProgressBar progressBar, int value, DateTime startTime); private delegate void UpdateDataGridViewDelegate(BindingList<SimilarityImages> images, DataGridView dataGridView, IList<Button> buttons, TextBox textBox); private delegate void DeleteImageDelegate(FileInfo fileInfo); private delegate void ShowGridDelegate(DataGridView dataGridView); #endregion #region "Start/stop search" //THE SEARCH BUTTON private void FindButton_Click(object sender, EventArgs e) { visible(); string folder = this.folderTextBox.Text; string filee = this.filetext.Text; DirectoryInfo directoryInfo; FileInfo[] files; try { directoryInfo = new DirectoryInfo(folder files = directoryInfo.GetFiles("*.jpg", SearchOption.AllDirectories);

);

this.exit = false; this.findButton.Enabled = false; this.cancelButton.Enabled = !this.findButton.Enabled; this.folderTextBox.Enabled = this.findButton.Enabled; } catch (DirectoryNotFoundException) { MessageBox.Show("Path not valid.", "Invalid path", MessageBoxButtons.OK, MessageBoxIcon.Error); this.folderTextBox.Enabled = true; return; } catch (ArgumentException) { MessageBox.Show("Path not valid.", "Invalid path", MessageBoxButtons.OK, MessageBoxIcon.Error); return; } ProcessImagesDelegate processImagesDelegate = new ProcessImagesDelegate(ProcessImages); processImagesDelegate.BeginInvoke(files, null, null); } private void CancelButton_Click(object sender, EventArgs e) { this.exit = true; this.findButton.Enabled = true; this.cancelButton.Enabled = !this.findButton.Enabled; this.folderTextBox.Enabled = this.findButton.Enabled;

60

} #endregion #region "GUI delegates body" private UpdateDataGridViewDelegate updateDataGridViewDelegate = delegate(BindingList<SimilarityImages> images, DataGridView dataGridView, IList<Button> buttons, TextBox textBox) { images.RaiseListChangedEvents = true; dataGridView.DataSource = images; dataGridView.Columns["Similarity"].DisplayIndex = 0; dataGridView.Columns["Source"].DisplayIndex = 1; dataGridView.Columns["Destination"].DisplayIndex = 2; if (images.Count > 0) dataGridView.Rows[0].Selected = true; buttons[0].Enabled = true; buttons[1].Enabled = !buttons[0].Enabled; textBox.Enabled = buttons[0].Enabled; }; SetMaximumDelegate setMaximumDelegate = delegate(ProgressBar progressBar, int value) { progressBar.Maximum = value; }; UpdateOperationStatusDelegate updateOperationStatusDelegate = delegate(string format, System.Windows.Forms.Label label, ProgressBar progressBar, int value, DateTime startTime) { progressBar.Value = value; double percentage = Math.Round(((double)progressBar.Value / (double)progressBar.Maximum), 3); format += " {0}/{1} ({2}) Elapsed: {3} Estimated: {4}"; TimeSpan elapsed = DateTime.Now.Subtract(startTime); elapsed = new TimeSpan(elapsed.Days, elapsed.Hours, elapsed.Minutes, elapsed.Seconds, 0); long estimatedTicks = (elapsed.Ticks / value) * progressBar.Maximum; TimeSpan estimated = new TimeSpan(estimatedTicks); estimated = new TimeSpan(estimated.Days, estimated.Hours, estimated.Minutes, estimated.Seconds, 0); label.Text = string.Format(format, progressBar.Value, progressBar.Maximum, percentage.ToString("P"), elapsed.ToString(), estimated.ToString()); }; ShowGridDelegate showGridDelegate = delegate(DataGridView dataGridView) { dataGridView.ResumeLayout();

61

dataGridView.Enabled = true; }; #endregion BindingList<SimilarityImages> similarityImages; private void ProcessImages(FileInfo[] files) { List<ComparableImage> comparableImages; comparableImages = new List<ComparableImage>(); this.Invoke(setMaximumDelegate, new object[] { this.workingProgressBar, files.Length }); int index; index = 0; DateTime operationStartTime; operationStartTime = DateTime.Now; foreach (FileInfo file in files) { if (this.exit) { return; } ComparableImage comparableImage = new ComparableImage(file); //comparableImages comparebleimaaa = new ComparableImage(file); comparableImages.Add(comparableImage); index++; this.Invoke(updateOperationStatusDelegate, new object[] { "Processed images", workingLabel, workingProgressBar, index, operationStartTime }); } this.Invoke(this.setMaximumDelegate, new object[] { this.workingProgressBar, ( comparableImages .Count +(comparableImages.Count -1)) / 2 }); index = 0; List<SimilarityImages> similarityImagesSorted = new List<SimilarityImages>(); operationStartTime = DateTime.Now; // int counn = 0; string[] selectedFiles = openFileDialog1.FileNames; // for (int i = 0; i <= 6; i++)

62

//{ //int k = selectedFiles[i].Length; //for (int i=0;i<selectedFiles //for (int i = 0; i < comparableImages.Count; i++) // { //string b=comparableImages [0] for (int j = 1; j < comparableImages.Count; j++) { if (this.exit) { return; } //string[] files = Directory.GetFiles(openFileDialog1.FileName); // if (comparableImages[0].ToString() == "rr.jpeg") //{ int ss = index1; ComparableImage source = comparableImages[ss]; ComparableImage destination = comparableImages[j]; //double similarity = destination.CalculateSimilarity(source); double similarity = source.CalculateSimilarity(destination); SimilarityImages sim = new SimilarityImages(source, destination, similarity); similarityImagesSorted.Add(sim); index++; this.Invoke(updateOperationStatusDelegate, new object[] { "Compared images", workingLabel, workingProgressBar, index, operationStartTime } ); }

similarityImagesSorted.Sort(); similarityImagesSorted.Reverse(); similarityImages = new BindingList<SimilarityImages>(similarityImagesSorted); List<Button> buttons2 = new List<Button>(); buttons2.Add(findButton); buttons2.Add(cancelButton);

63

this.BeginInvoke(updateDataGridViewDelegate, new object[] { similarityImages, imagesDataGridView, buttons2, folderTextBox }); //} } private void ShowProjections(Graph graph, ComparableImage comparable) { graph.ClearCurves(); graph.AddPlotCurve(Color.Red, comparable.Projections.HorizontalProjection); graph.AddPlotCurve(Color.Green, comparable.Projections.VerticalProjection); graph.Invalidate(); } protected override void OnClosing(CancelEventArgs e) { exit = true; } private void linkLabel1_LinkClicked(object sender, LinkLabelLinkClickedEventArgs e) { this.folderBrowserDialog.ShowDialog(); this.folderTextBox.Text = this.folderBrowserDialog.SelectedPath; } private void exitToolStripMenuItem_Click(object sender, EventArgs e) { } private void ShowSelectedImages() { if (imagesDataGridView.SelectedRows.Count <= 0) { return; } SimilarityImages sim = (SimilarityImages)imagesDataGridView.SelectedRows[0].DataBoundItem; //FileStream streamSource = new System.IO.FileStream(sim.Source.File.FullName, FileMode.Open, FileAccess.Read); FileStream streamDestination = new System.IO.FileStream(sim.Destination.File.FullName, FileMode.Open, FileAccess.Read); Image source = Image.FromFile(openFileDialog1.FileName); Image destination = Image.FromStream(streamDestination); // streamSource.Close(); streamDestination.Close();

64

string infoFormat = "Resolution: {0}x{1}\nSize: {2}kb\nFull path: {3}"; string resolutionFormat = "{0} ({1}x{2})"; sourcePictureBox.Image = Image.FromFile(openFileDialog1.FileName); //mainToolTip.SetToolTip(sourcePictureBox, string.Format(infoFormat, source.Width, source.Height, Math.Round((double)(sim.Source.File.Length / 1024), 1), sim.Source.File.FullName)); sourceLabel.Text = string.Format(resolutionFormat, "Source", source.Width, source.Height); destinationPictureBox.Image = destination; mainToolTip.SetToolTip(destinationPictureBox, string.Format(infoFormat, destination.Width, destination.Height, Math.Round((double)(sim.Destination.File.Length / 1024), 1), sim.Destination.File.FullName)); destinationLabel.Text = string.Format(resolutionFormat, "Destination", destination.Width, destination.Height); // ShowProjections(sourceGraph, sim.Source); ShowProjections(destinationGraph, sim.Destination ShowProjections(sourceGraph, sim.Source); } private void DeleteSourceLinkLabel_LinkClicked(object sender, LinkLabelLinkClickedEventArgs e) { if (imagesDataGridView.SelectedRows.Count <= 0) { return; } DialogResult result = MessageBox.Show("Delete the image?", "Confirm delete", MessageBoxButtons.OKCancel, MessageBoxIcon.Question); if (result == DialogResult.Cancel) { return; } SimilarityImages sim = (SimilarityImages)imagesDataGridView.SelectedRows[0].DataBoundItem; DeleteImageDelegate deleteImageDelegate = new DeleteImageDelegate(DeleteImage); imagesDataGridView.SuspendLayout(); imagesDataGridView.Enabled = false; deleteImageDelegate.BeginInvoke(sim.Source.File, new AsyncCallback(ShowDataGrid), null); } private void ShowDataGrid(IAsyncResult result) { this.Invoke(showGridDelegate, imagesDataGridView); } );

65

private void DeleteDestinationLinkLabel_LinkClicked(object sender, LinkLabelLinkClickedEventArgs e) { if (imagesDataGridView.SelectedRows.Count <= 0) return; DialogResult result = MessageBox.Show("Delete the image?", "Confirm delete", MessageBoxButtons.OKCancel, MessageBoxIcon.Question); if (result == DialogResult.Cancel) return; SimilarityImages sim = (SimilarityImages)imagesDataGridView.SelectedRows[0].DataBoundItem; DeleteImageDelegate deleteImageDelegate = new DeleteImageDelegate(DeleteImage); imagesDataGridView.SuspendLayout(); imagesDataGridView.Enabled = false; deleteImageDelegate.BeginInvoke(sim.Destination.File, new AsyncCallback(ShowDataGrid), null); } private void DeleteImage(FileInfo fileInfo) { try { List<SimilarityImages> toDelete = new List<SimilarityImages>(); for (int index = 0; index < similarityImages.Count; index++) { SimilarityImages item = similarityImages[index]; if ((item.Source.File.FullName.Equals(fileInfo.FullName, StringComparison.InvariantCultureIgnoreCase)) || (item.Destination.File.FullName.Equals(fileInfo.FullName, StringComparison.InvariantCultureIgnoreCase))) toDelete.Add(item); } similarityImages.RaiseListChangedEvents = false; for (int index = 0; index < toDelete.Count; index++) { SimilarityImages item = toDelete[index]; similarityImages.Remove(item); } similarityImages.RaiseListChangedEvents = true; similarityImages.ResetBindings(); fileInfo.Delete(); } catch (Exception ex) { MessageBox.Show(ex.Message); } }

66

private void OpenSourceLinkLabel_LinkClicked(object sender, LinkLabelLinkClickedEventArgs e) { if (imagesDataGridView.SelectedRows.Count <= 0) return; SimilarityImages sim = (SimilarityImages)this.imagesDataGridView.SelectedRows[0].DataBoundItem ; Process.Start(sim.Source.File.FullName); } private void OpenDestinationLinkLabel_LinkClicked(object sender, LinkLabelLinkClickedEventArgs e) { if (this.imagesDataGridView.SelectedRows.Count <= 0) { return; } SimilarityImages sim = (SimilarityImages)this.imagesDataGridView.SelectedRows[0].DataBoundItem ; Process.Start(sim.Destination.File.FullName); } private void DataGridView1_SelectionChanged(object sender, EventArgs e) { this.ShowSelectedImages(); } private void selectFolderToSerachToolStripMenuItem_Click(object sender, EventArgs e) { gbxFolder.Visible = true; } public void visible() { workingLabel.Visible = true; imagesDataGridView.Visible = true; workingProgressBar.Visible = true; panel.Visible = true; imagesLabel.Visible = true; } private void aboutToolStripMenuItem_Click(object sender, EventArgs e) { about ab = new about(); ab.Show(); } private void closeToolStripMenuItem_Click(object sender, EventArgs e) {

67

Application.Exit(); } private void viewGraphToolStripMenuItem_Click(object sender, EventArgs e) { label2.Visible = true; label3.Visible = true; sourceGraph.Visible = true; destinationGraph.Visible = true; } private void hideGraphToolStripMenuItem_Click(object sender, EventArgs e) { label2.Visible = false; label3.Visible = false; sourceGraph.Visible = false; destinationGraph.Visible = false; } public string ss; private void linkLabel1_LinkClicked_1(object sender, LinkLabelLinkClickedEventArgs e) { if (openFileDialog1.ShowDialog() == DialogResult.OK) { this.filetext.Text = openFileDialog1.FileName; sourcePictureBox.Image = Image.FromFile(openFileDialog1.FileName); // pboxAfter.Image = Image.FromFile(openFileDialog1.FileName); //openFileDialog1.ShowDialog(); ss = openFileDialog1.FileName.ToString(); this.filetext.Text = openFileDialog1.FileName; //string[] selectedFiles = new string[20]; //selectedFiles =openFileDialog1 .FileNames ; //for (int i = 0; i < selectedFiles.Length; i++) //{ // // MessageBox.Show("rrrr", +i); //} // int selectedFilesCount = selectedFiles.Length; // for (int i = 0; i < selectedFilesCount.c; i++) // { // MessageBox.Show("rrr"+i); // } } } private void Main_Load(object sender, EventArgs e) { } private void sourceGraph_Click(object sender, EventArgs e) {

68

} public int index1; private void openFileDialog1_FileOk(object sender, CancelEventArgs e) { DirectoryInfo directoryInfo = new DirectoryInfo(folderTextBox.Text); FileInfo[] files = directoryInfo.GetFiles("*.jpg", SearchOption.AllDirectories); string searchImage = openFileDialog1.FileName.ToString(); int se = searchImage.LastIndexOf("\\"); int aaa = searchImage.Length - se; string aa = searchImage.Substring(se + 1, aaa - 1); int count = files.Length; index1 = 0; for (int i = 0; i <= count-1; i++) { if (aa == files[i].ToString()) { index1 = i; } } } private void folderBrowserDialog_HelpRequest(object sender, EventArgs e) {

} private void sourcePictureBox_Click(object sender, EventArgs e) { }

} }

69

RGB PROJECTIONS:
namespace Processing { using System; using System.Collections.Generic; /// <summary> /// Represents RGB projections, horizontal and vertical. /// </summary> public class RgbProjections { private double[] horizontalProjection; private double[] verticalProjection; internal RgbProjections(double[][] projections) : this(projections[0], projections[1]) { } internal RgbProjections(double[] horizontalProjection, double[] verticalProjection) { this.horizontalProjection = horizontalProjection; this.verticalProjection = verticalProjection; } public double[] HorizontalProjection { get { return this.horizontalProjection; } } public double[] VerticalProjection { get { return this.verticalProjection; } } /// <summary> /// Calculate the similarity between two RGB projections, horizontal and vertical. /// </summary> /// <param name="compare">The RGB projection to compare with.</param> /// <returns>Return the max similarity value betweem horizontal and vertical RGB projections.</returns> public double CalculateSimilarity(RgbProjections compare) {

70

double horizontalSimilarity = CalculateProjectionSimilarity(this.horizontalProjection, compare.horizontalProjection); double verticalSimilarity = CalculateProjectionSimilarity(this.verticalProjection, compare.verticalProjection); return Math.Max(horizontalSimilarity, verticalSimilarity); } /// <summary> /// Calculate the similarity to another RGB projection. /// </summary> /// <param name="source">The source RGB projection.</param> /// <param name="compare">The RGB projection to compare with.</param> /// <returns>Return a value from 0 to 1 that is the similarity.</returns> private static double CalculateProjectionSimilarity(double[] source, double[] compare) { if (source.Length != compare.Length) { throw new ArgumentException(); } Dictionary<double, int> frequencies = new Dictionary<double, int>(); ////Calculate frequencies for (int i = 0; i < source.Length; i++) { double difference = (source[i] - compare[i]); difference = (Math.Round(difference, 2)); difference = Math.Abs(difference); if (frequencies.ContainsKey(difference)) { frequencies[difference] = frequencies[difference] + 1; } else { frequencies.Add(difference, 1); } } double deviation = 0; ////Calculate "weighted mean" ////http://en.wikipedia.org/wiki/Weighted_mean foreach (KeyValuePair<double, int> value in frequencies) { deviation += (value.Key * value.Value); } deviation /= source.Length; ////Maximize scale

71

deviation = (0.5 - deviation) * 2; return deviation; } } }

IMAGE UTILITY:
namespace { using using using using Processing System.Drawing; System.Drawing.Drawing2D; System.Drawing.Imaging; System.Diagnostics.CodeAnalysis;

/// <summary> /// An utility class to do simple image processing. /// </summary> public static class ImageUtility { private static object syncRoot = new object(); /// <summary> /// Resize an image in high resolution /// </summary> /// <param name="bitmap">The image to resize.</param> /// <param name="width">The expected width.</param> /// <param name="height">the expected height.</param> /// <returns></returns> [SuppressMessage("Microsoft.Design", "CA1011:ConsiderPassingBaseTypesAsParameters")] public static Bitmap ResizeBitmap(Bitmap bitmap, int width, int height) { Bitmap result = new Bitmap(width, height); using (Graphics graphic = Graphics.FromImage((System.Drawing.Image)result)) { graphic.InterpolationMode = InterpolationMode.HighQualityBicubic; graphic.DrawImage(bitmap, 0, 0, width - 1, height - 1); } return result; } /// <summary> /// Calculate the RBG projection. /// </summary> /// <param name="bitmap">The image to process.</param> /// <returns>Return horizontal RGB projection in value [0] and vertical RGB projection in value [1].</returns> public static double[][] GetRgbProjections(Bitmap bitmap) { int width = bitmap.Width - 1; int height = bitmap.Width - 1;

72

double[] xProjection = new double[width]; double[] yProjection = new double[height]; BitmapData bitmapData1 = bitmap.LockBits(new Rectangle(0, 0, bitmap.Width, bitmap.Height), ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb); int luminosity = 0; unsafe { byte* imagePointer1 = (byte*)bitmapData1.Scan0; for (int y = { for (int { byte byte byte 0; y < height; y++) x = 0; x < width; x++) blu = imagePointer1[0]; green = imagePointer1[1]; red = imagePointer1[2];

luminosity = (byte)(((0.2126 * red) + (0.7152 * green)) + (0.0722 * blu)); xProjection[x] += luminosity; yProjection[y] += luminosity; ////4 bytes per pixel imagePointer1 += 4; } ////end for j ////4 bytes per pixel imagePointer1 += bitmapData1.Stride (bitmapData1.Width * 4); }////end for i }////end unsafe MaximizeScale(ref xProjection, height); MaximizeScale(ref yProjection, width); double[][] projections = new double[][] { xProjection, yProjection }; bitmap.UnlockBits(bitmapData1); return projections; } /// <summary> /// Optimize the range of values. /// </summary> /// <param name="projection">The array to process.</param> /// <param name="max">The max value for the elements.</param> private static void MaximizeScale(ref double[] projection, double max) { double minValue = double.MaxValue; double maxValue = double.MinValue;

73

for (int i = 0; i < projection.Length; i++) { if (projection[i] > 0) { projection[i] = projection[i] / max; } if (projection[i] < minValue) { minValue = projection[i]; } if (projection[i] > maxValue) { maxValue = projection[i]; } } if (maxValue == 0) { return; } for (int i = 0; i < projection.Length; i++) { if (maxValue == 255) { projection[i] = 1; } else { projection[i] = (projection[i] - minValue) / (maxValue - minValue); } } } } }

COMPARABLE IMAGE:
namespace { using using using Processing System; System.Drawing; System.IO;

/// <summary> /// Represents an image and its RGB projections. /// </summary> public class ComparableImage { private FileInfo file; private RgbProjections projections;

74

public ComparableImage(FileInfo file) { if (file == null) { throw new ArgumentNullException("file"); } if (!file.Exists) { throw new FileNotFoundException(); } this.file = file; using (Bitmap bitmap = ImageUtility.ResizeBitmap(new Bitmap(file.FullName), 100, 100)) { this.projections = new RgbProjections(ImageUtility.GetRgbProjections(bitmap)); } } public FileInfo File { get { return this.file; } } public RgbProjections Projections { get { return this.projections; } } /// <summary> /// Calculate the similarity to another image. /// </summary> /// <param name="compare">The image to compare with.</param> /// <returns>Return a value from 0 to 1 that is the similarity.</returns> public double CalculateSimilarity(ComparableImage compare) { return this.projections.CalculateSimilarity(compare.projections); } public override string ToString() { return file.Name; } } }

75

SIMILARITY IMAGES:
namespace { using using using Processing System; System.Collections.Generic; System.Diagnostics.CodeAnalysis;

/// <summary> /// Represents a comparable images class. /// </summary> public class SimilarityImages : IComparer<SimilarityImages>, IComparable { private ComparableImage source; private ComparableImage destination; private double similarity; public SimilarityImages(ComparableImage source, ComparableImage destination, double similarity) { this.source = source; this.destination = destination; this.similarity = similarity; } public ComparableImage Source { get { return this.source; } } public ComparableImage Destination { get { return this.destination; } } public double Similarity { get { return Math.Round(this.similarity * 100, 1); } } [SuppressMessage("Microsoft.Globalization", "CA1305:SpecifyIFormatProvider", MessageId = "System.String.Format(System.String,System.Object,System.Object,System. Object)")] public override string ToString() {

76

return string.Format("{0}, {1} --> {2}", this.source.File.Name, this.destination.File.Name, this.similarity); } #region IComparer<SimilarityImages> Members public int Compare(SimilarityImages x, SimilarityImages y) { return x.similarity.CompareTo(y.similarity); } #endregion #region IComparable Members public int CompareTo(object obj) { SimilarityImages other = (SimilarityImages)obj; return this.Compare(this, other); } #endregion // override object.Equals public override bool Equals(object obj) { if (obj == null || GetType() != obj.GetType()) { return false; } SimilarityImages other = (SimilarityImages)obj; bool equals = this.Source.File.FullName.Equals(other.Source.File.FullName, StringComparison.InvariantCultureIgnoreCase); if (!equals) { return false; } equals = this.Destination.File.FullName.Equals(other.Destination.File.FullName, StringComparison.InvariantCultureIgnoreCase); if (!equals) { return false; } return true; } // override object.GetHashCode [SuppressMessage("Microsoft.Globalization", "CA1305:SpecifyIFormatProvider", MessageId = "System.String.Format(System.String,System.Object,System.Object)")]

77

public override int GetHashCode() { return string.Format("{0};{1}", this.Source.File.FullName, this.Destination.File.FullName).GetHashCode(); } public static int operator !=(SimilarityImages value, SimilarityImages compare) { return value.CompareTo(compare); } public static int operator <(SimilarityImages value, SimilarityImages compare) { return value.CompareTo(compare); } public static int operator ==(SimilarityImages value, SimilarityImages compare) { return value.CompareTo(compare); } public static int operator >(SimilarityImages value, SimilarityImages compare) { return value.CompareTo(compare); } } }

78