Cooperative Criminal Face Recognition in Distributed Web Environment

Hamid Tabatabaee Yazdi1, Amin Milani Fard2, Mohammad –R. Akbarzadeh –T.3
1
Department of Computer Engineering, Islamic Azad University, Quchan Branch, Iran, IEEE Student Member
2
Department of Computer Engineering, Ferdowsi University, Mashhad, Iran, IEEE Student Member
3
Department of Electrical Engineering, Ferdowsi University, Mashhad, Iran, IEEE Senior Member

[email protected] [email protected] [email protected]

Abstract follows. Section 2 reviews related works paradigms.

Section 3 discusses the proposed system framework
This paper presents a multi-agent framework for and finally, the conclusion is presented in Section 5.
distributed web-based face recognition using fuzzy
logic based result fusion mechanism. The model is 2. Related Works
designed in JADE framework and takes advantage of
JXTA agent communication method to allow agent Diogenes [3] is a web-based image search agent
communication through firewalls and network which takes advantage of the text/HTML structure of
address translators. This approach enables police web pages as well as the visual analysis of the images
stations to build and deploy P2P applications for personal image search and identification. The
through a unified medium to manage agent-based Missing Children Locator Agent is an example of
criminal face image retrieval from multiple another web-based system combining intelligent
distributed databases over the Internet. software agents with FR engines developed by the
ANSER team [4].
Keywords: Distributed face recognition, Multi-agent The software agents continuously search and
systems, World Wide Web, Fuzzy logic retrieve facial images and relevant information from
web sites on the Internet. A back-end FR engine
1. Introduction analyses each image file to detect and match each
face to a set of stored face images of missing
Facial recognition systems (FRSs) are computer- children. Successful implementation of the system
driven application for automatically identifying a demonstrates its ability to speed up labor-intensive
person from a digital image typically for security tasks and automate the processes required for face
matters. It does that by comparing selected facial recognition.
features in the live image and a facial database. Most However, in the above two systems, the intelligent
common recognition algorithms include eigenface, agent does not get real mobility when it traverses web
fisherface, the Hidden Markov model, and the pages. It can not migrate from host to host in a
neuronal motivated dynamic link matching. heterogeneous Internet environment which makes it
Newly emerging trends are 3D face recognition, difficult to incorporate with legacy systems when
and using the visual details of the skin. With the help need to access image databases in different formats
of these systems police can search for potential over the Internet. An active tracking vision capable of
criminals and terrorists in attendance at the event [1], detecting human faces from sequential images was
or child's disappearance in an attempt to identify the reported in [5] which integrates multiple information
abductor using a biometric facial recognition sensing from real-time and semi real-time agents. The
application [2]. real-time agents are used to extract and transfer useful
Web-based FRSs have received more and more information based on captured images for quickly
attention in recent years. In this paper, we are not detecting and tracking objective faces in real time.
going to explain a new recognition algorithm; Semi real-time agents work at the back-end,
however, a new distributed web-based FRS extracting the skin color region, determining the total
architecture is proposed using multi-agent mechanism evaluation values of facial probability for all face-like
and fuzzy logic concept. The paper is organized as objects, and detecting faces in sequential images.

978-1-4244-1968-5/08/$25.00 ©2008 IEEE 524

3. Facial Recognition Principles A newly-emerging trend in facial recognition
software uses a 3D model, which claims to provide
more accuracy. Capturing a real-time 3D image of a
Facial recognition software such as FaceIt1, can person's facial surface, 3D facial recognition uses
pick someone's face out of a crowd, extract the face distinctive features of the face -- where rigid tissue
from the rest of the scene and compare it to a and bone is most apparent, such as the curves of the
database of stored images. They have to know how to eye socket, nose and chin -- to identify the subject.
differentiate between a basic face and the rest of the These areas are all unique and don't change over time
background and then measure the various features of [1].
the face. Every face has numerous, distinguishable
landmarks, the different peaks and valleys that make
4. Distributed Information Retrieval
up facial features. FaceIt defines these landmarks as
nodal points. Each human face has approximately 80
The problem of finding information in scattered
nodal points. These nodal points are measured
sources located on different servers has become more
creating a numerical code, called a faceprint,
serious considering increasing number of databases
representing the face in the database [1]. The
company maintains the face print is identity specific on LANs and the Internet. The goal of distributed
and is not affected by aging, viewing conditions, and information retrieval (DIR) is to provide a single
superficial disguises. Some of these measured by the search interface that provides access to the available
software are: databases. This problem involves building resource
descriptions for each database, choosing which
• Distance between the eyes databases to search for particular information, and
• Width of the nose merging retrieved results into a single result list [6]-
• Depth of the eye sockets [7]. DIR includes four sub-problems of resource
• The shape of the cheekbones description, resource selection, query translation, and
• The length of the jaw line result merging.
In the past, facial recognition software has relied The resource description problem is how to learn
on a 2D image to compare or identify another 2D and describe the topics covered by each different
image from the database. To be effective and database. The resource selection means, given a
accurate, the image captured needed to be of a face query and a set of resource descriptions, how to select
that was looking almost directly at the camera, with a set of resources to search. The query translation
little variance of light or facial expression from the means mapping a given information need in some
image in the database. This created quite a problem. base representation, automatically to query languages
In most instances the images were not taken in a appropriate for the selected databases. Finally, after
controlled environment. Even the smallest changes in result lists have been returned from the selected
light or orientation could reduce the effectiveness of
databases, result merging is used to integrate them
the system, so they couldn't be matched to any face in
into a single ranked list.
the database, leading to a high rate of failure.
Let us consider m information servers, denoted by
Si and i=1...m; each of which provided with its own
information retrieval system (IRS). Here we assume
that the retrieval engines produce weighted lists of
retrieved documents as an indicator of the estimated
relevance of each document with respect to the query.
In order to provide user with a single ordered list of
relevant documents, the individual lists produced by
IRSs must be fused [8]. This fusion is not an easy
problem as IRSs produced their result with different
(a) (b) criteria and statistics such as average document
Figure 1. FaceIt software (a) compares the faceprint with other
length, text frequency inverse document frequency,
images in the database, (b) measures nodal points on the human
face to create a faceprint and find a match. and etc [9].

1
FaceIt® produced by IDENTIX® has recently merged with L-
1 dentity Solutions. www.l1id.com

525
 User Query specifies the fitness of each server with respect to this
query type. Formally, the properties useful to
characterize the subject matters of queries are
S1 S2 S3 ... Sm denoted as Uj (j=1..s); the fitness scores are the values
IRS1 IRS2 IRS3 IRSm associated with variables Vi (i=1..m), one for each
server Si. The fuzzy rules [8] identifying query types
D1,Score11 D1,Score21 D1,Score31 D1,Scorem1 are formalized as:
D2,Score12 D2,Score22 D2,Score32 D2,Scorem2
D3,Score13 D3,Score23 D3,Score33 D3,Scorem3
… … if U1 is Ak1 and … and Us is Aks then V is Bk
… …

in which k is the index of the rule, the Akj are fuzzy

subsets containing the values of variables Uj, V is an
Lists Fusion
m dimensional vector whose components are the Vi,
Figure 2. An overview of distributed information retrieval and Bk is also an m dimensional vector whose
Several works have faced the problem of components Bki, correspond to the fitness of server Si
distributed information retrieval and fusion [10]-[11]- as an appropriate source of document for the type of
[12]-[13], in which essentially consist of a three step query described in the antecedent of rule k. Once a
process. In the first step, a query Q is submitted to the user query has been evaluated by the IRSs on the
m servers and consequently m result lists would be servers, a matching between it and the query type
produced by each IRS. It is assumed that in these rules is performed, in order to compute the fitness
lists, each document has its score which indicates its scores for the considered query [8].
relevance to the query Q. The first step is to obtain a unique fitness score per
In the second step, a procedure computes the server for the desired query. To this aim the user
number Ni of documents that each server has to query is also described in terms of properties Uj, a
provide in the list. Considering N as the total number value aj is associated with each Uj and for each rule k
of documents to in the final fused list, we a firing level τk is computed: τk=minj[Akj(aj)]. To
∑
m
have N = Ni combine the fitness scores in the distinct rules, the
i =1
obtained information is employed as follows in which
Finally the third step performs fusion of those m p is the number of query type:
individual ordered lists and shows the user one
∑ τΒ
p
ordered result [8]. Some solutions to the problem of Β *
= k =1 k k
(1)
∑ τ
p
the lists fusion can be found in [10]-[11]-[12]-[13]-
k =1 k
[14].
The components bi* of vector B* denote the fitness
One strategy to decide Ni for each server is to
values of the servers for the considered query.
evaluate the appropriateness of the server information
Finally, to obtain the values Ni of documents to be
to the given user query. This is a knowledge intensive
contributed by each server, first the fitness values are
approach as some information about the content of
normalized:
the server is needed. In [10]-[11] the authors have
bi*
proposed an approach to determine Ni based on the αi = (2)
∑i=1 bi*
m
use of training queries to build both the content and
search behavior of each collection.
At this point the normalized values are used to
In [8]-[13]-[14] a knowledge based approach has
compute the number of documents to be retrieved
been proposed. This approach is based on the
from each server:
computation of a fitness score for indicating
appropriateness of each server with respect to the user Ni = αi N (3)
query. In order to compute this score, firstly a set of The number of documents to be selected from each
query type is defined; with each query type a set of server is proportional to the fitness of that server to
fitness score (one per server) is associated. the query [8].
A query type is formalized by means of a rule in
which the antecedent is the description of the query
by means of a set of properties, and the consequent

526
5. Proposed System Architecture A Web service can be published as a JADE agent
service and an agent service can be symmetrically
Multi-agent systems (MASs) as an emerging sub- published as a Web service endpoint. Invoking a Web
field of artificial intelligence concern with interaction service is just like invoking a normal agent service.
of agents to solve a common problem [15]. This Web services’ clients can also search for and invoke
paradigm has become more and more important in agent services hosted within JADE containers.
many aspects of computer science by introducing the Considering large-scale applications over separated
issues of distributed intelligence and interaction. They networks, agent communications has to be handled
represent a new way of analyzing, designing, and behind firewalls and Network Address Translators
implementing complex software systems. In multi- (NATs), however, the current JADE MTP do not
agent systems, communication is the basis for allow agent communication through firewalls and
interactions and social organizations which enables NATs. Fortunately, the firewall/NAT issue can be
the agents to cooperate and coordinate their actions. solved by using the current JXTA implementation for
Our proposed multi-agent system architecture is agent communication [19].
based on Java Agent DEelopment (JADE) framework JXTA is a set of open protocols for P2P
[16]. JADE is a software development framework networking. These protocols enable developers to
aimed at developing multi-agent systems and build and deploy P2P applications through a unified
applications in which agents communicate using medium [20]. Obviously, JXTA is a suitable
FIPA2 Agent Communication Language (ACL) architecture for implementing MTP-s for JADE and
messages and live in containers which may be consequently JADE agent communication within
distributed to several different machines. The Agent different networks can be facilitated by incorporating
Management System (AMS) is the agent who exerts JXTA technology into JADE [19].
supervisory control over access to and use of the In this work, the police officer after constructing a
Agent Platform. Only one AMS will exist in a single sketch from the aimed criminal, submits a query
platform. containing face image features (nodal points
Each agent must register with an AMS in order to information) to the broker agent. The broker agent
get a valid AID. The Directory Facilitator (DF) is the then sends query information to the search agents
agent who provides the default yellow page service in asking them to look for the answer in the available
the platform. The Message Transport System, also ontologies. Agents may also communicate with other
called Agent Communication Channel (ACC), is the agents and exchange information to broaden the
software component controlling all the exchange of search results. Our proposed architecture uses
messages within the platform, including messages different types of agents, each having its own
to/from remote platforms. The standard model of an characteristics.
agent platform, as defined by FIPA, is represented in 1. Broker Agent: After receiving the user query,
the Figure 3. the BA does a simple task sharing process for search
agents. The BA can also create search agents if
needed.
2. Search Agents: These agents will return an
ordered list of search results to the response agent.
3. Response Agent: This agent has the
responsibility to show the result of retrieved
information. To do so, RA collects SAs results, ranks
them and then shows them on the screen ordered by
relation percentage.

Figure 3. Reference architecture of a FIPA Agent Platform

JADE is capable of linking Web services and

agents together to enable semantic web applications.

2
Foundation for Intelligent Physical Agents (http://www.fipa.org)

527
 SIM ( fi ) s
Reques
Search page
μi ( f , s) = (5)
MAXSIM ( f ) s
, where SIM(if)i is the similarity value of feature
User vector f of image i in the server s, MAXSIM(f)s is the
highest feature vectors similarity value among all
Broker Agent Response Agent images in the server s database. A query <f,w> is
represented by feature vector fi and the corresponding
weights wi. The query is evaluated for a given image,
and then aggregation operators are used. Thus the
result of a query evaluation is represented as a fuzzy
subset of the archived image, given by
∑
Search Agent Search Agent Search Agent Search Agent
#1 #2 #3 #n Ris ( f ) = μW ( s, i, f ) / i (6)
d ∈I
This brings to light that fuzzy Boolean IR models
are more flexible in representing image contents and
Database Database ... Database information needs. When a query has to be evaluated,
#1 #2 #n the process of estimate of the appropriateness of each
server as the controller of the relevant information for
Figure 4. Proposed system architecture
the query can be done on the basis of a procedure
which performs a matching between the query and the
The response agent has the responsibility of fuzzy set of archived documents
computing the fitness scores of each server using a The matching procedure of a Boolean query
combination method of approaches presented in [8]- against the fuzzy term set proceeds in a bottom up
[21]-[22] which are based on associating each server way: first, the membership of each query feature
with a fuzzy set of image feature vectors in which vector to the fuzzy set is computed. The obtained
their associated membership values are interpreted as values (including the fitness of the server for each
their importance of the server. feature vector) are then aggregated by applying the
In other words the weights represent the fitness of connectives specified in the query (the AND is
the server with respect to the features expressed by interpreted as min, the OR as max, and the NOT as a
the associated vector. The fuzzy set can be interpreted complement). Figure 5 shows an example of
as a prototypal image or image type constituting a computing bi*, regarding equation 1, of the server Si
representative of the image in the archive of the for a given query Q.
server. The features and their fitness weights can be
defined either manually or automatically computed. Q = t1 AND (t2 OR t3)
An automatic selection of the image features can be ⇓
obtained on the basis of their normalized feature Bi* = min(μw(i,d,t1), max(μw(i,d,t2), μw(i,d,t3)))
value similarity in the archive on the considered
server; the computed value can then be used as Figure 5. Sample score computation
features membership values. An image is formally
represented as a fuzzy binary relation
6. Conclusion and Future Works
∑
Ris = μ is ( f ) / f
f ∈F
(4)
A multi-agent architecture for distributed image
, where Ris is the representation of the archive of retrieval on the web was proposed and a fuzzy logic
the server s image i∈I, the set of images, f∈F the set based approach was used for result lists fusion
of feature vectors, and μis I×F→[0,1] the membership mechanism. This model in contrast with other multi-
function of Ris. μis is again a dynamic function with agent models for web is considered to be more
μi(f,s) expressing the significance of the feature practical using hybrid JADE-JXTA framework which
vector f in image i on the server s. μi(f,s) can be allows agent communication through firewalls and
computed as a function of the normalized feature NATs. Some future works can be done on
value similarity as bellow performance evaluation of the retrieval mechanism

528
and meta-search development under this architecture [13] R. R. Yager, and A. Rybalov, “On the fusion of
seems to be a challenging matter. Also Grid based documents from multiple collection information
data retrieval is a very good testbed for such an retrieval systems”, Journal for the American Society
approach. for Information Science 49, 1177-1184, 1998.
[14] R. R. Yager, V. Kreinovich, "On how to merge sorted
lists coming from different web search tools," Soft
References Computing Research Journal 3, 83-88, 1999.
[15] M. Wooldridge, “An Introduction to Multiagent
[1] R. Johnson, K. Bonsor “How Facial Recognition Systems” Published in February 2002 by John Wiley
Systems Work,” Retrieved on 2006-06-18 & Sons. ISBN 0 47149691X.
http://computer.howstuffworks.com/facial- [16] F. Bellifemine, G. Caire, T. Trucco, G. Rimassa,
recognition.htm. “JADE Programmer’s Guide”, 21-August-2006.
[2] "Help find Madeleine McCann", Child Exploitation [17] JADE Board, “JADE WSIG Add-On Guide JADE
and Online Protection Centre, 2007-05-21. Retrieved Web Services Integration Gateway (WSIG) Guide”,
on 2007-05-21. 03-March-2005
http://www.madeleine.ceopupload.com/
[18] E. Cortese, F. Quarta, G. Vitaglione, P. Vrba.
[3] Y. A. Aslandogan and C. T. Yu, "Diogenes: A Web “Scalability and Performance of the JADE Message
Search Agent for Content Based Indexing of Personal Transport System”. Analysis of Suitability for Holonic
Images," in Proceedings of ACM SIGIR 2000, Athens, Manufacturing Systems, exp, 2002.
Greece, July 2000.
[19] S. Liu, P. Küngas, M. Matskin, “Agent-Based Web
[4] H. Wisniewski, "Face Recognition and Intelligent Service Composition with JADE and JXTA”,
Software Agents - an Integration System," Prepared Proceedings of the 2006 International Conference on
statement for the U.S. Senate Committee on Semantic Web and Web Services, SWWS 2006, Las
Commerce, Science and Transportation, May 12 Vegas, Nevada, USA, June 26-29, 2006
1999.
[20] J. D. Gradecki, “Mastering JXTA: Building Java
[5] Y. Aoki, K. Hisatomi and S. Hashimoto, "Robust and Peer-to-Peer Applications”, JohnWiley&Sons,2002.
Active Human Face Tracking Vision Using Multiple
[21] G. Salton, C. Buckley, “Term-weighting approaches
Information," Proceedings of SCI’99 (World
in automatic text retrieval”. Information Processing &
Multiconference on Systems, Cybermetics and
Management 24(5): 513–523, 1988
Informatics), Vol.5, pp. 28-33, Aug. 1999, Orlando.
[22] G. Pasi, G. Bordonga, “Application of fuzzy set
[6] J. CALLAN, “Distributed information retrieval”. In theory to extend boolean information retrieval,” in
Advances in Information Retrieval, W. B. Croft, Ed.
Soft Computing in Information Retrieval: Techniques
Kluwer Academic Publishers, 2000, pp. 127–150.
and Applications, F. Crestani, G. Pasi, Eds.
[7] L. Si, J. Callan, “A Semisupervised Learning Method Heidelberg, Germany: Physica Verlag, 2000, vol. 50,
to Merge Search Engine Results”, ACM Transactions pp. 21–47.
on Information Systems, Vol. 21, No. 4, October
2003, Pages 457–491.
[8] G. Pasi, and R. R. Yager, “Document retrieval from
multiple sources of information”, in Uncertainty in
Intelligent and Information Systems, edited by
Bouchon-Meunier, B., Yager, R. R. and Zadeh, L. A.,
World Scientific: Singapore, 250-261, 2000.
[9] R. Baeza-Yates, B. Ribeiro-Neto, “Modern
Information Retrieval”, Addison-Wesley, 1999
[10] E. M. Voorhees, N. K. Gupta, and B. Johnson-Laird,
“The collection fusion problem”, Proceedings of the
third Text Retrieval Conference (TREC-3), 95-104,
1994
[11] E. M. Voorhees, N. K. Gupta, and B. Johnson-Laird,
“Learning collection fusion strategies”, Proceedings of
the 18th ACM SIGIR Conference on Research and
Development in Information Retrieval, Seattle, 172-
179, 1995
[12] R. R. Yager, “On Ordered Weighted Averaging
aggregation Operators in Multi Criteria Decision
Making”, IEEE Trans. on Systems, Man and
Cybernetics. 18(1), 183-190, 1998

529