A Web Service Framework for Distributed

Multimedia Content Computing

Shilin Zhang Mei Gu

Faculty of Computer Science, Network and Information Faculty of Computer Science, Network and Information
Management Center Management Center
North China University of Technology North China University of Technology
Beijing, China Beijing, China
[email protected] [email protected]

Abstract—A distributed computing application can be used to do from space and is hosted by the Space Sciences Laboratory at
calculation-intensive multimedia content computing too, such as the University of California, Berkeley; And distributed.net,
video optical character recognition, shot detection and so on. which focuses on breaking various cryptographic ciphers [1].
Meanwhile,the Web is catalyzing a transition from PC-based In the past, users relied solely on their desktop systems to
software and file systems to Internet-based applications and Web
execute applications and store their personal data. Today,
services.We aim at the combination of multimedia computing
and Web technologies (Web Services) together that supports many desktop applications have feature-rich "software-as-a-
uniform naming, protection, and access for Multimedia Content service" counterparts, including Web-based email systems,
objects stored by Web services to accomplish distributed media editing tools, and office productivity suites. Web-based
computing. At a high level, it creates an integrated file and object services offer compelling advantages over traditional desktop
system from heterogeneous, multimedia content objects dispersed software. Fig 1 describes the contrast between PCs and Web
across the Internet. Based on this fact, we proposed a novel services.
framework of distributed multimedia computing in this paper. In To demonstrate the value of our approach, we have
this framework, the concept of workflow which can describe prototyped several new Web applications. Our experience
business processes logically is introduced. Some experiments
shows that it is possible to combine the ease of use,
show the effectiveness of the proposed framework.
publishing, and ubiquitous access advantages of Web services
Keywords- Multimedia Computing; Workflow; Load balance; with the organizational, protected sharing, and data processing
Fault Tolerance; Distributed computing. advantages of desktop systems. So a distributed computing
principle based on web service is proposed to do the complex
I. INTRODUCTION multimedia content analysis in this paper. In this framework,
the concept of workflow which can describe business
Supercomputers can be used to tackle Complex processes logically is introduced.
multimedia content computing. However, it is impossible for
many research organizations to afford the deployment and
maintenance of supercomputers because they are both
expensive and space consuming. Distributed computing
environment provides another alternative to carry out the
processing-intensive tasks due to its flexibility and scalability.
A distributed computing application can use two or more
networked computers to work together to accomplish a
complex task more quickly by splitting the task into many
small parts that can run simultaneously on multiple computers. Figure 1. PC VS Web service
So it is feasible for the distributed computing system to be
used to do multimedia content analysis. A variety of The remain of this paper is organized as follows: Section 2
describes the design of software framework, Section 3 and
distributed computing projects have been carried out in recent
Section 4 individually give our strategies about the load
years. Some examples of such projects are as follows: The balance and fault tolerance in our framework; the performance
Stanford University Chemistry Department Folding@home of our system is reported in Section 5; Finally, Section 6 draws
project, which focuses on simulations of protein folding to our conclusions.
find disease cures and to understand biophysical systems;
World Community Grid, an effort to create the world's largest
public computing grid to tackle scientific research projects
funded by IBM; SETI@home, which focuses on analyzing
radio-telescope data to find evidence of intelligent signals

978-1-4244-5273-6/09/$26.00 ©2009 IEEE

 II. FRAMEWORK Fig 3 shows a general architecture of distributed computing
framework, in which manager node performs the role of a
A. Web Service for distributed multimedia content computing
coordinator, divides a big task into many small jobs and also
Web Services are Internet-based, modular applications that assigns these jobs for the available worker nodes. Moreover, it
provide standard interfaces and communication protocols has to interpret and integrate the results returned from worker
aiming at loosely coupled service integration. Several key nodes into a meaningful final conclusion, at the same time
standards have emerged to define the advertisement, discovery, generate and dispatch new jobs according to workflow if
and usage of Web Services and to form the foundation for necessary. Manager node consists of network component, load
Web services: For advertisement, service descriptions based balance component and workflow component which can deal
on the Web Service Description Language (WSDL) [5] are with message transferring, task allocating and business flow
made available on UDDI registries for Universal Description, respectively. Worker nodes are client computers that can
Discovery and Integration [10]. WSDL and UDDI are receive dispatched jobs in form of message from manager
complemented by SOAP, a protocol for information exchange. node, processes the jobs and then return the results to manager
In contrast to conventional systems characterized by a node. It consists of network component and computing
monolithic structure or a set of components with proprietary component which can deal with message transferring and job
interfaces, the use of Web Services offers the following computing individually.
advantages. Software is not only provided in a component
oriented way, but as services with standardized interfaces to C. WorkFlow
support many different applications. This reduces In this framework, workflow is introduced to describe
programming complexity in two ways. First, separate business. A workflow is the profile of logical processes
components make software updates or changes transparent to describing a sequence of task. It shows us the processes of the
the application. Second, standardized interfaces allow third business to be done. And it is essentially a network of tasks.
parties to contribute their services with low integration effort Task, a specific computation or operation that we want to do
to applications. Revisiting our running example, Figure 2 and the base element in workflow, exists in the form of
shows one possible composition of Web Services to achieve program function. When running in distributed computing
the user preferred syndication of media streams in our use case. framework, a task will be split into many small parts to be
Before a Merging Web Service performs the combination of executed in many worker nodes at the same time. And we call
ticker and video, the video stream is transcoded and the text is the split task a job.
translated into the users preferred language. After merging, a Fig 4 shows a simple example of workflow where the
compression component adapts the source rate to channel and business contains four tasks. As we can see from it, task 2 will
device capabilities. be executed after task 1 is finished. And then, task 3 and task 4
will be handled as soon as task 2 is done. At the begin of the
procedure, each task will be cut into several jobs to be
executed in parallel.

1
Workflow
Profile
2

Figure 2. Media syndication through loosely coupled services

B. Framework Overview 3 4
Task

Job

Figure 4. Workflow example

III. LOAD BALANCE

In general, load balancing is used to dispatch tasks to
different processors in a parallel system, avoid some
processors being idle while others having more tasks for
execution, to get optimal resource utilization, throughput, and
Figure 3. Framework Architecture response time. A static load balancing technique allocates jobs
to processors at run time with no account of current network
load. Dynamic algorithms, more flexible, though more (2) Choose the worker with the highest hardware
computationally expensive, give some considerations to the configuration score among the selected workers.
network load before allocating the new jobs to processors. The Score of hardware configuration is defined as follows:
In our framework, load balance includes two categories: S =α • M + β •C  
storage load balance and compute load balance. Before you
begin to format your paper, first write and save the content as a α + β =1   
separate text file. Keep your text and graphic files separate  α , β ∈ [0,1] 
until after the text has been formatted and styled. Do not use
hard tabs, and limit use of hard returns to only one return at the Where S is the hardware configuration score of the
end of a paragraph. Do not add any kind of pagination worker, and the larger S is, more compute tasks will be
anywhere in the paper. Do not number text heads-the template assigned to the worker, Į and ȕ are weighted coefficients. M is
will do that for you? Finally, complete content and physical memory of the worker, and C is the frequency of
organizational editing before formatting. Please take note of the CPU Thus, we can adjust weighted coefficients according to
following items when proofreading spelling and grammar. the business between the memory and CPU.

A. Storage Load Balance IV. FAULT TOLERANCE

Storage load balance is considerable when task is disk Fault tolerance is the property that enables a system (often
consuming. We take the network load and the available disk of computer-based) to continue operating properly in the event of
the worker into consideration when doing storage load the failure (or one or more faults within) when some of its
balance. Storage load is defined as follows: components fail. Handling failures is an important theme in
distributed systems design. Failures fall into two obvious
Ls = α • Sa + β • Sc    categories: hardware and software. Hardware failures were a
α + β =1    dominant concern until the late 80's, but since then internal
hardware reliability has been improved enormously. Today,
 α , β ∈ [0,1]  problems are most often associated with connections and
Where Ls is the storage load point of the worker, and the mechanical devices, i.e., network failures and program failures.
larger Ls is, more storage tasks will be assigned to the worker. In our framework, we mainly concerned software failures:
Į and ȕ are weighted coefficients. Sa is available capacity worker failure and manager failure [4, 5].
percentage of the worker, and Sc is the percentage of capacity
used in a cycle that stands for the efficient of the worker. Here A. Worker Failure
a cycle can be one week, one month or others. Thus, we can Worker failure mainly happens in following situations:
adjust the weighted coefficients according to the business a) Connection lost because of network failures or power
between the available capacity and network efficient [2, 3]. outages;
b) Worker has no reaction because of task operation
functional problems;
Handling case 1, the worker is considered crashed in our
framework and removed from available worker node list of
manager server. All tasks assigned to the worker are reset to
waiting state, and therefore become eligible for scheduling on
other workers, waiting for being reassigned. Case 2 is closely
related to identifying (from the outside) worker is still doing
something useful or in some kind of infinite loop [4]. The way
we check the state of a worker is sending it a request. When
no reply is received within a certain amount of time, the
associated worker is considered to be crashed and will be
handled like case 1; otherwise, we think it is normal.
Figure 5. Storage Management
B. Manager Failure
B. Compute Load Balance We introduced the aid of a persistent backing store like
database to store the state of manager, if the manager crashed,
We applied compute load balance to make the best the new master can be started from the last store state.
utilization of workers when task is calculation-intensive. In
our strategy, hardware configuration such as frequency of PERFORMANCE
CPU, memory, and current number of jobs executing of Figure 6 and figure 7 demonstrate our system’s monitor
worker are taken into account. Hardware configuration is and function picture respectively.
overall static performance factor and current jobs number
marks the dynamic load of the worker. So our steps of finding
the load balance are as follows:
(1) Select workers having least number jobs;
 Table 2 and table 3 are our experiment results. Firstly, we
run standalone business program on a single worker node.
From Table 2, we can see that the business processing time of
workers, with different hardware configurations, ranged from
3012.609 to 5525.891 seconds, whereas Table 3 show us the
business consuming time in our distributed computing
framework that are composed of same worker nodes in table 2.
Its consuming time ranged from 1002.188 to 1233.453
seconds in different job units including time spent in message
exchanging and task allocating, here job unit refers to time
length in video cutting in task splitting. Comparing the two
Figure 6. System runtime monitor
tables, we can see business processing time in our distributed
computing framework is greatly reduced. Its average
consuming time is less than one third of that in standalone
experiment.
CONCLUSION
In this paper we have aimed at contributing to an emerging
topic: the combination of multimedia computing and Web
technologies (Web Services) together. Furthermore, we
proposed a framework of distributed multimedia computing
and introduced the concept of workflow; we also discussed the
strategies of load balance and fault tolerance in our framework.
Experimental results demonstrated that the proposed
Figure 7. System function picture framework had a good performance in the application of
multimedia process business.
Table 1, 2, 3 demonstrate our system’s performance.
REFERENCES
TABLE I. EXPERIMENT [1] David P. Anderson (2005-05-23). A Million Years of Computing.
Retrieved on 2006-08-11.
Experiment VOCR (Video Optical Character Recognition)
Business [2] LAP-SUN CHEUNG, YU-KWOK KWOK. On Load Balancing
Test Video CCTV News with 2278 seconds length in time and Approaches for Distributed Object Computing Systems. The Journal of
720*576 in resolution Supercomputing, 27, 149̄175, 2004:149-175.
[3] Mohammed A. M. Ibrahim , Lu Xinda, M. SaifMokbel.
Implementation Study of Dymnamic Load Balancing Algorithm of
In our experiment, we applied VOCR business processes parallel Tree Computation on Clusters of Heterogeneous Workstation.
on a news video with 2278 seconds length in time and Journal of DongHua University, (Eng. Ed. ), Vo1 㧚 22 㧘 No 㧚
720*576 in resolution to make comparison in time consumed 2(2005):81-86.
between standalone business programs and distributed [4] Jeffrey Dean, Sanjay Ghemawat. MapReduce: Simplified Data
Processing on Large Clusters. OSDI ’04: 6th Symposium on Operating
computing framework. Systems Design and Implementation, 137-149.
[5] McCarthy, J. (1960). Fundamentals of Fault-Tolerant Distributed
TABLE II. STANDALONE WORKER PERFORMANCE Computing in Asynchronous Environments. ACM Computing Surveys,
Vol. 31, No. 1, March 1999:1-26.
CPU Memory Time [6] A. Ankolekar. OWL-S: Semantic Markup for Web Services,2003.
consumed http://www.daml.org/services/owl-s/1.0/owl-s.pdf
Node1 P4 2.66GHZ 1024mb 5525.891s [7] W.-T. Balke, M. Wagner. Cooperative Discovery for Usercentered
Node2 Core2 6600 2.4GHZ 1024mb 3012.609s Web Service Provisioning. In Proc. of the First Int.Conf. on Web
Services (ICSW'03), Las Vegas, USA, 2003.
Node3 P Dual 2.8GHZ 1024mb 3901.797s
[8] W.-T. Balke, M. Wagner. Towards Personalized Selection of Web
Average 4146.766s Services. In Proc. of the 12th Int. World Wide Web Conference (WWW
2003), Budapest, Hungary, 2003.
[9] Balzer, T. Liebig, M. Wagner. Pitfalls of OWL-S – A Practical Semantic
TABLE III. DISTRIBUTED COMPUTING FRAMEWORK PERFORMANCE  Web Use Case. (under submission)
[10] E. Christensen, F. Curbera, G. Meredith, S. Weerawarana. Web Services
Worker Num Job Unit Time consumed
Description Language (WSDL) 1.1.,
Test1 3 30 s 1233.453s W3C,http://www.w3c.org/TR/wsdl.
Test2 3 60 s 1002.188s [11] W. Kellerer, M. Wagner, W.-T. Balke, H. Schulzrinne, Preference-
based Service Management for IP-Based Mobile Multimedia Signaling,
Average 1117.820s ETT, Wiley, July 2004.
[12] S. McIlraith, T. Son, and H. Zeng: Semantic Web Services.In IEEE
Journal on Intelligent Systems, IEEE, 2001.