Wireless Networks 10, 643–652, 2004

 2004 Kluwer Academic Publishers. Manufactured in The Netherlands.

Publish/Subscribe in a Mobile Environment

YONGQIANG HUANG ∗ and HECTOR GARCIA-MOLINA
Department of Computer Science, Stanford University, Stanford, CA 94305, USA

Abstract. A publish/subscribe system dynamically routes and delivers events from sources to interested users, and is an extremely useful
communication service when it is not clear in advance who needs what information. In this paper we discuss how a publish/subscribe system
can be extended to operate in a mobile environment, where events can be generated by moving sensors or users, and subscribers can request
delivery at handheld and/or mobile devices. We describe how the publish/subscribe system itself can be distributed across multiple (possibly
mobile) computers to distribute load, and how the system can be replicated to cope with failures, message loss, and disconnections.
Keywords: publish/subscribe, mobile networks, wireless ad-hoc networks, event systems

1. Introduction been tackled and solved, with some systems having reached
commercial maturity [24,28,30].
A publish/subscribe system connects together information However, almost all of the research on publish/subscribe
providers and consumers by delivering events from sources systems so far has concentrated on publish/subscribe sys-
to interested users. A user expresses his/her interest in receiv- tems in a fixed network. We argue that publish/subscribe
ing certain types of events by submitting a predicate defined systems are also advantageous in a mobile and/or wireless
on the event contents, called the user’s subscription. When a environment [11]. The anonymity and dynamism of pub-
new event is generated and published to the system, the pub- lish/subscribe allow the systems to adapt quickly to frequent
lish/subscribe infrastructure is responsible for checking the connections and disconnections of mobile nodes, characteris-
event against all current subscriptions and delivering it effi- tic of a mobile network. Asynchrony is helpful because mo-
ciently and reliably to all users whose subscriptions match bile devices are often turned off or disconnected from the net-
the event. work for long periods of time. Wireless devices have limited
The publish/subscribe communication paradigm differs capabilities and bandwidth. The multicasting nature of pub-
from traditional point-to-point models in a number of ways. lish/subscribe helps a system scale to thousands of units.
In publish/subscribe, communication is anonymous, inher- With increasing popularity of mobile handheld devices,
ently asynchronous and multicasting in nature. The system is there is a pressing need to extend publish/subscribe to a
able to quickly adapt in a dynamic environment. Anonymity mobile environment. As a sample application, in a mil-
means that the communication partners are not required to itary battlefield, thousands of wireless and mobile sensors
identify the party they want to talk to. For example, instead such as satellites and equipment sensors report all kinds of
of naming a publisher to receive events from, the subscriber information ranging from the location of enemy troops to
simply describes the characteristics of the events it wants to whether the engine of a tank has overheated. There are also
receive. Publish/subscribe is also inherently asynchronous many parties interested in receiving certain types of infor-
because the sender (publisher) does not have to wait for an ac- mation. An individual soldier may need to know the loca-
knowledgment from the recipient (subscriber) before moving tion of the nearest enemy troops, or whenever a missile has
on. The reliable transmission of events to the subscribers is been fired. The above scenario requires the deployment of
taken care of by the infrastructure. Publish/subscribe resem- a highly scalable and dynamic communication infrastructure,
bles multicast because it allows a publisher to send the same for which a mobile publish/subscribe system is an ideal can-
event to many subscribers with only one publish operation. didate.
Finally, the system can cope with a dynamically changing op- In this paper, we briefly review a few publish/subscribe
erational environment where the publishers and subscribers models and discuss how they might be adapted to a mobile
frequently connect and disconnect. environment. It is our hope that the issues we raise in this pa-
This combination of unique characteristics makes the pub- per will generate further exciting research, thereby giving this
lish/subscribe model well suited to a variety of application new field of study the attention it deserves. After presenting
areas, such as distributed information dissemination, finan- our framework (section 2), we start with the simplest model,
cial analysis, alert systems and factory automation. In the namely, the centralized approach (section 3). We then dis-
past decade many problems related to publish/subscribe have cuss distributed models that address the scalability problem
∗ Corresponding author. (section 4). Finally, we look at the use of replication and its
E-mail: [email protected] impact (section 5).
644 HUANG AND GARCIA-MOLINA

2. Framework

The first generation of publish/subscribe systems use either

group-based (also known as channel-based) or subject-based
(also known as topic-based) addressing. In the former [5,12,
21,27], a set of “groups” (or “channels”) are designated by
the system. Each event is published to one of these groups
by its publisher. A user subscribes to one or more groups,
and will receive all events published to the subscribed groups.
For example, in IP multicast [12], each group is identified by
a class D IP address. Subject-based systems [10,19,22,28]
are slightly more flexible. Each event is tagged with a short Figure 1. A content-based publish/subscribe system. The bubbles represent
“subject” (or “topic”) that describes its content. The subject is filtering of events, and are labeled with the respective filtering criteria.
either an arbitrary string or a string taken from an agreed-upon
domain. The subscriber defines its subscription in terms of
this subject line. In addition to an exact match, the subscriber all events reporting the location of any tank within a certain
can also ask for all subjects beginning with the word “jobs”, range. Note that, as the core of the publish/subscribe sys-
for example. tem, the EBS could be implemented as either a single server
In recent years a more flexible paradigm called content- (Event Broker) or multiple distributed ones working together.
based addressing has emerged. A content-based system gives Additionally, an Event Broker can be replicated to increase
more flexibility and control to the subscriber by allowing availability. Sections 4 and 5 discuss distributed and repli-
him/her to express his/her interest as an “arbitrary” query over cated architectures and their mobile adaptations.
the contents of the events. Therefore, instead of relying on the In figure 1, we use ci to denote the subscription criterion
publisher to classify the events into groups or subjects, the of user i. In other words, user i wants all events and only
subscriber is now able to define sophisticated subscriptions those events that satisfy ci . If a user’s subscription matches,
such as “give me all stock quotes for stock X issued between the event is forwarded to the Event Displayer associated with
time A and time B if the price is larger than 35”. A content- that user. The presence of a bubble labeled ci in the link be-
based publish/subscribe system has also been called condition tween the EBS and an ED implies that only events satisfying
monitoring systems or event notification/distribution/delivery ci passes through on this link. The Event Displayer is respon-
systems in various contexts. sible for alerting the user. In our example, the soldier will
However, the flexibility of content-based systems comes
be notified by a message on his/her mobile communication
at the expense of added challenges in design and implemen-
device.
tation [7]. Intuitively, because the subscriptions can be com-
Note that some of the event services surveyed in this paper
plex, figuring out matches between events and subscriptions
may provide additional functionality such as event stream ma-
is a lot harder than in traditional group- or subject-based sys-
nipulation. For example, some systems can trigger on events
tems, where usually a simple table lookup is sufficient.
to generate new events. In this case, a subscription might look
In this paper, we will assume a content-based system in
like: “generate a buy order when the price of stock X has
our discussions, because it is the more general and powerful
climbed for more than 20 percent for three straight quotes”.
model. For instance, group- and subject-based systems can be
regarded as special cases of content-based systems where the The ability to generate new events has been termed “content-
subscription syntax is restricted to simple tests on a specific based with patterns” [8], “event stream interpretation” [4] and
field of the event. “historical condition triggering” [17], among other things. In
this paper, we do not take into account any of the specific
2.1. Basic model system extensions such as this. Instead, we will focus on the
most fundamental functionality, namely, to route events from
Figure 1 illustrates the schematic of a basic publish/subscribe their sources to their targets efficiently and reliably.
system. It consists of one or more Event Sources (ES), an Although many systems do not adhere to the publish/sub-
Event Brokering System (EBS), and one or more Event Dis- scribe model exactly as defined here, they nonetheless share
players (ED). An Event Source generates events in response a lot of the goals and challenges of pure publish/subscribe
to changes to a real world variable that it monitors, such as systems. Recently, for example, data stream systems [2,6,15]
the location of an enemy tank. We assume that each event and sensor networks [13] have garnered attention in the re-
is labeled with its source and a consecutive sequence num- search community. The former filters and processes contin-
ber to facilitate our description. Other than that, we do not uous streams of data, while the latter coordinates large ar-
make any assumptions about an event’s content. The events rays of (usually wireless) sensors. However, it is beyond the
are published to the Event Brokering System, which matches scope of this paper to survey all relevant systems, and we in-
them against a set of subscriptions, submitted by users in the stead focus on extending the core event capabilities of pub-
system. For example, a soldier could subscribe to receive lish/subscribe to mobile environments.
PUBLISH/SUBSCRIBE IN A MOBILE ENVIRONMENT 645

ES can be a stock trading center giving out stock quotes. In-

dividual investors usually cannot ask these sources to monitor
a stock condition for them. Finally, the EB may need to store
a matched event and repeatedly attempt to resend it to its tar-
get as the target has gone offline. It is unreasonable to require
a mobile Event Source to prolong its connection to the fixed
network just because the event recipient is not connected for
the moment.
Likewise, the EB should be hosted on a separate device
from an ED as well. The PDA can be powered off or discon-
nected from the network most of the time to conserve battery,
Figure 2. Centralized architecture: one server does all the matching and making it unsuitable to host the EB, which needs to listen
filtering.
constantly for new events. Furthermore, the computer host-
ing the EB should be placed in the fixed network if possible,
3. Centralized architectures because otherwise when the central EB is disconnected, the
whole system would be paralyzed.
A centralized Event Brokering System consists of only one
Once we figure out where each part of the system resides, it
Event Broker (figure 2). The central EB stores all currently
would seem that we can simply rely on previous work on mo-
active subscriptions in the system. Every new event is pub-
bile networking [23] to provide us with connectivity between
lished to the EB, which is responsible for matching it against
the components and to hide the idiosyncrasies of mobile com-
all the subscriptions. Afterwards the event is forwarded to
munication. However, as we will illustrate next, there are is-
all Event Displayers whose subscriptions match. Represen-
sues unique to publish/subscribe in a mobile environment that
tative systems in this category include the SIFT Information
we have to consider.
Dissemination System [32] and active databases [9].
An important problem that any centralized system would Mobile/wireless devices can be frequently disconnected
need to address is how to efficiently match a new event against from the fixed network because they are off (running out of
a large set of subscriptions to figure out which ones match. battery or turned off to conserve battery), or they cannot be
Although the matching problem is challenging and interest- contacted (e.g., transient wireless communication problems
ing to study, it is beyond the scope of this paper. Interested or wandering into an area without radio reception). A good
readers are referred to [1,9,14,32] for more detailed discus- mobile publish/subscribe system has to deal gracefully with
sions. both the ES’s and the ED’s going offline. For example, when
Even though the central EB may be a performance bottle- a user is out of reach, it is reasonable to expect the EB to log
neck and a single point of failure, it is important to under- and queue the user’s events so that they can be delivered later
stand how it could operate in a mobile environment. In later when the user comes back online.
sections, we will look at how distribution alleviates the scala- Because a wireless device can be turned off or discon-
bility problem, and how replication helps with reliability. nected for long periods of time, a lot of missed events can
accrue in the meantime. Even if storage at the EB is not a
3.1. Mobile adaptation concern (which can be in an ad-hoc environment, for exam-
ple), the sheer amount of time and precious wireless band-
When adapting a centralized architecture to the mobile en- width required to transmit all of the queued events to the ED
vironment, we argue that, while the Event Sources and the when it reconnects might be unreasonable. In this case, the
Event Displayers can reside on a mobile device, the central EB can attempt to transform the set of events into an “equiv-
Event Broker server should if possible be placed on a separate alent” (or at least similar) sequence of events which take up
computer in the fixed network. Typically an Event Source re- less space [33]. The methods to accomplish this include sum-
sides near the real world variable it monitors, while an Event marization and selective discarding of events. Here knowl-
Displayer resides near the end user (e.g., on a PDA). Since in edge about the semantics of a subscription often helps.
a mobile environment both the information providers and the As an example, suppose the events in the queue contain
consumers tend to be mobile, the ES and the ED are likely to information about a certain stock. If the user is interested in
be placed on a mobile device. aggregates only, such as the total volume, then the EB could
The central EB, however, should reside on a computer sep- summarize the entire buffer of events with one event repre-
arate from the ESs or the EDs. There are three reasons why senting the overall sum. If the user is only interested in “re-
the EB should not, in general, be placed on the same device cent” activities, the EB can purge older events from the queue
as an ES. Firstly, the EB will likely require a fair amount of first. On the other hand, if the overall trend history is de-
computing resources for data logging and subscription match- sired, then the EB should probably reduce the sampling den-
ing, while an ES is usually a simple sensor device. Secondly, sity while still maintaining event samples from all time pe-
the Event Sources can be autonomous and do not allow the riods. For example, the EB could try to discard every other
users to store their own subscriptions there. For example, the event in timestamp order. Lastly, if the user is only interested
646 HUANG AND GARCIA-MOLINA

when the price breaks new highs, then the EB could retain
only the set of such “important” events.
More issues arise when an Event Source is disconnected.
One option of course is to have the ES queue all the events that
are generated when it is not connected. Such an option may
not be feasible, however, as the ES is often a low capability
device without too much storage. Moreover, the rate of event
accumulation can be very high at the ES, compared to at the
EB where only those events of interest to a disconnected user
need to be queued. Consequently, the ES will have to discard
some events once its buffer fills up. Unfortunately, without
knowledge of what subscriptions exist against its events, the Figure 3. Centralized architecture with quenching.
ES cannot perform more “intelligent” selection of events to
discard, and may have to resort to a standard scheme such as generated, the ES first checks it against call . If call (e) = false,
FIFO replacement (i.e., discard the oldest timestamped event that means no subscription will match e at the EB. Hence the
first). event is discarded (quenched) at the source. If e matches call ,
then at least one subscription will match, and the event is for-
3.2. Ad-hoc networks warded to the EB as usual. This quenching behavior is repre-
sented by the bubbles labeled call in figure 3.
The discussion in the previous paragraphs has assumed a tra- Note that in order for quenching to make sense it must
ditional mobile network where, although many nodes are mo- be much easier to figure out whether or not an event e
bile, some fixed network support is still available. Ad-hoc matches the combined call than to figure out the exact sub-
networks pose additional challenges to a publish/subscribe set of {c1 , c2 , . . . , cN } that matches, so that the Event Source
system. An ad-hoc network is formed by wireless devices does not have to duplicate all the work that is being done at
communicating without the benefit of a fixed network in- the EB1 [29]. Quenching has proved to be particularly effec-
frastructure. Ad-hoc networks are extremely useful in scenar- tive in reducing network traffic and the load of the central EB
ios where a natural disaster has wiped out the infrastructure, if a significant portion of the events generated do not match
or where rapid deployment is required and an infrastructure is any subscriptions.
not possible, for example in the battlefield. However, the appropriateness of using quenching in a mo-
The lack of a fixed network infrastructure in ad-hoc net- bile environment needs to be further examined. We have said
works implies that the central EB must also be mobile. Hence previously that an ES can be a wireless low capability sensor
we can no longer assume that an EB always stays connected. device. Thus it might not be feasible for the ES to evaluate
If an old EB becomes unreachable, it will have to be replaced. a complicated condition for every event generated. More-
In particular, when an Event Source wants to publish, it must over, informing the sensor of newly added or deleted sub-
first search for an existing EB. If one cannot be found, the ES scription could consume valuable wireless bandwidth. On
will initiate a protocol to elect a new EB. Likewise, an Event the other hand, effective quenching can also significantly re-
Displayer must periodically poll the EB and refresh its sub- duce the bandwidth needed to transmit events. Fundamen-
scription information. Otherwise, the old EB could have gone tally, quenching represents a tradeoff between the bandwidth
out of reach and a new EB elected without knowing about this
required to send all events and the computation power needed
ED’s subscription.
to match and filter events. Since a mobile device is usually
The above suggests that, at a given time, there can be more
limited in both resources, the answer is not apparent.
than one EBs in operation. For example, two EBs could each
Quenching can be a particularly attractive option when an
be servicing a disconnected partition of the network. Thus,
ES is disconnected, since it allows the ES to discard certain
when one EB becomes aware of the existence of another EB
events on the fly, thus reducing the potential size needed for
(for example, when two previously partitioned wireless sub-
the event queue. In the case where the queue eventually fills
nets come into physical proximity), a merging protocol might
up and discarding events becomes inevitable, knowing what
have to be invoked to combine them into one central EB. Al-
subscriptions there are in the system can also be advantageous
ternatively, both could operate in a coordinated fashion as dis-
in selecting which events to discard, as discussed earlier.
cussed in section 4.
However, quenching is also problematic in disconnection
since the system cannot contact the ES about newly added
3.3. Centralized with quenching
subscriptions when the ES is disconnected. Therefore, a rea-
Quenching has been proposed [26] as an enhancement to the sonable strategy might be a tiered approach as follows. At
straightforward centralized approach in fixed networks (fig- the beginning of a disconnection, the ES saves all events in
ure 3). An Event Source is given a “combined active subscrip- 1 A trivial example where this is true is the following. Suppose c1 =
tion expression” (call), which represents the logical OR of all (e.value > 10) and c2 = (e.value > 20). To figure out whether an event
the currently active subscriptions on the Event Broker. Essen- matches either c1 or c2 , it is sufficient to only test whether its “value” is
tially, we have call = c1 ∨c2 ∨· · ·∨cN . When a new event e is larger than 10.
PUBLISH/SUBSCRIBE IN A MOBILE ENVIRONMENT 647

the buffer, in case a new subscription not known to the ES

matches them. When the buffer overflows, however, the ES
can then start to discard events according to the quenching
criteria it has. Finally, if this still does not bring the situation
under control, the ES can take more aggressive measures such
as summarization.

4. Distribution

As explained in section 3, centralized systems are limited by

the capability of a single server, beyond which distribution
Figure 4. Distributed broadcast. The dotted lines are the path of an example
has to be used. This section illustrates two typical ways that event which satisfies c1 and c2 .
work is partitioned among multiple servers, and their exten-
sions to the mobile world.

4.1. Distributed broadcast

In distributed broadcast (figure 4), the EBS consists of M

Event Brokers, each responsible for a portion of the N total
subscriptions. The EBs are connected to each other by the net-
work and form a general connected graph.2 An Event Source
publishes a new event to any one of the M EBs. (In reality,
the ES probably connects to the nearest EB in the network.)
That EB is then responsible for forwarding the event to all
other EBs in the system (hence the name “distributed broad-
Figure 5. Distributed multicast.
cast”). The forwarding “broadcast” can be implemented with
network layer multicast [12]. Alternatively, the event could the EBs downstream from a particular branch. If the result
be sent along a “forwarding tree” rooted at the originating is false, that branch is “pruned” for this event. The behavior
EB, using unicast at each leg of the trip. is very similar to what happens at the routers in IP multicast,
When a new event arrives, each EB matches the new event hence the name “distributed multicast”.
against all subscriptions it is responsible for, and delivers the The need for partial matching implies that, unlike in dis-
event as necessary. Note that the matching and delivering tributed broadcast, it is no longer sufficient for each Event
workload at each EB is reduced compared to a centralized Broker to know about only its share of the active subscrip-
approach because, although each EB still processes all the tions. In the worst case, each Event Broker may need to store
events generated in the system, it only has to match them all the currently active subscriptions in the system. The Siena
against a fraction of the total subscriptions. The dotted lines system [8] proposes a solution where multiple subscriptions
in figure 4 give an example of the path traversed by an event can be collapsed into one condition, at the expense of re-
which matches c1 and c2 . An example of a distributed pub- stricted subscription syntax.
lish/subscribe system using broadcast is the SIFT Grid [32]. Unlike distributed broadcast, distributed multicast can no
longer take advantage of network layer multicast directly to
4.2. Distributed multicast forward events to needed EBs, because potentially complex
partial matching needs to be performed at each step. An effi-
Distributed broadcast can create a lot of network traffic be- cient implementation of the forwarding tree without using IP
cause events are flooded to all the Event Brokers. An alterna- multicast is given in [3].
tive approach, called distributed multicast (figure 5), prunes
the forwarding tree. Specifically, when an event arrives at 4.3. Mobile adaptation
each EB in the forwarding tree, it is forwarded onto one of
the EB’s outgoing branches only if the event might match a In addition to the challenges facing a mobile centralized sys-
subscription at some EB leading from this branch. In other tem, there are more issues associated with adapting a distrib-
words, the EB selectively forwards an event based on the re- uted publish/subscribe architecture to a mobile environment.
sult of “partial matching”. In effect, the event is matched Because EDs often move around, an ED may disconnect and
against the logical OR of all the subscriptions stored at all connect to a different EB quite often. When the ED recon-
2 Actual systems may connect the EBs in other topologies such as a hierar- nects to a different EB, two things need to happen. First, the
chical tree instead of a peer-to-peer graph. To be most general, our paper new EB needs to be informed of the ED’s subscription so that
assumes a graph structure, although our discussion is equally valid for other the routing tree can be adjusted to direct relevant events to
structures. the ED. Second, the new EB needs to obtain all the events
648 HUANG AND GARCIA-MOLINA

queued on behalf of the ED while the ED was disconnected to construct and maintain the interconnection network of the
and deliver them to the ED. For both tasks, the new EB may EBs.
contact the EB previously in charge of the user’s subscrip- Traditional distributed publish/subscribe systems are usu-
tion to obtain the information as part of a “handoff” proto- ally built as an overlay network of EBs over an IP-like net-
col [10]. working infrastructure which provides universal connectivity.
Alternatively, however, an ED can carry its own subscrip- In other words, they assume that the underlying routing fabric
tion information, and upload it onto the new EB when the ED always provides connectivity between any two nodes in the
reconnects. The advantage of this approach is that the ED system, and at a roughly constant cost. Thus, a new EB join-
can still receive new events even if the old EB is temporarily ing the system can choose any other existing EB to connect
down or partitioned from the new EB. (Of course the new EB to in order to start receiving events. Although this assumption
still needs to attempt contact with the old EB periodically to is reasonable in fixed networks, it is wasteful in an ad-hoc
cancel the old subscriptions.) environment.
The potential downside is that the ED may end up with Unicast protocols have been proposed for ad-hoc networks
more than one EBs monitoring the same subscription for it. [20,25], which give users the abstraction that any node can
Reference [16] proposes several schemes for mobile handheld talk to any node else. However, ad-hoc unicasts are actu-
devices which ensure that the ED receives the same message ally implemented with multi-hop broadcasts. If we build an
exactly once. For example, one variation requires the ED to ad-hoc publish/subscribe system purely on top of the unicast
keep a log of its past connections, which includes a timestamp framework, the wireless broadcast nature of radio is hidden.
and the id of the EB for each connection. Whenever the ED Instead, we believe that such a system should build directly
makes a new connection, this information is uploaded to the on top of lower level radio broadcast primitives due to the
new EB, which uses it to check for any potential danger of “many-cast” nature of the publish/subscribe mechanism. This
duplicate delivery. For instance, events generated after the way, with one broadcast operation, an EB can send an event
ED’s last previous connection can safely be delivered. More- to multiple other interested EBs.
over, if another EB cannot be contacted at the moment, but We define the connectivity graph G of a system as the
the log shows that the last connection to that EB happened graph whose vertices are the EBs, and where an edge ex-
“long enough” ago in the past, then queued events may still ists between two vertices if the two corresponding EBs are
be delivered without worrying about duplication. “neighbors”. Two EBs are neighbors if they can talk to each
The subscription handoff protocol needs to be designed other directly via radio, i.e., if they are a single hop away.3
carefully so that, as the new routing information slowly per- After a new event is published to an initial EB, it is for-
colates up the forwarding tree, no event from any potential warded from one EB to another towards its destinations. The
source is lost. Ideally the same event should not be delivered paths followed by all events from a particular initial EB con-
both to the old and to the new EBs. If that is impossible to stitute a tree, which is called a publish/subscribe tree (PST).
guarantee, however, mechanisms to eliminate duplicates will Because we have determined to only use radio broadcast
be needed again. primitives, two EBs should talk to each other only if they
In certain wireless systems, it is sometimes possible to are adjacent in the connectivity graph. Therefore, we effec-
further optimize the connection behavior by using an “inte- tively restrict the PST to be a spanning tree of the connectivity
grated” approach. Cellular communication systems, for ex- graph G.
ample, use base stations as access points for wireless devices Note that a PST is rooted at an EB where new events are
into the fixed network. A wireless device is controlled by one published. If multiple EBs in one system are allowed to pub-
and only one base station at any time it is connected. When lish events, which is often the case, we then must choose
it moves out of the range of an old base station and into the one of the following two options. First, there can be multi-
ple publish/subscribe trees, one per EB that can potentially
range of a new one, a wireless handoff protocol is invoked.
accept new event publications. Alternatively, we can require
Naturally, the base stations in such systems are ideal can-
any new event published to be passed to a global root EB
didates for Event Brokers in a distributed publish/subscribe
first via a unicast protocol. A possible optimization to the
system. In this case, subscription handoff can be handled
latter option above is to forward the event towards the root
as merely an additional step in wireless connectivity handoff,
along the publish/subscribe tree itself, allowing shortcutting
thus saving valuable time and resources.
to tree branches along the way, thus eliminating the need for
the event to be passed back in this direction. The tradeoff of
4.4. Ad-hoc networks the two options is similar to the shared versus per-source tree
discussion in ad-hoc multicast, and the user is referred to [31]
In a wireless ad-hoc distributed publish/subscribe system, for a detailed treatment. In the discussions that follow, we
multiple EBs reside on wireless mobile devices, and coop-
3 The connectivity graph is assumed to be undirected, precluding the situa-
erate with each other via radio to deliver events from their
tion where one EB can hear another, but not vice versa. We further assume
sources to interested users. Additional problems arise in such that G is connected, although the algorithms discussed next can be aug-
a system because the EBs can no longer be placed on fixed mented, for example, by adding expanding ring searches for the nearest
nodes as assumed above. We next discuss the problem of how EB.
PUBLISH/SUBSCRIBE IN A MOBILE ENVIRONMENT 649

assume a fixed root EB, say, EB1, where all new events are teria aimed to minimize overhead. S HOP PARENT is a greedy
published. The discussion can be easily extended to deal with local algorithm because no node needs to have global knowl-
any of the alternatives proposed above. edge about the system.
Obviously, in any given system, there usually can be many The implementation details of S HOP PARENT are given
possible PSTs (as many as there are spanning trees in the con- in [18]. In short, each EB periodically broadcasts a PARENT-
nectivity graph G). Some trees are considered better than oth- P ROBE to all EBs in its immediate radio neighborhood, giv-
ers because in these trees events travel a shorter distance, on ing its desired subscription. A recipient of the probe replies
average, to reach their destinations. with a PARENT-A DVERTISE message if it is capable of par-
enting the requester. The reply contains, in addition to other
Example 1. As a simple example, figure 6 shows a system necessary information, an estimate of how much its overhead
with three EBs. Assume both EBs 2 and 3 can receive events will increase if the requesting EB connects to it. The origi-
directly from the root, EB1. If they both do, the resulting PST nal EB selects among all the replies the one with the smallest
is given by the thick lines in figure 6(a). If, on the other hand, expected overhead. Because the probe is periodical, an EB is
EB3 chooses to rely on EB2 to forward events, we have the constantly searching for a better parent, and the tree can re-
PST in figure 6(b). Intuitively, the PST in figure 6(a) is “bet- configure itself as a result of changes in the system such as
ter” than 6(b) since those events of interest to 3 need travel node movements and failures.
only one hop instead of two. Because nodes consider subscriptions when making con-
nections, they can make better decisions, compared to tradi-
In order to meaningfully compare different PSTs, we need tional multicast algorithms. The following simple example
a metric to measure the “efficiency” of a PST. The metric illustrates this point.
is called the overhead of a publish/subscribe tree, and mea-
sures the total amount of “work” performed by all EBs in or- Example 2. Figure 7 shows a simple system with five EBs.
der to publish a given set E of events. For example, when EB4 is joining the system, and is trying to select either 5
a new event reaches a particular EB, the EB has to perform (figure 7(a)) or 3 (figure 7(b)) as its parent in the PST. If a
actions such as checking for matches, buffering the event and naive algorithm is used, it seems that 5 should be selected,
re-broadcasting the event for children in the PST, etc. These since it is closer to the root (EB1) by one hop. However, as-
actions are all factored into the overhead calculation. Ref- sume that EB4’s subscription is very “similar” to that of 3.
erence [18] more precisely defines the evaluation metric. We In fact, assume that the overlap is total, and both of them are
note that, not surprisingly, the tree in figure 6(a) has less over- interested in “sports.basketball.lakers” in a news group set-
head than the tree of figure 6(b). ting. On the other hand, 5 is interested in something totally
A publish/subscribe tree construction algorithm is the pro- different, e.g., “rec.cooking”. If 5 becomes the parent, all
tocol run by the EBs in order to determine which PST to use. “sports.basketball.lakers” events will need to pass through it
The algorithm aims to minimize the overhead of the result- as well. It turns out that, therefore, figure 7(b) represents a
ing publish/subscribe tree based on the metric presented ear- more efficient PST than figure 7(a). Because S HOP PARENT
lier. We propose a greedy tree construction algorithm called bases decisions on the subscriptions of the new EB and its
S HOP PARENT which is fully distributed. In particular, the proposed parent, it can produce the desired result.
PST is constructed by each EB running one instance of the
algorithm and making its own decision about which other EB In order to evaluate the algorithms, we run experiments
to select as its parent. The parent is chosen according to cri- based on simulation. We have come up with ways to simulate

(a) “Good” PST. (a) PST a.

(b) “Not-as-good” PST.

(b) PST b.
Figure 6. Two PSTs of the same connectivity graph. The lines represent the
connectivity graph, while the thick ones constitute a PST. Figure 7. Two PSTs of the same connectivity graph.
650 HUANG AND GARCIA-MOLINA

various facets of the system, from the connectivity graph G, ED is usually a program running on the PDA which the user
to user subscriptions and event publications. For example, we carries with him/her. Hence, the two streams of events gen-
use two models (below) to simulate subscriptions. The mod- erated by EB1 and EB2 will merge at the ED. Note that for
els correspond to systems of different characteristics. Yet, simplicity we use a centralized architecture as the basis for
they both have enough tunable parameters that allow us to replication. Although we do not discuss it here, replication
vary the experiments in interesting ways. can also be introduced in a distributed system like the ones in
The first subscription model, called Number Intervals, uses figures 4 and 5.
an interval on the number axis, e.g., [3.0, 4.0], or [100, ∞), With replication, a user is less likely to miss events. For in-
to represent a user’s subscription. Then, an event is repre- stance, suppose that EB1 misses some events from a particu-
sented by a single number, e.g., 3.2. The event matches a lar mobile source which can only communicate with EB2 due
subscription if and only if its number falls within the subscrip- to temporary network problems.4 Then the events can still be
tion interval. Thus, event 3.2 matches subscription [3.0, 4.0], matched by EB2 and delivered to the appropriate EDs. How-
but not [100, ∞). By carefully picking the intervals, we can ever, without any safeguards, replication can create “consis-
adjust the sizes of each user’s subscriptions, as well as the tency” problems in a publish/subscribe system. Specifically,
amount of overlap between them. the user may receive a sequence of events that are confus-
While the Number Intervals model can be mapped directly ing or even contradictory. As a simple example, without a
to many real world applications such as temperature range mechanism to eliminate duplicates, the same event may get
monitoring, our second model simulates other scenarios such delivered to the user twice, once from each EB. The user will
as newsgroups. In the Topic Tree model, a range of possible get confused if he/she relies on the events to keep track of
topics are arranged in a hierarchical tree. Each event is tagged an important count, such as the exact number of missiles that
with its topic, and each user is interested in a portion of the have been fired.
overall tree. Again, subscription scopes and overlap can be As another example, although it is not difficult to make
easily quantified and adjusted. a single EB always deliver events from the same source in
Reference [18] offers an in-depth study of S HOP PARENT. order, replication can often result in an unordered event se-
For instance, experiments show that the greedy S HOP PAR - quence when events from the two EBs are interleaved at the
ENT performs well in normal operating environments, with ED. For instance, suppose event number 3 is missed by EB1
less than 15% overhead compared to a global optimal strat- but received by EB2. It is therefore entirely possible for EB1
egy (which is, of course, impractical to implement). Further- to deliver the next event, say number 4, to the user before EB2
more, [18] demonstrates that being “subscription-aware” is could have a chance to deliver event 3. Out-of-order event
advantageous, and that its advantage increases with the over- streams can be a problem if the order of events is significant,
all amount of overlap between user subscriptions. for example to establish a trend in the movements of a stock’s
price.
We can define three desirable properties for a replicated
5. Replication publish/subscribe system: Orderedness, Consistency and
Completeness. The goal in general is to rule out deliver-
Replication can be used in a publish/subscribe system to ies of events to a user that could not have occurred with a
increase its availability and reliability when faced with non-replicated system. Intuitively, orderedness indicates that
server failures or network partitions. In a replicated publish/ events from the same ES are delivered to the user in the order
subscribe system (figure 8), a user’s subscription is monitored they are generated at the ES. Since a non-replicated system
by multiple Event Brokers independently. In particular, in fig- delivers events in this order, a replicated system that is or-
ure 8 we assume that two EBs, EB1 and EB2, simultaneously dered behaves similarly in this respect.
monitor the subscriptions for each user. On the other hand, For a replicated system to be consistent, the set of events it
we assume that there is still only one Event Displayer associ- displays to an end user over time must be a set that can pos-
ated with each user, because, as we have discussed before, the sibly be generated by a non-replicated system (although per-
haps in a different order). In other words, a user should not
be able to tell, from observing the events that are displayed
to him/her, that replication is being used (except for possi-
bly increased reliability and responsiveness). For example, a
replicated system that delivers duplicates to the end user is
trivially not consistent.
Lastly, completeness requires a replicated system R to
display all events that would be displayed by an equivalent
non-replicated system N had the single EB in N received all
4 We assume that events can be lost when they are sent from their source
to an EB. However, since we assume that the EB buffers and retransmits
events as necessary, the link between the EB and the ED is assumed to be
Figure 8. Replicated publish/subscribe. lossless.
PUBLISH/SUBSCRIBE IN A MOBILE ENVIRONMENT 651

Table 1
filtering algorithms are developed to guarantee consistency in
Properties satisfied by a replicated publish/subscribe system under various
ED filtering algorithms. such scenarios. Additionally, subscriptions defined on event
“joins” from different streams are also studied. The paper
ED filtering Orderedness Consistency Completeness
also investigates multiple subscriptions submitted by the same
√
No filtering X X
√ √
user that are interrelated and need to be monitored in a coher-
Duplicate removal X ent fashion.
Out-of-order and
√ √
duplicate removal X
6. Conclusion
events that were received by EB1 or EB2 in R. For example,
if an event matching the user’s subscription arrives at EB1 In this paper we discussed how to adapt a publish/subscribe
but is missed by EB2 due to network packet loss, then a com- system to a mobile operating environment. We described sev-
plete replicated system will need to ensure that the event is not eral architectures of a publish/subscribe system, starting from
discarded (see next on Event Displayer filtering) and is ulti- the simple centralized approach, to distributed ones with im-
mately delivered to the user. Completeness is a measurement proved scalability, and finally to replication that increases re-
of how effective a replicated system is at guarding against loss liability but may cause consistency problems. We discussed
of events in the network. Our three notions of correctness are issues and possible solutions specific to adapting the vari-
defined more formally in [17]. ous architectures to a mobile and/or wireless environment.
Obviously, if the Event Displayer simply passes along any We also sketched solutions to the more challenging problems
event it receives to the user, the resulting replicated system posed by ad-hoc networks. In presenting our work, we also
will be neither consistent (due to duplicates) nor ordered. Ta- surveyed some of the important work on content-based pub-
ble 1 summarizes properties satisfied by a replicated system lish/subscribe systems in fixed networks.
under various configurations, with the first row being when
no special processing is done by the ED. However, as we will
References
see next, some system properties can be enhanced or enforced
if the ED performs an extra step to filter out some events (e.g., [1] M.K. Aguilera, R.E. Strom, D.C. Sturman, M. Astley and T.D. Chan-
duplicates) before passing them on to the user. dra, Matching events in a content-based subscription system, in: Pro-
In the simplest example, the ED can implement a straight- ceedings of the 18th Annual ACM Symposium on Principles of Distrib-
forward “exact duplicate elimination” algorithm, in which an uted Computing (1999) pp. 53–61.
[2] B. Babcock, S. Babu, M. Datar, R. Motwani and J. Widom, Models
event is discarded by the ED if an identical one has already and issues in data stream systems, in: Proceedings of the 2002 ACM
been displayed previously. Identical events are identified by Symposium on Principles of Database Systems (2002).
examining their sources and sequence numbers, which infor- [3] G. Banavar, T. Chandra, B. Mukherjee, J. Nagarajarao, R.E. Strom
mation the ED keeps for displayed events. (The exact defi- and D.C. Sturman, An efficient multicast protocol for content-based
nition of “identical” is given in [17].) The modified system publish–subscribe systems, in: Proceedings of the 19th International
Conference on Distributed Computing Systems (1999) pp. 262–272.
properties under this ED filtering algorithm are listed in the [4] G. Banavar, M. Kaplan, K. Shaw, R.E. Strom, D.C. Sturman and
second row of table 1. As shown in the table, the system has W. Tao, Information flow based event distribution middleware, in: Pro-
gained consistency as a result. ceedings of the 1999 ICDCS Workshop on Electronic Commerce and
For situations where an ordered event stream is impera- Web-Based Applications (1999).
tive, an ED filtering algorithm has been proposed in [17] to [5] K. Birman, The process group approach to reliable distributed comput-
ing, Communications of the ACM 36(12) (1993) 36–53.
enforce orderedness of a replicated system. Essentially, the [6] D. Carney, U. Cetintemel, M. Cherniack, C. Convey, S. Lee, G. Seid-
ED records the last seen sequence number from each Event man, M. Stonebraker, N. Tatbul and S.B. Zdonik, Monitoring streams
Source and discards any new event that arrives out of order. – a new class of data management applications, in: Proceedings of the
The disadvantage of this algorithm, however, is that the sys- 28th International Conference on Very Large Data Bases (2002).
tem is no longer complete, since some events may be “unnec- [7] A. Carzaniga, E. Nitto, D. Rosenblum and A. Wolf, Issues in support-
ing event-based architectural styles, in: Proceedings of the 3rd Interna-
essarily” filtered out based on their arrival order rather than tional Software Architecture Workshop (1998).
their content. The tradeoff of completeness versus ordered- [8] A. Carzaniga, D.S. Rosenblum and A.L. Wolf, Achieving scalability
ness should be decided by the individual applications. The and expressiveness in an Internet-scale event notification service, in:
last row in table 1 gives the system properties under a com- Proceedings of the 19th Annual ACM Symposium on Principles of Dis-
bined filtering algorithm that guarantees both orderedness and tributed Computing (2000) pp. 219–227.
[9] S. Ceri and J. Widow, Active Database Systems: Triggers and Rules for
consistency. Advanced Database Processing (Morgan Kaufmann, 1996).
Reference [17] offers an in-depth study of replication in [10] G. Cugola, E.D. Nitto and A. Fuggetta, The JEDI event-based in-
publish/subscribe systems. For instance, it discusses systems frastructure and its application to the development of the OPSS WFMS,
with the ability to generate new events based on patterns in IEEE Transactions on Software Engineering 27(9) (2001) 827–850.
a stream of events. It is shown that such systems are usu- [11] G. Cugola, E.D. Nitto and G.P. Picco, Content-based dispatching in a
mobile environment, in: Proceedings of the Workshop su Sistemi Dis-
ally inconsistent, because event loss can often lead to diver- tribuiti: Algoritmi, Architetture e Linguaggi (2000).
gent perceptions between the two EBs about what constitutes [12] S.E. Deering, Multicast routing in a datagram Internetwork, Ph.D. The-
a triggering pattern. Consequently, more sophisticated ED sis, Stanford University (1991).
652 HUANG AND GARCIA-MOLINA

[13] D. Estrin, R. Govindan, J. Heidemann and S. Kumar, Next century chal- the International Workshop on Distributed Event-Based Sytems (2002)
lenges: scalable coordination in sensor networks, in: Proceedings of the pp. 619–624.
5th Annual ACM/IEEE International Conference on Mobile Computing [30] Vitria Technology, Inc., Businessware enterprise application in-
and Networking (1999) pp. 263–270. tegration, http://www.vitria.com/products/platform/
[14] F. Fabret, H.A. Jacobsen, F. Llirbat, J. Pereira, K.A. Ross and eai.html
D. Shasha, Filtering algorithms and implementation for very fast pub- [31] C. Wu, Y. Tay and C.-K. Toh, Ad hoc Multicast Routing protocol utiliz-
lish/subscribe systems, in: Proceedings of the 2001 ACM SIGMOD In- ing Increasing id-numberS (AMRIS) functional specification, Internet
ternational Conference on Management of Data (2001) pp. 115–126. draft (1998).
[15] J.M. Hellerstein, M.J. Franklin, S. Chandrasekaran, A. Deshpande, [32] T.W. Yan and H. Garcia-Molina, The SIFT information dissemination
K. Hildrum, S. Madden, V. Raman and M.A. Shah, Adaptive query system, ACM Transactions on Database Systems 24(4) (1999) 529–
processing: Technology in evolution, IEEE Data Engineering Bulletin 565.
23(2) (2000) 7–18. [33] Y. Zhao and R. Strom, Exploiting event stream interpretation in
[16] Y. Huang and H. Garcia-Molina, Exactly-once semantics in a replicated publish–subscribe systems, in: Proceedings of the 20th ACM Sympo-
messaging system, in: Proceedings of the 17th International Confer- sium on Principles of Distributed Computing (2001) pp. 219–228.
ence on Data Engineering (2001) pp. 3–12.
[17] Y. Huang and H. Garcia-Molina, Replicated condition monitoring, in:
Proceedings of the 20th ACM Symposium on Principles of Distributed
Computing (2001) pp. 229–237. Yongqiang Huang is currently a software engineer
[18] Y. Huang and H. Garcia-Molina, Publish/subscribe tree construction at Google Inc., Mountain View, CA. He received a
in wireless ad-hoc networks, in: Proceedings of the 4th International B.S. and a M.S. in computer science from Stanford
Conference on Mobile Data Management (2003) pp. 122–140. University, Stanford, CA, concurrently in 1996. He
[19] B. Kantor and P. Lapsley, Network News Transfer Protocol: A pro- also received his Ph.D. in computer science from
posed standard for the stream-based transmission of news, Request for Stanford University in 2003. His research interests
Comments 977 (1986). include scalable information dissemination, reliable
[20] S.-J. Lee, Routing and multicasting strategies in wireless mobile ad messaging systems, and mobile networking.
hoc networks, Ph.D. Thesis, University of California at Los Angeles E-mail: [email protected]
(2000).
[21] Object Management Group, CORBA services – event service specifi-
cation, Technical Report, Object Management Group (1997), ftp:// Hector Garcia-Molina is the Leonard Bosack and
ftp.omg.org/pub/docs/formal/97-12-11.pdf Sandra Lerner Professor in the Departments of Com-
[22] B. Oki, M. Pfluegl, A. Siegel and D. Skeen, The Information Bus – puter Science and Electrical Engineering at Stan-
an architecture for extensible distributed systems, Operating Systems ford University, Stanford, CA. He is the chairman of
Review 27(5) (1993) 58–68. the Computer Science Department since January 1,
[23] C. Perkins, IP mobility support, Request for Comments 2002 (1996). 2001. From 1997 to 2001 he was a member the Pres-
[24] PreCache Inc., Precache messaging engine, http://www. ident’s Information Technology Advisory Commit-
precache.com/tech.html tee (PITAC). From August 1994 to December 1997
[25] S. Ramanathan and M. Streenstrup, A survey of routing techniques for he was the Director of the Computer Systems Labo-
mobile communication networks, ACM/Baltzer Mobile Networks and ratory at Stanford. From 1979 to 1991 he was on the
Applications 1(2) (1996) 89–104. faculty of the Computer Science Department at Princeton University, Prince-
[26] B. Segall and D. Arnold, Elvin has left the building: A publish/ ton, NJ. His research interests include distributed computing systems, digital
subscribe notification service with quenching, in: Proceedings of libraries and database systems. He received a B.S. in electrical engineering
the 1997 Australian UNIX Users Group Technical Conference (1997) from the Instituto Tecnologico de Monterrey, Mexico, in 1974. From Stan-
pp. 243–255. ford University, Stanford, CA, he received in 1975 a M.S. in electrical engi-
[27] Sun Microsystems, Inc., Jini(TM) technology core platform spec – neering and a Ph.D. in computer science in 1979. Garcia-Molina is a Fellow
distributed events, Technical Report, Sun Microsystems, Inc. (2000), of the Association for Computing Machinery; is a fellow of the American
http://www.sun.com/software/jini/specs/jini1. Academy of Arts and Sciences; received the 1999 ACM SIGMOD Innova-
1html/event-spec.html tions Award; is a member of the Computer Science and Telecommunications
[28] TIBCO Software Inc., TIBCO Rendezvous, http://www.tibco. Board (National Research Council); is on the Technical Advisory Board of
com/solutions/products/active_enterprise/rv/ eGuanxi, Enosys Markets, Kintera, Metreo Markets, Morhsoft, TimesTen,
[29] P. Triantafillou and A. Economides, Subscription summaries for scal- Verity; and is a member of the Oracle Board of Directors.
ability and efficiency in publish/subscribe systems, in: Proceedings of E-mail: [email protected]