Distributed Lookup in Structured Peer-to-Peer Ad-Hoc Networks
Distributed Lookup in Structured Peer-to-Peer Ad-Hoc Networks
Ad-Hoc Networks
1 Introduction
Peer-to-peer (P2P) systems, in which peer nodes form a cooperative network
and share their resources (storage, CPU, bandwidth), have attracted a lot of
interest lately. Roughly speaking, P2P networks can be classified as either struc-
tured or unstructured. Unstructured P2P networks (e.g., the Gnutella [1] and
KaZaA [2] file sharing systems) have no precise control over the file placement
and generally use “flooding” search protocols, which typically generate a large
number of query messages. In contrast, structured P2P networks (e.g., Chord [3],
CAN [4], Pastry [5], P-Grid [6]) use specialized placement algorithms to assign
responsibility for each file to specific peers, as well as “directed search” protocols
to efficiently locate files.
Structured P2P networks using deterministic algorithms are based on the
distributed hash table (DHT) paradigm. Each data item is identified by a key
(data identifier). The DHT maps keys to the nodes of an overlay network and
provides facilities for locating the current peer node responsible for a given key.
DHT designs di↵er mostly by the way they maintain the network and perform
lookups: there is a fundamental trade-o↵ between the degree of the network
(the number of neighbors per node, i.e., the size of the routing tables) and its
diameter (the average number of hops required per lookup) [7].
Directed search protocols are particularly efficient, because they accurately
route queries toward the peers responsible for a given file. They require few
communication stages generating only little traffic, and do not produce false
negatives (i.e., the search fails only if there is no matching file in the system).
Therefore, they are particularly attractive for systems with limited resources,
where the cost of sending a message is non-negligible (e.g., in terms of energy)
and flooding is considered prohibitive.
Ad-hoc networks could greatly benefit from the efficiency of a directed search
protocol to locate content. They do, however, su↵er from limitations that make
the maintenance of a DHT impractical. Most notably, a node in an ad-hoc net-
work can communicate directly only with its physical neighbors, i.e., with the
nodes that are within its (radio) communication range, and communication be-
tween remote nodes must traverse multiple intermediate nodes. DHTs assume
that the peers form a fully connected network and that the neighborhood of
a node can be chosen according to its location in a “logical” overlay, rather
than to “physical” considerations. A simple mapping of a DHT design to an ad-
hoc network would be unrealistic, as every node would need to keep track and
communicate with a set of “logical” neighbors many steps away in the physical
network. There is a fundamental gap between the logical proximity defined by
the DHT overlay and the physical proximity imposed by the underlying ad-hoc
infrastructure.
In this paper, we study the feasibility of DHT lookup in ad-hoc networks.
We propose a DHT design that combines a minimalist logical overlay structure
together with adaptive routing mechanisms to quickly locate content. Nodes are
organized in a logical ring, similarly to Chord [3] or Pastry [5], but we do not
use logical long-range neighbors whose maintenance costs would be prohibitive.
Instead, we rely on the physical neighbors of the nodes traversed by requests
routed through the physical network to quickly converge toward the destination
in the logical overlay. We propose additional extensions, in which we consider
an extra level of visibility in the physical neighborhood (neighbors of neighbors)
and maintain a history of previous requests to dynamically identify and exploit
possible shortcuts.
We have extensively studied our algorithm by the means of simulations in
static deployment scenarios. We did not take into account mobility nor churn as
we are mostly interested in evaluating the feasibility of DHT lookup in ad-hoc
networks. Results demonstrate that our approach is indeed efficient and scales
well to large peer populations. It represents a promising alternative to existing
solutions based on unstructured P2P networks.
The remainder of this paper is organized as follows. Section 2 discusses related
work. We detail our algorithm in Section 3 and present results from experimental
evaluation in Section 4. Finally, Section 5 concludes.
2 Related Work
Peer-to-peer overlays have emerged from file sharing applications on top of the
Internet, leveraging from its routing infrastructure and the intrinsic peer-to-peer
property of the IP protocol. As already discussed in Section 1, the situation is
di↵erent in the case an ad-hoc networks. In particular, the DHT paradigm with
its notion of regular topology (often a ring) and the shortcuts (fingers) introduced
at the overlay layer, makes a direct mapping to ad-hoc networks difficult. Various
approaches are known from literature to achieve such mappings.
While GRACE (Global Replication And Consistency) [8] has not been specifi-
cally designed with ad-hoc networks in mind, it enables for mobile collaboration
by combining DHT properties with a layered architecture. GRACE supports
mobility in wide-area networks. The di↵erent layers or consistency levels are
interconnected through “consistency neighbors” that are logically close to each
other. Requests are routed along these neighbors. The lookup algorithm of the
system is based on Pastry [5]. This approach still relies on the standard Internet
infrastructure.
Pucha et al. [9] implement Pastry on top of the Manet routing protocol DSR
(Dynamic Source Routing)[10]. Three modifications are proposed as compared
to the implementation on the Internet: (1) the node join procedure is modi-
fied by expanding the ring search for locating distinguished bootstrap nodes in
charge of arrivals; (2) the Pastry ping metric is replaced by a distance metric to
reduce network load; and (3) the DSR protocol is modified to inquire about the
proximity used in the adapted Pastry routing.
Ekta [11] and MADPastry [12, 13] integrate the DHT paradigm with ad-
hoc network routing. Both approaches introduce the necessary functions at the
network routing layer. The principal idea of Ekta is to move the DHT protocol
from the overlay level to the network layer of the Manet : a one-to-one mapping
between IP-addresses and logical (DHT) node IDs is applied. MAPastry is built
on top of the AODV protocol (Ad-hoc on-demand vector routing) [14]. This
protocol aims to avoid full broadcasts as much as possible because these are
costly in ad-hoc networks when targeted to the entire network. MADPastry
creates clusters composed of physically close nodes sharing, at the same time, a
common overlay prefix. The nodes in a cluster are thus physically and logically
close. Routing is then based on the logical overlay node IDs.
All the approaches above su↵er from the size of the routing tables and the
complexity of setting up and managing connections with all the nodes that they
contain. In contrast, in our design, we only need to keep track of two remote
nodes, as explained in the next section.
Cramer et al. [15] propose the Chord based Proximity Neighbor Selection
strategy (PNS-CHORD). Here, the nodes are connected to their logical succes-
sors on the ring and through logical shortcuts to further nodes as usual in Chord.
These logical long-range neighbors are chosen according to physical proximity
in the ad-hoc network and are either one or two steps away.
Because the construction of the routing tables is based on physical proximity,
it may happen that the logical path pursued by a request traverses the same node
multiple times. This may only be prevented when the nodes keep track of the
requests that pass by and subsequently adjust their routing tables in case the
same request passes twice, but on di↵erent logical shortcuts. Our approach takes
into account of this situation and prevents thus such loops through the according
learning process.
Cell Hash Routing (CHR) [16] is a specialized ad-hoc DHT. CHR uses posi-
tion information to construct a DHT of clusters instead of organizing individual
nodes in the overlay. This approach groups nodes according to their physical lo-
cation. The routing between the clusters is done by position-based routing with
the GPSR [17] routing algorithm. A major limitation of this approach is that
nodes are not individually addressable, but only via the clusters.
Finally, we are aware of a parallel e↵ort to our’s by Rowstron et al. at Mi-
crosoft Research (Cambridge) that similarly aims to combine ad-hoc routing
and logical DHT-like addressing (Virtual Ring Routing, VRR). Based on the
(yet unpublished) information that we have obtained directly from the authors,
their approach di↵ers from ours in several ways. First, they build and proac-
tively maintain bidirectional routes between nodes. In contrast, our algorithm
always tries to find the best route at each node and sends the request along this
path. It is therefore possible to take into account locally profitable situations.
Second, VRR also maintains existing routes in a proactive way. Our solution
does not maintain existing routes, but previous routing decisions are kept in a
cache and can thus be reactivated if appropriate. This results in more efficient
routes to emerge at each step along a path and allows to faster reacting on
topological changes. Finally, VRR nodes use information of the physical paths
traversing the nodes for improving the routing. Our experiments, however, have
shown that such a caching policy results in small improvement, only. Indeed, the
cached entries are e↵ective only when a request’s path goes through the node
having registered them. In our approach, the nodes also capture communications
of the nodes in the physical neighborhood. The information acquired this way
implicitly includes routing decisions and thus allows avoiding paths to traverse
the same physical area twice. It has to be noted that this information is obtained
without any further communication overhead, since it may be captured for free
due to the characteristics of wireless networks, i.e. radio transmission.
1
In this paper, we denote by hop a logical link on the DHT overlay, and by step a
physical link of the ad-hoc network.
distributed in the logical space (see Figure 1). This diversity property is impor-
tant for the efficiency of our lookup algorithm.
1
12 2
12 11 6 7
9 5 3
11 3 1 2
4
10 8
4 Physical network
10
9 5
Logical links
8 6
Physical links
7
1: procedure Lookup(k, nd )
k : looked up key
nd : next logical hop
2: nj = arg minn2Vi [Li [{ni ,nd } (id(n), k)
3: if nj = ni then { We are responsible for k }
4: return ni
5: else if nj = nd then { Continue to nd }
6: nk next step on physical path to nd ;
7: send Lookup(k, nd ) to nk ;
8: else if nj 2 Li then { Go to logical neighbor }
9: nk next step on physical path to nj ;
10: send Lookup(k, nj ) to nk ;
11: else { Go to physical neighbor }
12: send Lookup(k, nj ) to nj ;
13: end if
14: end procedure
– If ni is closest to the key, then it is responsible for the key and the lookup
ends.
– If a physical neighbor is closest to the key, then it directly receives the
request.
– If a logical neighbor is closest to the key, then it becomes the new target nd
of the request.
– If nd is closest to the key, then the algorithm does not change the destination
of the request.
In the first three cases, the distance to the key decreases, either because we
reach the node responsible for the key, or because we send the request to a node
closer to the key. In the last case, we do not decrease the distance to the key
but we proceed toward nd and, if the request reaches nj , one of the first three
cases will apply. The assumptions on the underlying ad-hoc routing protocol
guarantees that a request sent to a node will eventually reach that node; thus,
the last case cannot apply forever. It follows that our algorithm converges and
terminates in a finite number of steps.
Ad-hoc networks are characterized by the fact that every node must participate
in request routing and forwarding. Therefore, a node sees traffic for which it is
not the target. We can thus exploit information about past requests to acquire
knowledge of remote nodes and improve lookup efficiency.
Each node maintains a cache that stores, for each observed request, the
searched key k and the destination of the message nd . Thereafter, the cache
entry (k, nd ) can act as a long-range neighbor: when a node receives a request
for a key closer to k than the node that would be selected by Algorithm 1, the
request is redirected toward nd instead.
This extension does not require any extra message to be sent. Further, as all
the physical neighbors of a node can listen to its messages (radio communications
are broadcast), information about past requests can be gathered passively. The
cache uses a least-recently used replacement policy and we have limited its size
to 256 entries in our implementation.
4 Evaluation
A. Simulation Setup.
B. Simulation Experiments.
One observes that the average values for networks of size 1,000 and 10,000
are roughly the same when applying the warm-up phase. This suggests that our
lookup algorithm provides a stable lookup time in average after some running
time (warm-up phase in the simulation), and up to some network size.
Figure 2 shows the average number of physical steps required to serve a
request. The use of the NoN and cache with warm-up strategies reduces the
number of physical steps by ⇠ 20% respectively ⇠ 50%. The introduced logical
shortcuts can thus be considered as very e↵ective.
The average costs of one logical hop in terms of physical steps are roughly
the same for all variants of the algorithm (see Table 2). The performance of
the lookup algorithm increases thus when the number of logical hops for a re-
quest decreases. Figure 3 demonstrates that our ad-hoc lookup algorithm indeed
reduces the number of logical hops to serve a request.
We observe that our algorithm always achieves the lookup in less than log N
hops or even less than 12 log N hops. This clearly demonstrates the efficiency
of the approach and confirms that the introduction of the cache further im-
proves performance. Moreover, the various warm-up phases applied show that
the ad-hoc lookup algorithm stabilizes over time: the longer it runs, the better it
1100
Basic
1000 NoN
C Wup-0
900
# physical steps per request
C Wup-2K
800 C Wup-50K
700
600
500
400
300
200
100
0
1000 10000 100000
# of nodes
log N
20 1/2 log N
Basic
NoN
C Wup-0
15 C Wup-2K
# of logical hops
C Wup-50K
10
0
1000 10000 100000
# of nodes
The ad-hoc lookup algorithm finds logical shortcuts through the selection of
physical nodes closest to the destination and with the help of the information
gained from previous requests kept in the cache. Intuitively this suggests that
shortcuts determined at the beginning of a logical path should approach the
destination the most. Indeed, the probability that some node in the physical
neighborhood is closer to the destination node than the next node on the log-
ical path decreases with the number of logical hops traversed. In other words,
the percentage of logical hops that terminate without a shortcut being taken
increases when approaching the destination. This characteristic is demonstrated
in Figure 4. The figure shows the percentage of terminated logical hops as a
function of the distance from the node responsible for the searched key.
40
20
0
1 2 3 4 5 6 7 8 9 10
Distance from destination node
Finally, Figure 5 shows the path of the same request in the same network of
10,000 nodes but with di↵erent versions of our algorithm. Coordinates represent
the geographical position of the nodes. Clearly, the best result is obtained when
using the NoN version of the algorithm with caching.
Logical path
Physical path
End of logical path
Start of logical path
Origin of the request
Destination of the request
Redirection on a cached path
Fig. 5. Path of a request with Basic (top left), NoN (top right), C Wup-2K (bottom
left), and C Wup-50K (bottom right) in a two-dimensional space.
5 Conclusion
Apart from very few known approaches, P2P systems for ad-hoc networks have
essentially used unstructured designs and flooding-based search protocols. It is
indeed challenging to build a structured overlay with a directed search protocol
on top of an ad-hoc network, because nodes have only limited communication
capabilities: they can exchange messages only with the nodes that are within
their (radio) communication range.
In this paper, we have studied the feasibility of building a DHT for ad-hoc
networks. The premise of such a design is to improve lookup performance and re-
duce the message overhead as compared to flooding-based protocols which yield
linear or worse performance. In our approach, we only maintain a minimalist log-
ical overlay structure but rely on the physical neighbors of the nodes traversed
during ad-hoc routing to quickly reach the object being looked up. Our simu-
lation results have demonstrated that a sub-linear lookup performance can be
preserved when relying on an ad-hoc network infrastructure rather than on the
standard Internet. It must however be noted that, to the best of our knowledge,
the observed sub-linear performance still has to be confirmed by an analytical
study.
References
1. : The Gnutella web site, www.gnutella.com (2006)
2. : The KaZaA web site, www.kazaa.com (2006)
3. Stoica, I., Morris, R., Karger, D., Kaashoek, M.F., Balakrishnan, H.: Chord: A
scalable peer-to-peer lookup service for internet applications. In: SIGCOMM ’01:
Proceedings of the 2001 conference on Applications, technologies, architectures,
and protocols for computer communications, New York, NY, USA, ACM Press
(2001) 149–160
4. Ratnasamy, S., Francis, P., Handley, M., Karp, R., Schenker, S.: A scalable content-
addressable network. In: SIGCOMM ’01: Proceedings of the 2001 conference on
Applications, technologies, architectures, and protocols for computer communica-
tions, New York, NY, USA, ACM Press (2001) 161–172
5. Rowstron, A.I.T., Druschel, P.: Pastry: Scalable, decentralized object location, and
routing for large-scale peer-to-peer systems. In: Middleware. (2001) 329–350
6. Aberer, K., Cudré-Mauroux, P., Datta, A., Despotovic, Z., Hauswirth, M.,
Punceva, M., Schmidt, R.: P-Grid: a self-organizing structured P2P system. SIG-
MOD Record 32(3) (2003) 29–33
7. Xu, J., Kumar, A., Yu, X.: On the fundamental tradeo↵s between routing table
size and network diameter in peer-to-peer networks. IEEE Journal on Selected
Areas in Communications 22(1) (2004) 151–163
8. Bosneag, A.M., Brockmeyer, M.: GRACE: Enabling collaborations in wide-area
distributed systems, 14th IEEE International Workshops on Enabling Technolo-
gies: Infrastructure for Collaborative Enterprise (WETICE’05) (2005) 72–77
9. Pucha, H., Das, S.M., Hu, Y.C.: How to implement DHTs in mobile ad hoc
networks?, the 10th ACM International Conference on Mobile Computing and
Network (MobiCom) (2004)
10. Johnson, D.B., Maltz, D.A.: Dynamic source routing in ad hoc wireless networks.
In: Mobile Computing. Volume 353., Kluwer Academic (1996)
11. Pucha, H., Das, S.M., Hu, Y.C.: EKTA: An efficient DHT substrate for distributed
applications in mobile ad hoc networks., 6th IEEE Workshop on Mobile Computing
Systems and Applications (WMCSA 2004) (2004) 163–173
12. Zahn, T., Schiller, J.H.: MADPastry: A DHT substrate for practicably sized
MANETs., 5th Workshop on Applications and Services in Wireless Networks
(ASWN2005) (2005)
13. Zahn, T., Schiller, J.H.: DHT-based unicast for mobile ad hoc networks., Fourth
Annual IEEE International Conference on Pervasive Computing and Communica-
tions Workshops(PerComW’06) (2006) 179–183
14. Perkins, C.E., Belding-Royer, E.M.: Ad-hoc on-demand distance vector routing.
In: WMCSA. (1999) 90–100
15. Cramer, C., Fuhrmann, T.: Proximity neighbor selection for a dht in wireless
multi-hop networks. In: Proceedings of the 5th IEEE International Conference on
Peer-to-Peer Computing, Konstanz, Germany (2005) 3–10
16. Araùjo, F., Rodrigues, L., Kaiser, J., Liu, C., Mitidieri, C.: CHR: a distributed
hash table for wireless ad hoc networks, 25th IEEE International Conference on
Distributed Computing Systems Workshops (2005) 407–413
17. Karp, B., Kung, H.T.: GPSR: greedy perimeter stateless routing for wireless net-
works. In: MobiCom’00: Proceedings of the 6th annual international conference
on Mobile computing and networking, New York, NY, USA, ACM Press (2000)
243–254
18. Ko, Y.B., Vaidya, N.H.: Location-aided routing (LAR) in mobile ad hoc networks.
Wirel. Netw. 6(4) (2000) 307–321