Summary
|
For Full-Text PDF, please login, if you are a member of IEICE, or go to Pay Per View on menu list, if you are a nonmember of IEICE. |
|
Load Balancing Scheme on the Basis of Huffman Coding for P2P Information Retrieval Hisashi KURASAWA Atsuhiro TAKASU Jun ADACHI Publication IEICE TRANSACTIONS on Information and Systems Vol.E92-D No.10 pp.2064-2072 Publication Date: 2009/10/01 Online ISSN: 1745-1361 DOI: 10.1587/transinf.E92.D.2064 Print ISSN: 0916-8532 Type of Manuscript: PAPER Category: Contents Technology and Web Information Systems Keyword: peer-to-peer, information retrieval, load balancing, Huffman-coding, Full Text: PDF(788.5KB)>>
Summary: Although a distributed index on a distributed hash table (DHT) enables efficient document query processing in Peer-to-Peer information retrieval (P2P IR), the index costs a lot to construct and it tends to be an unfair management because of the unbalanced term frequency distribution. We devised a new distributed index, named Huffman-DHT, for P2P IR. The new index uses an algorithm similar to Huffman coding with a modification to the DHT structure based on the term distribution. In a Huffman-DHT, a frequent term is assigned to a short ID and allocated a large space in the node ID space in DHT. Throuth ID management, the Huffman-DHT balances the index registration accesses among peers and reduces load concentrations. Huffman-DHT is the first approach to adapt concepts of coding theory and term frequency distribution to load balancing. We evaluated this approach in experiments using a document collection and assessed its load balancing capabilities in P2P IR. The experimental results indicated that it is most effective when the P2P system consists of about 30,000 nodes and contains many documents. Moreover, we proved that we can construct a Huffman-DHT easily by estimating the probability distribution of the term occurrence from a small number of sample documents. | ||||||||||
| ||||||||||
