A Versatile Crawler for Kademlia-Based Networks and Its Convergence Analysis

In recent years, Peer-to-peer (P2P) file sharing networks such as BitTorrent and eMule become more and more popular. BitTorrent and eMule deploy their distributed networks based on Kademlia, a robust Distributed hash table (DHT) protocol to facilitate the delivery of contents. The Kademlia-based networks and its measurement tools (i.e., P2P crawlers) have intrigued many researchers in the P2P community. However, to our best knowledge, few versatile P2P crawlers are developed for intensive measurement on Kademlia-based networks. In this paper we develop such a crawler, namely Rainbow. For the first time, we theoretically analyze the convergence of Rainbow, and obtain its convergence order which determines the time complexity of crawling. We then experimentally verify the convergence of Rainbow. Finally, we demonstrate that Rainbow can be applied as a versatile measurement tool to identify various detailed characteristics for Kademlia-based networks.