Gnutella Crawler Upgraded
January 17, 2003
Thomas Mennecke

Cheers to
UniteTheCows.com
From looking at LimeWire's host count, one would think that Gnutella was on the brink of collapse. During the last year, there has been a steady decline in population. For a brief time after Morpheus transitioned to the Gnutella network, LimeWire reported a population of just over 500,000 users; impressive for any community. However, this glory was short lived as users slowly began to migrate to other networks.
Those anticipating the death of Gnutella may have to wait just a bit longer. The host crawler previously used only counted clients using the 0.4 handshake, an obsolete communications protocol used by older Gnutella clients. This has lead to LimeWire's host count to report an inaccurate number of hosts. With the introduction of newer programs, fewer people have been using 0.4 compatible clients. In order to display more accurate information, the crawler has been upgraded to include clients that support 0.6 handshaking. Limewire developer Greg Bildson explains in the Gnutella Developers Forum:
If you follow the LimeWire crawl numbers, you should be noticing a spike in the data. The new numbers are more accurate than the recent trend.
LimeWire Network Graph
The major change is that we finally added Gnutella 0.6 handshaking support. Before this, the crawler was making 0.4 connections only and clients supporting 0.4 have been dwindling on the network. Using 0.6 headers also improves our chances of establishing a connection to ultrapeers. Plus, we can make use of the X-TRY headers.
There are two remaining issues with the crawls. The first is that we still don't connect to the whole network and you don't know when the network ends until you have nowhere else to crawl. The second issue is that not all ultrapeers generate pongs for their leaves. There should be a fair number of hidden leaf clients that are invisible to the crawl. We could solve problem number one by throwing more resources at it but problem number two is a client developer issue. LimeWire ultrapeers will generate pongs for their leaves when they receive an indexing ping (hop=0,ttl=2). I would encourage other client developers to do the same.
This story is filed in these Slyck News categoriesP2P Clients :: LimeWireYou can discuss this article here