Wednesday, December 19, 2007

Deja-vu all over again...

74.86.249.98 - - [19/Dec/2007:10:17:22 +0100] "GET /path/to/file HTTP/1.1" 200 10049 "-" "Mozilla/5.0 (compatible; Gigamega.bot/1.0; +http://www.gigamega.net/bot.html)"

Looking at the URL listed in the UA:

What is gigamega.net?
gigamega.net is a research project started by a group of Russian candidates from the cities of Saint-Petersburg, Nizhnii Novgorod and Novosibirsk.

Sounds familiar? Evidently the Russian sibling project of Litefinder, itself "a research project started by a group of Indian candidates from the cities of Bangalore, Patna and Jaipur.". In contrast to Litefinder "GigaMega" has no search spam function; the top page redirects to the URL in the UA string, which seems to be the only page on the site.

They also seem to have the same "company rules" about revealing which IP addresses they scrape from:

Can I learn the IP addresses, which Gigamega-Bot comes from?
Unfortunately, You can't since it is against the rules of our company.

Posted at 9:31 AM |Comments (0)