Research my 403s, GigaMega
Deja-vu all over again...
18.104.22.168 - - [19/Dec/2007:10:17:22 +0100] "GET /path/to/file HTTP/1.1" 200 10049 "-" "Mozilla/5.0 (compatible; Gigamega.bot/1.0; +http://www.gigamega.net/bot.html)"
Looking at the URL listed in the UA:
What is gigamega.net?
gigamega.net is a research project started by a group of Russian candidates from the cities of Saint-Petersburg, Nizhnii Novgorod and Novosibirsk.
Sounds familiar? Evidently the Russian sibling project of Litefinder,
itself "a research project started by a group of Indian candidates from the cities of Bangalore, Patna and Jaipur.". In contrast to Litefinder "GigaMega" has no
search spam function; the
top page redirects to the URL in the UA string, which seems to be the only page on the site.
They also seem to have the same "company rules" about revealing which IP addresses they scrape from:
Can I learn the IP addresses, which Gigamega-Bot comes from?
Unfortunately, You can't since it is against the rules of our company.
Posted at 2007-12-19 10:31:00 |Comments (0)