Penguin, small TECH.BARWICK.DE
Search Engines and Optimization
 

Recent posts

Categories

Archive

Syndication

 



Powered By

Info

2006-05-17 01:20:00

Neomo

This morning I found one of my sites had been subjected to a deep crawl by a bot naming itself "Francis/2.0 (francis@neomo.de http://www.neomo.de/)". The site seems to be an experimental but legitimate German-language search engine. The first hits from the bot were to robots.txt, the although the site's crawler information page doesn't indicate what entries it interprets, if any. Requests look like this:

85.10.204.13 - - [16/May/2006:19:19:09 +0200] "GET /robots.txt HTTP/1.1" 206 390 "-" "Francis/2.0 (francis@neomo.de http://www.neomo.de/)"
85.10.204.13 - - [16/May/2006:19:19:09 +0200] "GET /robots.txt HTTP/1.1" 206 390 "-" "Francis/2.0 (francis@neomo.de http://www.neomo.de/)"
85.10.204.13 - - [16/May/2006:19:19:24 +0200] "GET / HTTP/1.1" 206 1949 "-" "Francis/2.0 (francis@neomo.de http://www.neomo.de/)"
85.10.204.13 - - [16/May/2006:19:19:25 +0200] "GET / HTTP/1.1" 206 1949 "-" "Francis/2.0 (francis@neomo.de http://www.neomo.de/)"

Interestingly all requests returned with HTTP status 206.

Posted at 2006-05-17 01:20:00 |Comments (0)