As you can see above the bytespider useragent has sent thousands of requests before even visiting robots.txt, it started minutes after my server started up and has continued consistently for months.