Tag: robots

I had put together a list of search engine and other automated crawlers and bots together with their IP addresses which I was to not include in visitor usage statistics for a website, so only human visitors were tracked. The list was very useful both for showing only human visitors, but also to minimise the…