List Of Crawlers And Bots, And Their I.P. Addresses

I had put together a list of search engine and other automated crawlers and bots together with their IP addresses which I was to not include in visitor usage statistics for a website, so only human visitors were tracked.

The list was very useful both for showing only human visitors, but also to minimise the database table entries as I noticed that often more crawlers and bots visited sites than human and database tables can become necessaries huge, unless you actually have a use for crawler and bot access.

I kept the list of crawlers and bots after the site was discontinued, as it may be useful and it is listed below.

If you use it and especially if you believe that it should be kept up to date and can add new crawlers and bots, then please get in touch.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,


Share this:

Subscribe for updates

Enter your email address below. I’ll send you updates & news to your email address. You can unsubscribe at any time.






Leave a Reply

Your email address will not be published. Required fields are marked *