| NetHosted Community https://www.nethosted.co.uk/community/ |
|
| crawler blocking at server level. https://www.nethosted.co.uk/community/viewtopic.php?f=6&t=4291 |
Page 1 of 1 |
| Author: | themuttsknutts [ Fri Aug 14, 2015 1:12 pm ] |
| Post subject: | crawler blocking at server level. |
Just a quick suggestion. will it ever be possible for you to block certain crawlers and ip at your root like the SEMALT idiots who seem to have crawl from about 16 IP's and any number of others that chew up server resources and time? |
|
| Author: | NetHosted - Bret [ Fri Aug 14, 2015 1:51 pm ] |
| Post subject: | Re: crawler blocking at server level. |
Hi, If we're able to confirm that a crawler is 100% malicious and serves no legitimate purpose then it's something we can take a look at if you'd like to submit a ticket with any form of logs/problematic activity but unfortunately a lot of these crawlers do serve a purpose in one way or another which means we can't simply blanket ban them in most cases. Apologies! Kind Regards, Bret |
|
| Page 1 of 1 | All times are UTC + 1 hour |
| Powered by phpBB® Forum Software © phpBB Group https://www.phpbb.com/ |
|