|
HIGHSPOT |
|
|
HIGHSPOT |
Photo Gallery
Cyveillance Abuse
Anti Spam Tools
Spam Statistics
Usage Statistics
Slashdot
ars Technica
Fedora
Apache
Cyveillance have a robot that trawls through web sites looking for stolen intellectual property. The robot ignores the robots.txt exclusion protocol, originates from IP addresses that don't reverse lookup to Cyveillance and tries to look like an ordinary user by spoofing its user agent.
The robots.txt (defacto) standard is used, amongst other purposes, to stop robots getting stuck in dynamic pages and to stop robots generating costs for people who pay for their web services by the amount of data they transfer. By ignoring robots.txt, Cyveillance are seeking to make a profit by exploiting resources that other people pay for, much like spammers do. Cyveillance could avoid abusing peoples servers by sending people to look at pages that robots are banned from. Of course, this would increase their costs, just like spammers costs would increase by using ethical mailing practices. Cyveillance, like spammers, choose to ignore peoples wishes in order to make their money.
If you run a web site, you may want to grep your logs for visits from:
You may also want to firewall those addresses if you find that they have been abusing your resources for their profit.
The following blocks belong to Cyveillance as well. Their rDNS and whios information only points to the ISP, but they exibited behaviour exactly like the Cyvellance bot right after my firewall logs showed blocked access attemps from one of the above blocks.
So, not content with not identifying their bots, Cyveillance now resort to using various IP blocks that have no way of being traced back to them other than the fact that they abuse a site straight after a known Cyveillance block gets denied by the firewall. They act more and more like criminals instead of the good guys they claim to be. The fact that there is no rwhois or SWIP for these blocks is also probably in violation of ICANN policy.
The following block may also be Cyveillance.
There are more ethical companies that perform the same service, such as NameProtect, who identify their bot and obey the robots.txt protocol. Their robot is perfectly welcome on my sites. Cyveillance are firewalled whenever I find them.
As of Tue, 09 Feb 2010 04:00:23 GMT
Don't send mail to this address. Use the Javascript encoded address (glwebsite061126 at this domain).
Copyright © 2002 - 2009 Graeme Leith
