I don’t think it’ll do any good. I also ‘block’ Chinese crawlers from indexing my sites using robots.txt, but they completely ignore it.
Some services will continue to crawl, they just won’t tell the user where they got the source material from when requested. Services have already started doing that.
1 comment
I don’t think it’ll do any good. I also ‘block’ Chinese crawlers from indexing my sites using robots.txt, but they completely ignore it.
Some services will continue to crawl, they just won’t tell the user where they got the source material from when requested. Services have already started doing that.