Spam my blog and I report your site to Google for banning. You've been warned!
« Weird bot | Main | Some insights into spambot behavior »
January 10, 2005
First search engine to allow blocking of sections on pages
In the past we've had robots exclusion tags and robots.txt to tell search engines what to index and what to ignore. But with comment spam and trackback spam, we really need tags that allow us to tell spiders that sections of pages should be ignored.
Fybersearch (never heard of them before, who are they?) are the first to announce such an initiative.
Here's the explanation, and here's the press release
The problem with this, is that it's Fybersearch specific. What we need is a standard tag that more and more search engine spiders will choose to adhere to. Fybersearch is not important enough to warrant a tag specifically for them, in my opinion. But I WOULD use a tag Google, Yahoo and MSN supported!
The blog community has been screaming for such a tag for a while now, because of the problem with spam. PLEASE, somebody listen!
More on Fybersearch:
I checked one of my core search terms for another site, and Fybersearch returned my site, but nothing else. I guess that means they're just starting out, and are only spidering the top sites for now (yes, it's the top site in that search term).
They're also using Google Adwords.
Posted by Ann at January 10, 2005 01:11 PM
Trackback Pings
TrackBack URL for this entry:
http://www.annelisabeth.com/blog/mt-tb.cgi/55
Comments
Posted by: Anonymous at January 10, 2005 01:11 PM