Originally posted by piersk
See, I don't get the whole nofollow thing. I know that you can use a file called robot.txt and this will tell robots and crawlers etc where they can or can't go. However, whats to stop spam crawlers from carrying on burrowing deeper into a website??
Economics. Say a spamming organization derives a majority of its revenue from placing links to whatever crappy products as comments in popular blogs (then people click the links, etc... same as email spam as far as sales per million)...if the popular blog sites start using the nofollow attribute, search engines that respect it--google, whoever else--will no longer index the "followed" site linked from the blog. In other words, if a comment spammer has been using comments on a popular blog to beef up their Google pagerank , adding nofollow will in effect render all their comment spams worthless--as far as search engines go. People could still click the links and buy their garbage, but their revenue that came from search engine rankings will be reduced/destroyed.
For those still wondering the point, a better explanation than mine