The blogosphere is all a flutter about Google News' new comment-on-news feature (which I actually think is pretty nice). However, Robert Scoble, who posts on reasons for not allowing others to crawl the Google News site, seems to have totally missed the issue here in a way that makes me really doubt the fine figure of a blogger that he is perceived to be.
Robert's case for robots.txt is to prevent others from bootstrapping a news site out of your (that is to say, Google's) hard earned analysis and presentation. But this issue is about the new content that Google is now hosting. Comment on news is content - the news it self is aggregation of content from other sites, which is why Google's position is so, shall we say, ironic.
"Robert's case for robots.txt is to prevent others from bootstrapping a news site out of your (that is to say, Google's) hard earned analysis and presentation."
This assumes that the only reason to index Google News would be to build a competitor to Google News.
The customers using Spinn3r are building things I would have NEVER thought of.
Posted by: Kevin Burton | August 09, 2007 at 10:54 PM
I think it's a little weird whenever Google is greedy with its data, but only because they built (and continue to build) their business off of OUR data. I'm working on a "buzz aggregator" for the Semantic Web. Basically it will be aggregating blog posts and bookmarks related to the Semantic Web and will also allow users to comment on articles. Rather than keep the comments silo'd on my blog I'm going to open up user comments with SIOC. This will allow the conversation to be combined on the with the comments on the original author's blog.
Posted by: James | August 24, 2007 at 01:28 PM