[RESOLVED] News Aggregators

[Dawg]

I was looking at websites like Newser and gawker and I see they pull many articles from other news sites along with articles written by staff.

Do they have code that looks for stuff or are they paying people to scour the other sites for worthy items.

Anyone have any experience in this field?

Thanks

[Dawg]

Thanks, That makes sense. I never thought of that.

I figured they had some modified spider code that hit certain sites and looked for certain criteria.

[dalecosp]

Just finished a project that does this. Brad's hit it right on the head ... at least for our site(s).

If you prefer Javascript, Google's got an API and scripts already available as well.

[Dawg]

Thanks, can you provide the pseudo code so I can get an idea of how it flows?

[dalecosp]

Well, probably. Here's the trickier parts:

// Google News uses Julian dates!  We'll go back six months.
$julian2day=gregoriantojd(date('n'),date('j'),date('Y'));
$sixmonthsback=date('n',strtotime("-6 months"));
$sixmonthsagoday=date('j',strtotime("-6 months"));
$sixmonthsagoyear=date('Y',strtotime("-6 months"));
$julianback6=gregoriantojd($sixmonthsback,$sixmonthsagoday,$sixmonthsagoyear);


$url="http://news.google.com/news?q=$feed_term&daterange:$julianback6-$julian2day&output=rss";

$data=file_get_contents($url);

$x = simplexml_load_string($data);

foreach($x->channel->item as $item) {

   if ($num_stories<$max_stories) {
      story($item);
      $num_stories++;
   }
}

The story function just prints the chunks of the item object the way we want it to.

HTH,