Convertidor de HTML a RSS para las noticias de Globovisión.

Siguiendo con la fiebre de los Mashups hechos con Yahoo Pipes, llegué al punto en que decidí escribir un convertidor de HTML a RSS para Globovision. Como recordaran, Globovision no ofrece un feed RSS de sus noticias nacionales, lo cual es una verdadera lastima.

Asi que con un poco de imaginación decidí escribir este programa en Perl:

 #!/usr/bin/perl

 use strict;
 use LWP::UserAgent;
 use HTML::Parser;
 use XML::RSS;

 my $rss = XML::RSS->new( version => '0.9' );

 my $rssFile = "$ENV{HOME}/globovision.rss";

 $rss->channel(
  title        => "Globovision.com Venezuelan News",
  link         => "http://globovision.com/",
  description  => "Globovision.com news -
Brough to you by http://KodeGeek.com");

 # Be carefull with this one as nested elements can be ignored and
 # HTML normally is not well formed!
 my @ignore_tags = (
  "head",
  "h1",
  "strong",
  "form"
    );

 my $baseUrl = "http://globovision.com/";

 # We are only interested on the news from the first page,
as more news come up it will push older news
 my $newsUrl = "$baseUrl" . "history.php?cha=1&pag=1";

 use constant DEFAULT_TIMEOUT
  => 180;

 my $agent = LWP::UserAgent->new({
   agent => 'GlobovisionHtml2Rss.pl/kodegeek 0.1',
   timeout => DEFAULT_TIMEOUT
      });
 my $response = $agent->get($newsUrl);
 if (! $response->is_success) {
  die  sprintf "[ERROR]: Unable to retrieve the HTML
from '%s', Status: '%s'", $newsUrl, $response->status_line;
 }
 my $parser = HTML::Parser->new(
                         api_version => 3,
    start_h => [ \&start_a, "tagname, attr" ],
    text_h => [ \&get_headline, "dtext" ]
    );
 $parser->ignore_tags(@ignore_tags);
 my $headlineFlag=0;
 my $currUrl=undef;
 $parser->parse($response->decoded_content());

 $rss->save($rssFile);

 # ****** Functions used on the script *******

 # Get the headline
 sub get_headline {
  my $headline = $_[0];
  if ($headlineFlag) {
   $rss->add_item( title => $headline, link => $currUrl);
   $headlineFlag = 0;
   $currUrl=undef;
  }
 }

 # Identify news items
 sub start_a {
  my $tagname = $_[0];
  my %attr = %{$_[1]};
  if ( ($tagname eq "a") && ($attr{href} =~ /^news.php?.nid=\d+/) ) {
   my $url = $baseUrl . "/" . $attr{href};
   $url =~ s/&/&/g;
   $currUrl = $url;
   $headlineFlag=1;
  }
 }
 __END__
 =head1 NAME

 GlobovisionHtml2Rss.pl - Script to convert Globovision.com Venezuela
local news from HTML to RSS.

 =head1 DESCRIPTION

 I use a combination of Yahoo Pipes and Google Reader to keep me
updated about news of any kind. However, some websites like
 Globovision.com still don't have a proper RSS feed, so one day
I decided to create my own mashup :).

 =head1 AUTHOR

 Jose Vicente Nunez Zuleta (josevnz@kodegeek.com)

 =head1 BLOG

 KodeGeek - http://kodegeek.com

 =head1 LICENSE

 GPL

 cut

Lo más fastidioso de este ejercicio fué instalar EXPAT (para el procesamiento del XML del feed RSS) y el módulo parta crear el archivo RSS (me da un fastidio enorme aprender como es el formato resultante).

Mi proveedor de hosting gustosamente instaló el módulo XML::RSS. Después de probarlo un poco aquí les dejo las noticias de Globovisión para que la disfruten (planeo actualizar el lector de noticias cada 10 minutos para no matar a mi servidor).



Blogalaxia.com:globovision, rss, html to rss, perl, open source
Technorati.com:globovision, rss, html to rss, perl, open source