Scraping Hacker News: SUCCESS

10 Apr 2008

After failing miserably to scrape the craigslist blog and convert it into a feed it feels good to have won a round. I like reading the YCombinator news page but I don’t much like that there’s no way to see a preview of the articles in my feed reader. Just a headline, nothing else.

So I fixed that: http://github.com/JackDanger/hacker_news

Download and run that little Ruby script and you’ll have a working webserver spitting out an RSS feed for YCombinator’s news page and (assuming you have links) installed on your system you’ll even get the full article displaying in the feed.

I’ve got it running on a server at ‘news’ dot ‘9suits’ dot com. There's no caching yet so go easy on it. Caching is working. Have at it: http://news.9suits.com


Please if you found this post helpful or have questions.