Scanning the news

Date: Thu Nov 01 2007
You can get a lot of ideas for new content by watching the news. Whatever your topic area, other people are doing stuff around that topic. Whatever others are doing, will show up in some sort of news resources. Once you find a stream of news related to your topic, then it's a simple matter of writing little articles about items that strike your fancy.

Where, then, do you find the streams of news? And more importantly how do you sift through the news without drowning in information overload?

That this is the "information age" is apparent from the vast sea of information available to us. While the vast sea may seem overwhelming, it's quite possible to use some useful technology to create a virtual news clipping service that's highly tailored to your own needs.

The "news clipping" service is something which will scan newspapers and other news media for you, looking for articles of interest, and for a fee will clip out the articles and send them to you. This is a manual service that worked will in an analog age. Today we have digital communications and we have computers that can do the scanning for us.

Enter the RSS feed. While RSS had its initial popularity in blogging, it has been widely implemented on web sites of many kinds. In particular news organizations are extremely compatible with RSS. RSS excels at presenting the "latest content", which is exactly what The News is all about.


There are two sides to the technique which I am about to present.

First you find RSS feeds of interest. These feeds need to be focused precisely on your topic of interest. To be effective this technique needs to find exactly what you're looking for, with little extra fluff. As the geeks are fond of saying, you want a high signal to noise ratio, meaning that you want most of what you look at to be relavent to your interests.

Second, you aggregate those RSS feeds using the feed reader of your choice.

Blog search engines

There are a number of specialized search engines that focus on bloggers. You turn to the bloggers when you want the man-in-the-street point of view, rather than the predigested view you get from main stream journalists.

While you can subscribe to the RSS feed of particular bloggers, this isn't entirely effective. The blogger you choose may wander from topic to topic, and further by looking only to specific sources limits you to a limited set of opinions.

Instead what I do is go to the blog search engines, enter a search query, and then use the RSS feed provided by that search query. Today every search engine provides an RSS feed related to search queries.

For example, go to Feedster and enter a query into the search box prominently visible in the page. To the right-hand side of the page is an XML icon linking to the RSS feed for that search.

Some blog search engines are:

Social tagging

A recent gosh-wow kind of service has allowed the public to submit "tagged" items to shared repositories. and This is where the trend started. These services provide to the public a bookmarking where people add web links (bookmarks), and as they do they provide descriptive words. The words are the tags, and the tags help to organize the bookmarks. You can do searches on what was recently added under a given tag. Each search result has an RSS feed. The result is that if you subscribe to the RSS feed, you have an up-to-date view of what people are looking at.

Technorati looks for "tags" and category markers in blog postings. It can organize blog postings by those categories, and help you find blog postings which might not be found by a simple text search.

Mainstream news search

The mainstream news media sites all publish RSS feeds. While you can directly subscribe to those RSS feeds, you're going to run into the signal to noise ratio problem I described above. What's more effective is to use the news search engines.

Several services let you search the news. Rather than have the search be over the whole of the Internet, the search results come just from websites of recognized news media.

Here's a few:

All are used the same way. You enter search terms, and then on the search results page look for the link to the RSS feed for those results.

It should be noted that news search results are usually published under a terms of use prohibiting commercial gain from reuse of those results.

Aggregating the news

You don't read RSS feeds directly. Instead you use an RSS reader application. These are sometimes called "news aggregators" because they bring together content from multiple RSS feeds.

I use blogbridge. It is a very feature-filled application that has an interesting feature. It saves the list of feeds you're reading to a central server. You can then run the blogbridge application on any computer, tell it your account with that central server, and access the same set of feeds regardless of which computer you're on.

Bloglines does the same service, but as a web service from their web site. can be customized with RSS feeds. But the presentation of the results doesn't fit well with having lots of news feeds.

The Thunderbird email client includes RSS feed reading capabilities.

The planetplanet application allows you to build your own aggregation web site.

The drupal content management system has RSS aggregation features built in.