There are tens of thousands of podcasts available on zillions of topics, and there are podcast directories out there to help people find podcasts of interest.
The basic form of a podcast is an RSS feed which uses <enclosure> tags to link to the audio or video content.
It turns out that out-of-the-box Drupal does this. If you allow files to be uploaded, the attachment appears as an enclosure tag. However there are a few small details which are useful which aren't supported out-of-the-box. For example it is nice to let your readers view or listen to the content while browsing your site. Drupal doesn't support this out-of-the-box.
A podcast is an RSS feed, right? Generating feeds is one of Drupal's core competency. For example Drupal generates a unique RSS feed for every vocabulary item list. A simple way to implement a podcast is to use the vocabulary system to list the podcast nodes. For example if you use the vocabulary term 'podcast' on all your podcasts, then the RSS feed generated for the 'podcast' vocabulary term is the RSS feed for your podcast.
Here are a few useful modules:
RSS Remote Enclosure allows you to locate the media content "anywhere". Such as a "Content Distribution Network". Installation is simple, just unpack the module and enable it in the Modules administration page. A configuration page shows up in the administration section. On that page you select the content types for which RSS Remote Enclosure is enabled.
Once enabled for a content type, a few new entry boxes show up which allow one to specify the enclosure. The attributes are the URL, the size of the media file, and its Content-Type.
I'm interested in this because the site on which I want to do podcasts does not have the disk space to hold the media files. I can store the media files on my dreamhost account and refer to their URL's from the other site.
Embedded Media Field is meant to support linking to media on sites like YouTube. However it can be used to link media on any URL, such as when you use the RSS Remote Enclosure module. Embedded Media Field generates a viewer for the media, if it has a supported player for the media.
Views Podcast Says it has support for creating podcasts where the media is stored on an external server. I've not tried this yet.