I've been having a head-scratcher on a website for awhile and just found what the problem is. PROBLEM: When cron.php is requested, instead of running cron.php it redirects to some page on the site. This means the cron hooks aren't executed because instead some other page gets loaded. Drupal does a lot of maintenance stuff in the cron hooks, so it's bad news if they don't get executed.
Documentation of the normal setup for cron.php: http://drupal.org/cron .. I had done all that, and their troubleshooting section did not cover the problem I had.
A number of issues on drupal.org have this title: "drupal_goto breaks cron.php" This turns out to be the problem in my case. I had several old nodes with old content in them, had recreated that content in better form elsewhere on the site, and had added to the node body this code
<?php drupal_goto('/some/path'); ?>
The intent was to preserve any pagerank on the original page and be helpful to people browsing the site to redirect them to the current best content.
Unfortunately it is breaking cron.php (as noted).
The fix is simple.. remove the drupal_goto calls from the body of the nodes. Browsing the issues I also see several modules having the same broken behavior. An example patch to fix this misbehavior in a module exists here: http://drupal.org/files/issues/uc_webform_productize-453286.patch
The problem with that fix is those drupal_goto calls were made for a reason. I had several pages (140 to be exact) where some random innocent visitor from a search engine might land on the page. But in those cases I wanted to move the page elsewhere such as a different domain. In some cases the page was widely known and linked by external sites not under my control. This meant for visitors who would be displeased on landing on a "NO SUCH CONTENT" page especially if they clicked on a link somewhere else. My goal was to help these people by redirecting them to the new location for the page.
hmm... this is unresolved at the moment. I want for there to be a way to redirect visitors to the correct URL. It's clear putting drupal_goto into the node body is not a good idea. If anybody has a suggestion I'd love to hear (hint hint comment form below or else contact form linked at the top of the page).
PARTIAL RESOLUTION: I found that if the drupal_goto call is after the teaser (e.g. at the end of the node body) it doesn't break cron.php.