Potential and fix for Drupal xmlsitemap module to not index nodes

Date: Tue Apr 26 2011 Drupal »»»» Drupal Planet
I want to share the solution for a problem I (and it seems others) have had with the Drupal xmlsitemap module. The problem results in zero nodes being indexed in the sitemap which, of course, is an undesired result because the purpose of xmlsitemap is to reliably inform search engines of the nodes on your site so they can more easily index your site. Telling the search engine "I have no nodes" is a kiss of death in terms of getting traffic, that is until the search engine spiders your site and finds everything that way. But they don't spider sites every day.

I have xmlsitemap 6.x-2.0-beta2 installed on several sites. On all but one site the generated sitemaps contain lots of entries in proper xmlsitemap format, but on one site it contained only one entry, the site homepage. Hence it wasn't that the module was buggy, but something was off in my site.

I had tried uninstalling and re-enabling the module several times, the last time doing it carefully and manually checking that tables and variables were removed. But upon reinstalling the module the sitemap would still list only the one entry. Even after using the drush xmlsitemap-regenerate and xmlsitemap-index commands.

I filed my own issue: http://drupal.org/node/1139186 and then re-read closely another issue from last year which described the same behavior http://drupal.org/node/803756 ... way down the discussion thread on the second issue I found a question about whether content types were marked as "Included" or "Excluded" in the xmlsitemap settings. Turns out on my site that all content types were Excluded.

I don't know how they became Excluded, but upon marking some of them Included and then running drush xmlsitemap-index and xmlsitemap-regenerate the sitemap started to have entries.

Let's go over this carefully so you know what to look at:

In the xmlsitemaps admin area click on the Settings tab. You'll end up at admin/settings/xmlsitemap/settings .. Towards the bottom you'll see some collapsable fieldsets depending on which of the xmlsitemap submodules you've enabled. Uncollapse the fieldsets.

The columns are marked "Inclusion", "Priority", "Available", "Indexed", and "Visible"

The Inclusion column will read either Included or Excluded. The Available and Indexed columns will tell you a summary of how many nodes of which content type are included or not.

For each Excluded content type, click on the link and you'll go to the edit page for the content type. A collapsed fieldset named "XML Sitemap" will show up. Uncollapse it, and see the combo box. Change it to Included. Click Save. You'll return to the XMLSitemap settings page, so repeat this for each Excluded content type.

The same Included/Excluded setting exists for taxonomy terms and other things XML Sitemap will index.

I then checked on one of my other sites and found several content types, taxonomy terms, and menu's which were Excluded from being indexed. It appears the Excluded ones were all newer than the ones which were Included. Perhaps the default had formerly been to Include everything, and the default became to Exclude everything. I can't imagine why that would be since to my mind the default should be to include everything.

I hope this is helpful for others.