Choose eligibility before you generate XML
An XML sitemap is a machine-readable list of URLs and optional metadata. Start with an eligibility rule, not a crawler export: include a page only when it is intended for search, publicly retrievable, and the preferred version of its content. Keep faceted catalog URLs, internal search results, campaign variants, login destinations, and previews out unless they are deliberate public search destinations.
Public, useful pages intended for search with one preferred URL.
Redirects, errors, duplicates, noindex pages, and private routes.
Discovery help does not guarantee crawling or indexing.
List the canonical URL, not every way to reach it
A canonical URL is the version you want search engines to treat as the representative page among similar versions. Sitemap inclusion is a weak canonical signal, so it should reinforce—not contradict—redirects, self-referential canonical tags, and internal links. Select one stable HTTPS format and emit that exact version.
Check http and https, www and non-www, slash rules, case, print pages, filters, and tracking parameters. If a URL redirects, list only its final canonical destination. Give each independently intended localized page its own canonical URL; do not use a sitemap to collapse distinct localized content.
Write the complete scheme and host, not a relative path. Google crawls URLs as listed.
Sitemap, canonical tag, internal links, and redirect policy should name the same page.
Blocking a duplicate does not establish a preferred version.
Make the XML small, valid, and truthful
Use the Sitemap Protocol structure: UTF-8 XML with a urlset, the sitemap namespace, and a loc for each URL. Escape XML-sensitive characters in URL values. Automation should serialize URLs rather than concatenate XML by hand.
Each sitemap may contain at most 50,000 URLs and be no larger than 50 MB uncompressed. Those are limits, not targets. Split earlier if products, articles, locales, or content types create a useful operational boundary.
Use lastmod only for the page’s real significant-change date. Google uses it when it is consistently and verifiably accurate; a copyright-year update does not qualify. Never stamp every URL with generator time. Google ignores changefreq and priority, so omit them.
loc is required; lastmod is optional and must describe the page, not generation.
Gzip can reduce transfer size, but the decompressed file must stay below 50 MB.
Check XML well-formedness, fetchability, and listed-page behavior.
Use indexes to scale without losing ownership
A sitemap index lists sitemap files. Use one when the site exceeds a single-file limit or needs separate inventories. Group by editorial, product, or help-center ownership. It has the same 50,000-item and 50 MB uncompressed limits.
Keep child files on the intended site at stable URLs. The protocol generally ties eligible URLs to the sitemap’s host and location; cross-site arrangements need verified ownership.
Split around a testable purpose—content type, storefront, language set, or deployment boundary. Record the generator, owner, and expected population.
Submit the complete index rather than each child manually.
Index lastmod describes the sitemap file, not its pages.
Publish, submit, and interpret the results correctly
Host the sitemap at a stable, publicly fetchable HTTPS address, preferably near the root. Add an absolute Sitemap directive to robots.txt. Submit the sitemap or index in Google Search Console and Bing Webmaster Tools for processing details.
In Search Console, Success means the sitemap was fetched and read without errors. Has errors means it was read with errors; Couldn’t fetch means it was unavailable. Fix delivery, authentication, DNS, robots, redirects, or XML and resubmit.
Treat discovered-page counts as diagnostic input, not a score. A discovered page is not guaranteed to be crawled or indexed. Sample URLs against their canonical URL, response, and indexing reports.
robots.txt aids discovery; tool submission exposes processing reports.
Search Console lists report- or API-submitted files, not every robots.txt discovery.
Operate the sitemap as release-critical site infrastructure
Sitemap quality can regress when a CMS plugin changes, a deployment changes the canonical host, cache rules serve stale output, or a bulk job publishes incomplete pages. Make sitemap checks part of pre-release and rollback work. Do not silently replace a full inventory with an empty or malformed file.
Keep the prior known-good sitemap, flag unexpected count changes, and sample additions and removals before publishing. When paths change, coordinate redirects, canonical tags, internal links, and sitemaps as one change set. This is proactive uptime optimization: protect the discoverable public surface before crawlers and visitors find a broken path.
Pause publication for invalid, empty, wrong-host, or noncanonical sitemap output.
Keep the previous artifact and generator configuration ready for rollback.