Sitemap sync, daily re-crawl and nested sitemaps
Training now syncs sitemaps automatically, recrawls stale pages on a schedule, and handles nested sitemaps, CDATA blocks, gzipped feeds, RSS variants, and subdomain quirks. Streams URL inserts in batches so very large sites stay responsive.