fatinfo.com

When it comes to sitemaps and Yahoo, the search engine crawler just doesn’t want to do its work. For example if your website has over 100K urls, good luck at getting the website crawled by Yahoo thinking the sitemap.xml file is more than enough. MSN search engine is even worst at the lowest level of pages indexed by its crawls.

So far Google seems to be ahead of the pack indicated by the most pages crawled and indexed when you submit a sitemap.xml file. Then check back later with site:www.yourdomain.com to see how many pages are indexed correctly. The discrepancies are too large to ignore as one of the three can manage to deliver a good amount of natural search traffic while the Yahoo and MSN totally ignores these sitemap XML files and stop crawling after 100 URLs or so. As a webmaster, it is frustrating to note how Yahoo and MSN is always behind Google in making search indexing more accessible to everyone.

This is a shame even though Yahoo and MSN announced to the public that they too were adapters of the sitemap.xml standard to allow webmasters the flexibility to develop original content rather than worrying whether their websites were being indexed. This was understood as a blessing by webmasters every where, but Yahoo and MSN hasn’t done its job by playing its part of indexing XML sitemaps that webmasters have happily supplied to Google.

Come on Yahoo and MSN, stop playing catch up every time to Google. If you adapt the sitemap standard, then let your search bots go crazy on the sitemap.xml files instead of teasing webmasters throughout the world.