Fast cars and fast women are probably every young man’s dream. Fast loading and fast downloading are probably every web master’s dream. Driving in the fast lane with your foot pressed firmly to the floor reaching the maximum speed in seconds rather than minutes, picking the right highway to get to your destination means using the shortest route, just like intelligent web crawlers. You could take the scenic route and go sightseeing and discover new places, just like intelligent web crawlers. You could then tell your friends what you have seen and report back on interesting little diversions and tell of all the dead ends you have avoided, just like intelligent web crawlers.

Sitemaps are an easy way for webmasters to inform search engines about the pages on their sites, listing those that are available for crawling. Just like the navigation system in your car (providing it is up to date). A simplistic explanation of a Sitemap is that it is an XML file that is a list of URLs from a web site, written down in a method and a language that search engines can more easily understand, consequently this allows them to intelligently crawl the website, discovering new pages from links within the site and from other sites. This process is a bit like picking up a hitchhiker blindfolded, you never know who you might pick up, an old man, a young girl, a murderer, a celebrity crash victim or your mother, you do however learn about the new acquaintance after you have exchanged dialogue and learnt more about them. Web crawlers do this by gathering additional information (metadata) about each URL -when it was last updated, how often it usually changes, and how important it is relative to other URLs within the site etc.

Let us make you a site map