Page:Untangling the Web.pdf/29

  respects these mechanisms. Password protection, firewalls, and other measures will generally keep spiders from crawling a website and indexing it.

Robots Exclusion Page

Not every search engine has its own proprietary search program but instead relies upon another company's search service for its results. Most of these strategic alliances now involve Yahoo, Google, and Windows Live Search. All these partnerships are subject to change without notice; for more on these strategic alliances, see:

Search Engine Alliances

Knowing that Yahoo, for example, is the search tool behind a search engine can save you time because you can be pretty sure that using AltaVista Will get you similar (although not identical) results to the other search engines also powered by Yahoo. It is critical to remember that each service powered by a particular search engine produces different results even though they may all use the same core database. Why is this? Because the search interfaces have their own algorithms that decide how queries are run, how results are returned, or even if they query the entire database (most do not). In short, go to the primary search engine-Google, Yahoo, or Live Search for best results. Rh