Page:Untangling the Web.pdf/23

 several years ago, I noted a marked decline in both the quantity and quality of the Yahoo directory. The other major directory was and remains the, which has always powered the Google Directory and, ironically, now powers the Yahoo Directory. What distinguished the Open Directory from Yahoo was that, while Yahoo was heavily commercial, the Open Directory has always relied upon volunteers to populate and maintain it. Now that most of users' creative energy seems to have moved to wikis, the ODP is in what may be a permanent and ultimately fatal decline. Today, the most successful directories tend to be specialty directories such as NewsDirectory.com or yourDictionary.com, and vertical search engines, such as Business.com or MedlinePlus, which focus on a particular topic instead of trying to catalog the entire Internet.

Directories were almost always a part of the portal concept. Portals were all the rage for a few years, while search was considered the Internet boondocks-no one was terribly interested in the boring (and unprofitable) technology of search. So where are portals now—those one-stop handy-dandy Swiss army knife web sites that tried to do and be all things to all people? Most of them are gone, thanks in large part to Google's ascendancy. With its clean, spare look, Google changed the face of Internet search by moving away from the portal concept to pure search. While it is true that Google offers a directory as well as other types of searches-Image, news, shopping, groups-Google's focus has always been on web search. Google's new look, which debuted in April 2004, included removing the directory tab from the Google home page, further evidence of the decreasing importance of directories. Although there is growing criticism of the "" of websites, Google continues to be the standard by which most sites are judged.

The rapid and dramatic decline in web directories is only partially attributable to Google's success. The other explanation for the waning of directories is the Tristram Shandy paradox. The Life and Opinions of Tristram Shandy, Gentleman is a nine-volume 18th century novel in which Tristram Shandy tries to record every detail of his life but discovers his task is hopeless because it takes him one year to document only one day. As Shandy writes an additional day, it takes him an additional year to complete the events of that day. Such is the fate, to a somewhat lesser degree, of those who seek to compile an Internet directory. By the time the information in the directory is researched, compiled, and published, the Internet has changed and made much of that information obsolete.

I believe Yahoo's decision to metamorphose from directory to search engine was in part a result of a tacit recognition of the Tristram Shandy paradox. Yahoo just couldn't keep up with the Internet's changes and it became too costly to try. Creating and maintaining a directory is an extremely manpower intensive endeavor, which flies in the face of the Internet model of relying on automation and technology. Undoubtedly, Yahoo's changes were largely driven by Google's enormous financial success. Yahoo sat by for years and watched as Google's popularity (and revenues) increased as Yahoo's stagnated. "By the late '90s much of [Yahoo's] focus was actually diametrically opposed to search, which is supposed to Rh