Search Engines Considered Harmful

In this talk, we discuss the widespread use of Web search engines and its potential impact on the ecology of the Web. Recent studies show that a significant portion of Web accesses are referred by search engines. Furthermore, the Web-search market is increasingly dominated by a small number of key players. What are the implications of this heavy reliance of Web users on search engines in their pursuit of information? For example, given that search engines return currently "popular" pages at the top of search results, are we somehow penalizing newly-created pages that are not very well known yet? Are popular pages getting even more popular while new pages are being completely ignored? We first show that this unfortunate trend indeed exists on the Web through an experimental study based on real Web data. We then analytically estimate how much longer it takes for a new page to attract a large number of Web users when search engines return only popular pages at the top of search results. Finally, we develop new ways of ranking Web pages that can significantly reduce the potential bias of existing ranking metrics. We believe that our proposed ranking metric has the potential to alleviate the "rich-get-richer" phenomenon and help new and high-quality pages get the attention that they deserve.