NCSA Steps Back from Yahoo Debate
The NCSA distanced itself over the weekend from a study posted on its Web site that called into question Yahoo's claims of a larger search engine index than rival Google. The organization has now issued a disclaimer that appears before visitors can view the actual study.
The report came under fire due to its methodology, which paired two random words together and counted the number of result pages returned. Critics claimed that this method would not return any kind of real world document, and in the case of Google, was even returning other word lists or dictionary results.
Also, Professor Vernon Burton's name has now mysteriously disappeared from the list of authors, leaving two students, one of whom the NCSA says "previously" worked for Burton. No reasoning was provided for the removal of Burton's name.
"The study was done outside the scope of any NCSA core projects," the NCSA said in the disclaimer. "When first published online, staff at the NCSA noted several issues with the study, and some revisions have been made to the document to reflect several of these concerns."
"Please note again that this study is not an NCSA publication and was not conducted as part of any NCSA project or under the supervision of NCSA," the statement reiterated.
Jean Veronis, a linguist in France and director of the Centre Informatique pour les Lettres et Sciences Humaines, led the argument against the study in his blog "Technologies du Langage" (which translated means Language Technology or Technologies of Language).
"Yahoo's indexing of long documents is nowhere near as deep as Google's," Veronis arguied. "As a result, even if Yahoo is not lying about the size of its index in terms of the number of documents, this could partly explain the smaller number of documents returned for certain search requests."
Furthermore, Veronis said the probability of "real documents" containing two of the randomly selected words were "virtually zero."
Neither Google nor Yahoo have publicly commented on the search engine size debate.