Wall Street Journal - Google article by Lee Gomes
I've noticed a lot of visitors to my blog searching for this article, so here it is...


Google Is Most Popular Search Site,
But Others Sometimes Do It Better

Is Google still the best search engine on the Web? It's a question you might want to ask God.

Do so by typing "God" into the Google (www.google.com) search bar. The results you get back, at least as of late last week, aren't exactly the high-quality sort on which Google built its reputation.

The top two listings had nothing to do with religion, but were an MP3 music site and a software operation called PHP-Nuke. Francisco Burzi, who runs the latter, says he and some pals tricked Google by having their personal Web sites all link to PHP-Nuke while making references to "God."

It sounds like the bad old days of Internet searching, when porn sites routinely turned up near the top in nonporn searches, a blight Google has been praised for ending.

Is Google slipping? More likely, it's that the chinks in Google's armor are more readily apparent, simply because so many people are out there banging on it.

Still, there are other ways to search the Internet than through Google, something Google's many fans might not appreciate. Some search industry gurus even preach heresy: that Google isn't the field's technology leader anymore.

First, though, a brief history of search.

In the Web's early days, if you wanted to know about "mortgages," the first generation of search engines would show you pages with references to the term. But, as porn sites quickly discovered, this approach is easily fooled, say by putting "mortgage" somewhere -- or dozens of times -- on your page.

Then, Jon Kleinberg, a Cornell University computer scientist, realized a better approach would be to forget about the contents of a page and concentrate instead on the people linking to it. It's known as "link analysis," as opposed to the earlier "text analysis." Prof. Kleinberg ran an IBM research project that tried to write software that would find the Web "communities" around a particular topic, like mortgages. You'd then go to that community, and see what sites it thought were best. A good idea, but the IBM crew couldn't figure out how to do it fast enough.

Enter Google, which in the late 1990s came up with its own variation of link analysis. Google's soon-to-be-famous "Page Ranking" system listed Web sites by their popularity, on the assumption that the best sites were those with the most people linking to them. It was slightly different from what IBM was trying because with Google, everyone, in effect, had a shot at voting at the best page, rather than a presumed "community of experts." It worked, and Google quickly became the No. 1 search engine. It holds that position today for many reasons besides its technology, like its clean design.

Lately, the Google folks have been downplaying the page-ranking system in describing their advantages -- if only because everyone else is now doing it. In fact, all search engines nowadays take many things into account when deciding how to list the pages in response to a query.

Though it is used more than any other site, Google actually has many competitors. One of the most technically interesting is Teoma. Founded by Apostolos Gerasoulis, a Rutgers University computer scientist, Teoma has figured out how to quickly find the communities that the IBM crew was looking for. Teoma was bought by AskJeeves in 2001, and now powers AskJeeves searches while also maintaining a separate site.

The advantages of this approach might not be immediately apparent to the casual visitor to the Teoma.com site. Pay attention, though, to the "refinements" you see on the right side of the screen after a search. Teoma's software has, in effect, found the "community" associated with your search, and is listing what related topics that community is "discussing." For "power blackout," the refinements Friday included "electrical surge" and "cost of downtime."

It's a great way to learn about a topic, or to find the precise thing for which you are looking, without having to actually go to a lot of links. AltaVista has a similarly useful, if slightly less complete, offering. But Google doesn't. That's one reason that Prof. Kleinberg says Teoma's technology has lately eclipsed Google's.

How's that? Something better than the vaunted Google? Draw your own conclusions.

What's indisputable is that the world of search engines is scene to more innovation and competition than many folks realize. There's even stock to fund this; the recent, reality-challenged run-up in tech stocks has increased AskJeeves's stock price almost 20-fold.

While Google is expected to go public soon, the economic rules of the search world are still being written. Will Google become the "Microsoft of search?" Or are the dynamics of the marketplace fundamentally different, in a way that will allow for two or three (or more) profitable players? How "locked in" do people get to a particular engine? How much better than Search Engine X does Search Engine Y need to be to get people to start using it?

The answers are all coming soon to a search engine near you.
WSJ.com - Portals
Andy Beal | permalink | +marktd |