Why is web search such a hard problem?

Disclaimer: This is my personal blog. The views expressed on these pages are mine alone and not those of my employer.

Modern web search engines can be traced back to AltaVista, which was originally developed in 1995 by 2 DEC employees, Mike Burrows (google) (recently moved from Powerset to Microsoft) and Louis Monier.

Historically when you think about web search, you had to think about 4 dimensions:

  1. Ranking
    • Put the best answer in the first few positions
      • (As the indexes get bigger and data types keep growing, finding the best answer gets complicated)
  2. Comprehensiveness
    • The web keeps getting bigger and bigger everyday
      • (Can search engine’s find, access and index data fast enough?)
  3. Freshness
    • Just like the weather, web pages change at an alarming fast pace
      • (Can search engines update their own indexes fast enough?)
  4. Presentation
    • Make it attractive, quick to scan and support the query process
      • (Out of the gate, a search engine needs: spell check,  also try, titles and abstracts and highlighting)

Google has added a fifth dimension “SPEED”

  • When fast page load times are combined with fast page scans, users are able to spend more time focused on the problem and less time waiting on technology.

18 years ago Louis Monier while working at DEC designed today’s popular Search Engine Results Pages (SERP’s) and times are finally changing. Startups as well as the big 4 are trying to accomplish alot more with their SERP’s. The historical layout to the right appears to be breaking down.

‘Blended Results’ pages are coming

The presentation of blended information is forcing a change and potentially providing opportunity for those willing to take on the 800 lb gorillas. Below is one of gorillia’s current attempts at blended SERP.


The loss of the straight line down the left and the inclusion of a single photo dramatically changes the usability of this SERP. When you include a partial second column for ads and a sponsored link section at the top of the page, focus becomes an issue. You can see from the heat map below that we loose the golden triangle and eye movement seems to focus around the photo.

How are startups tackling this problem


SearchMe’s beta UI leverages Apple’s cover flow concept along with several other UI elements.


Kosmix’s alpha UI represents a larger snippet and tons of related links.


Quintura adds a tag cloud UI element to the left of the traditional search results


Viewzi has multiple UI’s depending on the type of SERP you think you need.

Danny Sullivan reviews several of these sites at searchengineland.com.

What to do?

A single query today may yield 100’s of videos, 1000’s of images and millions of URL’s. A new design grid has to be created, one that is organic enough to handle multiple data types, rigid enough to convey a consistent structure and visual enough to work across multiple demographics.

I have attached a wireframe to start the discussion, let me know your thoughts?

The wireframe above supports multiple data types on a single page and forces them into containers based on their data or semantic type. The containers themselves are also dynamic, visible only when appropriate. All containers can be minimized and maximized with a single click.

3 Comments on “Why is web search such a hard problem?”

  1. franz says:

    hi, great article.

    but well, about your grid: it’s a back to the window paradigm.

    people using search engines want to find, not to search. with this grid they have to search again in their viewport what is maybe best for them. they are mostly not interrested into different medias (website, pics, video, …) all they really want is the perfect match to their (information) need. thats why google now has oneboxresults, because sometimes a pic or news is the best result to a “shakira” search query.

    here is my contribution to the new search engine serps

  2. menro says:

    I enjoyed the face search you mentioned above, but it does not solve the blended results problem. I find windowing the data to be an alternative method to scan for content, blending takes allot more precision, though it can be done the pagination rules are significantly more difficult in a dynamic environment.

    The problem with google’s blended search is the design, if i know what i want i can ask the search engine for it, but if i enter Shakira. i might want her latest MP3, photo, video, a Verizon phone, fan pages, etc…

    Blended search has to deal with all of these different datatypes.

  3. […] If you look under the covers you quickly realize this is not the case. Web Search is comprised of 5 key dimensions, ranking, comprehensiveness, freshness, presentation and speed. Yahoo, Google, Microsoft and Ask […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

Join 2,208 other followers