Why

Disclaimer: This is my personal blog. The views expressed on these pages are mine alone and not those of my employer.

Browsers use a technology called a Rendering Engine to determine how to layout a web page. Rendering Engines use a technology called a Parser to determine the structure of the content. The Rendering Engine is at the heart of a web browser and it is one of the reasons when viewing web pages in different browsers that they look different.

Well guess what, a Web Search Engine uses a similar technology. Web Search Engines use something called a Crawler or a Spider to walk the Internet and retrieve web pages. Prior to inserting the data into the index, the software deconstructs the web page using a similar technology to a browsers Rendering Engine. This can be  beneficial to a web page owner. For example a web page that has lots of inbound links can help convince a web search engine to send thousands if not millions of people to visit its web site. This is one of the reasons Wikipedia is always returned in the top 10 for so many searches.

Google – Bad Cop

Image via WikipediaThe same technology can also be used to hurt a website. If two websites have similar content and one has a high PageRank and the other has a low PageRank the one with the higher PageRank will get 100% of the search referral traffic from Google.

Google Juice as it is called leverages PageRank as well as many other techniques to evaluate a web site. Some of the techniques are common sense, others are known by a small handful of SEO experts and others are only known by those working at Google.  Now this wouldn’t be such a big deal if Google did not own 70% of search referral traffic and if the other top search engines did not use a similar technique to Google’s PageRank in order to rank results.

To illustrate the Bad Cop techniques, lets review a recent issue that was aired publicly between Twitter and Google. As the story goes Google asked Twitter to change the Bio links on its user profile pages to nofollow. Doing this meant each user who worked hard to build up followers on Twitter could not leverage those links to promote their own blog, or companies website. Twitter ultimately did what Google wanted and changed the link to nofollow.

What I find interesting about this issue is how Google’s almost monopoly on Search is acting like an oligopoly. Google’s Bad Cop techniques are actually benefiting all of the search engines that leverage PageRank style indexes.

The consumer is the one losing out in today’s search market, yes search is better than it was 5 years ago, but that does not mean it cannot be significantly better. The Consumer needs a voice and they need to understand why certain pages are ranked higher than others.

How do we change this oligopoly? Do we have options?

The consumer does have a choice. Up and coming search engines like the one I founded Me.dium enables the end user to influence the search results. If you do not like what you see after typing a term or phrase into the Me.dium web search engine, download the toolbar and surf to the web pages that you think are better.

Similar to Wikipedia, where the people get to create, update and police the information. Me.dium users validate the web pages prior to being returned in search results. Power to the People.