As Google Becomes Web's Gatekeeper,
by Michael Totty and Mylene Mangalindan - Wednesday 26th February, 2003
Sites Fight to Get In
On October 17, 2002, SearchKing filed suit in the United States District Court, Western District of Oklahoma, against the search engine Google. SearchKing's claim was that Google's tactics to prevent spamdexing constituted a tortious interference with contractual relations. Click to read more on Wikipedia under Legal Precedents or search for "SearchKing"
Google works by sending "crawlers: across the Internet and using its 10,000 computers to update its index of about three billion Web pages. A site is ranked largely by the number of other sites that "link" or point to it, making Google a kind of online popularity contest. Each link is treated as a vote, with more weight given to links from more important sites, and a site with more votes will have a higher Google ranking.
Because of its importance, Google can make or break businesses that sell over the Web. It's the new :"location, location, location" for online retailers, for whom ranking at the top of a Google search is the Web equivalent of landing a choice corner on Miracle Mile or Fifth Avenue.
Some optimizers and Web sites claim that Google is unfairly punishing them. Last fall, an Oklahoma Web company, SearchKing, Inc., saw its ranking plunge on Google after it began selling online advertising that tried to capitalize on Google's formmula for ranking sites. In effect, SearchKing was offering its clients a chance to boost their own Google ranking by buying ads on more-popular sites. SearchKing filed suit against the search company in federal court in Oklahoma, claiming that Google "purposefully devalued" SearchKing ad its customers, damaging its reputation and hurting its advertising sales.
Google won't comment on the case. In court fillings, the company said SerachKing "engaged in behavior that would lower the quality of Google search results" and alter the company's ranking system.
Google, a closely held company founded by Stanford University graduate students Sergey Brin and Larry Page, says Web companies that want to rank high should concentrate on improving their Web pages rather than gaming its system. "When people try to take scoring int their own hands, that turns into a worse experience for users," says Matt Cutts, a Google software engineer.
Efforts to outfox the search engines have been around since serach engines first became popular in the early 1990's. Early tricks included stuffing thousands of widely used search terms in hidden coding, called "metatags." The coding fools a search engine into identifying a site with popular words and phrases that may nt actualoy appear on the site
Another gimmick was hiding words or terms against a same-color background. The hidden coding deceived search engines that relied heavily on the number of times a word or phrase appeared in ranking a site. But Google's system, based on links, wasn't fooled
Mr. Brin, 29, one of Google's two founders and now its president of technologh, boasted to a San Francisco search-engine conference in 2000 that Google wasn't worried about having its results clogged with irrelevant results because its search methods couldn't be manipulated.
That didn't stop search optimizers from finding other ways to outfox the system. Attempts to manipulate Google's results even became a sport, called Google bombing. Pranksters would try to bump a site to the top of Google by creating a series of oinks using a particular search phrase. In an early example, a Stanford University student in 2001 created a series of oinks to make a friend's Web site appear on Google for the phrase "talentless hack." In another highly publicized instance, anyone searching on Google for particular obscene phrase would call up a site that sold President George W. Bush memorabilia.
Search-enigne optimizers also found they could boost their clients' sites by creating Web sites that were nothing more than collections of links to the clients' site, called "link farms." Since Google ranks a site largely by how many links or "votes" it gets, the link farms could boost a site's popularity.
In a similar technique, called a link exchnage, a group of unrelated sites would agree to all link to each other, therby fooling Google into thinking sites have a multitude of votes. Many sites also found they could buy links to themselves to boost their rankings.
Ms. Homan, the leatherwear retailer, discovered the consequences of trying to fool Google. The 42-year-old hospital laboratory technician, who learned computer skills by troubleshooting her hospital's equipment, operates her onloine apparel store as a side business that she hopes can someday replace her day job.
"The big search engines determine the laws of how commerce runs," says Mr. Massa.
When she launched her Exotic Leather Wear store from her home in Mesa, Ariz., she quickly learned the importance of appearing near the top of search-engine results, especially on Google. She boned up on search techniques, visiting online discussion groups dedicated to earch engines and ready what material she coud find on the web.
At first, Ms. Holman limited herself to modest changes, such as loading her page with hidden metatag coding that would help steer a serach toward her site when a user entered words such as "haltertops" or "leather miniskirts." Since Google doesn;t give much weight to metatags in determining its rankings, the efforts had little effect on her search results.
She then recieved an e-mail advertisemetn from AutomatedLinks.com, a Wirral, England, company that promised to send traffic "through the roof" by linking more than 2,000 Web sites to hers. Aside from attracting customers the links were designed to improved her site's search-engine rankings by taking advantage of Google's link-popularity system.
AutomatedLinks claims to be different from other link-popularity schemes, such as link farms. FOr an annual fee of $22, AutomatedLinks will outfit a customer's Web site with an invisible piece of code that contains links to hundreds of other AutomatedLinks coients, according to its Web site. The service creates a kind of daisy-chain of "links" or mutual references, where the sites vote for each other and increase their rankings on Google.
In theory, when Google encounters the AutomatedLinks code, it treast it as a legitimate referral to the other sites and counts them in toting up the sites' popularity
Shortly after Ms. Holman signed up with AutomatedLinks in July, she read on an online discussion group that Google objected to usch link arrangmetns. She says she immediately stripped the code from her Web pages. For a while her site gradually worked it way up in Google search results, and business steadily improved because links t her site still remained on the sites of other AutomatedLinks customers. Then, sometime in November, her site was suddenly no longer appearing amonth the top results. Her orders plunged as much as 80%.
Ms. Homan, who e-mailed Google and AutomatedLinks, says she has been unable to get answers. But in the last few months, other AutomatedLinks customers say they have seen their sites apparently penalized by Google. Graham McLeay, who runs a small chauffeur service North of London, saw revenue cut in half during the two months be believes his site was penalized by Google.
The high-stakes fight between Google and the optimizers can leave some Web-site owners confused. "I don't know how people are supposed to judge what is right and wrong," says Mr. McLeay.
AutomatedLinks didn't respond to requests for comment. Google decliend to comment on the case. But Mr. Cutts, the Google engineer, warns that the rules are clear and that it's better to follow them rather than try to get a problem fixed after a site has been penalized. "We want to return the most relevant pages we can," Mr. Cutts says. "the best way for a site owner to do that is follow our guidelines."
Google has been stepping up its enforcement since 2001. It warned Webmasters that using trickery could get their sites kicked out of the Google index and it provided a list of forbidden activities, including hiding text and "link schemes," such as the link farms. Google also warned against "cloaking" - showing a serach engine a page that's designed to score well while giving visitors a different, more attractive page - or creating mulitiple Web addresses that take vistors to a single site.
To stay one step ahead of the Web sites, Google frequently tweaks its search algorithm - the formula it uses to rank sites. It also relies heavily on the search-engine optimizers themselves to report egregious examples of trickery.
While Google approves of optimizers that help a site improve its content or design, it recently revised its Webmaster guidelines to warn against firms that among other things, promise to guarantee a top Google ranking.
As evidence of the dangers of running afoul of Google, optimizers point to Oklahoma City based SearchKing, an online directory for hundreds of small, specialty Web sites. Search also sells advertising links designed both to deliver traffic to an advertiser and boost its rankings in Google and other search results
Bob Massa, SearchKing's chief executive, last August launched the PR Ad Netowrk as a way to capitalize on Google's page-ranking system, known as PageRank. PageRank rates Web sites on a scale of one to 10 based on their popularity, and the rankings can be viewed by Web users if they install special Google software. PR Ad Network sells ads that are priced according to a site;s PageRank, with higher-ranked sites commanding higher prices. When a site buys an advertising link on a highloy ranked site,k the ad buyer could see its ratingts improve because of the great weight Google gives to that link.
Shortly after publicizing the ad network, Mr Massa discovered that his site suddenly dropped in Google's rankings. What's more, sites that participated in the separate SearchKing directory also had their Google rankings lowered. He filed a lawsuit in Oklahoma City federal court, claiming Google was punishing him for tying to profit from the company's page-ranking system.
A Google spokesman won't commment on the case. In its court filings, Google said it demoted pages on the SerachKing site because of SerachKing's attempt to manupulate search results. The company has asked for the suit to be dismissed, arguing that the PageRank represents its opinion of the value of a Web site and as such is protected by the First Amendment.
"The big search engines determine the laws of how commerce runs," says Mr. Massa, who is persisting with the lawsuit even though the sites have had their page rankings partly restored. "Someone needs to demand accountability."
Google is taking steps that many say could satisfy businesses trying to boost their rankings. Google has long sold sponsored links that show up on the top of many search-results pages, separate from the main listings. Last year, the company expanded its paid-listings program, so that there are now more slots where sites can pay for a prominent place in the search results. Many sites now are turning to advertising instead of tactics to optimize their rankings.
That's what Ms Homan has done to keep up visibility for her Web site; Now she pays as much as $70 a month to appear in Google's paid listings. "You have no other alternative but to pay," she says. "I'm really frustrated that I don't have any other resourse. I've got to be in there somehow."