It's one of those marketing conundrums: The bigger the Web gets, the smaller your chances of being caught by one of its search engines.
Adding to that problem is the fact that many marketers may be underestimating the significance of ranking highly on search engines--the importance of which is highlighted in at least one recent study.
In a survey of about 5,000 respondents, 50% found Web sites they were looking for by performing keyword searches on search engines, according to Eric Knight, VP-marketing for Outrider.com, an Internet services firm in Farmington, Conn.
Only 18% of respondents found sites they were looking for by clicking on banner ads. Seven percent visited Web sites after learning about them from radio, TV and print media advertising. Another group, 12%, first learned about Web sites by word of mouth. The remaining 13% of respondents found Web sites through "other" means.
In the early days of the World Wide Web, developers focused on building sites that didn't crash and were fairly easy to navigate, said Charles Watty, senior Web producer at Grapevine, an Internet services agency in Redwood City, Calif.
Early users of the Web most often had to type the exact address into a browser in order to reach a Web site.
Then came the first search engines, starting with WebCrawler and Lycos in 1994, designed to sift through the increasing deluge of information on the Internet. Yet by 1997, companies had a hard time finding their own sites in search engine results.
That's when a small Joplin, Mo., firm, FirstPlace Software, found a way to let companies sidestep the hours of research needed to record their search engine rankings. The company's $149 WebPosition Gold software package is now one of the top-selling automated products for monitoring and optimizing search engine results.
Software like WebPosition Gold helps Web developers pay close attention to issues that affect how a site is ranked. These include the placement of relevant keywords throughout the site so they can be spotted easily by search engines, as well as encoded terms called "tags" that help search engines identify a Web site's content.
Developers must often create additional pages to serve as pathways around features such as animation that prevent auto- pmated search engines from viewing Webpages. Yet companies in the business of search engine optimization contend that the business is still in its infancy.
"A small sliver of the market pays attention to the need for this," said Fredrick Marckini, CEO of Internet services company iProspect.com. "We're still a cottage industry."
Further complicating search engine optimization is the fact that search engines do not all function the same way.
Some are automated computer systems, known as spiders, that scan the Internet and "record" the content of individual Web sites. Spider-based search engines, such as AltaVista, AOL Search, Excite, HotBot, Lycos and MSN, generate lists of Web sites by category so that listings appear ranked according to relevancy in response to any keyword search.
Directories, including About.com, LookSmart, Open Directory, Snap and Yahoo!, are a different breed of search engine. Teams of editors review Web sites for each directory, then decide whether to index the sites and how to categorize them.
Directories disregard tags and other aspects of a Web site's code, paying careful attention instead to the Web site's content and design, and such problems as broken links and pages under construction.
Mark Olivieri, Outrider's director of product development, cited the success story of Capsugel, a subsidiary of Pfizer Inc. that dominates the market for two-piece gelatin capsules used to package pharmaceuticals.
Capsugel approached Outrider last year for help boosting traffic to its Web site, Capsugel.com. The site received about 750 visitors per month in 1999--lackluster even by b-to-b standards. So Outrider built a new site that was optimized "from the ground up" to achieve top search engine rankings. Traffic on key pages on the Capsugel site has since tripled.
Though other marketers have enjoyed similar successes, optimization specialists say it's best not to promise too much to clients when submitting sites for registration. Powerful software can help optimize sites for the various algorithms search engines use to rank their listings, but achieving high rankings is still unpredictable and can never be guaranteed.
Steve Schepke, VP-marketing of Des Plaines, Ill.-based Meandaur Internet Marketing, said, "If we can get a site into the first screen or two of results 25% of the time, then we feel we've done a good job."