E-mail lists are becoming databases. Web ad networks and ad servers build databases to target banners. Your business runs databases on customers and prospects.
Understanding what these databases are telling you, and using that knowledge in developing strategies, is becoming the heart of your job.
Most databases are called SQL (structured query language) databases, but that doesn't mean they're the same. SQL is a computer language, like C or Basic, that was invented by IBM Corp. in the 1970s for accessing one of its relational database products, System R. Relational databases are those in which each entity is stored in a separate table to avoid duplication.
In the past 20 years, nearly all relational database vendors have implemented support for SQL commands. So products such as Microsoft SQL Server and Oracle 8, as well as IBM's own DB/2, are called SQL databases.
SQL gives you a common way to manipulate databases, says John Ashenfelter, technology adviser to the University of Virginia and author of Choosing a Database for Your Web Site, published this year by John Wiley & Sons, New York.
Making a connection between disparate databases requires another technology, Open DataBase Connectivity (ODBC), he says.
ODBC translates between databases, says Benny Bollay, president of ExperTelligence, Santa Barbara, Calif. If you have an Oracle database in one department, an SQL Server database in a second and a FoxPro database in a third, ODBC can connect them. The only problem is that if you try to do this in real time, say, from a Web site, ODBC translation will slow things, Mr. Bollay says.
But just linking databases doesn't give you useful data, Mr. Ashenfelter says. "There has to be some consistent intellectual connection between databases, some consistent field in common," he says. "Otherwise you get garbage" when you try to combine data sources.
Jim Sterne, president of Target Marketing, a Santa Barbara, Calif., marketing consultancy, says that to get the most value from a database, marketers need to get involved with their technical people before it is built.
"Sit down with the data modeler at the beginning and ask all the questions you might want to ask in the future, so the model will be sufficient to the task," he says.
The result of this exercise should be a complete database design that includes indexing, Mr. Ashenfelter says.
"Indexing means you take a field you know will frequently be searched and you point back to it," he says. "The data may be ordered by last name, but maybe you want to sort it by phone number." The last name and phone number in this case would be index terms.
For databases to be linked, some piece of information they hold in common is needed, preferably an index term, Mr. Ashenfelter says. This could be a phone number or a Social Security number. Because phone numbers change, Social Security numbers -- or, in the case of businesses, Dun & Bradstreet numbers -- are most commonly used as index terms, he says.
When database engineers talk about "dirty data," they often mean databases that are indexed using hard-to-translate terms like client numbers, which are useful for only one company, or names, Mr. Ashenfelter says.
There's a lot wrong with using names as index terms, he says. For instance, umlauts in some German names and apostrophes in some Irish names can cause problems in name indexes. So can translating from languages such as Arabic -- is it Qadafi, Khadafy or Khaddafi? And what if people change their names?
The result is that when such databases are combined, you may have many entries referring to the same individual, Mr. Ashenfelter says.
Exchanges with XML
An enhancement to the Web's HyperText Markup Language (HTML), called the eXtensible Markup Language (XML), is making databases easier to exchange, says Mr. Bollay, who produces a database search engine called WebData.
"XML helps unify the movement of data and queries," he says, through hidden tags. Oracle, the largest database vendor, said last November it will support XML in all its products.
"But there's still a lot of work to be done in this area because you have to standardize the tags," he says.
Again, for instance, take last names. The tag associated with that field could be "last" or "lastname" or "name" in different databases.
The good news is these differences are being ironed out, Mr. Bollay says. The bad news is they're not ironed out yet.
Mr. Bollay says he hopes XML will make WebData a "Yahoo! for databases," building a marketplace for them.
But the solution to database problems isn't a search engine, it's knowing what you want from a database before you query it, Mr. Sterne says. "I want a Lieutenant Commander Data at my side.