We're just as likely as anyone else -- maybe more so -- to scour Twitter and other social media for consumer insights, but it's not exactly a perfect science.
Witness the weekly Twitter-fueled "X-Factor" predictions from General Sentiment, which crunched a lot of tweets this week to infer that fans had given contestants Stacy Francis and LeRoy Bell the fewest votes and put them on the chopping block Thursday night. It predicted Mr. Bell would be sent home.
Both were actually safe.
"There are some limitations," General Sentiment CEO Gregory Artzt said today. "The classic limits are the ones you're familiar with already, that this is not a traditional research study with a representative population. That's what traditional research people will tell you about anything in social media. However, we know Twitter and social media in general are shifting and becoming much more representative of the U.S. population, and the world population, so that argument is losing steam."
"The other place where people don't quite capture the value in this is by looking at overly simplified models," Mr. Artzt added.
And that 's what General Sentiment is putting against its weekly "X-Factor" predictions: simple tweet volume counts.
"If we wanted to do a better job of this we would layer in sentiment, which is of course key to our business," Mr. Artzt said. "Maybe some of the reactions for LeRoy Bell, although he didn't get much volume, were very positive. The other thing you can do with this is to try to make it a little bit more representative, to understand that certain singers might appeal to younger or older audiences."
In the world of "The X-Factor," Ms. Francis and Mr. Bell are both members of the "Over 30s" category.
"There might be an audience for the older contestants that 's not as much online," Mr. Artzt said. "You could make this more like a traditional representative study and weight each Twitter mention for an older contestant higher."
For an exercise like guessing "X-Factor" eliminations, though, General Sentiment will probably stick with the simple analysis. It's not like a client is paying for a deep dive on the subject. And its predicted bottom four contestants did prove to include the eventual bottom two, so even straight-up tweet-counting isn't that far off.
"I believe you have to start simple," Mr. Artzt said.