Why Images on the Web Are a Data Blindspot and Potential Gold Mine

When then-Google exec Vic Gundotra took the stage at the company's annual developer conference, he showed a photo of the Eiffel Tower. Many in attendance automatically recognized the landmark -- and so did Google's computers. Since that milestone in May 2013, Google, along with Yahoo, Facebook and other companies, has been struggling to get at a gold mine of potential data: the millions of images flitting about the web. Image-recognition technology is the next frontier for companies that have built multibillion dollar businesses on their ability to convert data into more personalized content for audiences and better targeting for advertisers. "There's this treasure trove of data these companies are sitting on in the form of the visual web. That data for the most part is uncategorized. It's a black box," said Justin Fuisz, CEO of Fuisz Media, a company that uses image recognition to attach branded calls to action to objects in videos. Images account for roughly half of the content in the average Facebook news feed, the company said last year. That share has likely risen and not just on Facebook but across the web as visual social networks like Pinterest continue to grow and publishers like Vox Media and BuzzFeed post more image-heavy articles. Some companies have tried to sidestep the problem by tapping the so-called metadata around an image such as captions and article text to understand its contents. But that's not always reliable or as useful as knowing what a photo actually contains. "We've done some research and found that 85% of the time textual data does not support the actual brands that are showing up in an image," said Ophir Tanz, CEO of GumGum, an in-image ad company. That's why companies like Google and Yahoo are beefing up their ability to tackle the problem. This year alone Google, Yahoo, Pinterest and Twitter have each acquired companies that have developed technologies related to image recognition. Google and Pinterest declined to comment for this article. Twitter did not respond to requests for comment. Facebook, which bought facial-recognition firm Face.com in 2012, isn't applying its technology to ad targeting but is using it for content curation, like whether to show "meme" images (like a cat photo overlaid with text) in someone's news feed if that person has shown an interest in memes or considers them spammy, according to a Facebook spokesman. Yahoo -- which has been able to use its photo-sharing site Flickr to hone its image-detection chops -- is able to pick out and categorize specific objects like a pair of boots in an image or video in order to provide more context about a page's contents for ad targeting, said Alex James, director of research at Yahoo Labs. Yahoo's technology can also help to refine search results, like determining whether an article about a jaguar refers to the luxury car brand or a big cat. Image-based ad targeting has come to Twitter through an outside party. Image analytics firm Ditto Labs is able to pick out photos posted to Twitter -- either directly or through Instagram -- that contain a brand's logo and collect the Twitter handles of the people posting those photos into a list that an advertiser can then submit to Twitter as the audience it would like to show ads, said Ditto Labs CEO David Rose. While unlocking images' data is becoming a reality for these tech companies, for marketers it remains a fringe issue. But that doesn't mean marketers' agencies don't see the potential for image-based ad targeting. "Our clients have never given it much thought. But as soon as it's available at scale, it's going to be huge," said Doug Kofoid, president-global solutions at Publicis Groupe's VivaKi. Pinterest is talking with agencies like Horizon Media and Starcom MediaVest Group about how its image-recognition technologies can help their brand clients. "When we are talking in theoretical terms about [Horizon Media client] Weight Watchers for instance, we can identify who is pinning Weight Watchers specific materials -- food, recipes -- but also any type of food content that we think would be a qualifier for a potential Weight Watcher prospect. So it becomes very clearly about the actual content that's pinned," said Horizon Media Chief Digital Officer Donald Williams. "The biggest surprise I had was the richness of those images and how much personal information is coming across in the richness of those images," said Starcom MediaVest Group's global research lead Kate Sirkin. "It really does start to give you a good flavor of interests and lifestyle and passions of the consumer."

Why Images on the Web Are a Data Blindspot and Potential Gold Mine

Featured Stories