Deepfake Taylor Swift is like the Oprah of Facebook giveaways—everyone who clicks gets an iPhone, a Dutch oven, a $500 gift card. Except the fake Swift ads are social media scams involving offers for iPhones, stainless steel Stanley tumblers, Le Creuset cookware, Visa gift cards, or anything to induce a click from an unsuspecting fan.

The ads are phony, using deepfake artificial intelligence to impersonate the superstar on platforms, including Facebook and Instagram. The ads reflect a darker side of AI that is starting to corrupt the ad world and beyond, causing trouble for brands.

Similar ads have used other fake endorsements—from actress Selena Gomez, billionaire Elon Musk and podcaster Joe Rogan. In one Facebook ad, an AI-generated voice of Rogan asks fans to enter a crossbow giveaway. Musk’s fake ad claims it will double a person’s bitcoin if they first send the cryptocurrency to him.

The ads are “deepfakes, cheap fakes and shallow fakes,” said Philip Cowdell, global chief strategy officer at social media and digital video marketing firm Channel Factory. Cowdell has studied the rise of AI in advertising, where generative AI has made creative tools more accessible for legitimate marketing. But the technology is also wreaking havoc on the online information landscape in ways that are detrimental to brands, celebrities and the general public discourse.

Besides Meta, which owns Facebook and Instagram, AI-generated marketing material has appeared everywhere from YouTube to TikTok. The ads sometimes use sophisticated deepfake technology that recreates a celebrity’s image with almost complete accuracy. Other times the fakes are less convincing, what Cowdell would call “cheap fakes,” and they don’t require generative AI at all. For instance, a scammer could use real video of Swift and dub over it with any voice, and that could still fool consumers, Cowdell said.

“All this disinformation is now cheaper and easier to make and disseminate, and it is going to fuel polarization,” Cowdell said. “It used to be hard and expensive, and now it’s cheap and prevalent, and social media is the easiest way to push it out to the world. That’s not good for consumer confidence.”

Also read: General Mills combats AI deepfakes

Exciting and serious

A search across platforms shows that AI-generated fakes are everywhere. The ad world has helped spearhead AI adoption, embracing tools such as ChatGPT for text generation and Midjourney for image creation, and they streamlined ad production.

Top brands from Coca-Cola to Mondelēz have accepted the AI call, using the technology to upgrade their marketing.

Last year, Coca-Cola, created a website where consumers could generate images using its marketing assets. The site is called “Create Real Magic,” and shows how some brands see AI as a way to democratize access to their advertising and give people power to shape how they interact with the companies.

At the same time, though, bad actors have used similar tools to build fake ads, misleading websites and spin disinformation. And the largest digital platforms, such as Meta, Google, Amazon, TikTok and more, are having trouble separating the good AI from the bad, according to ad leaders, who are trying to figure out how to incorporate AI while being vigilant about abuse.

“We’ve been banging on about the absolute wonders and unlimited potential of AI, particularly in the creative world and the business world,” said Ellie Bamford, chief strategy officer, North America at VML. “It’s an exciting moment, but this is a very serious moment.”

Bamford is particularly concerned about disinformation during the U.S. elections this year and with the proliferation of deepfake pornography. There needs to be regulations, Bamford said. 

“It’s time for a pivot and for us to really discuss the serious impact of what this is going to do,” Bamford said. “We need better detection and removal policies.” 

‘Celeb-bait’

Meta and Google declined interviews for this story. They did point to their ad policies that prohibit scam ads, and newer rules that address manipulated media.

Meta, for instance, has a lengthy policy against “fraud and deception,” and said it removed 1.3 billion accounts from October 2022 to December 2022 for violating those rules, its most recent stat. Meta considers the fake Swift ads a form of “celeb-bait,” and these types of scams have plagued the internet for years, even before the recent AI boom, a Meta representative said in an email. Meta is investing significantly to combat such scams, but the attackers are always getting more sophisticated. For instance, a fraudulent ad could fool Meta’s detection systems, using a method called cloaking, which hides the content of the ad from the moderation system.

On Google’s YouTube, meanwhile, there have been ads that use AI-generated voiceovers that mimic popular personalities. There also is a whole genre of AI-generated videos from accounts that create commercials and use brands’ logos and jingles, including Oreo, Arby’s, Chevrolet, Burger King and others. In most cases, the AI videos appear harmless, or even funny, but in some cases they insert politically charged messages or push cryptocurrencies, such as Dogecoin, the popular memecoin.

A Google spokesperson said that the platform requires creators to label synthetic material accordingly, a policy it announced last year. Google said it removed 5.2 billion ads in 2022 for policy violations and suspended more than 6.7 million ad accounts in 2022. The Google rep also said that the company is aware of the emerging “deepfake celebrity endorsement” trend and that it is investing heavily to combat it.

Risk to reputation

Brands and advertisers are worried about the abuse of AI and how it could affect their operations, according to Nicole Denman Greene, VP analyst at research firm Gartner. A recent Gartner study found that 55% of brand reputation leaders said generative AI poses a risk to their brands, Greene said. Part of that risk is the proliferation of misinformation created with the help of AI and deepfake scams that co-opt their brands.

“There’s a lot of work for brands and their agencies to do right now,” Greene said. “They’re going to have to manage both the consumer and business risks as generative AI content and adoption increases.”

The fraudulent Swift ads promoted Le Creuset cookware, a brand that Swift has been known to like. Other social media ads, with digitally altered celebrity endorsements, hawked iPhones, as well as gift cards for Visa and Walmart.

“You need to monitor your brand, and you need to understand what is, and what isn’t, part of your brand [identity],” Greene said. “That will help you when generative AI content of your brand is out there.”

Gartner found that 60% of chief marketing officers will likely adopt technology that can help authenticate content, which could be used to prove when digital material is actually from the brand. Brands could also use this technology to authenticate user-generated AI content. They should also enhance online monitoring to watch out for when trademarks are being abused, Greene said.

Cybersecurity software firm McAfee is working on a deepfake audio detection system. The firm found that nearly 70% of Americans said deepfakes make it hard to trust what they see online, according to a recent study by McAfee. “AI-generated voices can be used for a wide range of purposes,” said Steve Grobman, McAfee’s chief technology officer. “Some of them can be nefarious, traditional scams that can be enhanced by a celebrity hawking a product.”

Another crafty scam that spread online recently featured MrBeast, the popular YouTuber, according to Grobman. The video used real footage of MrBeast, aka Jimmy Donaldson, but inserted deepfake audio to promote a gambling app. It was one of many online scams that misappropriated MrBeast’s image.

In some cases, these scams are directly related to advertising, Grobman said, because they prompt consumers to download apps or visit websites that monetize by serving ads.

Misinformation minefield

Some advertisers have been taking deepfakes seriously. Last year, General Mills tapped Zefr, a third-party brand suitability measurement firm, to monitor ad campaigns on social media to quantify the level of AI-generated misinformation. Zefr delivers post-campaign reports from TikTok, YouTube and Meta to understand the level of misinformation that appeared near an ad. What is and isn’t misinformation is defined by standards set by the Global Alliance for Responsible Media.

There is a dilemma, however, in advertising regarding how to handle generative AI. Deepfakes have opened access to marketing potential, especially for small businesses. In 2022, Queen Latifah teamed up with Lenovo and allowed her AI likeness to be used in small business advertising in the U.S. and Canada. That’s the type of celebrity deepfake that most brands can get behind, said Bamford of VML.

“Creatively, with our agencies, deepfakes have been used as an interesting creative tool to create scale and to unlock access to celebrities,” Bamford said. “It can be tapped into as a positive thing, and I think that’s really interesting, but there is a whole other side of this and it’s worth calling attention to.”