The following exchange took place on Twitch in April: “The question is are white people being replaced in America and if so is that a bad thing,” a Twitch moderator said. A Twitch streamer chimed in: “That’s what it seems like to me, a lot of immigration, no matter how many large, large, groups of Americans gather to push back against it … Why would they want to force us to allow foreigners in against our will.”

They were discussing the “Great Replacement," a theory that suggests there is a plot in America and other Western countries to limit the procreation of white people and allow Black and brown immigrants to grow in numbers. Those who believe this often blame Jewish people for masterminding the plan. 

This particular exchange on Twitch, which appeared in the platform’s “Just Chatting” category of videos, was preceded by pre-roll advertisements from Hyundai, Dr Pepper, Discover, Meta and Doordash, advertisers on Twitch that likely did not know their ads would appear before a debate about the conspiracy theory that veered into racist fearmongering. The “Great Replacement” has now become the latest brand safety concern related to social media as the subject was in the news for inspiring a radicalized gunman who killed 10 people in Buffalo, New York, over the weekend. Most of the victims were Black. The shooter left a racist-filled manifesto behind.

The shooting was livestreamed for nearly two minutes on Twitch, before being shut down. Video clips made their way to other social platforms such as Meta’s Facebook. The aftermath of the mass shooting has sparked a new round of soul searching in the worlds of media and social media about the responsibility of platforms to police their services, to tone down violent rhetoric, and the role brands play in inadvertently funding some of this content.

Twitch, which Amazon acquired in 2014 for $1 billion, features plenty of streams that touch on controversial issues in the “Just Chatting” category.

“Brands do need to take this into account,” said Kayla Gogarty, associate research director at Media Matters for America, a media watchdog group. “With their funding, their advertising money, they have the ability to hold … platforms accountable. We continue to get empty promises. They keep telling us they’re doing what they can to stop misinformation, extremism, and yet still they have these horrible incidents happen.”

Videos of the Buffalo shooting were clipped and sent to Facebook feeds, Gogarty said, generating potentially millions of views. A Meta spokesperson said that the platform "quickly designated the event as a violent terrorist attack" and removed the perpetrator's account, as well as links to a "manifesto" and videos. The shooter claimed he was radicalized “by infographics, shitposts, and memes,” in the manifesto that went online. He also allegedly had a YouTube account.

Platform and brand responsibility

Attempts to reach Twitch were unsuccessful. The “Just Chatting” stream from April where the “Great Replacement” theory was debated was unaffiliated with the shooting, but ran programmatic ads from major brands. Twitch is a vibrant video streaming community, mostly for gamers to share their skills in front of 31 million daily viewers. The company is attempting to become a mainstream media platform, one ready for conversations around primetime NFL games. Amazon has exclusive rights to the league on Thursdays in September for the first time.

A Hyundai spokesperson said that the car company was working with Twitch to fix any inappropriate placements going forward. “Hyundai works constantly with Twitch and the other digital platforms it advertises on to ensure its ads run with content that follows established protocols and filters,” the spokesperson said by email. “We are ensuring this content is on a blocked list moving forward.”

Amazon is trying to clean up Twitch by creating more official content by curating channels under corporate supervision, and opening it to advertisers. “For Twitch, With Twitch” was one of the new programming tracts that Amazon revealed at its NewFronts ad sales event earlier this month. The timing around NewFronts makes the Buffalo attack all the more sensitive for the company as it is looking for strong brand support.

“They are very active in recruiting and funding more structured shows in categories that are easier to gain ad money for,” said one digital media ad buyer, who spoke on condition of anonymity about the brand safety ramifications for Twitch following the shooting.

“The most important person at a platform in this year’s NewFronts cycle is going to be the platform’s head of brand safety,” said Lou Paskalis, president and chief operating officer at Mobile Marketing Association Global, an industry trade group. “Media buyers need to understand what the platforms are working on, and specifically how are we going to be mitigating these issues.”

In an industry that relies on sponsorships and advertising dollars, the conversation has turned to the responsibility of brands, which have had concerns about social media platforms for years. Platforms like Amazon’s Twitch, Google’s YouTube, Meta’s Facebook and Instagram, Twitter, ByteDance’s TikTok and Reddit have all worked with industry groups like the Global Alliance for Responsible Media to create some guardrails. Meanwhile, to show how much this debate about moderation has permeated social media, look no further than Elon Musk, who has offered $44 billion to buy Twitter. Musk has made looser moderation a cornerstone of his argument for why he should buy the site, claiming the platform is too strict in taking down certain forms of speech.

YouTube has been dealing with the brand safety question ever since there was a brand boycott of the service in 2017, when sponsors were worried about ads appearing on extremist and terrorist videos. YouTube has overhauled its policies in the intervening years to prevent ads from showing up on videos that promote racist theories. The video site has some leeway for permitting the publishing of videos around certain subjects on the grounds of “free speech,” but it tries to block those controversial subjects from carrying ads. 

“We have long-held Community Guidelines that make clear we do not allow hate speech on YouTube,” a YouTube spokesperson said by email. “Our hate speech policy specifically prohibits content that promotes violence or hatred against individuals or groups based on attributes like their race, nationality, or immigration status. This includes some claims made under the so-called ‘Great Replacement Theory.’”

Last year, Facebook started testing “topic exclusions” in its feed, which are meant to limit brands’ exposure to subjects they consider sensitive.

In many ways, the Buffalo shooting shows what Musk critics were worried about, if he ultimately takes over Twitter, according to an advertising executive, who spoke on condition of anonymity. “It really puts Elon Musk's Twitter rhetoric in sharp relief,” the advertising executive said. “It doesn't just feel out of step, but it feels like the Silicon Valley conventional wisdom from 2006 is way out of date. The rest of the world has moved forward, and in fact Twitter had previously been a prime mover in accepting that platforms should play a more active role in safety matters.”

Don’t miss the latest news. Sign up for Ad Age newsletters here.