Inside Edelman’s large language model and how it tracks brand trust

When you think of the companies building large language models (LLMs), your mind probably goes to cutting-edge tech firms—Meta, Google, OpenAI and so forth. But Edelman, the PR consultancy, has surprisingly joined these ranks with its own model built to help brands comprehend their level of trust with consumers. Dubbed “Archie,” the LLM took nearly one year to develop, requiring the use of open source code and training data provided by pre-existing models, as well as significant investments in time and labor. The overall costs have contributed to Edelman more than doubling its spend this year as compared to last year, which itself substantially outweighed the company’s spending for the year prior. Last month, Edelman’s Brian Buchwald, who oversaw strategy for the company’s trust product business, was promoted to help lead Archie. The new global chair of AI and product is now focused on using the LLM to track how brand trust is influenced by scores of factors, including journalism and social media. Archie can also predict with high accuracy how trust may be impacted in the future, delivered through real-time recommendations for Edelman’s clients. Ad Age sat down with Buchwald to discuss the reasoning behind Archie, how it works, how it was built and where it fits into the greater AI trends impacting the advertising industry. The conversation also zeroed in on how Edelman is using its model to concretize an abstract but vital metric like trust. “We’ve gotten to something that’s pretty objective,” Buchwald said. “It is kind of this weird concept of, ‘Well, trust is intangible.’ But it is actually measurable.” This interview has been lightly edited and condensed for clarity. Why did Edelman create its own LLM? Edelman has been investing in trust for over 20 years. And there’s always been this question of, how do you make that more actionable? How do you take something that’s big and global and make it relevant to particular brands or executives? So about two and a half years ago, the company started to invest in its own surveys around trust, as well as collect digital data from earned media and social. Edelman then built machine learning models that, with some level of accuracy, could see digital data today and predict survey scores tomorrow. What’s happened with [the advent of] LLMs and generative AI is the possibilities for what you can create have become much greater, and the time to market has decreased exponentially. So we basically burned the boats. We said, “Instead of continuing to invest in the models that we’ve created over the last few years, we’re going to go build our own LLM focused on trust.” And we developed a new platform that enabled us to more accurately tie the digital data to those surveys, with 97% accuracy. How do you measure an abstract concept like trust? We have a robust database of surveys and quantitative research [on topics] associated with trust, such as ability, dependability, relevance, integrity and purpose. We understand how each of these actually contribute to this measure of trust, so that for any industry, I can show you how important it is to be seen as competent versus innovative, or have high integrity versus being purposeful. We have definitions for each of those pillars, which contribute in their own way up to an overall trust score. And then I have 3 million articles or pieces of content a day that I’m scoring on those pillars and overall trust. It’s one thing to tell [a brand] that the digital data is telling us their trust is going to go up or it’s going to go down. It’s very different for me to be able to use Archie and say specifically, “These articles are contributing to or detracting from your trust. These specific topics, these specific journalists, these specific publications are either aiding you or diminishing trust amongst your audiences.” I have to ask: Am I in the system? [laughs] You are in the system, yes. So could Archie tell a brand when it’s right to, say, jump in on a social media trend? It will give you an understanding of whether a trend or a meme is actually increasing or decreasing trust in a brand or in a category. Now, there’s always going to be some level of human involvement. So, like, how do you tap into it? That’s a question that a strategist might need to answer. There’s a creative application that a creative is going to likely apply. But yeah, absolutely—this is part of what it does. For instance, we have clients who are running social media campaigns on TikTok and Instagram. And what the platform allows us to do is to show them whether [the campaigns] are creating a sugar high for the brand, or whether these promotions and these influencer partnerships are legitimately creating elevated brand positioning and with whom. How else is Edelman using these insights to help clients? There are teams that use the platform to create insights reports and intelligence briefings, to pitch stories and to inform social content. There are teams that are using it to understand a crisis, for instance, what is actually happening to either contribute to or erode trust, such as whether an executive statement is actually helping the cause or hurting the cause. And if something is happening at an industry level, [it is being used] to understand whether it’s impacting the brand more or less than their competitors. So it’s pretty multi-utilitarian. It’s still a new enough platform that we haven’t pushed it down one particular vertical or use case, we’ve tried to make it relevant enough for multiple use cases. What have been some of the challenges of creating your own LLM versus using a licensed model like ChatGPT? Our cloud costs went up threefold virtually overnight. We were able to bring that down by becoming more efficient, but costs went up pretty significantly. We also had to hire folks onto the team that probably didn’t exist in any comms agency previously, like machine learning operations engineers. And we had to increase our discussions with our legal team to make sure that we weren’t doing anything that could trip any wires regarding plagiarism or unlawful use of third-party data. Do you foresee more players in the advertising industry opting to create their own LLMs over leveraging existing ones? I think you have to figure out who you are. We have proprietary IP in trust that we can leverage in a way that no other company in the world can, and that creates a proprietary advantage for us and hopefully for our clients. If you don’t have underlying IP that is uniquely valuable, I don’t know what a custom LLM is going to do for you. I’d say the totality of the investment is the other piece that companies need to be aware of. It’s having the product designers and product managers; the machine learning engineers, data engineers and software engineers; the lawyers and finance professionals who ensure that everything you’re doing allows for the progressive expansion of the capability while also safeguarding the business. It’s not a toy. This is about a holistic institutional investment in AI. And you got to go all in, you can’t do it part way. See all of Ad Age’s 2024 Creativity Award winners here

Inside Edelman’s large language model and how it tracks brand trust

Featured Stories