How a creative technologist used deepfake tech to resurrect a legend

For our AAPI Heritage Month Creative Excellence Series, we are excited to give the stage to Shan Jin, creative technologist at Goodby Silverstein Labs, the agency’s internal innovation department that focuses on emerging technologies. Born in Shanghai, she earned a degree in software engineering and began as a developer. Her creative passions however eventually led her to the ITP program at New York University’s Tisch School of the Arts, where she was able to learn how to leverage tech to create art. From there, she landed at experiential design firm Fake Love, which was eventually acquired by The New York Times. Now, at GSP, she spends much of her time creating with machine learning, evident in projects for The Dali Museum and Pepsico’s Frito-Lay as well as her agency’s “Respond2Racism” TwitterBot created to fight the anti-Asian racism that emerged during the pandemic. 

Here, she shares one project for The Dalí Museum that utilized deepfake tech to bring Salvador Dalí himself back to "life" —even enabling him to take selfies with visitors at the institution. 
 

I’m originally from Shanghai, China. As a good kid in a Chinese middle class family, life was pretty straightforward—I went to a first class university in China, I majored in Software Engineering and graduated with a job as a developer. I think that was where my life started: on the day of graduation I told myself that I’ve fulfilled everything my parents expected of me and can go on and live my own life now.

Two years later I quit my job, and then the whole world opened up for me. Back then I hated coding so much and swore that I would never ever code one day in my life, but my journey at New York University soon changed my opinion. At NYU’s art school, I was introduced to what is called “creative coding,” and an epiphany moment emerged, and all the dots connected. Technology can be used as a medium of creative expression. 

Very soon after I started at Goodby Silverstein, I got the chance to work on “Daií Lives,” a project that would become a milestone in my career. The “Dalí Lives” project brings the dead artist Salvador Dali back to life using deepfake technology. In the Dalí Museum in Tampa, Florida, people would be able to interact with an engaging life-like Dalí on a series of screens throughout the museum.

“Dalí Lives” is the dream project that every creative technologist could ask for. It’s particularly important to me because it’s a perfect combination of art and technology. It’s a playful, interactive experience while also educational and meaningful. People learn about Dalí’s life and his art, they are surprised when he takes out his phone and takes a selfie with them. Partnering with the Tampa Bay Times, we also pulled their front page everyday so Dalí was always reading up-to-date newspapers. 

There’s also a social aspect of it—it’s the first time I knew of that deepfake technology was used for good, after people had seen all of its application on fake news and celebrity porn. Recent years have witnessed an explosion and the huge progress of machine learning technology. We can now generate fake human faces that are indistinguishable from real ones. We can have AI write essays and taglines, compose music and generate images just from text prompts. But how do we utilize all these emerging technologies and make meaningful applications out of them? This is a question that all creative technologists are always wondering, and “Dalí Lives” answers that.

Read: 30 years after his death, A.I. brings Salvador Dalí back to 'life'