<\/p>\n
Listen to the episode on Apple Podcasts<\/a>, Spotify<\/a>, Overcast<\/a>, iHeart<\/a>, PlayerFM<\/a>, Podchaser<\/a>, Boomplay<\/a>, Tune-In<\/a>, Podbean<\/a>, Google Podcasts<\/a>, Amazon Music<\/a>, or on your favorite podcast platform.<\/p>\n Read the Show Notes<\/a><\/p>\n Watch on YouTube<\/a><\/p>\n [00:03] BW: Hello and welcome back to Pixels and Paint. We have a very special guest today, Christian Burke of Refik Anadol Studios. Christian, can you introduce yourself for listeners who may not know you?<\/p>\n [00:24] CB: Certainly. Hi, nice to meet you all. I\u2019m Christian Burke, the lead data scientist at Refik Anadol Studios. I started working for Refik in 2018 when I was a sophomore in college. Over the past five years, we\u2019ve led various AI, art, and web projects. I\u2019ve primarily managed the data collection, data processing, and a lot of the backend work for our art pieces, especially on the machine learning and AI side.<\/p>\n [00:53] BW: How did you first connect with Refik?<\/p>\n [00:56] CB: We actually met randomly. He asked if I could do data collection, like downloading the internet and collecting images to train AI models. I agreed, and the rest was history.<\/p>\n [01:14] BW: So, you were already studying data science at the time?<\/p>\n [01:18] CB: Yes, I was at Duke University studying computer science.<\/p>\n [01:23] BW: When you began your journey in data science and computer science, what did you envision working on?<\/p>\n [01:31] CB: Initially, I wasn\u2019t certain. I started teaching myself computer science around 12 or 13 years old. I was drawn to the accessibility and abundance of information available online. It empowered me to learn and apply anything I wanted. I originally imagined working primarily with web design, website building, and back-end servers. However, I found my way to data science when the opportunity presented itself.<\/p>\n [02:19] BW: Were you studying data science specifically when you started working with Refik?<\/p>\n [02:27] CB: No, I was only a sophomore in college and hadn\u2019t delved deeply into specialization. I\u2019d taken entry-level courses and was involved in statistics, math, and probability. I was learning data science, but it wasn\u2019t my explicit intention from the outset.<\/p>\n [02:46] BW: That first summer with Refik must have been significant if it influenced you to change your major and continue working with him.<\/p>\n [03:00] CB: Absolutely. It was an invigorating experience. I had previously dabbled in data collection and processing. For instance, my first program fetched stock ticker data from Yahoo finance and analyzed those numbers. So I had some interest in data collection and visualization. That summer, Refik tasked me with downloading numerous images of New York for an art project. In 2018, I gathered about 153 million images of New York City. At the time, this was the most extensive dataset ever utilized for artwork.<\/p>\n [03:46] BW: Do you have any background in the arts?<\/p>\n [03:50] CB: Yes, I was quite involved in drawing and painting during high school and was an active participant in my school\u2019s art program. I even won a scholastic art award among other achievements. However, in college, I couldn\u2019t allocate much time for personal artistry. I did study ancient Greek archaeology, focusing on patterns on pottery and burial distributions. This kept me connected with the art domain, but I primarily concentrated on technical subjects.<\/p>\n [04:30] BW: Have you tackled any personal art projects since leaving college?<\/p>\n [04:43] CB: That\u2019s a tough question. I\u2019d say no, not directly. However, I believe everything in life can be interpreted as art, including writing code. There\u2019s an inherent style and artistry to it. While I may not have created traditional art myself, I do see a strong artistic component in web design, programming, and such. Working closely with other artists, providing them data, and ensuring everything looks cohesive has its artistic elements.<\/p>\n [05:18] BW: I want to discuss Unsupervised, your piece in the MoMA. How did that collaboration begin?<\/p>\n [05:38] CB: They first approached us in 2020, expressing interest in the web3 world and our previous work. The exact context escapes me. In 2021, we launched the NFT project with MoMA named \u201cMoMA Unsupervised.\u201d We released about 6,000 NFTs: an edition of 5,000, nine editions of 100 pieces each, and a few unique pieces. We initiated this collaboration amidst the NFT boom in 2021. While the NFT realm provided support, there was no equivalent representation in the physical world.<\/p>\n There remain uncertainties regarding data usage, AI, web3, and what the museum should accept. After the successful NFT collection, MoMA reached out about a year later. They appreciated the NFT collection and proposed a physical installation in their lobby. They granted us access to high-resolution data from their collections, which enhanced our model. We then installed a 16\u00d716 meter display in MoMA\u2019s lobby around November 2022.<\/p>\n [07:23] BW: Yeah, I\u2019ve been there and it is really amazing to stand in front of the biggest digital art screen I\u2019ve ever seen. How did you get a screen that size? Where did it come from? Did you or Refik decide to go this big?<\/p>\n [07:46] CB: Yeah, we\u2019ve been installing big screens around the world for quite some time. This isn\u2019t new to us. Most of these screens are made from custom panels. If you get close, which you probably can\u2019t see when it\u2019s on, there are littler panels stitched together. This screen is 16 meter by 16 meter. We\u2019ve done ones even larger, but what\u2019s special about this screen is its resolution. I think it\u2019s a 2K or 3K resolution screen. So it\u2019s really nice to have something this high quality and large.<\/p>\n [08:29] BW: How has life at the studio changed since that piece debuted?<\/p>\n [08:35] CB: There\u2019s been more public attention, and also some controversy. With topics like web3, AI, and digital art, there\u2019s always some debate. Many people come to the studio praising our work, but there are also those who critique digital art. However, the feedback has been mostly positive. The installation was set to be up for three months, from November to March. But due to its reception, it\u2019s been extended several times and is still up. What\u2019s meaningful for us is seeing people\u2019s emotional reactions to our art. Watching children play in front of it, or loved ones sharing moments\u2014it\u2019s a touching experience. We\u2019re glad to have shared that with so many.<\/p>\n [10:00] BW: Yeah, I have a question that I\u2019ve been asking recently that some people have a hard time answering. I\u2019ve seen some of the negative reviews of Unsupervised, calling it something like a fancy lava lamp. I think that was the New Yorker or the New York Times. What does art do? Why make art at all?<\/p>\n [10:35] CB: It was one of my favorite quotes to ever read, calling the unsupervised piece an intelligent lava lamp. For us, art is all about the human impact. We\u2019ve focused on public art from the start of the studio. When it comes to the actual meaning and purpose of art, people take away whatever they want from it based on their own perspectives. What we really focus on is the human experience, allowing people to feel emotional reactions. The MoMA is a great example. We did another piece at K\u00d6NIG GALERIE back in, I think, 2021 or early 2022. It was in a brutalist architecture museum with a massive screen at the end of a long chamber. People had intense emotional reactions there. Some cried tears of joy or sadness. For us, the human impact of art and allowing people that space to heal, grieve, or feel is what\u2019s important.<\/p>\n [12:14] BW: When were you most surprised by somebody\u2019s reaction? You\u2019ve probably seen a range of reactions by now, but what was the one that most struck you?<\/p>\n [12:33] CB: It was at K\u00d6NIG GALERIE. A couple who had lost a loved one recently were sitting there. They had such an intense reaction that they were crying together, appreciating the artwork, and feeling the emotions. It was a profoundly sad and beautiful moment.<\/p>\n [13:02] BW: There\u2019s a popular meme of sentiment I want to pivot to. It goes, \u201cI thought machines would relieve us of manual labor, but instead they\u2019re painting and writing poetry, while I\u2019m worried about losing my job.\u201d Why have the arts found themselves seemingly in jeopardy first?<\/p>\n [13:37] CB: That\u2019s a really interesting question. First off, I don\u2019t really feel like the arts are in jeopardy from AI. AI removes many technical limitations for participation in the arts. While I\u2019m not a talented illustrator, I can use AI software to create illustrations. But artists have a unique ability to discern what\u2019s attractive and appealing. I don\u2019t believe you can replace the artist with AI. AI replicates what it has seen before; it doesn\u2019t innovate or think like an artist. An artist always remains the most vital part of the process. When we use AI, there\u2019s a strong influence from artists. AI lowers the technical skill barrier and increases productivity. For instance, a concept artist could produce hundreds of drawings per day instead of just a few.<\/p>\n [15:34] BW: I interviewed a concept artist, Andre Riabovitchev, who worked on the early Harry Potter films, and he\u2019s really embraced AI. Another artist here in Portland, Oregon, Chazz Gold, suffered a brain injury and lost the use of his right hand. He turned to photography, but AI became the tool that let him create as he did before his injury. I\u2019ve seen firsthand how AI has changed artists\u2019 lives. What\u2019s your view on artists\u2019 training data? It\u2019s a controversial subject.<\/p>\n [16:53] CB: Training data, like using art pieces to train a model, is a complicated issue. Data ethics in AI is a sensitive area. We\u2019re concerned about data sourcing, its use, and online collection methods.<\/p>\n If someone fine-tunes a model on a specific artist to mimic their work and publishes it, that\u2019s not the best use. The MoMA gave us rights to use their data to create our own artwork. Training a unique model and applying various processes meant we weren\u2019t copying another artist\u2019s work. It\u2019s about fair use: taking something and modifying it enough to make it new. We can mimic someone else\u2019s work with AI, but we can also diverge. The MoMA was significant because they recognized our work as new and different.<\/p>\n [19:00] BW: I want to discuss some specific Reef at Ganadal studio pieces, especially the Winds of Yawanawa. Can you tell me about that piece?<\/p>\n [19:17] CB: The Winds of Yawanawa is a collection we created in collaboration with the Yawanawa tribe, an indigenous group from the Amazonia. We\u2019ve been connected to them for a few years. Refik has visited them multiple times and deeply appreciates their culture and practices. For this project, we used patterns from their jewelry, fabrics, and clothing to create a new piece of work. Importantly, 100% of the proceeds go directly to the Yawanawa people. They\u2019re using these funds to build a new village, well, and school. It\u2019s been an inspiring project for us.<\/p>\n [20:15] BW: How did that relationship begin?<\/p>\n [20:18] CB: I\u2019m not sure how Refik first met them. He somehow connected with them, expressed admiration for their culture, visited them in the Amazonia, and traveled extensively to spend quality time with them.<\/p>\n [20:44] BW: Can you share more about their culture and what distinguishes them?<\/p>\n [20:52] CB: They\u2019re an indigenous tribe from the Amazonia, known for their herbal medicines and dedication to preservation. Their music and language are deeply unique. We\u2019re currently focused on language preservation as there\u2019s concern that languages from indigenous tribes are disappearing. We\u2019re exploring ways to safeguard their culture and language. They practice distinctive herbal medicines and have a rich musical and linguistic culture. They also have representatives who travel globally to share their practices. They\u2019re deeply spiritual and it\u2019s truly inspiring to witness.<\/p>\n [21:49] BW: You mentioned music and language. Are you working with audio, and what does that entail?<\/p>\n [22:00] CB: We work with audio, video, and text, which are our three benchmarks. Text encompasses some of the language projects too. We\u2019ve been involved in audio for a long time, using AI-generated audio in our studio. Often, in our installations, the background audio is AI-generated. For instance, we recently worked on a project with Dvorak, training a custom model to produce AI-generated Dvorak pieces.<\/p>\n [22:36] BW: That\u2019s fascinating. I recall a 1995 interview with Brian Eno in Wired Magazine. He predicted that someday you\u2019d request a new piece by, say, Brahms, and it would be created for you. This vision seems to be materializing years after his prediction.<\/p>\n [23:09] CB: Yes, the advancements in audio models over the recent years have been significant.<\/p>\n [24:03] BW: At the 2022 PyTorch conference, you and Refik hinted at some works in progress, including a Rumi piece combining Dreams, audio, and text, and a Mozart piece reminiscent of a project by Maria Finkelmeier, who uses Unreal Engine for song modeling. What\u2019s the status of these pieces?<\/p>\n [24:44] CB: Both of those pieces have been released as immersive exhibitions worldwide. Rumi, for instance, was showcased in Istanbul. It\u2019s an immersive room where Rumi\u2019s dance influenced the artwork, and we incorporated a rich image archive provided by the client. This project marked one of our initial uses of diffusion.<\/p>\n [24:52] BW: Can you tell me more about this?<\/p>\n [25:15] CB: We showcased this artwork globally. Traditionally, we\u2019ve used GAN trainings, but diffusion has become prominent recently. This was one of our first projects using diffusion outputs. Rumi received an overwhelmingly positive response, especially given its cultural significance to Turkey. As for the Mozart project, it took place, I believe, in Germany. We utilized a similar approach to Rumi\u2019s, incorporating generative audio from Mozart\u2019s recordings. We also integrated a valuable image archive of Mozart-related content, examining text and old songwritings. It was another successful installation.<\/p>\n [26:06] BW: How did you incorporate text in the Rumi exhibition?<\/p>\n [26:11] CB: We created a model from the text, which influenced the fluid simulation in our artwork. In some pieces, there are large sweeping motions. We connected words from Rumi\u2019s texts and other data using a network of nodes. A graph-searching algorithm was then applied to this network to influence the fluid simulation.<\/p>\n [26:39] BW: What is unsupervised scan or diffusion?<\/p>\n [26:45] CB: Unsupervised refers to GAN. What\u2019s interesting is that it\u2019s a real-time GAN. Inside the MoMA lobby, we added motion tracking, audio recording, and a weather station. For any AI model, you need input parameters to yield outputs. We harnessed real-time data from the space itself, using it as input parameters for the GAN model, which greatly influenced the art piece.<\/p>\n [27:11] BW: That\u2019s innovative. I recall a mention in the PyTorch chart about an EEG headset for mental health.<\/p>\n [27:26] CB: Mental health is crucial to our work. Art often has therapeutic effects. We\u2019ve been researching EEG data and recently ran a project at the MoMA where participants wore EEG caps, and we recorded their brain data while they viewed the artworks. This aimed to gauge the emotional and cerebral response to art. An exciting direction we\u2019re exploring involves the EEG data from the brain cap. We intend to use it to guide the art experience. So, if someone reacts a certain way to one piece, we\u2019ll tailor the subsequent content based on that reaction.<\/p>\n [28:39] BW: Are there partnerships with psychologists or medical institutions for this progression?<\/p>\n [28:50] CB: We had a medical researcher lead a recent project. While I can\u2019t provide specifics about partnerships, having medical experts on board is crucial.<\/p>\n [29:08] BW: Do you have a timeline for when we might hear more about this? I\u2019m intrigued by the intersection of EEG, mental health, and art.<\/p>\n [29:23] CB: We\u2019re aiming to release many of these tools at the start of next year. We\u2019ve been working on a project named Dataland. It represents much of our recent work, offering immersive experiences that highlight our capabilities. We\u2019re planning its release in 2024, when more of our developments will become public.<\/p>\n [29:47] BW: Could you elaborate on Dataland? Is it open yet, and how can people access it?<\/p>\n [29:56] CB: I must be discreet about it. However, we\u2019re establishing a physical space in downtown LA, slated to open within the next year.<\/p>\n [30:09] BW: That\u2019s exciting. I\u2019ve often heard the term \u201clatent space\u201d concerning AI. Could you clarify its meaning for our audience?<\/p>\n [30:24] CB: Absolutely. Taking the GAN model as an example, when you train an image-based model like a GAN, you have input data consisting of real images and an end component comprising computer-generated images. Latent space is the intermediary area that translates the input data into these outputs. Picture latent space as a 3D field you navigate. Depending on where you are in this space, you\u2019ll encounter various outputs. For instance, one region might produce images of flowers, while another offers images of buildings. Essentially, latent space is the bridge between input data and computer-generated images.<\/p>\n [31:38] BW: I\u2019m familiar with the difference between diffusion and GAN from past research, but I\u2019m a bit rusty. Can you clarify the distinction and the process of choosing between them? Is it akin to choosing between oil and acrylic?<\/p>\n [31:58] CB: Your comparison is apt. Diffusion models are text-to-image-based, so they focus on the prompts you use and the resulting outputs. GANs, on the other hand, are image-to-image-based. Unlike diffusion, there isn\u2019t a direct input query in a GAN to obtain a specific output. That\u2019s the primary distinction, although there are other technical nuances.<\/p>\n [32:34] BW: I\u2019m intrigued by your work with AI and scent. Can you shed light on this area and its future direction?<\/p>\n [32:47] CB: AI in scent is indeed fascinating. Beyond just scent, our research has revolved around the \u201chyper multi-model\u201d concept. We aim to explore different facets of a single concept \u2013 like a flower \u2013 through AI. So, considering a flower, we\u2019d investigate its AI-generated image, scent, sound, and even texture. This multifaceted approach offers diverse ways to interact with models and derive various outputs. For scent, perfumes are crafted from a mix of bases and chords. Depending on an image\u2019s representation in a GAN model, we can convert it to a chord series and formulate a perfume accordingly. A memorable example is a Bulgari installation in Italy. It involved a mirrored cube outside a Bulgari shop, where we introduced scent into the room in real-time. As viewers watched projected visuals, we had a scent machine delivering corresponding fragrances, such as floral during a flower scene or aquatic notes during a water scene.<\/p>\n [34:34] BW: How do you train a scent model?<\/p>\n [34:38] CB: Training a scent model is unique. Without revealing too much, in the AI realm, data can be converted into a number series, which is foundational to data science and machine learning. An image is transformed into what we call an embedding by processing it through a neural network. This embedding\u2019s data can be used in various ways, essentially crafting a translator that turns an image vector into a scent.<\/p>\n [35:24] BW: So, can that embedding be input into scent machines to produce the actual fragrance?<\/p>\n [35:34] CB: Exactly. It\u2019s akin to translating from one language to another. The image\u2019s embedding is processed by the translator, which then instructs on the needed quantities of various chords to create the scent.<\/p>\n [35:51] BW: That\u2019s incredible. The scent technology is mind-blowing. And you mentioned texture, can you elaborate on that?<\/p>\n [36:03] CB: Some aspects of that are still confidential, but we\u2019ll share more openly at Data Land next year.<\/p>\n [36:14] BW: That\u2019s exciting news! On another note, have you begun exploring Apple Vision Pro\u2019s applications and its implications for art?<\/p>\n [36:28] CB: Apple Vision Pro is truly groundbreaking. The standout feature they offer is the quality of the visualizations. Currently, high-resolution VR requires a powerful PC and a top-tier headset, both connected. In the AI art context, display quality greatly impacts the overall experience. Viewing digital art in low resolution isn\u2019t ideal unless it\u2019s a specific style like pixel art. The Vision Pro promises significant changes in content presentation, and I\u2019m eager to see its evolution.<\/p>\n [37:42] BW: So you haven\u2019t started working with their OS yet?<\/p>\n [37:52] CB: No comment.<\/p>\n [37:55] BW: I\u2019m trying to uncover a lot of secrets from you.<\/p>\n [37:58] CB: I\u2019m sorry.<\/p>\n [38:05] BW: I\u2019d like to understand the ideation process behind many of these projects. Do you begin with finding interesting data, or do you have ideas first and then seek the required data?<\/p>\n [38:25] CB: It varies by project. There are generally a couple of main types of projects we undertake. One is self-directed, where someone, often Refik, suggests training a model with specific data, like Coral data. We then search for and collect the necessary data and proceed with training. There are many online resources with open-source data available, Flickr being one example. Another approach involves client-specific projects. For instance, AT&T Dallas provided us with data they wanted visualized, and we developed visualizations based on that. These represent the two primary ways we categorize our projects and source our data.<\/p>\n [39:37] BW: Can you tell me about the AT&T project?<\/p>\n [39:41] CB: Of course. The AT&T project in Dallas involves a massive screen on the exterior of their downtown building. We collaborated on several pieces for this space. One involved unique visualizations of cellular data that were transformed into art. Another was for Clint Eastwood\u2019s 50th anniversary celebrations, where we showcased images from his archive. The works are displayed on the AT&T Center\u2019s external screen in downtown Dallas.<\/p>\n [40:17] BW: What\u2019s been your favorite project so far?<\/p>\n [40:24] CB: MoMA is undoubtedly a significant project we\u2019ve handled, both historically and personally. Another favorite of mine was the inaugural project I participated in, Machine Hallucinations New York City. It was my introduction and involved an impressive dataset that was quite innovative for its time. Nature Dreams at Conan Gallery also stands out due to its extraordinary exhibition in a unique space. And then there\u2019s Machine Hallucination\u2019s Coral Dreams, where we set up a giant screen on a Miami beach, offering viewers a unique beachfront experience.<\/p>\n [41:26] BW: As a collaborator with Refik, do you have personal projects you work on?<\/p>\n [41:33] CB: Given our busy schedules at the studio, I rarely find time for solo projects. I head the AI and data science team, as well as the web and web3 teams. A lot of my side projects focus on bridging the gaps between these teams to enhance the experiences we offer.<\/p>\n [42:02] BW: What\u2019s the forecast for machine learning and artificial intelligence in the upcoming year?<\/p>\n [42:10] CB: The trajectory for AI has been unpredictable over the recent years. One of the driving forces behind AI\u2019s rapid advancement has been its growing accessibility. Previously, AI\u2019s reach was limited as it wasn\u2019t user-friendly for those without technical expertise. Our emphasis is on crafting tools to mitigate this. For example, we\u2019ve developed the GAN Browser, allowing users to navigate latent space and interact with a GAN model using just a PlayStation controller. This kind of accessibility has empowered even kids to engage with AI models.<\/p>\n AI tools like ChatGPT have gained popularity because of their user-friendly nature. Looking ahead, I believe the focus will shift towards offering broader access to these models while ensuring transparency in their operations. This includes educating users about data sources, collection processes, ensuring ethical practices, and fostering a beneficial, informative interaction with AI.<\/p>\n [43:55] BW: What can our listeners look forward to from Refik Anadol\u2019s studios in the near future? Besides Dataland, is there anything else?<\/p>\n [44:10] CB: We\u2019re diving deep into projects in the web3 space. Recently, we\u2019ve activated burns for many of our collections, transforming still images into moving artworks. This has been positively received by our collectors. We have several studio projects releasing soon. In fact, this morning, it was announced we\u2019ll showcase a collaboration with the weekend at the Sphere in Las Vegas. We\u2019re thrilled about this, so do check out the article if you haven\u2019t. Several other projects in the web3 space are on the horizon.<\/p>\n [44:56] BW: That sounds exciting. I\u2019m especially eager for Dataland. Might have to plan a trip to LA.<\/p>\n [45:02] CB: We\u2019ll be unveiling more about Dataland in the coming six to eight months. We promise it\u2019s going to be intriguing.<\/p>\n [45:10] BW: It\u2019s been great having you on Pixels and Paint, Christian. Anything else you\u2019d like to share? And where can our listeners connect with you?<\/p>\n [45:27] CB: I appreciate the opportunity. My advice is simple: embrace AI, but use it ethically. For those wanting to connect, I\u2019m on Twitter as@christianburke0, and also available on LinkedIn and Instagram under the same handle.<\/p>\n [45:49] BW: Thanks for joining us. I\u2019m genuinely excited about visiting Dataland.<\/p>\n [45:55] CB: Thank you. We\u2019ll be waiting.<\/p>\n Listen to the episode on Apple Podcasts<\/a>, Spotify<\/a>, Overcast<\/a>, iHeart<\/a>, PlayerFM<\/a>, Podchaser<\/a>, Boomplay<\/a>, Tune-In<\/a>, Podbean<\/a>, Google Podcasts<\/a>, Amazon Music<\/a>, or on your favorite podcast platform.<\/p>\n Read the Show Notes<\/a><\/p>\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\nEmotional Responses to Public Artworks<\/h2>\n
\n
\nData Ethics<\/h2>\n
\n
\nDiscovering the Yawanawa<\/h2>\n
\n
\n
\nFrom Rumi to Mozart<\/h2>\n
\n
\n
\n
\n
\n
\n
\n
\n
\nDataland<\/h2>\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n