Shari Frilot, Curator of New Frontier at Sundance Film Festival and ArtPrize's Kevin Buist delve into exciting developments at the intersection of filmmaking and Virtual Reality technology, discussing the dynamically immersive potential that these new tools offer to artists and filmmakers.

Kevin Buist: For those who aren’t familiar, could you explain a bit of what the New Frontier section of Sundance is and how that fits into the overall event?

Shari Frilot: New Frontier is a section of the Sundance Film Festival that is dedicated to featuring works that really experiment with, expand, and traverse the boundaries of traditional narrative storytelling. The section is comprised of two showcases: one is a lineup of films that are presented in theaters, and then the other showcase is a venue that we build to present an exhibition of different kinds of works, manifesting at the intersection of art, film, and technology. Performance art and music is a part of it, too.

Sundance is a discovery festival and New Frontier allows us to follow the artists as they radically innovate the form of storytelling. Once we realized that interconnectivity in digital culture was greatly affecting how storytellers manifest their work, we built this showcase to augment the film lineup so that we can keep up with the artists and the voices that were coming on the scene.

Courtesy of Sundance Institute.

KB: This year’s New Frontier featured a lot of virtual reality. Could you talk a little more about that and the potential that that technology offers to artists and filmmakers?

SF: The aim was that New Frontier would contribute to a more capacious and expansive media ecology. It was a way that Sundance could not only meet the opportunities that storytellers were already embracing, but also serve to create new opportunities through provoking and promoting innovation. That’s been happening, and the revival of Virtual Reality is a great example of that happening.

In 2011 a journalist came across my desk: her name is Nonny de la Peña. She is an accomplished journalist—had written for Time, Newsweek, and other major outlets—and had just joined a USC program for professional women who wanted to evolve and expand their practices by embracing a different field of practice. And so she had come to USC to do that, with a specific interest in creating more immersive ways of reporting the news. I caught up with her and paid her a studio visit.

In addition to the work she had been doing in Second Life, she had been working with a creative technologist named Palmer Luckey, who was a teenager at the time, to create a kind of virtual way of telling the news. She was focusing on this report that she had been working on about hunger in Los Angeles. For this particular piece, she captured a live recording of a hunger line in LA where a guy fell out from a diabetic coma, and the ambulances came and they took him away. She then took that live audio and worked with Palmer, who was a big virtual reality (VR) nerd coming from the gaming world, to create a 360-degree simulated world of the hunger line using Unity game engine, and formatted it for a VR prototype that Palmer luckily was developing. So here is a journalist coming together with a technologist from the gaming world to create an entirely new way of telling the news.

When I experienced that piece, I was blown away—if I’d read this report in the news, I’d be like, "Well okay, that happened." Even if I’d seen it on TV I’d be like, "Wow look at that," but when I got inside the goggles of this prototype, this incident happened to me! It wasn’t that I learned about it, or just saw it, it actually happened to me: I was there, this guy fell out right in front of me, I tried to get him to the front of the line and a woman yelled back at me. It was interactive, so it was a tremendously powerful way to communicate this incident that happened.

I spoke to Nonny afterward and told her, "This is amazing, but it took this massive room, with a bunch of motion sensors looking at me, and me wearing like this massive backpack and helmet and people carrying cords behind me and it took 15 minutes to do it. How are we going to bring this to Sundance with all the lines and people? And how is this going to address your mission as a journalist, which is to educate the wider public?"

She was like, "I don’t really know about that, but what I do know is that what you felt experiencing this report is what I’m going after. That immersive feeling, the fact that it actually happened to you, that you were in that person’s shoes, that’s what I’m going after."

The passion was so clear with her that I decided to take the leap and brought it to New Frontier in 2012. It was phenomenal to see the audience response. People were deeply moved—a number of audiences were crying, dropping to the floor to help this guy. It was kind of a revelation, in terms of how effective this prototype was as a way to redefine what it was to tell the news.

KB: Reading up on what’s been featured at New Frontier recently, particularly thinking about Virtual Reality, I was thinking about it in terms of the intersection between new media artists—contemporary artists who are using new technology— and filmmakers. I hadn’t really thought that much about journalism, but it makes sense, because they’re thinking about narrative.

SF: The focus was on storytelling and finding innovation in visual storytelling at this intersection. Because the evolution of the digital landscape essentially expanded the role of media makers—they would not only create works that would feed the entertainment industry, but they would effectively create the visual language individuals would use to communicate with one another in a networked society—the community of storytellers New Frontier would intersect with expanded tremendously. I started engaging with architects, technologists, journalists, data-visualists; all kinds of different visual storytellers would land in the crosshairs of our exhibit.

In terms of the VR story, the 2012 edition of New Frontier brought a lot of press attention to what Nonny and Palmer had achieved. Palmer was interested in creating a consumer version of this headset, so he capitalized on this success by establishing a Kickstarter after the festival with a set a goal of raising $250,000. He raised $2.5 million on that Kickstarter and immediately went into production on a consumer headset that would become the Oculus Rift.

I circled back to him in 2013, once he had that prototype up and had started going to gaming conferences and challenging developers to create content for it. I was like, "What have you got? We have to see this thing through!" Joe Chen came to the office and showed me everything Oculus had and I made a selection of 4 works for the 2014 festival, including showing 3D films in a VR cinema app, called Oculus Cinema, and a high-end multiplayer game called Eve: Valkyrie.

I was also talking to Chris Milk, who is ever the early adaptor, as he was figuring out a way to retrofit a 360 degree video that he shot of Beck to format it for the Oculus DK1. Jonathan Minard and James George, the filmmaker/technologist team that created the interactive modulated documentary called Clouds that I selected for the showcase that was made entirely out of code, also seemed right for formatting for the Oculus, so I encouraged them to expand their work and create a module formatted for VR.

So in 2014 we had a small line-up of work on Oculus' first edition development kit, focusing on storytelling. In November before the ‘14 festival, Oculus managed to raise a $75 million round of financing. That’s when I realized, ok this is not just amazing and fresh tech, this is going to be big. This new medium is coming at us like a tsunami. So I decided the thing to do was not only showcase the work, I would also showcase the technology as something important for our festival audiences to pay attention to.

Two months later, Facebook bought Oculus for $2 billion, and the field of VR has been leapfrogging, taking quantum leaps every three months since then, with major companies like Samsung and Google and Sony jumping into the fray. When 2015 came around there was a riper field of artists and technologists coming to work together to use VR to tell stories. I continuted to encourage experiments in putting technologists and filmmakers together such as Rose Troche and Morris May, who made the VR work, PERSPECTIVE.

The focus of the 2015 edition of New Frontier was not so much about the VR technology as it was about the different ways VR could be engaged with to tell stories. I put a lot of effort and thought into creating a very diverse lineup that reflected different stories from different fields of practice. The development of VR had been so quick—storytellers, technologists, and industry were all muscularly embracing the medium—I felt like we were on the precipice of an historic moment. New Frontier was in a unique position, being ingrained in the story of VR, so I made the decision to build a slate that majorly engaged the technology. I selected 11 VR-driven works, hoping to show festival audiences the different kinds of ways that VR is being engaged to tell stories.

Courtesy of Sundance Institute.

KB: There’s a lot of talk about the current moment being on the cusp of something really big with VR. Do you think that is accurate or is it industry hype driving that conversation? Is it really the dawn of a new age of storytelling?

SF: Well, I think it’s both. Certainly the industry has been waiting for something like this for a while, and they have made a pretty deep investment into this. VR absolutely has the potential to affect our society in pretty deep ways. If you step back and look at the technology itself, it’s profound.

In general, stories told through media effectively serve to rewire our brains. Radio has done this, television has done this, our mobile devices have done this, but not in the way that VR does it. VR is able to engage with a part of our brain that has to do with the ancient evolutionary story of what our body is and where our body is in space. So it engages an unconscious level of our neurology. Because of the neuroplasticity of our brains, the technology has this tremendous deep interaction with how our brains work and how it forms in various ways. It’s a profound interaction, and presents a set of challenges other mediums don’t (i.e. motion sickness), but it also presents enormous opportunities.

I do think this is a pretty significant corner turned in how a media that is poised for mass access can really affect the formation of our brains and the formation of our society. But let’s see how it goes. Right now, it’s all in its infancy—it’s a baby industry and art form right now, even with all of these big names attached to it. And industry has advanced past the state of content creation, so right now it’s all about content creation—the stories, the games, and the applications. Sundance is always going to be interested in engaging with the storytelling aspect of this. And right now, it’s a golden age for content creators. It’s an open field. It’s a brand new world.

KB: Thinking about this stuff in the context of Sundance leads me to think about it as a new medium for cinema. But there’s also temptation to think of this as a platform for gaming, or contemporary artworks. The more I think about that, and the more I listen to you talk, all of those categories could be insufficient for what it could become. Do you see it as an extension of cinema? Is that how it plugs into Sundance? Or is it breaking from that?

SF: We’re about cinematic storytelling, and so we’re engaging this medium on that level, but I think Mark Zuckerberg even said this—and I actually do believe its true—VR is not so much a media platform, it is a form of computing. Computing has changed our lives. Computing is how we watch movies, how we learn our news, and how we communicate with one another.

Instead of plugging into a laptop as a way to access the internet, we will likely access social networks through VR. Hospitals are embracing VR as a way of creating therapy for patients. So kind of like when YouTube became the second-biggest search engine for knowledge next to Google (then Google bought them), VR doesn’t have only to do with the entertainment business, it brings the business of storytelling into the center of every business.

KB: In this moment, like you said, Microsoft, Google, Facebook obviously, maybe even Apple…

SF: Samsung.

KB: Yeah, Samsung. So they’re making these big moves into it. Are you concerned at all that we could be in this brief golden moment for independent creators before it gets very big and commercial?

SF: Hmm, I’m not so much concerned as I am excited about that brief golden moment! [laughs.] You know, life is made of golden moments. We have the moment that we have now and we make the most of it—that’s all we’ve got anyway. So, yes, tomorrow is not going to look like today, certainly. I’d say in another year or so, the opportunities that content creators have right now will look very different in two years. So it is a very special time; I want to speak to that. The time is now to jump into this stuff because it’s a wide open field.

KB: To critique my own question, as big and bloated as the Hollywood machine is currently, things like Sundance still exist to help small independent voices find an audience.

SF: We can assume that a structure will emerge within this field that will effect access to opportunities and possibilities, but what we can’t assume is what that structure is going to look like, or that it will look like anything we’ve seen so far.

Technology has affected how movie companies have developed, and we’re just starting to see that take off in terms of studios vying and establishing online identities and points of access to their libraries and new properties. We may see some carry-over of that model—the online model, which is still relatively new for studios—into the VR space.

But interestingly, VR kind of came out of journalism. To watch the newspapers and publications in the last several months embrace VR as a platform, and reconceiving themselves ramping up their structures to be media distributors, this shows how the future is not going to look like the past. It’s provoking companies that were thinking one thing about themselves, and they’re now thinking about themselves in a totally different way. We’ll see what happens, but I do think there will be a reshuffling of industry that’s going to feel pretty fresh, at least for a while.

KB: You’ve been working with the blending of film and new media for quite a while. Can you explain a bit of your background, and how you think the landscape for independent storytellers, artists, and filmmakers has shifted?

SF: Well. I was an engineering major before I decided that I wanted to make artwork with moving elements in it. That’s how I came to make films. Once my films started being selected for film festivals, I wasn’t very happy with how my films were exhibited, and that’s why I was provoked to take on creating a festival to create platforms I would be excited by as a filmmaker.

I brought all of that with me, my interest in technology, my art practice, my interest in the cinematic moving image, my interest in systems. My personal journey was to create spaces for the kind of work that I did and create opportunities for the communities that I belong to. I belong to a class, a whole cohort, of artists looking and plotting in ways along the same realms.

So much has shifted! Certainly, over the course of my own career, the level of interconnectivity has exploded. I remember having to hack into the internet every time to log in—it was before Netscape. Now, logging onto the internet is completely taken for granted at this point. And how we communicate with one another in society is rapidly moving from the written form and cinematic events to images themselves being the packets of info that we exchange between one another. Ten years ago we weren’t witnessing entire structures/buildings being projection mapped, and transforming the building into some kind of narrative landscape. That requires the kind of computing power we have today—it requires even the vision to be able to do that, that comes out of the technology and the evolution of the media landscape.

KB: Right. In between the Sundance festival that happens every January, for the rest of the year, what does it look like for you? Where are you going? What are you looking at? How are you scouting the projects that end up at New Frontier?

SF: Every year it seems like it looks different. I used to have time off in between festivals, but now there is no time off. I travel around a lot, I attend art and technology Biennials—I went to Venice this year, I plan to attend ISEA in the summer. I also do a lot of studio visits all around the world.

I’m talking to media professionals and colleagues in various fields, I go to companies that are developing new technologies, startups—checking up to see what they’re doing and how they’re doing it. I surf the web a lot. I work with a researcher who searches the web with me.

I talk to creative communities and, at the same time, I’m listening to where our film community is at Sundance—what is fascinating the filmmakers, the storytellers. What do they know, what are they not aware of? I’m crafting the program to meet that core audience of ours: filmmakers, film professionals, and cineasts.

The show that I generally put together is not as much of a conceptual show as it is cinematic. The work that I bring into this lineup must speak the language of cinema, so that when audiences walk into the New Frontier space they don’t feel like "I haven’t read the right books to go to this show." It has to be a continuous experience of leaving the cinema and entering the New Frontier space and thinking, "Oh, this is the same language, but in a different form."

By Kevin Buist on