Jessica Brillhart, Immersive Director, on VR and AR

So, you started your company this year. My great question: So, this actually ties into my past, actually. I was at Google for years. I started as their first filmmaker with the Creative Lab. I moved on five years later into the Google VR team, which is now the Google AR VR team. I don't know; they might be different now. I don't know, but so I became the principal filmmaker for VR at Google, which again is, they talk about titles, yeah, the fanciest of titles. It was very cool.

So, I helped develop Jump, which was their VR live-action and kind of capture ecosystem for live-action VR footage. I was making stuff and working with the engineers, and the more that I was working there, I think I found it was very, very tough to be reactive. In the beginning, it was super easy, because it wasn't like VR was still pretty new, and no one really knew what we should be doing with it yet.

Then, once it started to solidify, it was much harder to turn the big ship towards the things that I thought were important. Namely, you know, I thought that there were something really interesting parallels between the machine learning team and the Google Brain team and also what was happening on the VR team, and just in terms of the mediums, how similar they were both in how they become more present in our lives. How they both kind of were like, "Oh, everyone was really excited." It was all, "No, it doesn't work," and then it was like, "Wait, we found this weird thing that actually makes it work great."

So, like for VR, it was like cell phone technology. Actually, we have it in our pocket; we can make this stuff work. And with machine learning, it was like, "Right, logic is wrong." There's actually preventing us from doing the right thing and teaching these systems how to learn.

Suddenly, they both are on this trajectory, and from a creative standpoint, you could see some really interesting stuff coming from both teams. I felt that there were a lot of ways that we could work together on stuff. I also felt that there were lots of interesting pockets of artistic pockets where we could create content. It wasn't going to be like, you know, with the big studios; it wasn't going to be like the big IPs. It was really going to be in these places that needed it probably more than Hollywood.

Thinking about ways that it could be functional and helpful for people, yeah, but also be artistically—you didn't have to be boring, and it's not just functional. Yeah, precisely. Function is actually good, because, like, films actually serve a function; it serves a purpose. You know, and yeah, VR could also do it. Any sort of immersive content can.

I thought it was all, and I had before also like worked with—I had worked on some stuff kind of envisioning what the future would be like when Glass was around. Yeah, and it clearly worked, because everyone now is like, "Glass? No, we're great." But for me, it was like I'm really interested in this other layer of immersive stuff. But I also believe that it's not one technology or another technology; I believe it's all kind of going towards something— they're all networked together.

So, a combination of that belief and wanting to be more reactive, and honestly, just working on the stuff that I felt was important to me to work on, once that all sort of became more clear, I felt that once I had left Google, that the best course of action for me was to actually create my own company. So, 'For a Pictures' is the company we crave for short and has all the letters in it.

Yeah, meaning 'weight crai crai,' but I didn't want all the letters in my name, because I always thought that was a little weird for me. I mean, it exists in the ecosystem, and some people wear it really nicely. For me and my company, I was like, "I don't know." Then I was in Paris with my partner, and he had said, I had asked him, you know, what's the word for "true"? He had no idea. I was looking for a name, so he immediately was like, "Hold on." He took a pen, he wrote it down, he goes, "That's what it is."

And the word for true in French is 'vrai,' so VR AI means true—it actually means real, depending on how you use it. So, at that point, I was like, "Okay, well, I can't not," and so, you know, this makes absolute sense. I started the company in January, and it really, really does in a very odd year but wonderful way, reflect the stuff that I care about, which is how all these letters, all these—you can pick it apart even and think, you know, it's visual arts, it's mixed reality, it's all these various things that we talk about will all come together, work together in really wonderful ways, and actually lead to whatever this new immersive environment will be.

I think, you know, right now we're seeing each as like these separate paths. I think it's really not going to matter. I think you see the same thing happening in science. It's happening right now where, you know, physicists and mathematicians and CS people are all learning Python, and like, it's converging into one thing, right? Just their computers, right?

So like, it may be your deep, deep, deep dream project is like the most clear example, I think it is. Yeah, I mean, that was really funny, because that's, so in Seattle, so the Jump team I think is still in Seattle; I don't know. But the engineers that are working on the computer vision people that are working on the Google live-action VR rigs are planted in Seattle at the same office. It used to be on the same floor as the Google machine learning team. That was literally adjacent, like next to each other, never really not intentionally.

Well, they're both computer vision teams, okay? So, I think that was the whole—that was the reasoning behind. I don't know. But I would visit the Seattle team, and then I was fortunate enough to make friends with this particular machine learning team AI team. I was run by this guy Blaise Aquaria—our cue is he—he was amazing. So, him and some friends of mine who were working on this team were literally sitting two feet away from us.

I had gotten an email from Clay, actually on the Google VRI team, saying the ARVN team or whatever had emailed me and put me in touch with them, saying, you know, they’re actually announcing Deep Dream, and they really need a video person to help them out. So, you know, they were asking me what I had. I originally just showed them stills of the stuff that I actually filmed in VR.

We started having this conversation around—I was trying to push on them the idea that, well, what if you're able to dream up on top of the stills of these VR clips, then surely you could do it in such a way that we can actually experience it in VR? Yeah, and it was a bit of a back-and-forth thing. I'm like, "Well, the fidelity might not be that great; it will be low res, I don't know if it's gonna be interesting."

Then finally they dug. Fritz, who was working on the team at the time, he actually gave it a shot, worked his magic, and we were like, "Oh, this is actually kind of compelling." Yeah, we don't know what it is; I think we're just like dreaming up one stuff. It was really fun because I think there was no expectation.

It didn't fit on a roadmap; it was literally just us working together to see what we can come up with. We actually worked with Russ Goodwin on having his system, which was trained on Faulkner and Vonnegut separately, to actually write prose about what it's all. Then it would recite it on using this voice, Myra, which is an Irish-speaking, Irish dialect-speaking Apple voice, you know, a backup Mac OS voice, which made it really just unbearable, because it was like—and we both sort of like, this is kind of interesting, but it sort of calls to mind like "less is more" in a VR space, you know?

That's like once everything's like kind of acid-trippy, like hearing this like Irish American-speaking fabricated lady speaking Vonnegut interpretations to you just isn't ideal. Right?

Okay. How on earth? So, we sort of used it with caution. Yeah, I mean, it makes sense when you like change out every variable; it just completely throws you for a loop. Yeah, it's all kind of closed but sort of nonsensical. It is. I mean, I think what's great about what Russ does is that it actually does—it is very poetic, but it gets dark very quickly, because, well, Vonnegut is a little bit more upbeat, but Faulkner is, you know, like, yeah.

So, I consider her as "she," because I hear Myra all the time every day. Yes, but she sort of says these phrases that are like sometimes kind of like normalized about like the world, like, "You know a man with a hat on a hilltop, you know, why is he there? What is he doing?" She'll go off on that, and then she'll be like, "The darkness... it was for me," or something, and you're all just like, "This is weird."

Again, it's like us and Turk. I mean, what's interesting is that it's all pretty surface stuff, and the depth is stuff that we bring to it, which I really thought. I think that part, to me, is very fascinating, like how we interpret it is really where the story and the emotional, like, foundation comes from.

So, this is actually like a big topic I wanted to jump into, which is like framing, right? So, in cinema, obviously, it's a director and cinematographer working together to create the picture that transmits a story to you.

Sure, but in VR, like obviously, you have a certain amount of control, but how do you think about that act, like, the framing in the storytelling? Oh, so I don't think it's storytelling, and I think that's the main problem is that people think, you know, I think story is extremely important to the medium. I think storytelling is kind of like—it's sort of like you want to get to something that's important, so you take a pill to make you feel that way.

Like, say like a sleeping pill: "I need to go to sleep; I need something to help me sleep, so I take a sleeping pill." I don't recommend that as something you should do; it's just something that some people do. I think storytelling is the act of taking that pill.

So, it's like, yeah, you may take that pill; you might, you know, you might exercise before you go to bed. It's a means to an end, but it itself is not the end. So, for film, storytelling was the way that you got to story, and a lot of mediums, the telling part was actually really important because you couldn't actually take those people and put them where you were thinking.

And now you're in a medium where you can, so telling doesn't really work anymore. You don't need that. It's like if you think of it as, you know, kinetic versus potential story, right? So, kinetic is storytelling; it's the idea that actively I am trying to take you and put you over there, so it's this kind of active thing that's occurring.

For VR, the way that I've been looking at it is it's more potential story; it's I'm crafting worlds where if, when you go there, the story is something that you can decode, right? So, it's like how do these worlds transmit the story or represent the story, right? So that, no matter how you interact with that world, you're able to sort of decipher or get to the core values of what that story's trying to transmit.

Okay, it's a bit in the same line as in like an Aesop fable in a way; it's like slow and steady wins the race. If that's what you're going for, it's not about the turtle or the bunny; it could be anything. Yeah, it's about that's the core value of the story; that's what we want the takeaway to be.

And so, in my mind, in VR or any immersive content, if you can get that piece, that truth, transmitted to that person, it doesn't matter what kind of experience they have; they walk away with what was most important. You're in the first place.

I mean, how do you think about that in the context of, you know, video games? Do you think that is just a transition to immersive content? Do you think it's—it's fair for certain games? Already, I think certain games provide.

What's interesting about games for me is this idea of flow, which is it's very musical in nature. The best games are where they give you the capacity to explore at your leisure. You know, some of my favorite games have been like something like Myst. You know, I actually loved Red Dead Redemption a lot, because, you know, yeah, okay, it's really stressful this, like, this like rail that we're putting you on.

Maybe you just want to take a break and train some horses over there, and that's fine. Then you can duck back into the story when you want to. This idea that you can be in a completely explorable place—you can do all sorts of things that you want, but then there is this rail that you could go on and duck in and out of—I think that to me is very compelling.

Myst had that same sort of rail as well. It is definitely a different kind of game if it's like, okay, you do have this, and that's all you can do, and you can't move forward until you do that, like a Super Mario Brothers is an example—any of the earlier games, literally, yes.

Yeah, that's cool, sure. I mean, I think that's something that we need to be aware of as creators in the space too when we talk about games. We don't mean if there's no—it's not a catch-all; like different games serve different purposes.

Pawns in different in Super Mario Brothers which is different than Myst, which is different in some ways, or a lot of ways, from Red Dead Redemption to like Red Dead Redemption at least gives you new elements that you're used to dealing with. Like, it works sort of similarly, the way the world works.

Like, okay, that's a gun, that's an evil person, that's like, it's easy to kind of decipher it, and Myst, the whole point was that you had no idea what you were doing pulling levers. The first thing you do is there's a book on the ground, and then you go and then there's a lever, and you don't know what the lever frickin' does, you know, and nothing happens.

You're like, is this gonna be the rest of the game? Yeah, and it is! And many of those games, like pre-internet, were so hard; now you just go on YouTube and it takes two minutes to figure out what you're doing.

Yeah, I know. I know, actually, when am I—there was an old game from the 90s called Amber: Journeys Beyond, which actually has been an inspiration on a recent project, and it's really great, because it was just a really great game. It was sort of a Myst-ish copy made by two guys; I don't think they made anything after that.

But they ended up—we can't play it anymore, or at least you have to have an emulator and a piece of hardware that will help you run it. But I have a bunch of playthroughs on YouTube, and I actually sat there with a glass of wine and watched my—you know, I remember it was such a beautiful experience, actually, where I'm like, okay, I actually love watching other people play these games, and the commentary is really beautiful too.

I think one of my recent—my own and my recent favorite games is from Davy Reid, and the Beginner's Guide. It's wonderful because it literally sets it up and says this is a 45-minute game, because most of these games, I really don't know who's gonna take off my life.

But it's basically, you're on rails, but it's him, and he kind of gives you—he voiceovers and it's about this other game creator who's created kind of these weird psychological, like, very basic games. So like it's weird; it's like he built this character that you never see, and the game—this character is someone you explore through the games—pieces of games as his character is you, yeah.

And Davy's taking you through it as if he knows this person, but then you start to question whether or not this narrator actually knows this person or not or if this person gave Davy the permission to showcase his games like this.

So, suddenly you start to question everything as you're going through this like kind of game. It's, I think it's really brilliant; I think any game that kind of takes the format and then through your interactive—through your agency, kind of makes you feel these different things is, uh-huh, I think that's a huge win for games especially. And I think that is where immersive needs to go.

Florence, which I think just won the Apple Design Award for like best designed game, this guy Ken Wong, who was the designer on Monument Valley as well, started his own company in Melbourne. He's an amazing creator, and this game Florence is all about using—you use your phone the way that you would normally use your phone, but the story—you feel for the story based upon the way that you would feel for like conversations and interactions with your phone.

So, moments of frustration is like, you know, not hearing back from the person for like three minutes, yeah, like that kind of thing; they can see all those things. Yeah, absolutely, like how fast you reveal things, you're like— the whole idea of swiping; it's just very smart. Yeah, and I think that's the thing—it's not we try to shove a lot of this stuff through preconceived notions and conventions; we're embracing how technology works now and using those limitations as a means of telling a story in some capacity or understanding a story in some capacity.

I think that's really fascinating. Have you played—the game? This is really embarrassing because I met the creator. I think it's called like Black Box or something, the iPhone game, where's all the puzzles? For instance, like one of the puzzles, you have to put your phone in, and the temperature has to hit certain, and then you unlock it. That's brilliant!

Yeah, that's good stuff like that where I'm like, that's just so—because it makes you question your relationship with a device, totally. You know, and I think that's an interesting thing, because that story—yeah, is potentially as good as some of the films that you've seen. Of course, you know, and it again, it's like it's not 1:1 but it still gets you to the same place mentally and emotionally.

So, okay, so when you left Google, did you have a project in mind or were you just going to client stuff? I wasn't sure what I was gonna do; I just knew that I needed to go. I think that what I ended up doing first was I took—I kind of got a lay of the land, sort of explored what other people were doing, talked to some folks, thought about the things that I wanted to do.

I think for me, I needed a little bit of time to sort of get a sense of what was out there. It's difficult, because when you work for a company for that long, you're like, oh okay, like my problem was water, water everywhere but not a drop to drink, kind of where it's like, "Oh, I have all this stuff."

Yeah, but I can't do anything with it. And then one of my limitations was like—beyond that was, you know, I had seen where interactivity was going, a lot of ideas for that. I could see where augmented reality and VR could talk to each other, where machine learning techniques in VR could talk to each other. I couldn't do any of that other than the stuff that I could kind of sneak through.

I think in frustration, I made what I believe to be sort of like a VR GIF-type thing, which was the Weather Channel project that I did. That was my favorite one. I know I don't need to be down on, you know, there are other really ambitious ones, but that was my favorite.

Yeah, well, it's most people's favorites because they get it; it's like immediately apparent. It's—and it's also like it's hilarious. Yeah, yeah. And it's one of the things that people are always asking was, "Well, how do we make comedy in VR?" They try to bring comedians on it, and I'm like, it doesn't have to be a literal thing.

You know, it doesn't—I mean, it does not have to be like slapstick or like here's, you know, frickin' like Jerry Seinfeld telling you a joke. Like it's literally just like, what is the human condition? Like what have we created for ourselves in light of spaces and experiences that are just inherently funny?

Oh, and the Weather Channel just happens to be like watching the Weather Channel. It's one of those things—oh, totally! It was very Adult Swim, that's like the little infomercials. Yeah, it was really great, and I think that to me—and I made that in a day.

Yeah, that's not surprising. No, but I mean it, considering how long usually it takes to make VR, it's insane, and I can show it —tell you one of my phones, I don't care if they watch it on YouTube in low-res or high-res. Like it's one of those things that you could just show anyone, and I love that stuff, and I still think that that stuff's very important as well.

So, there's all that stuff and just this idea that I felt that I—again, being reactive, seeing how everything works together, creating a wide variety of stuff. I think I did wanna, but I also really wanted to create. Like, I had to work in a 3D environment or like a three-strike 360 stereo environment, which for me was actually really great, because it forced me to understand how something like just looking around was still very interactive.

It wasn't—and I because I think that—and how we actually experience a space and how those spaces engage with us—that is an important part of this medium. I don't know if I wouldn't necessarily come to that conclusion if I had made immediately like volumetric and sixth-off, and you can, you know, you can speak whale.

So, for me, I think that was—it was important to have those limitations so that I could see within her; I could explore within the medium just some of the basic building blocks of what made it so special. But that said, I really wanted to make volumetric sick stuff, whale-calling VR experiences. I just wanted to do more, and I want to work with different technologies and different VR companies and really expand what I was doing.

I felt as someone who was in the space early enough and who had the fortune of sort of being around the medium and its sort of second resurgence, I felt that I had the opportunity at that moment to actually do some good.

So, what's coming out next? So, I can't—yeah, I was thinking about what I could talk about, and I can't. I feel like I’ve been told I need to be patient; I'm probably not talking about anything yet.

But I think I can say that one of the things that has been really amazing is the response to me going solo-ish was really—it was pretty good. I mean, it was like I got so many people coming up wanting to work with me, and you know, pretty like big names and IPs and so on.

So the projects that I have, I can say that there are four projects. One is with a pretty big IP, which is a series, which we're in development with right now, still working out how that will work, but we were excited about it.

Then, I feel like that's really interesting because that's both—it’s something that explores the biopic and how we reimagine that sort of experience in VR in a way that feels like a journey; it feels like you're going somewhere special.

Yeah, so that's sort of the crux of the series. The awesome because biopics are so bad usually; the music ones are so disappointing every time. Well, the first one out of the gate will be a music one. Great! So, that should be fun.

Oh, you blow the doors off of it. Yeah, yeah, thank you. No, I think it's difficult because I think music has a—I mean, has a capacity to transcend and be emotionally valuable as well, and so I think that for me was an important part.

Plus, this particular person is a very epic cultural icon, so it's been a very interesting experience getting to know who this person is on a deeper level, many, many levels. The second project is with an architect who's pretty well known, and it's sort of—a combination of—it's more exploratory, kind of figuring out what we should be doing with his work.

The third is a game, a fringe game, probably more in the Weather Channel from here, which I'm very excited about, because it’s ridiculous. I think that if there were needed to be a game that would help do what Pong did for folks, where I wanted to have a game that was simple enough that people could all understand and play and would also be sort of an introduction to like what immersive stuff was.

Come on, Pokemon Go to me is still a bit complex for folks; like, I feel like it’s—I think it totally helped people get the hang of things. I'm thinking about like what can be uniquely VR, like, you know, not taking an original IP like, yep, like a Pokemon, sure.

Yeah, it's more like can we create something that is from the ground up, something that was built for this space? Okay, based upon like thinking about the way that defaults are built, thinking of like thinking more in that kind of retrograde, like basic geometry.

Oh, yeah, not like complex character world. Yeah, but complexity comes with it. Longer conversation. Oh, so, and I think the fourth was—the fourth one that was—so I'm trying to think; it's really hard to keep them straight. That's okay.

And the fourth, the fourth one is actually an immersive audio project. I think that immersive audio is great. I think we've talked about to perform it.

Yeah, I feel that there's, you know, as the visual fidelity changes all the time, and as someone who creates in the space, it's annoying because like every time a headset comes out, you have to rehab sport.

Well, like it refers specifically for a live action; I think it's probably a bit easier for probable. I'm gonna hear back from people beginning—is Nadia's you're in volumetric or sixth off—but the audio guys were like, "We good?" You know? They just like lean back like, "Yeah, I'll be exported as whatever kind of like you need to. 5.1? Here you need like, you know, what kind of a Masonic, what kind of spatial eyes, like, file format?"

And it's very—I'm not—this is not to put—their extremely talent; like the bright—the people who are talented are amazing, of course—who know how to do it are incredible.

Yeah, it just feels like that stuff is so important in terms of selling you being in a space, and it's again people are trying to think of it second. So, I want to be part of the—I want to lead, help lead the charge, in like its first for me.

Yeah, I totally agree with you; that's—people don't realize that you could shoot this podcast on a Motorola RAZR from ten years ago. Yeah, video, yeah, and then use these mics, and that would be good enough. Yeah, audio is so much more important than HD anything. Absolutely, yeah, and no one gives it any love.

No, I mean, like I'm thinking about the Weather Channel piece again; it's like, can you really hit it out of the park? Understood. Yeah, yeah.

All right, let's go into all the Twitter questions. So, first question. Sure. Andrew Peterman asks, "How long until we'll be able to create 3D 360 video from cell phone hardware and some kind of software, ballpark guesstimate?"

Oh, I mean, you can kind of do that now. Yeah, I think it's pretty good already. Yeah, it's a storytelling challenge or not, whatever it—whatever what's the word une is no story time.

I mean, I can speak to what is the—what's—I mean, the question is like, rather than storytelling, what would you prefer to say? Oh, oh God, I don't know. World building, that's the one that I can think of. Okay, it's weird again; it's like the director's question—we don't really know what to call it, so we just—it could be anything. Close enough.

Okay, cool, story crafting, narrative. Narrative is good; you know, creating a narrative. I'm trying to think back to all the all the decks I've made in the past couple months, and I think there are a few where I've definitely used narrative, story crafting, story crafting experiences. I think time will tell.

Yeah, not the most important thing; another—no, it's all semantics. It's like you just make stuff that's meaningful.

Oh no, totally. It's like the hardest part is any creative person. You're like, "Dude, I know you want to put me in a box," so here's the box, but I'm gonna do then I actually created. This is—I love this because I haven't used this phrase in a while, but I came up with my earlier—my earlier days of working. I was trying to explain how I edited World Tour together, and the thing that I had called it was probabilistic experiential editing, which the acronym is P.

And I remember what I was putting this together for—for a presentation, I noticed it, and I was like, either I can pretend like that's not the acronym or I have to call that [ __ ] up. So, I basically called it out and was like, "I know, yeah."

But in the beginning of every media, you know, people are gonna say some weird [ __ ] and it may not last, but here you go. And yeah, people—actually, the response was very positive, because I think sometimes people are so serious, like, "Well, this is what this is called," and it's like, "Well, it doesn't have to be; it could be called anything."

We can call it cheeseburgers and it'd be like, "Yeah," and if you have the confidence and you make cool stuff, no one's gonna be like, "Oh, that's not wrong, like Jessica's wrong."

Yeah, dude, I make the cool stuff!

Another VR future question: Matt asks, "Where do you see VR in ten years?"

I don't see it being called VR. Okay, sure, like, I just don't see it. I think that it's all gonna go to a different kind of place. I think, yeah, I think I'll be called something else.

Mm-hmm. Related, Michael Hodapp asks, "Does VR or whatever you want to call it still have long-term mass adoption potential, or will the market shift to AR, you know, so magic leap Apple glasses stuff?"

No, I think that's already happened in a lot of ways. Like I think you could see that just in the shift of Google's like cut; like, yeah, it's like it was Google, it was Cardboard, then Google VR, then Google VR, here, then Google AR VR.

So it's like, you know, they don't relieve; it's like people are still there trying to be like, "Okay, what's gonna be the most—what's gonna be the easiest thing for people to grok?" Though there’s probably—there’s an AR project that I'm working on, or will hopefully be working on that is, um, it's similar where it's, you know, everyone has phones in their pockets; everyone can download apps.

So, there are these two things that are like easy enough for people to grok, where it's like, "Okay, I get it; don't—it's not like I have a new piece of hardware that I have to learn." I think that I still have a hard time finding compelling content on AR in the AR sphere as well. So, I think similarly to like where VR is gonna be in ten years, I think that a lot of these technologies, which are by themselves, have weaknesses.

Weak spots, yeah. Well, when together, kind of fulfill each other's gaps, fill each other's gaps. So, like, you can imagine, okay, AR is great, because it's the most accessible thing.

Yeah, you build a relationship with a particular app that leads you into the VR space, because now you actually—because with VR, it's like you kind of need people to care enough to want to do it. So, if you have an AR experience that is compelling enough that does speak to you on an emotional level, and the fact that VR—a VR experience can be there to help expand upon that—like that's a simple, like, off the top of my head way that those two could work together.

Yeah, is it compelling to you, the notion that in a pick a number of years, people will be in VR for a significant percentage of their day?

I mean, people kind of are, aren't they? And with computers and cell phones, like we spend so much time on the internet.

Yeah, me—if you really want to do that, you could probably go to that level of like, well, we do—we spend so much time not here as it is.

Yeah, so it's like—I mean, we've seen the effects of that in some way. It hasn't destroyed us, but it has sort of—it's had ramifications to the way that we work, live, and right relationships, right?

So I think—I had a—I think I told you that story about the hedgehogs and the plasma screen. Oh yeah, I went to the lounge at Heathrow, and I saw these plasma screens with these hedgehogs on them for some sort of AI system advertisement.

I'm thinking like, "Blade Runner" called this in a very just, you know, kind of semi-dystopic way, like, look at, it's just everywhere, and it's all over the place, and I'm like, "It still is; it just has hedgehogs," right?

So, it's probably not gonna be anything that's gonna just—it's docked pop; it's probably not, "We destroy humanity!" like we do a great job doing that.

Oh, yeah, we're on track! I think the technology itself is something that challenges the way that we exist and will provide us with ways to have conversation around that. I don't think itself will be the cause of any sort of insane thing.

I’m actually not like—I’m not negative about it. I think people are just interested in altering their perception. And like, you mean you see it when people like watch folks on Twitch.

I think you see it with, like, you know, deprivation tanks. Have you tried those? Yes, they're really awesome; they are amazing.

Yeah, I mean, that to me, that's what's so great about, like, when people are like, "Oh, when you go into VR and you do a billion things," like, "No, you don't do anything."

Like sometimes, people just want to hang out, like, oh, for sure, and just like chill for a second. That's the thing I think that's very valuable for, you know, for VR right now at least, is the idea that you can focus.

Yeah, and it does have that meditative quality that's really great. I also think that, you know, the potential for having it in hospitals, having it in places where people are; they're sitting anyway, and it's all whole.

Like, the idea that you can have someone in a VR experience and not feel so bad is a great—it's a great idea. My dad, I gave my dad an Oculus Go recently; he loves it. My mom's been taking photos of him using it. He fell asleep in one, which was really pretty entertaining.

And that's—I mean, to me, I'm like, that's how I know it's gonna be fine, because I mean, it's just, it literally is—it's just it looks, he's fine, like it's just him being like, "Yeah, I was watching this thing, and I fell asleep. I don't know what he was watching; it didn't ask it was my project."

But I think that there's something really—there's something really sweet about that where it's like it's just another thing for us to experience the world.

Yeah, just falling asleep in front of your TV.

Yeah, I'm reading a book or whatever.

Yeah! So, I don't like—I do think that there's, for example, I actually realized I needed glasses when I was starting to make VR, because I couldn't rectify the—I couldn't—the stereo justice.

No, no stereo disparity. So I couldn't actually—it took me a while for me to see my left and right eye for close-up. Like, oh, wouldn't match, so they'd still seem like they weren't working. And I sent it to an engineer, and I said, you know, that is my text, right?

Because it doesn't seem to look right. And he looked at—I apparently looked at it and said, you know, like, well, that's you know, the font's kind of weird. And I was like, "What do you mean the [ __ ] doesn't work?"

He's like, "Oh, yeah, no, it looks great." He's like, "Are you sure your eyes are okay?" And I went to an eye doctor, and that sure enough I had like—I had a stigma thing.

So now I'm funny. Yes, it's weird, so I mean even that kind of—like I feel like there's some interesting ways of how there's all sorts of things that will come to the surface from once people start to like actually adopt and experience this stuff, right?

I do think from a VR standpoint right now, people are sitting there with their VR headsets, but I—you know, all the headsets are going towards volumetric like, you know, positions. Wireless, so I imagine they're being, you know, I'm much, much more interactive, much more engaging, like you're not just sitting there; you're gonna actually move around.

Have you been to that—I think it's in like Utah or Las Vegas? It's like some laser tag type thing where you have a VR headset on? Oh, it's the—oh my God, why am I blanking on it?

Yeah, they have a Star Wars experience too. I'm gonna get reamed for this; I don't remember the name of this company.

But it's a—yeah, they basically do kind of a location-based entertainment. Yeah, they do. I really feel bad about this—I want to say Valve, but that's not true at all!

It's like, although all the game talk, I feel like they actually—yeah, they do. I think that stuff's really cool. I think it fits within, again, it's like it's a different genre of VR for game where I feel like that's a— that's a very, what they've what they've figured out, you have to dedicate like a space for that kind of thing.

Because what you're doing is you're letting people move, seemingly move through a whole experience. I do believe that taking that first step in a VR space for a lot of people is really—it's powerful.

It's a powerful thing to feel. I—we were running some very, very basic—like if we took the three, six, the three like World Tour film that I made, if we put it in a space and like kind of created like evenly, if it was like a rudimentary like box with like what the floor would look like and gave you space to move, I did that, and I lit almost like—I was very emotional for me because I remembered what it was like to be there on a level that seeing it just couldn't do anymore.

You could see it and be like, “Yeah, I was there.” And then you move, and you’re like, “No, but I was really there.” And that's where it's like holy [ __ ]—like when we're able to actually capture our own personal moments like that where we can relive them, that to me is like kind of terrible.

I mean, I've said it before where it's like, okay, you know, part of the human experience is that we forget, of course. So when you can't forget anymore, what happens when you can't?

Like where you—because I don't—I believe in the sanctity of experience in the human experience. That's my own opinion on it; I don't think that we should have all the world's experiences accessible to people all the time.

I think that's a terrible idea, and I don't like the way that that's been marketed as such. Like, I think that there are things that are poetic to forget and should be forgotten in some way.

And I worry that my worry of this is that it goes in the opposite direction where everyone's like, “Remember everything! Experience everything! Access to everything!” Yeah, once we index all of our memories and be perfect as long as there's Hedgehog's on a plasma screen saying AI systems, and it's an app, like, I—as long as those exist in that kind of way, I think there's hope, yeah.

Because it's like that actually brings it to a place where it's like this stuff is being adopted but in a way that's like—it’s essentially harmless.

It's like once you start thinking—once you start getting into that like the bare bones of like why people are trying to do this, like what the end goal is for some of this stuff, that stuff gets a little weird.

But like I think right now we're in a really nice—we’re in a reasonably nice place with some of this.

Well, I think it's like whenever you are a nun VRA are, so it's bringing people joy at this point. So I don't think it's really seeded fear in ways that other technologies have.

I think you know, you're—we were talking about this like it's only AI until it's cleaning your floors, and then it's a Roomba. Yeah, they're like, "My friends—I know!"

It's like we have so much AI intelligence systems around us now. I speak to Google Home every day; I would, you know, speak to Siri. It's a—like it doesn't—fine, you know, it does—it doesn't even faze me at all to do that.

But ask me five years ago if I would talk to a little speaker in my kitchen and ask it to set the timer for me for grilling, it would be like, I wouldn't really understand what I was talking about.

Let's go into the craft. So Virginia Pegado asks, "How can a traditional storyteller adapt to VR?"

And actually, I don't know this about you either—were you making films before? Yeah, I was a filmmaker.

So my quick background on my life was that I was a film—I studied film at NYU, and at the same time I was running across the street to Crump, the Courant Institute of math.

I mean, tell me that's—Oh, okay! And so, you know Courant, so I see you, you know, I had a minor in computer applications—not computer science.

Because I got to Perl and I hated it, so I was like, “No way in hell am I gonna like spend my time.” Thank you so much! Languages—turn people off; I can't stand it.

But I, so I would go back and forth between these two schools—I didn't know about ITP, never heard of it. So, I'm either—and if I had, I would have totally gone there.

So it was this kind of—sorry, I went to both schools, did that. I have shorts; I worked at Apple for a little bit. I worked at a place called UV Factory as their lead editor for a bit, and then I ended up at Google.

Google was—it was always like that back and forth like science art, science art. And then art technology film, I guess were the two. I then Google had a position open for a filmmaker or film; it was like film editor at first but then turned to a filmmaker when I got there.

So I started working there with the Creative Lab, which at the time was like ten people. The one in Chelsea? Yeah, exactly!

And now it has become like a huge—it’s a very important part of Google, actually. They create magical things about the—you know, the brand and like they—they kind of tell Google's story about their products, and I was part of that.

You know, how do we use films? How to be artful about these films so it's not talking heads and engineers that look really nervous?

So I ended up working with them for five years before I went into VR, and actually the VR team, the people there working on the camera, I think found that it would be better if they just tried to look internally to see if someone could use the rig so they wouldn't have to rely on external stuff; it'd be more cost-effective to try to find me than, you know.

I remember they had emailed me—I actually kept the email, because it was huge turning point, because I was actually feeling kind of bored, to be honest, like these films, yeah, because it was a bit like—again, it's that idea that I wanted to do more and felt like what I could do more.

And then when the VR—I wasn't really interested in VR at all, to be honest. I just wasn't; I didn't think it was for me, or I didn't really think about it at all.

But then a bunch of my friends started working in it; like Erin Koblin and Chris Milk started with Verse at the time, which is now within Sashka Unseld went to Oculus, and started a story studio.

So there were all these people that I had known who were starting to get more involved with VR, but I again, I wasn't thinking about it. And then sure enough, I get invited to see this rig and experience the footage. There was one clip that it was a test footage of the clip that the engineers had filmed of just themselves hanging out in Seattle, and it was brilliant.

Like it was wonderful, because I'm like trying to make films about these engineers, and it's really hard. You know, I feel like I was mostly successful in doing it, but it was hard to get engineers to be as relaxed—you know—egg normal, or you talked about what they love about this stuff. Because when they do, they totally make sense.

Yep, I think they were used to like well, I have to talk like this, and I'm not, so I do like, no idea what you're talking about! But then I think seeing that, I was like, right, this is actually—this is the truth—yeah, of that.

So, to really quickly answer the question, how do traditional storytellers adapt to VR? Right, you abandon the telling. Yeah, as hard as that is. You think about what I do actually is I draw two Xs, and in brackets, I write down what I want to—what—the story that I want to tell, okay?

And I use a whiteboard; I love whiteboards! I don't know if it's the Google thing that I love whiteboards, because it's sleep—it’s non-precious—you can just like write and then erase it, and it doesn't exist.

So I write down even if it's long-winded and crazy; I'll write down what I believe this story is, and then I'll look at it and start to hack away at it until I get to that truth statement. So, I'll like—it'll be some—like if it’s a tourist in the hair, it's like the entirety of that narrative, and then it's like, you know, slow and steady wins the race might be the end result, you know?

So I reduce—I mean, I kind of feel you're sort of sculpting at that point. You're sort of like here’s my the story. The story I would tell is this big block of clay; within that clock of clay is the actual meaningful bit.

Yes, I mean, that's like the Michelangelo quote, right? It's like I see it in the marble and then I bring it out, literally what you're saying, okay?

Well, then I think that's—aim for that, and it's also thinking about, you know, once you get to that truth, and it's really about holding it; it's like your north story. And it's about—for me, after that, once you get to that, you have to also think about the flow of the experience, thinking about the cadence, how people come into spaces, how they relate to other objects, how they relate to other people, thinking about the various elements that you want to bring into it.

So it's a layered thing. Is there inner—I like thinking about the tech as well. Like it's not film in the sense, okay, you edited in Final Cut or Premiere or whatever, you export it, you put it on FTP and you send it to somebody, right?

It's like the process is very much in flux; yeah, so a lot of the things you have to consider are who's my audience? Where is this going to go? Which had set what kind of limitations are there for the headset?

Are they creative limitations? Are they annoying limitations? And then you really have to understand like, you know, how those limitations play into the truth that you're trying to transfer over.

And again, thinking how is that truth constantly represented in this experience? How do I make sure that, regardless of where someone is engaged with either what I believe is front/back, wherever—like, how is it even at its worst still able to transfer that over to somebody?

So yeah, it's a lot of like really the preemptive stuff—it's like kind of figuring out, you know, what is the superpower for the person in the experience?

Like in a game—oh man! Okay, you know, I mean like in a game, you're like, okay, I can jump, I can throw a fireball if I have one eye; I can go this way but not that way.

So being—giving people the time to understand that first and foremost is also important, but understanding what those things are and how that evolves over time is part of the narrative too.

And like, I don't know if it's like the companion cube in Portal, where you're given an object that's meant to help you and does help you, but then it's incinerated like three layers in because the computer wants you to feel to it.

Yeah, and you do, and it works. And so it's that kind of like when you’re given this gift to do something incredible, and then, you know, halfway through, you're—it's taken away from you.

Yeah, I don’t know— I don't want to spoil Red Dead Redemption, but like, so sorry at this point. Okay, so there's the second one coming out soon.

Okay, but you know, you're playing this character John Marston, and you get to the end, and you realize that you can farm; like you get your wife back and your kid back; and you know you've been trying to help the FBI as this outlaw for the entire game.

You beat your kind of enemy former colleague, and then they let you go, and you get your wife back and your kid back, you can farm, and then the FBI comes back in a rampage and kills you, and you can't win!

There's no like—and you're just like, "Holy [ __ ]—look what happened!" And then you die! That's it! And that's life, and there's nothing you do about it!

And then for the last little bit, you play your son. Okay, and then you play the son, and then your son is now taking revenge on his father's death, and he turns out—in a way, the whole Red Dead Redemption is about the son not redeeming his father, not the father redeeming whatever else, which was always a little bit, you know, like, I guess like getting back to add his gang for having abandoned him.

I don't know, but the real redemption's with the son, and that's when you're like, "Wow!" Okay, that's when you're like—because you're suddenly like terrified; you're like, I did everything that I could and you did!

Yeah, and you still failed. You still failed, right? Because life—that's sometimes life. And that's how you set up a second game!

That's how you set up! All right, let's really go rapid fire through these questions. So Ken also asks, what key but non-obvious thing is missing from VR to become mainstream? People not just focusing on entertainment and games.

I think that a lot of what I've been doing has—I mean, granted, one of the pieces I'm making is entertainment. But thinking about education and thinking about various other me—you know, professions that could use infrastructure, architecture like, you know, the other project I'm working on, thinking about—I don't know where this stuff could actually be useful.

Don't do those things artfully. I think that we kind of just try to focus on doing this one thing, and it forgets like—but you know, we could just find being inspired by other places where like, yeah, like I've gotten—I've had people come up to me saying like, "Is VR important to us?" And sometimes I'm like, I actually hope that might work through the stuff with them, and I say, actually, no, is I useful?

It might be useful in the future. I think right now it's—it would be weird if you did it. I think like this is a, you know, as technologies expand in popularity so quickly.

No, yeah, they just become hammers, and people are looking for nails! Wait, block train—yes, we haven't black—just white? Why do you say that?

No, it's for sure. It's like, this was driven by AI. I really— I don't think—AI was. You're like, why would you do that? It's like there's no reason.

Yeah, it's like kind of, you know, I think that that's it's more figuring out where you can have—where that stuff, that technology, myself included, where we could have like the most impact and still be creative and still explore.

And I think people who really do want this from other sectors are also really open-minded, at least the people I've been working with, to explore those. How it could be really interesting and wonderful for people, yeah, totally.

All right, next question. Matt MacCabe asks, what are some of the most exciting or scariest parts of social VR, and what is the storytelling potential of social?

You also, what would a social—VR social—VR is—well, it means a lot of different things. That's another—well, okay, so it's social could mean like it's a way for your friends or followers to basically—it's about the idea of like sharing the experience with another person; that's ultimately what it got you.

Okay, so it could be like either you're represented by avatars in the space, and there’s multiple, or it's multiple people being able to do the same experience at the same time. It could mean all those things at the same time.

I did Alt VR, I think that's what its basis—sorry, Old Space VR. And for the first time, I think a month ago, I just was curious. I just hadn't been able to do it for a while.

So I did it using the Oculus Go, and I was changing my avatar because you're a robot, and then it can be like three or four other things, and meanwhile, this couple comes along, and they start talking to me, and I am looking over at them as they're talking to me, and I'm still changing my av—I'm like, and I kind of like give him a look, and I look away, and I—for me, because you know you're thinking, okay, it's a game, and stock it up, but then you realize they're actual people behind that.

Yeah, and I just probably looked like someone who just gave him the cold shoulder like, "Oh, it's you!" and just like look. I was like, "I don't—I don't even know how to talk; I'm still a freakin' robot."

Like, I don't know how to do this, and I look in the distance, and there's another robot, and that robot's looking at a fire, and it's hearting the fire, because you can basically shut up, throw like a motor cons at things.

So it's like heart, heart, heart, heart! Yeah, one's throwing a stick. And all I could think of was, this is beautiful. This is a beautiful thing because everyone's just like, I'm just gonna try out a bunch of stuff, of course!

But this is probably where chat rooms were for like AOL back in the day, where you're like, this is great; we're all like one big happy family learning from each other. But then there's also like, then the creeper comes through and destroys everything for everyone.

So I think that the fear is that people—there are creepers out there. There are, you know, there's been a lot of talk about, you know, women being mistreated in that space as well.

I think the same patterns of what we've experienced before in digital space is probably gonna be the same problems we would potentially deal with. Yeah, no, and again, it's like, it's more affecting because it's like it's trying to emulate real life to some extent.

So these experiences are a bit more—they stick to kind of this role. Yeah, yeah, so we have to be very careful about that sort of thing.

I mean, it's one of the reasons why I think games with guns are—I mean, it's hard because I think that VR experiences should not be following in the same footsteps of games in that regard, because holding a gun to someone's head is very different than holding a controller, so that that character can hold a gun to someone's head.

It's like once you actually put a gun in the hands of a kid in a space, it starts to get—like, there's heavy responsibility there, and I think that's where we do—you would need to be very careful and very responsive to how we create these things.

Well, let's end on a happy note. Tony Kisar asks probably the most pertinent question: "What kind of dog do you have?" This is from your Twitter bio, I think, right? You have a dog.

Yeah, I have a German Shepherd mix. I adopted him when he was five months old; his name is Fisher. He's great! I actually—he hates VR!

When I’m in VR, he gets very upset; he's just like, "Where'd you go?" Like he's kind of like, "What about me?" I think for him it's like, he’s started to get used to it, but he does this whole like—he'll lay down in the middle of the floor near where I'm doing it and kind of huff like, "I'm here."

Yeah, but yeah, he's cool! One of my favorite—talking about memories and so on like that, like I've actually—the benefit of having been able to work with the Jump rig so early on was that I was able to take a lot of the prototypes, and I took one home.

Oh cool! Yeah, and I filmed my family and my house—kind of where we were at this part of my life where I grew up and where my parents moved after various other stints in other places.

There's a shot of me playing with my dog and like just us like running around and, you know, right now it's like yeah whatever, that's silly—that's dumb.

But I imagine myself, you know, years from now, you know, looking back and being very happy. Yeah, seeing that happened to me too, that, you know, I've had—you know, one of my producers passed away, actually, the person who worked on World Tour with me passed away like a few years ago.

And one of the things that I had to grapple with was like do I give his parents this VR footage? Is that okay? And his, you know, we figured out a way to do that where there—he had a friend of his actually was working at Jaunt, and so his Jaunt friend was also there kind of, but it's that question, too, where it's like, it's so wonderful to have these experiences, like to be able to remember some of the things that matter to us a lot.

And I think, you know, even though it's, you know, what I'm trying to do is trying to combine these three different technologies or more, it's also about what's the conversation all the stuff will with reality and how we live our lives. So, I think I'm really excited to kind of keep exploring that stuff too.

Me too! Yeah, well thanks for making time.

Yeah, thanks for having me on this podcast, and thanks for you—it thanks for the questions too. It was really good, yeah, dog one—one of the best!