How VR is changing the game of cinema | Danfung Dennis | Big Think

Yes, it’s actually too early to really have any hard and fast rules of VR. I think it’s maybe like the internet in 1998 where you could see that it was coming, but it’s pretty hacked together. Everything from the cameras, the stitching pipeline, to the headsets themselves—they’ve been at this prototype phase, literally taking GoPros and sticking them together to make a 360-degree camera—to manually aligning images so they stitch correctly, and popping phones into plastic holders. I mean all of that is going away and is being replaced with the real tools.

We’re getting real VR cameras, we’ve built an automated stitching pipeline, and we’re going to have headsets that are affordable and easily accessible. And so I think we’re really learning from just the difficulty of the technical challenges of creating VR, and some of those challenges are melting away. The “language” of it is actually much harder. How do we use this effectively? We’re finding things that do work. We’re finding a lot of things that don’t work.

But we know that these experiences need to be longer. They’ve been pretty short, under ten minutes. We’re finding that the longer you spend in VR – and it has to be comfortable – the more immersion and feeling that you are in that world, the more your mind starts to accept it. And so we’re creating potentially 40 minutes of content, up to 60 minutes where you can go in, have these long durations and come out and have these profound experiences within that timeframe.

We know that the visual fidelity needs to get much better. Right now we’re at 4K by 2K, and when stretched out 360 degrees it looks low-res. We need resolutions of 8K by 4K and frame rates of 60 frames per second. We need perfect synchronization between all of these cameras, and ambisonic audio where an audio source sounds like it’s actually coming from this position. When all of these factors start combining, we’re going to have these high-fidelity experiences where the fluidity of emotion that can transfer in these worlds is going to be unparalleled.

And so I think we’re just beginning on this curve of VR where the technology and the storytelling are starting to come together, where we’re passing the prototype phase and we could actually use it to create these profound experiences—people come out after even ten minutes, come out of a headset and they will say, “I was so moved by that.” And a year later will come back and say, “that experience changed my life.” I haven’t heard that before in films and images, that a short experience can have such a profound and lasting impact.

And so I think there is this real potential that we’re just starting to crack. Right now it’s a little bit of radio on the television; it’s an entirely new medium, so a lot of the cinematic language from traditional documentary and cinema has to be rethought in this new medium of VR. From the basics of composition – there is no frame. You’re working in a 360-degree environment. Cuts can be very abrupt and kind of take time for the viewer to reorient themselves in a new scene.

So even some of these real basics change. So we’re still learning what this language looks like, but we know it’s very spatial. You are feeling like you’re actually there; this sense of presence can be very strong. You are trying to interpret, your brain is interpreting these scenes as real and your body is reacting to them as they’re real as well. So you can have these very intense or meditative experiences in VR depending on how you use it.

We first thought that the whole crew had to disappear from a shot, and so we would set it up on a tripod and everybody would leave, and we’d get what we’d get. But then that was very limiting to shoot everything on a tripod. We really wanted to move the camera and move with our subjects. So we decided to leave our camera operator in the shot carrying our camera, which we really streamlined in weight so that we could put it on a stabilized gimbal. This allowed us to move the camera th...