Founded in 1999 by post-production engineer and inventor Michael C Kaye, In-Three has already secured a major foothold in the motion picture industry. But to date, 3D cinema has never been more than a recurring short-term gimmick, mainly because of the eye-strain, eye fatigue and headaches generally associated with viewing. This is what 3D enthusiast Michael Kaye refers to as Bad 3D. For a number of years now he has been experimenting with ways of creating 3D in software without the numerous factors that contribute to an ‘uncomfortable’ viewing experience. “Bad 3D can still produce a really neat 3D effect,” Michael explains. “But Bad 3D is hard to watch. You can get a sense of the effect, but for some reason it’s really hard to look at for very long. In other words, it encompasses poor left/right construction of the frames in the stereo pair. A very minor amount of disparity between the left and right image will cause your eye muscles to be strained. We’ve carried out tests where if you watched Bad 3D for one minute the eye strain could last for over an hour. So the big problem from the very beginning was, ‘how in the world would you ever be able to do two-hour movie?’”
Understanding 3D
After shooting 3D for a number of years Michael sold his interest in Encore Video in 1995 in order to concentrate on his passion for the format, developing many of the concepts and technology that now forms In-Three. Like James Cameron and Vince Pace (the team behind the modified Sony F950 Fusion 3D camera), Michael has also built 3D camera systems. “Not for the purpose that they do: to shoot 3D,” he explains, “But to analyse and understand 3D. In the beginning stages of developing the software it helped to understand the mathematics of what needed to be done. I’d shoot 30 shots of one thing and throw 29 away. You try it lots of different ways and keep the good one. You try to figure out why that one worked and the others didn’t. It’s just that 3D is not as simple as some people think, where you just take two cameras and put them together. What’s hard about 3D for us is to reconstruct it out of nothing – that’s where you really learn about 3D.”
The In-Three approach to Stereoscopic conversion is to treat the original captured 2D footage as the left camera or the image that will be viewed by the left eye on screen. The right camera image, or the image that will be viewed by the right eye on screen will contain all of the necessary parameters required to create depth within the image that was not recorded by the camera’s single perspective. “The beauty of this is that the original image is always intact; it’s completely unchanged, we don’t do anything to it, so it’s in its pristine form,” explains Michael. “What we’re doing is creating the right perspective. It’s really restoring the depth back into the image that was never there. In order to do that one basically has to recreate all the aspects of the image by way of the perspective, shape, and so on. Everything has to be connected together in a way that makes sense. It could be done the other way too, I suppose, but we’re creating one perspective to make up that Stereoscopic pair. It’s not necessary to make a left and a right out of the original. It would be twice as complex to do that.”
An immediate advantage that this could present for theatres is the ability to project a 2D version in one screen while simultaneously projecting a 3D version in another, both from the same source. Of course, larger screens (60ft wide or greater) require a degree of recalculation, as the extra image width increases the left/right eye image displacement. Objects in front of the screen appear closer to the viewer, while objects behind the screen get pushed further back. If the backgrounds in 3D are pushed back too far, beyond the physical visual limits, it causes the viewer’s eyes to diverge outward, resulting in eye fatigue. “There are a whole set of rules that go along with 3D that do not apply to 2D. If you violate any one of those rules you can cause yourself to have Bad 3D. And that’s an unfortunate reality. We’re very anti-Bad 3D at In-Three, everything we do you can watch for hours and you don’t get eye-strain or eye fatigue. It’s not just chance, or because we say so. It’s because there are real quantifiable absolute factors that we avoid within our software process.”
So now filmmakers can decide whether to shoot 3D or go 3D in post. What are the advantages of the latter? “Well just building a stereo camera system based on the basic principles of human vision does not necessarily guarantee Good 3D. There are a number of factors that are not considered in the very broad scope of that thought process. You have to consider the fact that our eyes – even though for all practical purposes they’re a given distance apart – are also a given focal length, even considering people who wear glasses. When you are photographing in 3D, focal lengths are all over the place. When the focal lengths drop – in other words you go wideangle – the distance between the cameras needs to increase. Conversely if you go telephoto – you zoom in – the cameras want to go closer together.”
Innovative camera engineer Vince Pace has been working with James Cameron to develop an Active Convergence system to attempt to accommodate this factor when shooting with a stereo camera set-up (see separate article). This allows the two lenses to cross and uncross much as our eyes do when tracking objects moving closer or farther away. “There’s a difference between our eyes doing it and two cameras doing it for the purpose of reproducing the image for our eyes to see,” explains Michael. “If the left and the right cameras angles are toward/away from each other to the subject matter you get an equal and opposite keystoning in each the left and right camera. So you end up with vertical disparity, from the centre of the image outward. Now that vertical disparity can never be gotten rid of, it’s in there forever. It’s going to cause a lot of visual confusion, so it’s another source of eye fatigue. We’ve built cameras both ways and I argued that with Jim and he said: ‘How would you do it?’ The cameras have to be dead parallel, no matter what. So he said: ‘Okay but then how do you converge?’ You converge them after the fact, in post. ‘But then you’ll lose resolution’ and I said well yeah, but I’d rather lose some resolution than introduce an artefact that can never be gotten rid of.”
James Cameron used In-Three’s services as an addition to his usual palette of stereo filmmaking tools when the facility was asked to Dimensionalize some 2D video images captured by robotic undersea vehicles which were then blended with 3D acquisition shots. In-Three has also run comparative tests for James Cameron and other filmmakers between 3D acquisition footage and Dimensionalized footage taken from a single camera. “We give it to our Dimensionalists. We never show them the 3D. We just give them the left camera, and to them it’s 2D. We don’t want them to be persuaded one way or the other by seeing any 3D results. We want to see what they come up with to make it look real. Then we compare the shot that was photographed in 3D by immediately cutting to the shot that was Dimensionalized. It’s quite stunning to see the difference. The one that doesn’t hurt your eyes is the one that is Dimensionalized – always. The one that looks more realistic is the one that is Dimensionalized. Jim sat there watching it over and over and over again trying to figure it out.”
Conducting the 3D process in post production offers far greater control than is available to any DP attempting to shoot 3D in the real world. “When we look at an image, every single shot, every scene, is heavily scrutinised, and it’s scrutinised not just by one human Dimensionalist. We have at least six people in the end to make the judgement call and the reason we do that is because we don’t want one person to adapt. Usually the best consensus is achieved when the image first pops up on the screen. We always watch them on a big screen and we try the shot many different ways. You can Dimensionalize a scene and say, well this works great, then you can do it another slightly different way that also looks great, so we cut between them while we’re looking at them up on the screen and say, which one ‘feels’ right, including what it cut from, what it’s cutting to and so forth. We can adjust everything while we’re watching. In the post environment you have all that control, whereas if we were committed to shooting something on a set with two cameras, it would be incredibly difficult.”
Although shooting 3D in HD gives the camera crew and director more opportunity for greater feedback than 35mm film, it still leaves some grey areas that only experience can fill. “Now, if I’m out there on a set and I’m shooting 3D, I’ve got my camera set up and I’ve changed my focal lengths and I’ve converged them towards the subject matter, but the problem that I have is that I don’t know how to predict where that subject matter out at infinity is going to end up on screen. What’s the actual physical displacement? What does it need to be mathematically to make your eyes look out parallel to one another at infinity? I have no clue. I mean, I have an idea, but when I’m on the set and I’m shooting it I don’t know whether it’s happening. You can’t see it on a 20in monitor. When you come to view it, if it’s got a problem what do you do then? Now here’s the difference when you’re Dimensionalizing – software knows that. Our software won’t even allow you to do that. It’ll say stop. It’s a numbers game. It’ll push everything where it needs to be in the reverse direction.
Having said that we are not, by any means, advocating that everything needs to be Dimensionalized and nobody should shoot 3D – that’s absurd – I love shooting 3D myself. All I’m saying is that it’s very hard to get it right.”
So what about CG? Surely if everything has been created within a virtual world then the process to convert to 3D should be reasonably straightforward. “You could take a look at Polar Express, which was a very successful movie in 3D, but there are a lot of shots in there that are eye-strainers, big eye-strainers. It’s hit and miss. So you’re watching as every cut goes by and it’s good shot, good shot, bad shot, bad shot, good shot, bad shot, bad shot, I mean it’s all over the place. We were stunned when we saw Polar in 3D because we’d also assumed that as it was CGI every shot would be perfect, but it wasn’t. Those 3D rules also have to apply even in the virtual world. You land a virtual second camera in software and render that out, but it also has to be placed properly. So it’s a matter of understanding 3D. We have a really good sense of it because we are creating it out of nothing. We’re restoring it back in and you have to really understand the properties of it to do that.”
Polarisation
Although the focus of In-Three is the generation of Good 3D content, they also recognised that an appropriate system for theatrical presentation was required that would do justice to their high calibre of work and so invested expertise and money with Nu-Vision in the development of ‘active’ glasses. This polarisation method of 3D uses infrared to control LCD shutters within the glasses to determine which eye is viewing the screen at any one time. “Our preference is for the ‘active’ for viewing because you get the best 3D,” explains Michael. “Whenever we end up looking at something on a silver screen with ‘passive’ it’s a big let down. Which is unfortunate, because a lot of people adapt to it and think it’s great. We could never see thousands of screens in movie theatres adopting silver screens. They were problematic in the past and they’re still problematic now. You rely on the viewing angle of the glasses themselves to cancel out the left and right opposing image from the screen, and of course that makes the seating position and the seating angle very, very critical. The extinction ratio* at best is something like 20 to 1. ‘Active’ is ten times that and every seat is a good seat because the shuttering of the lenses ensures a discreet left and right image that is not restrictive to the angle, no matter where you are, and you don’t have the graininess of the silver screen because it uses a matte white screen. It works. We use it all the time at our workstations.”
So for fans of the 3D format these are extremely exciting times. Stereoscopic cinema is being taken much more seriously than ever before as major directors, producers, designers and both camera and software engineers throw their weight behind it. Are we on the brink of a completely new approach to cinema that may come to be the main form of all cinema in the future?
“Honestly, we’ve only scratched the surface with our process, we’ve done hours of material, yes, but it’s still a new process. Our first fully Dimensionalized movie isn’t even out there yet. We’re really happy with the results that we’re able to produce right now, in these beginning stages, but if you think back to when ILM did Jurassic Park, it looked fantastic. But where they are now from where they were then is vastly different – and so will we be. We’re out of the proof of concept mode.”
* Extinction ratio refers to cross-talk ghosting that may result if one eye’s image is not properly blocked when the other is displayed. Theatres may want to exploit the 3D opportunity to get people back into cinemas, but the studios also know that a huge chunk of their revenue comes from the home video market. Good news for studios: the ‘active’ system is adaptable for home entertainment. Luckily, there’s also some good news for the theatres. It will probably be at least another five to seven years before the technology is available. Up until very recently, display development in the Far East has focused on glasses-free Auto-Stereo displays. The big downside to these displays has been the extremely limited viewing angles. So, for the time being at least the 3D experience will only be available at cinemas and IMAX screens.
 |
David Valentine.David Valentine is a freelance writer and filmmaker currently working with arts organisations and education providers to support community filmmaking projects for young people. He is also a proponent of Free Media. |