Stereoscopic Composition

The 3-D or stereoscopic cinema is an extension of the quest for three-dimensionality.  Creating three-dimensional images has always been a part of the cinema, and its creative people have tried to control and exploit it to tell the story.  The current interest in the stereoscopic cinema is not a radical departure from what has come before; the stereoscopic depth cue is one of many, but is the only cue that requires two eyes to perceive.

The first rule of stereoscopic filmmaking is to make images that it doesn’t hurt to look at.  Do no harm.  The second rule is to make beautiful images. It is possible to make stereoscopic images that “hurt” and are unpleasant to look at.  There are two general sources for this: improper transmission of the image (capture or generation means and the pipeline to the projector, the stereoscopic selection device, the screen, and the eyewear) and poor content creation.  Here I assume that we have a neutral stereoscopic transmission system that doesn’t muck up the image.  Modern theatrical stereoscopic projection systems achieve this transmission neutrality.  They are, on a consistent basis, real-world products that don’t require continual tweaking and adjustments (like the systems used in the 1950s in theme parks).  Given this, the quality of the image is up to the content creator.  That’s what this article is about: trying to help the content creator produce the best-looking stereoscopic movies.

Depth Cues

As noted, from their inception movies (like still photography) have been three-dimensional – three-dimensional in the sense that they’ve had the same monocular cues that are used in painting.  These cues are well understood, and most of them were enunciated by Renaissance painters (even though you can see examples of them in art that is thousands of years old).  For example, aerial perspective:  The distant hills have a blue-grayish cast because of the intervening haze.  Interposition:  You know that an object is in front of another object because you can’t see through it and it occludes what’s behind it.  Relative size:  You know that people who are in the distance aren’t tiny; they’re just far away.  (That’s the basis for a trick that’s been used in feature films, in which forced-perspective sets use little people.) 

One cue cinematographers have used to control the three-dimensionality of their shots is depth of field.  For example, if the background is out of focus and there is a person in the foreground, you’re going to pay more attention to the foreground. This cue didn’t exist before the existence of photography.  An image is sharp where the lens is focused, and sharpness falls off on either side of the focus distance.  There are a number of other depth effects that cinematographers use and these have to do with camera movement, aerial perspective, lighting and composition.  The director wants to direct your attention to a specific part of the composition.  With movie screens that run between 20 and 70 feet wide there is a lot of room for your eye to wander.  The stereoscopic depth cue can help filmmakers achieve even more three-dimensionality and help to direct the audience’s attention.

In recent years, new depth cues have come to light, such as depth of field.   The textural gradient was enunciated by the psychologist Gibson.  As an example, consider a lawn:  When you’re close you can see the individual blades, but in the distance they blend together and are less distinct.  You know the blades of grass are all more or less the same size, and as they get further away they get smaller and indistinguishable (so you might consider this to be a subset of relative size). 

The perspective cue is determined by the distance that the camera is from the subject.  The closer the camera is to the subject, the stronger the perspective cue.  The farther from the subject, the weaker the cue.  The perspective cue has to do with the juxtaposition of foreground and background objects, and the distance between foreground and background.  If the foreground image is large relative to the background, it appears to be closer.  Stressing perspective is a condition that is best achieved with wide-angle lenses. The closer you get to something, the wider angle lens you need in order to produce an image of the size that you desire, and that will then cause the background to appear to be smaller. 

Stereoscopic images profit from short focal length lenses because properly used, they heighten the depth effect.  On the other hand, when you use long focal length lenses you have to be farther from the object to have an image of the desired size, and that tends to flatten the image’s perspective effect.  Long focal length lenses will not produce a strong stereoscopic effect, generally speaking.  It may require an extension of the interaxial separation in order to produce a stronger stereoscopic effect, and even then the image may be compromised.  That is, objects may look like they are made of cardboard.

The geometric perspective cue is a relatively new cue as far as the human race is concerned.  Before people built things like roads and buildings it could not have existed.  You have to have straight lines for this cue, and those don’t often appear in nature.  A commonly cited example to illustrate this cue is railway ties going off into the distance, meeting at the horizon.  But any building, any room, typically has these geometric perspective cues and they can strongly emphasize the stereoscopic effect.  Another extrastereoscopic cue that is worth talking about is motion parallax.  Motion parallax is produced by the relative rejuxtaposition of foreground and background with the passage of time and can be created by a traveling camera (and not by zooming). To generate this cue it does not matter which direction the camera is moving – vertically, horizontally, or diagonally. 

Obviously the difference between the planar and the stereoscopic cinema is that the stereoscopic cinema involves two images.  The stereo depth cue derives from the distance between corresponding image points of the left and right images.  The distance between these corresponding points is called parallax.  Parallax can be measured by placing a ruler on the projection screen and one can say that the left and right image corresponding points are, for example, two inches apart, or 18 pixels apart, or whatever.  However, the most meaningful measure of parallax in terms of understanding image comfort is angular parallax, because angular parallax directly correlates with retinal disparity. Angular parallax is measured by the angle formed from the observer to the projected corresponding points.

Retinal disparity is similar to parallax, but is measured by comparing the left and right retinas.  If you could superimpose the retinas’ images when you are looking at objects in the real world, corresponding points will have some horizontal difference, and this is the analog of parallax.  It is disparity that produces binocular stereopsis and it is angular parallax that directly relates to retinal disparity, because it takes into account the observer’s distance from the screen.  That’s an important concept, because there are certain values of angular parallax that you don’t want to exceed for comfortable viewing.  Given a constant screen parallax as measured in pixels, angular parallax will be different for different people sitting at different distances from different size screens.

To sum up:  The most important extrastereoscopic depth cues for our purpose are perspective (the relative juxtaposition of near and far objects and geometric perspective) and motion parallax.  That is not to say that the other depth cues aren’t important, but these turn out to be extremely effective in emphasizing the stereoscopic depth cue.

Cinematography

The two additional creative controls required for creating or capturing a stereoscopic image are setting the distance between the lenses (called the interaxial) and the ability to set the zero-parallax plane, which some people call the plane of convergence.  Changing the distance between the lenses increases or decreases the parallax values and the strength of the stereoscopic image. For wide-angle photography it turns out that you can use reduced interaxial separations – significantly less than the interpupillary separation; and for lots of on-set cinematography you need reduced interaxials.

Less is more (or better) when it comes to parallax.   When looking at a projected stereoscopic image, as a rule of thumb, if the image is not greatly doubled without the eyewear on, the image is easier and more pleasant to look at.  There will be occasions when there is lots of parallax, but not consistently because if that’s the case the image is probably going to be hard to look at.  Remember, the depth effect is determined not only by parallax but by the extrastereoscopic cues. Setting the zero parallax plane, which can be accomplished in photography or post, determines what is at the plane of the screen and not the overall depth effect.  Setting the zero parallax plane deeper within the scene moves the entire image outward toward the audience but does not change the strength of the stereoscopic effect. Setting the zero parallax plane closer to the camera moves the entire image away from the audience but does not change the strength of the stereoscopic effect.  Improperly set it will only add to the parallax values and make the image harder to look at.

We can change the desired strength of the stereoscopic effect by various means: changing distance between the lenses, getting closer or farther away from the object of concern, changing the focal length of lenses, and by playing with the extrastereoscopic cues mentioned above.  The most basic way to control the strength of the stereo effect is by altering the interaxial separation.  A greater separation produces more parallax; a smaller separation, less parallax.  On the set, adjusting the interaxial to less than the interocular separation is often required to prevent the images from looking elongated or to control the parallax values to within acceptable limits.  But there are situations when you have to set the interaxial in feet or yards.  For example, I once mounted the left and right cameras on the ends of the wings of a biplane to get good depth effects of the city when flying over San Francisco. 

In order to compose stereoscopic images properly you have to think about what will be happening in projection. Projection can be thought of as having two regions: screen space and theater space, plus a boundary condition – the plane of the screen.  Screen space is defined as the image region within the screen, and theater space is the region in the theater.  The plane of the screen is at zero parallax.  Screen space has positive parallax by definition (sometimes called uncrossed parallax), and theater space has negative parallax (or crossed parallax).

The paradox of stereoscopic cinematography is that there is less parallax budget available to stereo-optical infinity in screen space, than off-screen in audience space. As a rule you don’t want background parallax points to be very much greater than the interpupillary separation, because that can be uncomfortable to look at.  But off-screen parallax for theater space effects can have values of many inches, or even a foot or two on occasion.

There is a depth range formula that was originally published by Helmholtz, and then elaborated on by Spottiswoode and Spottiswoode.  It is a guide to how parallax changes as a result of camera and projection parameters.  It states that the amount of screen parallax is proportional to the interaxial separation, the focal length of the lenses, and the size of the projection screen.  It also states that parallax values are inversely proportional to the difference between the distance to the zero parallax plane and the most distant objects in the shot. Notwithstanding Helmholtz et al., in order for stereoscopic cinematography to be a creative medium it’s got to be intuitive.  If it’s not in the gut, it’s not going to succeed as an art form.  If you have to rely solely on tables and calculations, composing stereoscopic images is not going to work.  Until the electronic or digital incarnation of the stereoscopic cinema, creators were stuck with slide rules and calculations.  This is no longer necessary.  This is the single most important difference in terms of creative freedom and control for the modern stereoscopic filmmaker, and a major reason why it is succeeding.  You can see what you are doing without having to wait for the dailies. And you can readily make changes.  CG animators have it all over the cinematographers because they can change every variable at will.  After all, they create the world they are shooting.

 

P = Mfctc  (Do-1-Dm-1)

The maximum parallax P, for a setup is given by this equation. M is the linear magnification of the projected image, fc is the focal length, tc is the interaxial separation, Do is the distance to the plane of convergence or the zero parallax plane, and Dm is the distance to the farthest background point.  (A similar equation predicts off-screen parallax.) For example, a typical magnification M might be about 500 (the ratio of the width of the screen to the size of the image written on the DLP image engine), fc 10 mm, tc 20mm, the distance to the zero parallax plane Do is 3000 mm, and we’ll take Dm to be a great distance so it drops out.  P is then equal to 33 mm, about half the average interpupillary — so we don’t have to worry about divergence.

Post Production

For live action cinematography, post can play a part in making the image aesthetically superior and to correct for camera generated errors. The major creative stereo parameter that can be controlled in post is the setting of the zero-parallax plane, by horizontally shifting the left and right images and correcting whatever the cinematographer or stereographer decided was appropriate at the time of photography.  This may be required to improve flow within a sequence because a shot can be seen in context with adjacent shots.  This process may require cropping some of the image, because the horizontal shifting of the images relative to each other may lead to running out of image at the frame’s edges. At this moment the most serious deficiency in the stereo post arsenal is the inability to routinely control or adjust the strength of the stereoscopic effect by interpolating or generating new parallax information.  It’s available, in effect, by using image conversion or synthesis, but someday it will be a routine process.

A recent development is the addition of the floating window, which can mitigate vertical surround conflict of cues and thus greatly extend the parallax budget by allowing for off-screen parallax values. The floating window also causes us to reevaluate the notion that the plane of the screen is absolutely set by the zero parallax plane and the screen surround.  When using floating windows, one could argue that there is no screen plane. 

Floating windows work by printing appropriate vertical edges onto the frame adjacent to the left and right screen surround.  These new edges generate parallax to create a new window so that the stereo window no longer corresponds to the screen surround.  The new window is floated out into theater space.   

It is imperative that those involved in the editorial process (and also at the time of image capture or generation) are able to view the projected image on a screen approaching the size of real world theaters.  Some productions routinely use a 30-foot-wide screen, but in sweat boxes something on the order of 15 feet is more common.

What it comes down to is that it’s hard to know what a stereoscopic image will look like until you see it on a big screen.  Working on desktop monitors may be better than nothing, but there is no substitute for projection on a theater size screen to gauge the overall look, to understand the effect, and to know if you have a comfortable image. 

Projection

From the filmmaker’s point of view, what’ counts is how the image looks on the big screen.  That’s as true for the planar cinema, and for considerations of sound or color.  The content creators’ entire effort is devoted to how the image will look and sound in a typical cinema.  For the stereoscopic cinema issues of comfort are important, and assuming proper image capture techniques and technology, and a good projection system, both the comfort and the aesthetic quality of the image are in the hands of the filmmakers.

One comfort-related concern has to do with the A/C breakdown, which describes the lack of correspondence of the viewing conditions of a stereoscopic movie with what occurs in the visual field. Accommodation relates to focusing of the eyes, and specifically refers to the change in shape of the eyes’ lenses produced by a set of muscles to produce sharp focus.  Vergence (or convergence) is the rotation of the eyes to achieve fusion, or single vision, on the object of attention. In the usual visual world accommodation and vergence are coordinated so that the eyes are converged on that on which they are focused.  It’s a habituated response, with each function controlled by a separate neurological pathway and a separate set of muscles. On the other hand, when looking at a plano-stereoscopic projection in a theatrical cinema, the eyes are focused on the plane of the screen and converged for the parallax values of the composition that capture one’s attention.

In a theater one is typically sitting tens of feet from the big screen, and in this case the eyes are accommodated at infinity more or less.  In such a case it takes very large values of parallax to produce a breakdown of accommodation and convergence. 

Well-composed stereoscopic images projected on the theatrical screen, for the vast majority of people in the theater (remember angular parallax is the best measure), will not produce a breakdown of accommodation and convergence.  If parallax values are measured in inches instead of feet (most of the time, with the occasional expectation of some off-screen effects) on a big screen, the image is going to be pleasant to look at.  For whatever reason, many people who like to talk or write about the 3-D cinema are hung up on the ill effects of A/C breakdown.  But it is not much of a factor in theatrical projection.  It may apply to the first row or two, but people who sit in those rows know they are in for a special experience and seek it out.  The amount of parallax on the screen is a function of image magnification.  The larger the image is projected, the bigger the parallax values.  (But that will involve people sitting farther from the screen, so the angular parallax may not be increased that much for many seats in the house.)

Advertisements

5 Responses to “Stereoscopic Composition”

  1. treehaus1 Says:

    Hi Lenny,

    this is very interesting – especially the interaction of focal length/interaxial.

    It would be nice to have high interaxials for the foreground (to keep the heros round!) and low for the background (to avoid divergence), especially for deep stages. Is there a trick, or do we need to bend light? Anything I should re-read in Foundations?

    Best Regards,

    simon sieverts
    treehaus
    munich

    • lennylipton Says:

      The CG people like Phil McNally at Dreamworks and Rob Engle at Sony have got it knocked. They use multiple rigs or cameras throughout the shot to sculpt the various elements of the scene. FredFred27 in LIve-Action has the right idea.

  2. treehaus1 Says:

    Hi Lenny,

    thanks for your feedback.

    Who is FredFred27 ?

    Best Regards,

    Simon Sieverts

  3. olegalexander Says:

    Dear Lenny,

    First let me say that you are a god amongst men in the field of 3D cinema. I’m currently reading your book, Foundations of the Stereoscopic Cinema, and glad I discovered your blog as well.

    I’m currently in preproduction on my own short VFX film, PRIMITIVE.

    http://olegalexander.wordpress.com/2009/06/11/primitive-previs/

    I’ve recently decided to shoot the live action in 3D, so I’m reading as much 3D cinema theory as I can get my hands on, as well as doing practical tests with my video cameras and inside a 3D application with virtual cameras.

    I have a few questions for you, if you don’t mind:

    1. I understand that the ultimate measure of 3D viewing comfort is the “angular parallax”. In your book you say that up to 1 degree of divergence is acceptable. In your opinion, what is the maximum allowed convergence angle? (Maybe I missed the answer in your book.)

    Also, what is the standard screen width and viewer distance from screen used in the industry to calculate the vergence angles for theatrical films?

    2. The 1/30 rule for interaxial works well for stereo pairs viewed on a computer monitor. However, it must clearly be modified to work for 3D cinematography. Some say it should be the 1/100 rule for 3D cinema. But even then, the denominator could be smaller if the background is closer than infinity. How do you feel about this rule? Is there a quick and dirty rule you’d recommend to a novice 3D filmmaker (who hasn’t yet developed enough of a 3D sense and still clings to formulas) to use on set (besides the max parallax formula you’ve given above)?

    Thank you!
    Oleg Alexander

    • lennylipton Says:

      Becasue I am good at one thing doesn’t make me a god. I hope these articles are helpful. I want to help people to understand what’s going on in my little corner of the wuniverse.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: