Stereoscopic Displays: Yesterday, Today and Tomorrow

This was the basis for a presentation at the 3D Workshop sponsored by the USDC and Insight Media in South San Francisco, on November 16th, 2006.

Stereoscopic displays have a curious history, because the invention of the fundamental display medium and the discovery of the depth sense of binocular stereopsis occurred simultaneously. Although there were illusions in antiquity that each eye sees different images, there was no clear understanding of what this meant in terms of depth perception until Sir Charles Wheatstone presented On Some Remarkable, and Hitherto Unobserved, Phenomena of Binocular Vision at the Royal Philosophical Society. Wheatstone invented the mirror stereoscope and demonstrated the existence of binocular stereopsis, or “two-eyed solid-seeing,” by presenting drawings that had ambiguous depth cues. My favorite, and possibly the simplest, is a line drawing of a flight of stairs. It’s impossible to tell its volumetric extent from looking at a single view, but there is a visualization revelation when viewed in Wheatstone’s stereoscope. Suddenly you can unambiguously judge depth.

There are a number of other depth cues, many of which have been understood since people began to draw and paint. Most of the monocular depth cues were enunciated in the Renaissance. For example, one depth cue is interposition. You know that I’m in front of what’s behind me because you can’t see through me. There are others such as geometric perspective, motion parallax, aerial perspective, and relative size. These monocular depth cues and others like them, plus the depth cue of binocular stereopsis–the only depth cue we have that depends on having two eyes–form the basis for depth perception. They all work together to build an image of the world.

After Wheatstone’s publication he commissioned a photographer to take stereoscopic pictures, and soon thereafter other people began to do the same thing. Sir David Brewster, who invented the kaleidoscope, also invented the lenticular stereoscope, a decisive improvement and the antecedent of not only the ubiquitous View-Master, but also head mounted displays, and other modern stereoscopes.

Stereoscopes and stereo cards were the basis for a brisk business in the Victorian era. The stereoscope was the TV of its day and was used for a full range of imaging from journalism to pornography. But the use of the stereo cards and the stereoscope faded with the proliferation of newspapers and magazines with their photomechanical reproductions that could speedily disseminate photos.

It occurred to people that stereoscopic displays might be useful in certain professional fields, and experiments took place for medical imaging and aerial mapping. From the Second World War on, stereoscopic aerial mapping became important for civilian and military uses. It was based on images that were captured photographically and displayed in a device called a stereo-plotter. Measurements could be made based on trigonometry, and the height or depth of objects could be accurately gauged by a photogrammetrist.

Stereo vision has been important to the human race because we’re both predators and technologists. Predators tend to have stereoscopic vision, and prey tend to have peripheral vision: predators with eyes facing forward, and prey with eyes on the sides of the head to get a more panoramic view. The stereoscopic depth sense has played a significant role, not only in our ability to hunt but also in our ability to advance technology. It has helped the human race to develop technology, from agriculture (weeding, for example) to creating devices (making tools, and building with those tools). Stereopsis is also a source of pleasure, because viewing the world with this depth sense is fun.

The rise of electronic displays, with the invention of television, is one of the most important changes in the history of the human race. Television is a medium of importance because our viewing of media changes the way we view the world. The cathode ray tube, originally the basis for a laboratory instrument, was developed into the key component: the display device used in television sets and computer monitors. Viewing conventional electronic displays has this in common [in common with what?]: They can be viewed without any disadvantage by a person with one eye compared to a person with two eyes, because they cannot convey stereoscopic information.

The same comment applies to virtually all visual media. Just about every man-made image we look at is three-dimensional in the sense that it has the monocular depth cues, but it lacks binocular stereopsis. I’m talking about paintings and drawings, snapshots, magazine illustrations, newspaper pictures, billboards, photographs or photomechanical reproductions, or electronic displays: These are invariably planar or two-dimensional, despite more than 170 years of development effort. Sculpture and theater, which have been around for thousands of years, do intrinsically involve the depth sense of stereopsis to be fully appreciated.

A number of workers, including myself, recognized the lack of stereoscopic capability in electronic displays. In the early 1970s I set out to do something about it. Inevitably, then, this talk is also about my work, since I am to some extent responsible for the creation of the electronic stereoscopic industry with my founding of StereoGraphics® Corporation and my inventions of the monitor and projector ZScreens® and CrystalEyes®; and in terms of what’s happening with digital projection for the motion picture industry, the perfection of the projector ZScreen for REAL D has created a renaissance in the stereoscopic cinema.

StereoGraphics in a 20-year period or so sold hundreds of thousands of stereoscopic display systems to scientists and engineers. Research organizations concentrating on the development of new drugs find the use of our stereoscopic displays to be invaluable; people in oil and gas exploration use our stereoscopic displays to evaluate their seismic data; and people in mapping for government and civilian work use stereoscopic displays. Another major field of use is for presentation of computer generated mechanical designs. The most glamorous application is the use of CrystalEyes for driving the Rovers, remotely operated vehicles, on Mars.

In no case was StereoGraphics able to do much about creating a need for stereoscopic applications. These applications were originally recognized and validated by people in the field, and we supplied the products that met their needs. In addition to the efforts on the part of StereoGraphics, Tektronix seeded the market for stereoscopic displays. Having a company like Tektronix in the field (even though they eventually left it) validated the market for StereoGraphics. Tektronix left the field, I think, because they are an electronics company, not stereographers. StereoGraphics had the secret ingredient for success: a critical eye when evaluating the quality of stereo images viewed using our hardware. As a stereographer I am as concerned about the requirements for creating beautiful stereoscopic images as I am with designing the displays.

Stereoscopic displays are used professionally wherever they can demonstrate that they are necessary to do a job. I consider our displays to be useful because they are a form of extreme visualization that is employed when the conventional modalities of computer visualization or video reach their limit. To design a molecule or evaluate an aerial map demands the use of a stereoscopic display. But these displays would not succeed in the marketplace if, in addition, they didn’t produce beautiful images. That’s a factor that is not often taken into account, and yet it’s the hidden reason for the success of professional stereoscopic displays.

The professional stereoscopic display business undoubtedly accounts for hundreds of millions of dollars in aggregate sales including selection devices, monitors, projectors, computers, and software. The problem in terms of running a stereoscopic specialty organization like StereoGraphics is that it is a niche market company that manufactures selection devices, and the total market for these devices is less than ten million dollars. Although the selection devices enable the viewing of stereoscopic images on electronic displays, most of the money earned comes from the sale of costly software, projectors, and computers. The lion’s share of the business doesn’t go to the selection device, which is viewed and priced as a computer peripheral.

Another current that will be followed in this discourse is the theatrical cinema. William Laurie Dickson, who was Edison’s assistant, broke off from Edison and developed a stereoscopic camera and projector system. A number of other early workers attempted to make stereoscopic images work for theatrical motion pictures. Attempts were made by several studios in the twenties and thirties, and I recently saw a Pete Smith MGM comedy that was shot in the ‘40s and printed in anaglyph and viewed with eyewear with red and blue lenses.

There have been other early attempts to commercialize the stereoscopic cinema, and one is of key importance because it is the direct precursor of the most important modern displays. Laurens Hammond (who invented the Hammond organ) contributed to what is called the “eclipse” or “occlusion” system, wherein images are alternated rapidly using two film projectors. His system was used for only one film in 1923 that played on Broadway. Everybody in the audience looked through spinning mechanical disks that served as shutters. This is the antecedent of the modern field-sequential displays developed by StereoGraphics and others that use occlusion for selection, but today the mechanical shutters have been replaced with something better–electro-optical shutters. Liquid crystal shutters are tremendously better, and replacing two projectors with a single CRT monitor, and later with a single DLP projector, is also a key to marketplace acceptance.

The marketplace for stereoscopic movies in theme parks was initially demonstrated in the United States at the World’s Fairs at Flushing Meadows Park in 1939 and on Treasure Island in the San Francisco Bay shortly thereafter. These systems were the first in this country to commercially exploit polarized light for image selection, which was enabled by the perfection of the sheet polarizer by Land and others. The Festival of Britain in the early ‘50s exhibited 3D movies that also used polarization for image selection. Then in 1952 and 1953, 50 or so stereoscopic movies were made by the theatrical film industry to counter the decline of ticket sales that occurred after the Second World War. Movie attendance was down by half, attributed to the rapid acceptance of television. The opportunity for 3-D projection was based on the fact that in those days motion picture projection involved what is known as a changeover: There were two projectors in every both, and a transition was required at the end of every reel. It was possible to synchronize the projectors and to use them for the required the left and right images. The two projectors created this opportunity, but it was also an opportunity for disaster since it requires careful calibration and constant monitoring of the projectors to make this approach work. However, in theme parks in the last quarter century, as pioneered by Disney, stereoscopic movies have flourished. It’s one way to differentiate the theme park from the mall, and some care and attention has gone into the production and projection of these movies.

Lately the most important change in the projection of stereoscopic movies has taken place as a result of efforts by my colleagues and me at REAL D, the successor to StereoGraphics. The liquid crystal or polarizing modulator, which I named the ZScreen (for Z, the third dimension), has been used by our customers for projection for many years. REAL D significantly improved the ZScreen, and its sales and marketing people succeeded in making deals with exhibitors as well as Disney and Sony Imageworks.

The REAL D system is linked to digital projection and, unlike Hammond’s system that required two projectors, only a single digital projector is required. DLP projectors have no mechanical pulldown and in fact can refresh the image virtually instantaneously; which is the key to their successful application to stereo. We’ve been able to provide the theatrical cinema with a means for projecting stereoscopic movies that is of excellent quality and is essentially foolproof. The image quality of the REAL D projection system is the best anybody has seen–in fact, it’s good enough to sit through and watch a feature film. Moreover, once it is set up it requires no tweaking and very little maintenance.

Today there are hundreds of thousands of people who are using StereoGraphics or similar products to view images on monitors, or using DLP projectors in industrial locations or in the theatrical cinema. In fact, the theatrical cinema has exposed many millions of people to good stereo. Recent films released simultaneously in both planar and in REAL D stereo have done better in 3D by a box-office factor of about three to one.

The introduction of technology is a crap shoot. I can think of one example that I was a witness to: the change from monochrome to color for computer displays. There were people who argued that monochrome was sufficient and that color would be a distraction. Nevertheless, these same people would be watching color television. So despite any objection, now apparently ridiculous, color was introduced, and accepted. Hindsight can make fools of experts.

The same kind of hesitation occurs with regard to stereoscopic displays. There are rational reasons to resist stereoscopic displays, not the least of which is that on some occasions the displays are badly designed or the content is poorly created, in which case the result will be discomfort. Yet another reason is that it costs more money to make stereoscopic displays and to produce content for them. Another reason has to do with the fact that in most types of stereoscopic displays people have to wear eyewear. Yet another reason has to do with the questionable value of a stereo display in the context of some applications such as word processing or spreadsheets.

In order to understand which technology streams might be followed to produce successful stereoscopic display products in the future, it is worth understanding something about the past efforts and how they have influenced the present. There are several major currents for stereoscopic display technology. One follows the work of Wheatstone and Brewster: Two planar images are required, and when viewed with a proper device they’re seen stereoscopically. These are the direct antecedents of the systems being sold by StereoGraphics, for example, or the REAL D projection system. These are called plano-stereoscopic systems, because they involve two planar images.

Assuming we are dealing with a plano-stereoscopic system, how the image is encoded or multiplexed is important. Color encoding is one basis for stereo displays, and that is called the anaglyph system. Another way to encode the image information is the line-sequential technique, in which states of polarization are spatially alternated. Line-sequential has been the basis of a process that has been shown by Arisawa, and if the lines are sufficiently fine, each of the eyes sees a continuous, decent quality stereo image. This interdigitation technique as combined with polarization for image selection has its antecedents in work I trace to Rehorn in the 1950s.

Another major technology current uses more than two images. It is the panoramagram, classified as an autostereoscopic display that does away with eyewear as a selection device because image selection takes place at the plane of the screen. There have been a number of workers in the past 100 years who’ve been responsible for advancing the art of panoramagram displays. Ives comes to mind as the man who invented the panoramagram requiring its multiple images. Other work has been done substituting raster barrier with refractive optics for increased brightness. The panoramagram has been widely used for book illustrations, product packaging, and posters, and it is the basis for a major current of autostereoscopic work that has been shown by StereoGraphics, Philips, and others.

Another autostereoscopic current is the volumetric display, in which the image has actual spatial extent, like a sculpture. This has its antecedents in the work of the Lumière brothers, who took a series of transparencies using large view cameras with shallow depth of field. By stacking transparencies in a frame and rear-illuminating them, a three-dimensional image could be seen. The depth of the image is limited in extent to the depth of the display itself. This idea has been advanced using polymer-dispersed liquid crystal screens that turn on and off in synchrony with a rear-projected image, or spinning devices such as one shown by Texas Instruments several years ago that had laser beams projected onto a rotating surface. Some of these displays are charming, but their size and cost present limitations.

What about holography? The image of a miniature Princess Leia floating in space, assumed to be a hologram, has become part of our collective myth and taunts stereoscopic display designers. But holographic technology may have plateaued. That doesn’t mean that some smart person won’t come along and vastly improve it, but to make holograms work for moving image electronic displays will take the ability to squeeze much more information through the existing pipelines and a substantial improvement in the image quality of the hologram itself.

Will the stereoscopic display of the future be based on past technology or will it require some radically new invention to turn the display market on its heels and make every display potentially a stereoscopic display? Will the work of Wheatstone, Brewster, Ives, Land, the Lumières, and others be improved upon? There are several other currents extant, but I think these represent the dominant currents and they show up again and again as the antecedents of modern work, but it is doubtful that one technology will work for all applications such as theatrical projection, television, or photomechanical reproduction.

What encourages the acceptance of stereoscopic imaging? As noted earlier in this talk, the imaging media that we use in our society is predominantly planar. It may be three-dimensional in the sense that it can have all of the traditional monocular depth cues, but it isn’t stereoscopic. Will there come a time when stereoscopic displays can be applied universally? My answer is that that time will come, but making short-range predictions is more difficult than making long-range predictions. And short-range predictions are dangerous, because very soon they can be proven to be incorrect, thus turning the sage into a fool.

So I’ll take the coward’s way out and say that in the long range I believe that stereoscopic displays will prevail for most imaging. I believe that we will live in a world where everything from textiles to billboards to electronic displays and entertainment media will have the potential for displaying successful stereoscopic pictures, and these will be viewed without glasses. This includes television and the theatrical cinema. I think that when the technology is sophisticated enough to present good-quality stereoscopic images, then murals, shirts and skirts, in fact everything that can take an image, like refrigerator magnets, whatever it is–everything will have the potential of being a true stereoscopic three-dimensional display. Although the focus of this talk is electronic displays I think traditional media will also become stereoscopic; but, as noted above, I don’t assume that there will be one pervasive technology for all display types.

But what technology will allow for the universal and successful adoption of stereoscopy to electronic displays such as digital projection, computer graphics, or television? It may not be based on a technology current that is already known. Until now, the history of the medium over the past more than 170 years has been one of developing and improving that which has existed. For the first five or six years of my work, about 30 years ago, I studied the various stereoscopic display technologies and picked the field-sequential medium as the path. And I was right.

In parallel with my work, for more than 30 years I experimented with autostereoscopic display systems that were based on the work of Ives–essentially a panoramagram video display using lenticular optics. I turned my attention fully to that with a firm commitment seven or eight years ago, and began the development of the SynthaGram®. It was the availability of flat panel displays that made this possible. Philips has taken a similar technology path–one that they may eventually adapt for the home market. Philips is obviously a formidable organization with a good history of licensing and good relationships in the field. But I wonder if Ives’s basic technology can provide the basis for a superb quality stereoscopic glasses-free display. In order to achieve excellent depth and viewing angle, combined with good sharpness, I estimate that the underlying flat panel display needs roughly a thousand times the number of pixels presently available on the highest quality panels. Moreover some fundamental improvements need to be made to the panoramagram optics.

If I had to chart the technology course for the stereoscopic display that might own the future, one long-shot guess is that it might involve technology using negative index refraction materials. Negative index refraction materials, which were described theoretically decades ago, have now been produced in small quantities in laboratories. They have not begun to be exploited, and we could be decades and decades away from the appearance of any display work that is based on their application. My instincts tells me that the successful autostereoscopic display of the future could involve a break in the technology continuum and involve a radically new technology. Barring that, Ives’s panoramagram has a fighting chance.

Another barrier to making a prediction about future developments is the rapid increase in stereoscopic invention activity. There was a time, not long ago, when I collected all the patents in the field in folders, as what used to be called paper but we now call hardcopy. Up through the late 1980s a single folder of patents for one year took up about an inch. Today that folder is a foot or more thick. (I stopped collecting the paper patents and it’s now all digital files.) The proliferation of ideas on how to make stereoscopic displays, especially autostereoscopic displays, has become more voluminous and richer in scope and applications. What used to be an isolated field has gone through a transition, one I played some part in, and is now the subject of increased university and commercial activity. People smell money.

This world we live in is a series of mysteries, and in our quotidian cares we do well to avoid facing them so we can focus on survival. But for me this occasion is a moment of reflection. Some of the mysteries unfold and some remain opaque. In this brief discussion of a technology I love, I hope I have contributed to the former.

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: