Wednesday, April 1, 2015

Life after Legend3D

The colors of Barry Sandrew, part 2

This is the second installment and conclusion of the colorful adventures of neuroscientist, digital colorization pioneer, animation innovator, 3D conversion trailblazer, producer, entrepreneur, philanthropist, public speaker Dr. Barry Sandrew. In an attempt to osmose his success, the LA Animation Examiner poses additional queries about his vivid career path.

LA Animation Examiner: What are your immediate and long term goals with Legend3D and your other entertainment-oriented, entrepreneurial pursuits?

Dr. Barry Sandrew: I left Legend3D in the good hands of three extremely capable professionals: Matt Akey, CMO; Anthony Lopez, CIO; and my son Jared Sandrew, CCO/CTO. The guys are very busy right now with over two years of film backlog and with sufficient reserve capacity to expand to take on more. In addition, during the past year, Jared has been focused on a proprietary conversion process he invented that turns any 360 degree, 2D footage into the most immersive 3D virtual reality I've ever experienced. I believe this will become an exciting new vertical production pipeline for Legend3D. In fact, the most recent Legend3D 360 work has convinced me that virtual reality is a real product that could change our lives in many exciting ways.

As founder and equity holder, I continue to cheer Legend3D on to even greater achievement. I’m also involved in several for-profit and not-for-profit boards. I'm a Charter Member and member of the Board of Directors of The Indus Entrepreneurs (TiE), which is the largest entrepreneurial organization in the world; I'm on the Board of Advisors of EvoNexus, the only pro-bono start-up incubator in the country; I was recently elected to the inaugural Board of Managers of the Los Angeles Chapter of The Visual Effects Society (VES); I continue to serve on the Board of Governors of The International 3D and Advanced Imaging Society; I'm a perennial 3D judge at Camerimage, the largest film festival in the world celebrating cinematography and the cinematographer. Additionally, I've served as Chairman of the Feature Film section of The Tesla Foundation Group, which endeavors to establish standards and best practices for the use of drones in filmmaking.

Finally, I’m in the process of assessing several new entrepreneurial endeavors that I expect will become my central focus for the next several years. More information on that will be disclosed in the near future.

LAAE: Does animation and visual effects play a part in your work with the Tesla Foundation? If so, how?

DBS: I got involved with The Tesla Foundation Group through my interest in the use of drones for feature filmmaking. Currently, it is illegal to employ drone technology in the US to shoot a movie, because the Federal Aviation Administration (FAA) has not yet developed regulations for its exploitation. In fact, there are only seven production companies that have exemptions for professional drone filmmaking. I believe the FAA’s failure to establish standards and best practices for drones in film production is driving more entertainment-related business and jobs out of this country. I helped the Tesla Foundation Group with, and participated in, their first Drone Expo in December (2014), and I drummed up support for the movement within the Hollywood community.

LAAE: What is your advice to others who aspire to follow in your multi-hued footsteps: scientist, producer, entrepreneur, public speaker, philanthropist?

DBS: Don’t get type cast into a career. Keep your eyes open for new opportunities that relate to your passions. Don’t be afraid to take on measured risks. Most important, when you’re starting out, network, network, network . . . and expect to pay your dues. Often, it is necessary to accept unpaid internships for experience and new contacts. Another way of building a network is by joining non-profit organizations that relate to your pursuits. Successful professionals frequently join non-profits for both personal or business marketing and for giving back [to the community]. By becoming a highly visible member of key non-profits, you'll likely be doing something positive to help others, and in the process, you'll be benefiting your career in many diverse ways. Finally, be aware of and learn how to exploit quality professional social media like to further expand your professional circle.

LAAE: Thank you for your time, insight, and advice, Barry. In terms of your accomplishments, there is no gray area. You are not only the Superman of American movie preservation, you are a career chameleon! It will be a pleasure to chart your progress as you conquer additional fields in the near future.

Sunday, March 29, 2015

Statistical Management In Film Production
As many of you know, I am in the business of making films. The process of film production is largely run by creatives and lacks the management discipline (and training) found in more conventional factory operations. It is a frequent error to believe that filmmaking is a "work to quality." Instead, almost all commercial film production today is an endeavor of "work to cost." In the case of studio films, there may be the added pressure of "work to schedule." Since budgets rarely flex, schedules and quality have to become plastics. We have three points of control; quality, cost and schedule. Change any one of them and the other two move, like it or not. All of this can create a toxic environment of stress, where there exist ambiguous quality specifications, a delivery date promised by sales and not production, and the entire process is operating on a fixed budget.

This yields the classic management solution of making people work longer hours for the same pay with the expectation of level quality. This behavior, which I call "management by panic", is ubiquitous in visual effects and film production. Its energy is the aggressive drive toward deadlines made by others, quality judged by others and a pressure cooker business model of fixed costs. One visible example of this management from time/budget pressure is the recent death of Sarah Jones and the subsequent imprisonment of the film's director. 

This industry wide environment, where there is a drive to shove quality, schedule and quality all at once, leads to a lot of schedule guessing. Schedule guessing is not evil in itself, but leads to failed projects, unhappy workers, underbid projects and an environment of stress, where management by panic leads to poor judgment. 

There is an alternative. 
In visual effects, we have the ability to add code into our tools. This is called "instrumenting the software" and yields hard cold productivity data to allow accurate predictions of vfx delivery schedules. This is not a common current industry practice. However, by having unbiased data about worker productivity, shot complexity, probability of reworks, and the time trend analysis that comes with this, we can create an environment of rational and sane process control. 
If you are a manager that bullies your workers, or a worker who feels they are pushed to "just give me the delivery date!", the correct answer is in the form of probability and statistics. Brush off your old textbook and look at polynomial regressions, scatter plots and infographics. Use this to empower your project to yield beautiful precision and boring predicability. For example, when you install that new whizzy version of software, see if a productivity bump actually occurs and if so, did the savings outweigh the cost.

I feel it is time to bring visual effects and film production out of the garage and into the adult world of statistical factory management. To do anything else is unreliable and management by panic and potentially toxic to those you work with.

Management without data, is not managing at all.

Wednesday, January 21, 2015

How Colorization Critics Contributed To The Tragic Fate of Our Public Domain American Film Heritage.

The Colors of Barry Sandrew, Part 1

A perspective few understood regarding the colorization of black and white classic feature films. The critics of colorization never realized they were contributing to the deterioration and inevitable loss of much of our black and white American film heritage. Barry Sandrew, Ph.D.

Neuroscientist/digital colorization pioneer/animation innovator/3D conversion trailblazer/producer/entrepreneur/philanthropist/public speaker Dr. Barry Sandrew knows when to speak and when to let color - particularly the on-screen variety - speak for itself.

In that same spirit, the LA Animation Examiner proudly presents the vibrant Barry Sandrew, in his own words. The following is the first of a two-part, exclusive interview.

The LA Animation Examiner: Could you describe your segue from biological science to the art and science of motion picture production?

Dr. Barry Sandrew: I earned my doctorate in neuroscience from State University of New York at Stony Brook in 1977 and immediately won a three-year National Institutes of Health (NIH) postdoctoral fellowship at Columbia University, College of Physicians and Surgeons. After completing that fellowship, I joined Harvard Medical School and Massachusetts General Hospital (MGH) as staff neuroscientist. Among my many interests at MGH was medical imaging: an industry that was in its formative years during the early 1980s. In 1986, several entrepreneurs who had learned of my work in 3D brain imaging approached me with an interesting problem. They asked me to invent a process for colorizing black and while feature films.

The businessmen recognized that, if they were able to colorize movies that had fallen into public domain, then they would own brand new, 75-year copyrights on the colorized versions of those films as derivative creative works. Today, that new copyright is good for 95 years. I would not have taken the gentlemen seriously had they not secured some very impressive people on their board of directors, including two-time Oscar winning composer, Al Kasha; Emmy winner Peter Engel, one of Hollywood's biggest and most successful TV producers (creator of Saved By The Bell); and studio executive Bernie Weitzman, best known as one of the key execs at Desilu Studios in its heyday under Desi Arnaz and Lucille Ball. So, I had confidence that these people were legitimate players in Hollywood.

Three years prior to my meeting the aforementioned industry group, an analog colorization method emerged and created significant controversy, largely due to the poor results of the process. The business men understood that, if a superior quality digital procedure could be developed, they could own the colorization field. At the time we crossed paths, I was unaware of movie colorization as an enterprise. However, the technical issue that they presented to me was intriguing, and frankly, the solution - while not obvious - came to me almost immediately . . . like the proverbial light bulb. It took me one night to write an extensive white paper describing a unique system for digitally colorizing black and white movies, along with a detailed account of the corresponding pipeline.

Having gone through several failed research and development (R & D) attempts and a great deal of investor money, the entrepreneurs were desperate to find a solution, and after reviewing my white paper, they felt that they finally had found their guy. They asked me to conceive the system, plus start a new company that would have the capacity to colorize all public domain films in Hollywood. While entrepreneurship was attractive to me, I explained that I had a significant career in neuroscience, and that it is almost impossible to leave a lab, the grant application process, and the peer-reviewed publishing world, and then expect to return without consequence. They responded to me with a financial and equity offer that I could not refuse. After careful consideration with my wife Lori, I decided that I could not ignore the opportunity. I saw the risk as minimal, so I took a leave of absence from my staff position, left my family back in Boston, and opened a colorization R&D lab, American Film Technologies (AFT) in San Diego. I chose San Diego because the new mode was very labor intensive. I recognized that a colorization production facility must be close to a low cost labor market. Therefore, locating 30 miles north of Tijuana, Mexico was a key factor. 

Once in San Diego, it took five months to build a colorization workstation capable of digitally capturing an entire feature film of sequential black and white movie frames from the existing broadcast recording standard of the time, one-inch tape. That had never been done. Since there were no acceptable PC server and network solutions, I captured the black and white frames onto Bernoulli Disks . . . thousands of disks, each bar coded with SMPTE time code of the 48 frame sequence that could be stored on one of them.

Sid Luft, Judy Garland's ex-husband and manager, became a fan of the gestating procedure. He provided a great deal of black and white footage from Judy's concerts and TV shows to colorize as demos. In April, 1987, after colorizing several minutes of his footage, I held a press conference at Universal City (in Los Angeles) to display the work. Once the reporters saw Judy’s natural skin/makeup color and Frank Sinatra’s blue eyes on the TV screen, they were sold. That summer, we won our first movie project, Bells of Saint Mary's, from Republic Pictures. Within a month, I had forty of my proprietary workstations built, hired and trained a bunch of people, and we completed the colorization of that Bing Crosby classic in six months for a Thanksgiving release. The colorized version of Bells of St Mary’s became the highest rated holiday film of that season, and Republic Pictures gave us several more films to revamp. Meanwhile, Ted Turner’s people expressed interest in improving the quality of the films that he was colorizing. Ted wanted to move away from the inferior analog process that he originally supported. When he visited my San Diego studio in January of 1988, he was extremely impressed with both the approach to and the quality of our product. He subsequently awarded 36 films per year for five years to AFT. 

The rest is history. AFT became the premiere colorization studio in the world. The company colorized the vast majority of Ted Turner’s MGM films, as well as black and white titles for Disney, Warner Brothers, Republic Pictures, Fox, Sony, Universal, and Paramount. I also colorized content for broadcast and cable networks, including CBS, ABC, HBO and TF1. In 1989, I took AFT public and, in 1991, I expanded my colorization technology and pipeline by devising the first digital paperless animation system. It was implemented in several animated shorts, such as Gahan Wilson’s now iconic theatrical classic Diner, the ABC/McDonald’s TV special, The Magic Flute, and the TV series Attack of the Killer Tomatoes for Fox Children's Network. In 1992, I developed the first ink and paint and compositing method competitive with Disney’s revolutionary CAPS system. Ours was the practice of choice for several minutes of An American Tail: Fievel Goes West for Steven Spielberg’s Amblin Entertainment. Later, it was used to apply all color, compositing, and visual effects for Spielberg’s first digital animated theatrical feature film We’re Back: A Dinosaur Story.

LAAE: Did your groundbreaking movie colorization process cause any controversy amongst film fans? If so, how did you manage the related public relations efforts?

DBS: There was always controversy. The congressional hearings with Woody Allen, James Stewart, and John Huston drew a great deal of attention. But the drama really heated up after I colorized Casablanca for Ted Turner. I think that was our sixth film with him, but in my opinion, it should have been about the 106th film. We needed more experience. Nevertheless, the resulting debate only popularized the process, attracted much more work from around the world, and it emboldened my biggest client, Ted Turner, to colorize his MGM library. Frankly, if consumers did not want colorized content, I would have been out of business. The fact is, AFT (and later, my visual effects and digital media company Legend Films) were very successful. People not only bought the colorized films, but they became collectors and fans of our particular colorization.

What the critics did not understand was that, when a film goes into public domain (PD), it belongs to everyone. That's a good thing, because after a studio has enjoyed a monopoly on selling the film at a premium price for close to a century, the film then falls into public domain where it is available to anyone for exploitation. This makes the movie attainable to the world at a cheaper price. The same is true in the pharmaceutical business in which a company patents and sells a certain drug without competition for as long as the patent survives. After the patent expires, anyone approved by the FDA can produce a generic version of the drug at a lower price. This is beneficial for the consumer. The bad aspect about movies is that, unlike the pharmaceutical business (which has the FDA as watchdog), virtually anyone can obtain a print of a public domain black and white film, regardless of the quality. They then can slap it on DVD's to be sold at major retailers. Because anyone can package and sell a PD film, there typically are several versions of any specific PD film in the marketplace: all unrestored and in poor condition. The retailers do not care about the caliber of the product. You see, with so many easily accessible, inferior versions, there is absolutely no monetary incentive for anyone to restore the black and white PD treasures, our American film heritage.

However, colorization actually subsidized the restoration of PD black and white films. The higher retail price (and ultimately, profit margin) allowed me to spend the necessary money to restore the black and whites too. Since we always included the fully restored black and whites along with the colorized films, consumers had a choice of iterations in one package. Without colorization to justify that premium sale, the black and white movies never would have been refurbished to an acceptable condition.

LAAEDr. Sandrew, you are the Superman of American movie preservation! Thanks for using your considerable, multi-faceted, potent array of powers for good. Undoubtedly, the readers look forward to Part II of your story . . . coming soon . . . in glorious color.

To read the entire article and learn why classic PD films are deteriorating, click here!

Monday, December 22, 2014

Colorization Revisited

The Revolving World of Color

Originally posted by WAYNEGERARDMADDEN  Dec. 22, 2014 

Even though Color Television was a staple of the American diet from 1958, the UK had to wait until John Newcombe won the Men’s singles at Wimbledon nearly a decade later, before the first color images were projected onto European TV screens.

John Logie Baird, the inventor of television, first demonstrated color television in 1927 but despite its popularity, high prices and the resulting scarcity of color programs, the format was not as widely used in its initial outing. As time moved on, however, cheaper alternatives and formats were created in order to make the expansion of color film a reality for all mediums and budgets.

Barry Sandrew, PHD, an internationally recognized entrepreneur, digital imaging expert and visual effects pioneer invented digital colorization in 1987. “I invented digital colorization in 1987 at my company, American Film Technologies, as an alternative to the very poor quality that was being delivered using the initial analogue process.”

But new technology is not necessarily always greeted with Universal acclaim. Back in the mid-80s, there was a well-known and controversial campaign to colorize classic movies which led to heated public debate. Ted Turner (the media mogul and founder of CNN) spearheaded a movement in which he believed classic films, among them Orson Welles “Citizen Kane” and Frank Capra’s “It’s A Wonderful Life”, should be colorized for the benefit of future generations. He believed that classics such as these were being ignored by viewers in favor of color film.

This is an argument that goes back to Thomas Edison, in the late 20th century, who colorized film by hand as he perceived that audiences would not enjoy his own black and white work. It’s a popular misconception; therefore, that color film was a more modern cinema trait. Victor Fleming’s 1939 musical fantasy “The Wizard of Oz” is one of the most well-known pieces of the time to have been shot purposely in color (using two tone Technicolor), although the basic technology had existed for almost 40 years beforehand, with the earliest examples going back to 1902.

Technology aside however, there were also plenty of examples where black and white films were colorized in pre-production. Howard Hughes and Alfred Hitchcock created films in black and white, only to remake them in color, citing both artistic and financial reasons.

Kevin Shaw is a colorist, with over 30 years’ experience in the industry. He believes there is some truth in the notion of a stigma towards black and white on film. “There is a feeling that black and white is “missing” color rather than being seen as an alternative medium,” says Shaw, “and I believe this inclination persists today.”

Sandrew argues the very objections to colorization is what made it popular. “The fact that colorization was a “hot button” issue actually helped AFT become more popular with both clients and fans of colorization,” says Sandrew. “If people didn’t like colorization they didn’t have to buy nor watch it. Guess what…they did both!” Before Turner, there had already been a counter argument that color film was damaging to cinema. French director Francois Truffaut argued that color should not be used at all, making a statement in 1978 that “color has done as much damage to cinema as television.

It is necessary to fight against too much realism in the cinema; otherwise it’s not an art. From the moment that a film is in color, it’s not cinema anymore.” Shaw agrees somewhat with Truffant’s statement. “A director has to have a good, artistic reason to use black and white, and that doing so invokes a particular meaning to the project, it is a genre in its own right so not appropriate for all concepts.” Certainly there are examples of this. Alfred Hitchcock shot “Psycho” purposely in black and white to meet what he felt was the required tone for the film, as did CBS Productions with Rod Sterling’s original “Twilight Zone” television series. In both cases color was offered and refused as a medium.

In the 21st century, it’s perhaps hard to imagine a time when directors had such creative control over their own productions, especially productions with such important financial outcomes for their respective studios. Barry Sandrew agrees that films such as these are classics, which he says have been helped significantly by colorization. “The directors and actors were paid for their work,” says Sandrew, “that does not give them perpetual creative rights to the film. They do not own the film nor did they, in most cases, put up a dime to make the film. Colorization has actually served to subsidize the restoration and preservation of some of our most treasured public domain black and white feature films. In that regard, I believe that colorization has actually done a great service to these classics.”

Professor James O. Young, of the University of Victoria, wrote an academic paper entitled In Defense of Colorization in 1988 that stated once a work is modified it is no longer able to express its creators original intentions. His paper was written at a time when the first Turner produced color films came under criticism, namely because low quality colorization, restricted by the technology of the day.

But Young still believes its possible colorization can have a negative effect on a film’s artistic merit. “Appropriation from earlier art ought not to be prohibited,” he says. “Artists frequently borrow from their predecessors in a variety of ways and do so (some of the time) with good aesthetic results.”

Shaw argues that, regardless of 21st century technology, the same rules of artistic filmmaking merit apply. “We design images to fit the concept, to be emotionally evocative and to be easier for an audience to interpret.

It is a creative decision that should be made at the creation stage.” It would seem that the image (and color choice) of a film is therefore essential to its concept. Sandrew agrees with this up-to a point. “When I re-invented colorization in 2000 the film critics were no longer saying that the work looked shoddy or unrealistic. Instead they were saying that the colorization looked so natural. This raised a new concern from critics that the quality was too high and young people would never know what the film looked like in its original black and white format. So surprisingly, high quality colorization earned negative feedback as well!”

Ted Turner had the ability to spearhead a movement which, he personally felt, was of benefit to his consumer and made the business steps necessary to do this. But perhaps more ironically, as Young concludes, “the use of colorization has increased the distribution of original black and white versions, since these are now often packaged with color versions.”

Ironically, the greatest benefits come to those who feared their legacy might be jeopardized in the first place. While there’s no doubt life looks better in color the facts seem to indicate that audiences would rather see them as they were originally intended, sparks of geniuses intact. Colorization, like everything else in Film, has its place.

Tuesday, November 25, 2014

Why 3D Will Dominate Cinema In The Future

Originally published in Forbes ARTS & ENTERTAINMENT

We are five years into the emergence of 3D as a major part of modern film viewing, and yet every year there remains the droning sound of some pundits predicting the demise of cinema’s 3D “fad.” When 2013 was predicted to be the first year witnessing a decline in 3D box office, entertainment media was quick to suggest this was finally evidence of the press’ accuracy in insisting 3D was declining/dying/dead. But of course, despite the best efforts of opponents, 3D continues to contribute massively to global film receipts and won’t be leaving any time soon — a point I’ve had to make in the past, you might recall.

Indeed, Transformers: Age of Extinction, Guardians of the Galaxy, Dawn of the Planet of the Apes, Captain America: The Winter Soldier, Maleficent, The Amazing Spider-Man 2, and other movies received substantial boosts from 3D pricing worldwide. It’s particularly notable that The Amazing Spider-Man 2 was able to pull down $709 million worldwide due to a large amount of help from 3D that was best of any superhero film that’s been made, allowing it to rank as one of the year’s top-grossing films despite mixed reviews and the press focusing on a lot of negativity in reporting on the film (my own reaction to the film was decidedly more positive, as my review makes clear).

The next several years will see an expansion of 3D’s power at the global box office, with a series of brand new Star Wars films, Avatar sequels, and a huge growing slate of franchises and sequels in the popular superhero genre now that DC and Marvel characters will parade across the big screen at a rate double that of previous years (a total of more than 25 different superhero franchises will actively exist by 2020, as incredible as that seems). More than 70 3D films hit theaters in 2015 and 2016. And despite a modest leveling off of domestic audience attendance at 3D cinema since its modern reintroduction, it has held steady at home while foreign audiences continue having enormous appetites for the format. China in particular seems to love 3D.

I spoke with Legend3D ‘s founder, chief technical officer and chief creative officer Dr. Barry Sandrew before he departed Legend3D for his new position as chairman of the feature films section at Tesla Foundation Group (he also remains a member of the International 3D & Advanced Imaging Society). As my loyal readers will remember, I’ve spoken with Barry in the past about 3D cinema and innovations. So, when I learned he was leaving Legend3D, I reached out to him again for his thoughts about where things are headed with 3D in the coming years, and how new technologies in film and home entertainment will change cinema in the near future. In addition, Legend3D’s Stereo VFX Supervisors Tony Baldridge and Matt DeJohn, and Creative Director and Stereo VFX Supervisor Jared Sandrew also chimed in with some of their thoughts and experiences as well.

We discussed advances in post-conversion compared to native-3D, 4K and High Dynamic Range (HDR) advances that will change home viewing, theatrical laser projection, the genres most benefiting from 3D, and how 3D was used in films like The Amazing Spider-Man, Maleficent, and Transformers: Age of Extinction. Read on!

BARRY SANDREW: In many cases conversion is more efficient than filming in native 3D and in other cases a hybrid approach, using both native and conversion, is the preferred route. However, most filmmakers will agree that conversion is particularly advantageous in VFX-heavy films.

A natively captured 3D camera shoot creates a left and right eye image, where each image is generated from one of two cameras separated by a set amount called an interaxial distance. Visual effects elements are created based on the storyboard and subsequently, the natively captured stereo pair. Each VFX element must itself be either rendered in stereo — a costly process which essentially doubles the VFX vendor’s workload — or converted to a stereo pair and then precisely positioned and composited into each natively captured left and right image.

During the compositing process, the VFX elements must be adjusted to fit the captured images rather than the other way around. Bear in mind that each natively captured image, left and right eye, has inherent differences such camera tilt, focus, lens aberrations, color temperature, flares and surface reflections. These stereo image mismatches have to be taken into account through the life of a natively captured VFX shot.

TONY BALDRIDGE (Legend3D stereo visual effects supervisor): Sometimes the native capture cannot be resolved because the image mismatch is so bad. When this happens we have to take the unresolved differences and bake them into our VFX, which can open up a whole new can of worms. At times you can get away with it. Most of the time it just makes more sense to throw out one eye and convert the shot using the other eye along with the VFX.

On the other hand, in the Legend3D conversion process we’re able to easily interface with the vendor, share VFX elements and import the useful layers directly into our software 3D environment. We then manipulate our highly segmented (rotoscoped) 2D imagery into 3D to match the VFX data, all in one step. Once everything is placed within our 3D environment we can then create two or more virtual cameras to marry both the segmented 2D captured elements and VFX elements into a seamless, perfectly aligned, stereo image pair—a left and right eye differentiated only by the disparity our artists introduce during the creative process.

Conversion also offers greater flexibility in post-production. With the film completely shot in 2D, the Legend3D visual effect supervisor can manipulate and enhance the imagery in stereo, based on the storyline. They can draw attention to objects or subjects of interest in a shot by effective use of volume and interaxial separation, creating a wide or narrow lens look regardless of the lens actually used in the shot. We can affect the POV of a character or influence the audience’s perception of a shot by blocking it forward out of the screen or placing it deep into the screen to get a desired effect based on the storyline as well as the direction of the filmmaker.

BARRY SANDREW: Since digital TV made its appearance in the beginning of the last decade, the consumer electronics industry has been attempting to redefine the entire home entertainment experience. Sometimes they hit it right as in LCD and plasma flat screen technology as well as in the more recent introduction of today’s OLED TVs. However, at other times their strategy while correct in direction is more lemming like in execution, where TV manufacturers seem to follow each other in an over exuberant introduction of technology when neither the consumer nor the industry is ready for it. Indeed, I’ve written in my blog that the introduction of 3D TV was too much, too fast.

Of course, this year was the year of Ultra High Definition or 4K TV. And while I don’t believe that 4K alone is sufficient to appreciably drive new TV sales, the introduction of 4K provides less of a departure from the normal viewing habits of consumers than 3D and it will likely be a sought after feature once prices come down close to where they are now for HD TVs.

What the consumer will soon come to understand is that 3D survived the consumer electronics industry miscue and today it has become a standard feature in many large screen TVs, particularly those coming out of LG and Samsung. In fact, it’s difficult to purchase a 4K TV today that is not also 3D. The feature is not hyped and marketed like it was when 3D was first introduced but it’s there nonetheless. And once it’s in the home, it will be used.

I believe that the real advantage to 4K is that it further enhances 3D. It makes 3D better because it delivers full 3D resolution in TVs that require passive glasses. Likewise, 3D enhances 4K because most consumers don’t fully appreciate the difference between HD and 4K, despite the 4x higher resolution. TV aficionados may take offense to that statement, but I’m referring to the average consumer.

The technology that I see on the horizon, that will enhance both 3D and UHD into a uniquely superior visual experience, is High Dynamic Range (HDR). HDR significantly increases the color gamut of an image to more accurately simulate the resolution of the human eye, both in the number of colors as well as the number of shades of each color. The technology also increases the TV brightness by a factor of 40x or 50x. Considering the fact that we recognize TV as a source of bright-transmitted light, a 40 or 50 times brightness boost might seem to be an exaggeration. However, if you look around you during daylight, individual objects reflect significantly more light to your eye than can be reproduced on a TV screen. With HDR, the appearance of a scene or image becomes strikingly natural as it would in the real world. It’s something you have to experience to fully appreciate it but once you’ve seen it, it’s difficult to imagine ever watching TV without it. In fact the mantra at NAB this year was “we don’t want more pixels, we want better pixels,” a sentiment I wholeheartedly agree with.

What does all this have to do with digital cinema? Simply put, once HDR is married with UHD and 3D, the home TV viewing experience will far surpass that of current theatrical exhibition.

Since the advent of color TV, theater exhibitors have been living under a cloud of paranoia that the home entertainment experience will keep moviegoers away from their theater screens. That proved not to be true for color TV, nor did high definition influence theatrical box office numbers in a negative manner. However, I believe that once UHD, 3D and HDR find themselves in the home, theatrical exhibition will be significantly impacted for the first time in history. The bottom line is, the home entertainment experience will eventually establish a new quality bar that theatrical exhibition cannot compete with given their current installed base of technology.

Today, the resolution of most theaters is HD or 2K. In addition, the brightness reflected off of the theater screen is typically 50% darker than it should be and 3D glasses further reduce the brightness by at least one stop.

However, I see a bright spot in all of this, no pun intended. The remarkable advances that are coming to the home entertainment experience will force theatrical exhibitors to significantly upgrade their technology to laser projection. Laser projectors are the only way to bring higher resolution, greater brightness and high dynamic range to the theater screen. We’re talking about at least five years before theatrical exhibition will find this a tangible threat, but if planning is not started now, I believe theatrical exhibition will ultimately suffer. It’s possible that the costs of these upgrades will lead to a greater consolidation of theater chains with only the best capitalized surviving, but the advantages to the movie going consumer will be amazing.

BARRY SANDREW: Perhaps. In my three-part blog post “Cinematic Realism Considered,” I discuss how HFR and higher resolution significantly changes the cinematic experience for the audience. The question is whether that change is correct for all films or whether it will simply become a creative call by the filmmaker for selected films and genres. The standard cinematic frame rate of 24 fps that we’ve become used to over the past 80 years creates a subliminal detachment of the audience from the action on the screen. This translates into a sense of storytelling rather than immersive reality. If the use of high frame rate translates the cinema experience into a unique form of pseudo-reality that immerses the audience it will likely produce a different and perhaps unwanted cinematic experience for the moviegoer.

The bottom line is that HFR will likely continue to be used and will certainly be refined over time. I believe that variable frame rates will become the best use of the visual effect where different types of shots will benefit more from a certain frame frequency than others. Ultimately it will become yet another creative tool for the filmmaker.

Keep in mind, I make a distinction between the immersion created via HFR and immersion created by 3D. I believe 3D can create an immersive experience without giving up the storytelling feel of 24 fps. However, HFR appears to be an all or nothing influence on the cinematic experience.

BARRY SANDREW: It was clear that the filmmakers on The Amazing Spider-Man 2 were not at all shy about the 3D depth. The industry went through several years where filmmakers tended to be conservative, not wanting to appear gimmicky. However, The Amazing Spider-Man 2 showed that you can enhance 3D without being gimmicky.

MATT DEJOHN (Legend3D stereo VFX supervisor, who worked with Sony stereographer Ed Marsh on The Amazing Spider-Man 2): We identified the moments where 3D would add to the story and paid special attention to those shots. For example, when Spider-Man’s spidey-sense is tingling, time slows down for him and he becomes acutely aware of his surroundings and the challenge he’s facing. We supported that story beat by expanding the overall depth and by making sure even minute details were well-defined within that depth. This gave the viewer a heightened sense of spatial awareness, just like Spider-Man has. Beyond the big 3D moments, it was a general decision that being bold and maintaining a generous depth throughout was the way to go for a film like this.

[The] color grade certainly makes the 3D effect pop. The more texture, contrast, and color definition that an image has, the easier it is to perceive the 3D effect. On the other hand, vivid colors and high contrast tends to exaggerate any slight imperfections from the conversion process so we knew we had to maintain a very high quality bar throughout the film.

BARRY SANDREW: We partnered with Disney and the filmmakers in the conversion of about 7 minutes of select footage on Maleficent. The footage selected was mostly hero shots such as close-ups of the key characters Angelina Jolie and Sharlto Copley.

JARED SANDREW:  (Legend3D creative director and stereo VFX supervisor): Increasingly, we are finding that directors and studio executives appreciate the fine stereo detail that Legend puts into human faces, as well as our accuracy in 3D facial relations and natural 3D volume. Although many of the films we work on are heavy VFX tentpoles, both Legend3D’s software process and talent is becoming known for its ability to create the most realistic humans in 3D.

BARRY SANDREW: Indeed they are. If you look at the highest grossing box office films of all time and you remove Titanic that was a re-release, 10 out of the top 15 were 3D — all billion dollar-grossing films. Of the highest grossing films in 2013, 13 out of the top 15 were 3D. This trend is continuing in 2014 and it’s expected that the same will ring true over the next 24 months when there will be more 3D films released (>70) than ever before. I look at these numbers and have to laugh whenever I hear a critic proclaim the death of 3D, pondering what planet or alternate universe they’re living on.

It seems domestic audience appetite for 3D has leveled off in the last couple of years. Do you think it’s going to remain where it is, or do you see it changing — either up or down — in the coming years, strictly in terms of domestic audiences? And what are the primary reasons for changes in domestic audience attendance?

BARRY SANDREW: I think that the domestic audience is getting tired of the same genre of 3D films. If you introduce something unique like Gravity, the domestic audience rallies to the theater, but if one genre tends to define the medium it can get old, if not anticipated. I’m hopeful that the studios will begin to explore a wide variety of genres in 3D. I think then, the domestic audience will more readily accept 3D as a creative enhancement to the story rather than a spectacle.

BARRY SANDREW: I believe that animation and some sci-fi films have benefited significantly from 3D. Hands down, Gravity was one of the greatest beneficiaries of the medium. I wholeheartedly agree with the filmmakers when they said that if you saw Gravity in 2D you only saw 20 percent of the film.
Unfortunately, I believe 3D has been locked into a certain class of genre. However, there are other genres that have been successful. The Great Gatsby was beautifully executed in 3D as a native capture. I consider the visuals and storyline greatly enhanced by the addition of stereo in that film.  Post conversion has shown without doubt that live action films with little high-octane VFX can also benefit from 3D. One only has to look at Top Gun and Titanic.

Are there any genres that you think should be relying more on 3D, like regular action films outside of the superhero and sci-fi genres for example?

BARRY SANDREW: I believe that any and all genre of films would benefit from 3D if crafted correctly. Indeed, 3D creates an intimate immersive experience that translates as readily to love stories as it does to suspense thrillers, etc. I’m hopeful that over time, the greater sophistication gained by leading 3D filmmakers will open the gamut of genres considered for 3D. To stigmatize 3D by restricting it primarily to superhero, animation, or sci-fi films cheats the movie-going audience and stunts the advancement of the craft. Likewise, it’s a mistake to think that one size fits all… each genre and each film within each genre has to be looked at as a unique opportunity to use 3D in an effective immersive manner. To become formulaic in the creative use of 3D does a disservice to the medium and the audience.

BARRY SANDREW: As with any new advance in filmmaking the greatest creative innovation in 3D cinema will come as a result of filmmaker experience. The more a filmmaker is allowed to experiment and hone his/her craft, the more the medium will evolve. As I mentioned before, I see the industry stagnating in formulaic 3D genre.  To truly progress the industry needs to explore 3D in every genre. The movie going audience also has to achieve a greater degree of sophistication. Unfortunately a significant portion of the movie going public still rates the success or failure of a 3D movie by how many objects are thrown out of the screen. I find that disheartening. Until that perception is tossed and a more refined appreciation for the immersive nature of 3D is achieved, I believe the medium will not fully progress.

BARRY SANDREW: Tony [Baldridge] was the stereo visual effects supervisor responsible for Legend3D’s creative and technical conversion production of Transformers: Dark of the Moon and Transformers: Age of Extinction. In both cases, Tony worked hand in hand with Corey Turner, one of the top stereographers in the business and currently vice president of post production at Paramount. [Note: Read my Forbes interview of Corey Turner here!]

I feel that Tony defined the role of stereo VFX supervisor at Legend3D 4 years ago during the conversion of Dark of the Moon. That film was the most complex and technically challenging project Legend3D or any of our competitors had faced at the time.  Tony stepped up to that challenge and created a unique conversion team that was technically malleable enough to turn on a dime, yet proficient enough to hit every deadline.

When we were first awarded the contract on Transformers: Dark of the Moon I made a strategic decision to bring on board Scott Squires, the former ILM CTO and one of the most admired visual effects supervisors in the industry. He was new to the conversion process but working with Tony Baldridge, Scott was able to contribute to our overall pipeline design and effectively liaise between our Legend3D team and both the Digital Domain and ILM [Industrial Light & Magic] VFX teams.

I found inspiring the dynamic collaborative effort between our production and pipeline teams — defined by Tony and Scott— and the innovative software development directed by them together with my technology partner, Greg Passmore of Passmorelab, perhaps one of the most brilliant software engineers and physicists I’ve known in my career.

During the conversion process Tony established the protocol for importing and handling all the VFX assets in a manner that we still use today. He was able to effectively restructure our pipeline and software to interface with both VFX vendors. Indeed, during the conversion process we were constantly exchanging assets between Legend3D and both VFX studios.

TONY BALDRIDGE: One key lesson learned was that our conversion pipeline had to be highly adaptable. On a daily basis, ILM and Digital Domain turned over many VFX scripts along with accompanying digital assets. Each shot was treated as a giant puzzle. We had to determine how to plug the pieces into our own pipeline in a timely and efficient manner. By the end of the project, when the majority of VFX assets were transferred to us, we had gotten pretty good at the process. What really helped us was the tight working relationship developed with Digital Domain and ILM and our technical ability to adapt.

BARRY SANDREW: Transformers: Dark of the Moon was a hybrid project where our conversion work had to be intercut in a true checkerboard manner with natively captured shots.

TONY BALDRIDGE: The challenge was to make the multiple formats, both 2D conversion and 3D native capture, intercut seamlessly which we accomplished in spades.

BARRY SANDREW: At the end of the project, the fact that we finished two weeks ahead of time is testimony to the talent and knowledge of our team headed up by Tony and the skill and experience of Scott.

BARRY SANDREW: We basically dissect and analyze the stereo IMAX footage to understand the native capture of 3D. We then take that information into consideration when converting the 2D footage so the two different sources of stereo imagery seamlessly blend. It’s ultimately up to the visual effects supervisor to make it all work together and ensure that all sequences blend.

TONY BALDRIDGE: IMAX isn’t the only format we came up against that required special attention. We also had to contend with a smorgasbord of image capture devices Michael Bay used, including iPhone and GoPro. It’s tricky to make all those formats make sense in 2D let alone 3D. Bay’s not afraid to experiment, that’s for sure, and we’re happy to take on the challenge of making it all fit as he intended.

One thing I noticed is that, in the sequences filmed in IMAX 3D Michael Bay obviously set up the scenes and action in ways to maximize the 3D effect and provide great depth and weight to the visual effects in a way that maximized the impact of the 3D imagery; but then, in the scenes that required post-conversion 3D, it seems he was still directing those scenes to provide for a lot of placement in foreground and background to achieve the same sort of 3D effect. We’ve talked in the past about this sort of attention to detail in Dark of the Moon, but can you speak to it again with regard to this latest film?

TONY BALDRIDGE: Michael Bay figured out during the production of Dark of the Moon that if you block a shot specifically for stereo the audience will be able to better translate the 3D. You can actually process what’s happening through several layers of smoke, explosions and fighting robots easier in 3D than in 2D when the shot composition and blocking is thought through from the beginning. … [T]he pacing of Age of Extinction was different and designed to make the 3D experience much more immersive. Michael came to recognize that you can’t always achieve the desired feeling during fast cuts because it’s often difficult for the audience to know what’s going on. Michael clearly honed his 3D craft on this latest Transformers tentpole and it shows in nearly every aspect of the visuals he put on the screen.

What were some of the more difficult sequences you had to tackle this time around, or ones that presented particularly unique challenges you maybe hadn’t directly confronted in quite the same way before?

TONY BALDRIDGE:  [A]t this stage in the game, each film presents unique challenges, but no surprises. There are never any insurmountable issues at Legend3D during the conversion process.

BARRY SANDREW: I believe it’s our overall philosophy in conversion production that sets us apart. From the software and technical development to our artist training, we’ve managed to create an academic approach toward the problems of conversion while maintaining a bottom line vision.
We’ve created sophisticated technology that doesn’t intrude on the creative process. It allows artists to be artists, rather than technicians or engineers.

We’ve found that talent and technology are essential ingredients in delivering conversion that accurately interprets the filmmaker’s vision, but perhaps the most essential thing we bring to the table is the in-depth experience our team has gained on an incredible variety of films that we’ve converted over the past five years. You can have the most talented artists and the best technology but without experience, it’s impossible to create exceptional conversion.

Thanks to Barry and everyone at Legend3D for taking so much time to discuss all of this with me!