Saturday, August 23, 2014

Master of Conversions: Legend3D's Barry Sandrew Goes Deep.

Master of conversions: Legend3D’s Barry Sandrew goes deep

Aug 18, 2014
- By Doris Toumarkine

'The Lego Movie' is one of this year's hits that turned to Legend3D.

The Hollywood career of Dr. Barry Sandrew, who founded 3D-conversion and visual-effects giant Legend3D, moves from black-and-white to color and now to state-of-the-art 3D, as if mimicking the evolution of the movie industry itself.

Carlsbad, Calif.-based Legend3D, leveraging Sandrew’s many patents and inventions, specializes in 2D-to-3D conversion technology and services. Working at the cutting edge, the company has been integral to bringing that additional third dimension to many features over the past five or six years, including such recent and current international blockbusters as Transformers: Age of ExtinctionThe Amazing Spider-Man 2,The Lego Movie and Maleficent. Older sparkling jewels in the Legend crown include Hugo and Life of Pi.

Sandrew is also Legend’s chief technology officer and chief creative officer. Indicative of his importance in the 3D tech world, he was inducted last March into the International 3D & Advanced Imaging Society board of governors. The organization is the 3D industry’s largest and most important group. It represents the interests and goals of the 3D business internationally with a mission to advance the creative arts and sciences of stereoscopic 3D and 4K. (Disney, DreamWorks, Sony, Paramount and IMAX are just a few of the Society’s big-name founders.)

Sandrew’s journey to 3D is Hollywood-worthy. A PhD neuroscientist, he invented Legend’s proprietary 3D software after more erudite beginnings as an East Coast academic on staffs at Harvard, Boston’s famed Massachusetts General Hospital, and Columbia’s College of Physicians and Surgeons.

His career first took a hairpin plot turn in the late ’80s that even a Hollywood script couldn’t imagine: He devised the proprietary software for the controversial but jaw-dropping colorization process and made an extraordinary leap to the West Coast to build it. His inventions for Hollywood emerged from work done from 1978 to 1987 at Harvard Mass General Hospital. There he established neuroscience labs including an imaging lab centered on MRI, CAT and PET scans of the brain.

Although at the hot center of the 3D frenzy powering blockbusters worldwide, Sandrew remains more soft-spoken scientist than Hollywood player. “I’m immersed in science, but left [Boston] in 1987 to come to California to invent colorization,” he recounts. “Back then, some entrepreneurs approached me who wanted to digitally colorize black-and-white movies. The colorization process at that time was analog, producing extremely poor color quality. They understood that if you colorize a public-domain film, the colorized version is eligible for a new copyright that they would own. They knew about me and my medical imaging work, which is how I was brought into the colorization business.”

Out West, Sandrew founded American Film Technologies (AFT) and served as chief technology officer as inventor and patent-holder of the first all-digital colorization process that was used to colorize more than 250 motion pictures, including “essentially all of [Ted] Turner’s work.” The process most memorably brightened (literally) Ted Turner’s cable channels but frightened, even horrified, film purists. Deemed by many as tampering with an auteur’s vision, colorization faced a considerable backlash, but Sandrew—Our Man from Academe—weathered the storm: “I was naive about all [the controversy] because I was from such a different field, so it didn’t affect me. The irony is that the more critics complained, the more successful colorization became.”

After leaving AFT, Sandrew was a founder of Lightspan, Inc. in 1993, one of the largest educational software companies in the U.S. selling K-6 “edutainment” DVDs meeting all state standards to entire school districts. Then came Legend Films, which he founded in 2000 when he reinvented the colorization process with digital tweaks made possible by significant advances in computer technology.

Says Sandrew, “I invented colorization in the early ’90s and was approached in 2000 by a friend, Jeff Yapp, formerly president of Fox International Home Video, who wanted me to invent colorization again but in a more refined manner producing far superior color using the latest technology. So I invented a new process for colorization and we did a lot of work on public-domain and major studio catalog titles. Then in 2006 when my technology partner Greg Passmore showed me 3D technology for the first time, I was blown away and I immediately had a sense of where it was going. So I diverted my R&D away from colorization to 3D.”

The advanced colorization work, in fact, enabled the platform from which he created Legend3D’s proprietary 3D conversion software. Simply put, the jiggling of pixels enables both platforms. Colorization and 3D processes are, in fact, related and indeed “similar in the early stages.”

Legend was renamed Legend3D in 2010 when, again empowered by his proprietary software, Sandrew shifted gears from colorization to 2D-to-3D conversion.

The majority of Legend3D clients are Hollywood studios and their first-run features. But things old could be new again. Legend’s proprietary 3D conversion makes it possible to convert any 2D film to 3D, including the hundreds or maybe thousands of musty or newer features that sit in distributors’ libraries and comprise their backlist. If content is king, Legend3D’s software could enable any feature to re-enter the marketplace as 3D royalty.

But such a possibility raises the question of what kinds of films—what genres—are appropriate for conversion, assuming the 3D conversions would make these films more appealing to audiences. Asked this inevitable question, Sandrew ventures, “We don’t really know yet which are best for the format, because to date the industry has been focused on limited genres and maybe just about anything will be enhanced with 3D. Look at [the 3D conversion of] Top Gun, which is really more of a love story than anything and it’s great in 3D.” He adds that he’s a fan of catalog titles and believes that many franchises like the MatrixPirates of the Caribbean and Harry Potter films would work as conversions. In fact and no surprise, Sandrew doesn’t name any film or kinds of films that shouldn’t be considered good candidates for 3D conversion.

Mainly, the company is after the big fish: “Our sales force,” says Sandrew, “is focused on the new studio films emerging, not so much the catalog titles.”     

Legend plays well with IMAX: “In Transformers [Age of Extinction], Michael Bay used many digital camera formats including IMAX and we had to match the IMAX look, but there was no problem. IMAX is not an issue for us al all.”

Sandrew cites IMAX as a company that is aggressively moving toward leading-edge display technologies. “They were already delivering the brightness required for theatre audiences and now they’re installing laser projectors that will improve the theatre experience even more. This is happening in China and South America and in five years will be ubiquitous.”

According to Sandrew, Legend has a “very limited number of competitors” and, like the competition, the company finds its clients through “a very short list. We all know what films are coming out, so we’re all bidding against each other on the same titles. We’re often brought in at the end after the 2D shoot.”

While all company projects involve Sandrew’s proprietary software for the conversion of 2D material to 3D, work varies according to whether features are “native” 3D films (begun as 3D camera-captured productions) like Hugo and the first Amazing Spider-Man, “hybrids” (a combo of material captured during production as either 2D or 3D) like Transformers: Age of Extinction, or full 3D conversions of original 2D films like The Amazing Spider-Man 2Man of Steel and Top Gun.

Without understanding the mechanics, many of us take 3D for granted, already knowing that each eye captures a slightly different perspective or angle that the brain next processes into a single image. Or is that it? Sandrew helps give a little more, er, depth to the 3D magic:

As he explains it, stereophonic, so familiar to music lovers, enables an individual to hear two audio feeds but experience these as one. Similarly, stereoscopic is the underlying quality of 3D, the experiencing of two visual feeds as one. The eyes converge to see the stereoscopic effect of two distinct visual captures so that the actual perception of 3D only happens inside the brain. The eyes do see two different objects, but the brain takes over creating one additionally endowed with depth, volume and solidity. In 3D, the whole is definitely greater than the sum of its parts.

If perception is at the heart of 3D, misperception is at the heart of the notion that 3D popularity is sagging, Sandrew contends. These notions are floated in the media, with blame placed on fickle audiences, ticket upcharges, inadequate screen brightness or even the productions themselves.

“[Declining interest] is more misperception than reality,” he continues, reminding that 3D grosses have hit an all-time high and that of the top 15 performing films of all time, ten were in 3D, and that 2013 was a stellar year when 13 of the 15 top-grossing films were 3D. He predicts that this year and 2015 will mark the biggest performances ever of 3D product.

And not to be forgotten, he adds, is that “while 3D continues to grow here, it is even much more popular internationally, especially in China.” 

Sandrew continues: “‘Jaded’ is a good word for what might be a cooling of audience enthusiasm, but look at China, where the upcharge and glasses are signs of a premium experience instead of what some in North America deem an annoyance. But in other parts of the world, it’s that coveted premium experience.”

But 3D is not just for theatres. 3D TV has hit the home front, although not yet claiming any victory. Sandrew is also a big believer in 3D TV for couch potatoes and sports-bar fans and does not buy into the perception that 3D at home is “dead in the water.” The stalled status of 3D for smaller screens doesn’t bother him: “The consumer-electronics industry introduced 3D TV too fast and in too grand a fashion—consumers weren’t ready for it. But today it’s difficult to purchase a high-end flat-screen TV that is not 3D-enabled. So, as people buy 4K TVs, the installed base for 3D TV will grow. Once these are in homes and the high dynamic range is available, people will use them [for 3D viewing].”

Technology keeps roaring along, but exhibitors too can get active and need not just be vigilant on the sidelines awaiting the latest and greatest. They have options today, says Sandrew, for improving the cinematic experience for their audiences. Eventually, he notes, 4K will be further enhanced by the introduction of high dynamic range (HDR) both in and out of the home. “At NAB 2014, the mantra was: We don’t want more pixels, we want better pixels, brighter pixels with a greater color gamut which more precisely captures what the eye sees.”

But, he cautions, “these advances will ultimately make TV a superior viewing experience than what audiences will be able to get in theatres.” He reminds that “unlike IMAX, traditional theatre screens are typically not bright enough and 3D glasses further darken the movie screen.” But as home viewing improves, he speculates, “this will force exhibitors to upgrade to laser projectors which raise the resolution bar, improve brightness and introduce high dynamic range. Right now, theatres can offer filmgoers passive glasses and make sure the venues are clean. Perhaps number one is cranking up the projector bulbs to a minimum of six foot-lamberts so that the images get brighter. They now tend to be dull.”

Filmmakers too are doing their part by "being less shy about 3D,” he says. “In the early days, there was the belief that pushing 3D would make movies appear too gimmicky, but look how well 3D works with so many films. The Amazing Spider-Man 2 is a good example of how far 3D can be pushed without appearing gimmicky.”

As for that all-important question of when 3D viewing without glasses might be possible for cinema audiences, Sandrew suggests that “glasses-free is somewhere out there but not imminent. It will eventually happen, but not with the technology that exists now.”

Taking a more in-depth look at Sandrew’s conversion technology, it comprises three steps: separating the different elements (or planes) in a shot, giving these depth and volume, and painting in the gaps between the objects and the backgrounds from which they were pulled forward. Rotoscoping (manipulating individual frames) is common in the first step, separating the elements, but Sandrew also applies his pixel-specific process developed from the company’s colorization work, which entails his creative staff determining a sampling of pixels in specific objects to be separated. The tedious frame-by-frame work is then outsourced to one of Legend3D’s India facilities.

Early last year, Legend moved into 60,000-square-foot state-of-the-art studio headquarters in Carlsbad and the company recently announced the opening of their new 3D L.A. Hub in Hollywood, which Sandrew describes as “a one-stop finishing solution for the stereo conform and post needs of our clients. We’ll be state-of-the-art with our color-grading capabilities.”
Known in-house as the Legend3D Hub, the facility will offer a superior screening facility and allow filmmakers to review their 3D conversions in continuity on the latest RealD screen technology, and do conversion adjustments in real time. The Hub, still awaiting its formal name, will be equipped with a full editorial infrastructure and a 10-gig pipeline to the Carlsbad headquarters.

So 3D is thriving, but wait a second—this is 2014 and technology advances every second. When talking 3D and immersive images, the subject of holograms inevitably arises. Might these ever become part of the film production or conversion process? “A year ago I put it in the sci-fi category,” says Sandrew, “but I have associates who have worked out some interesting possibilities. Maybe it’s ten years off before that technology can be done, but there’s some potential.”

Let viewers and the industry beware: Blade Runner2001: A Space Odyssey and Dick Tracy were also once in “the sci-fi category." 

Sunday, June 29, 2014

3D Film Conversion in 2014-2015

3D Film Conversion - New Challenges

Originally published in CIO Review
To the naysayers proclaiming the demise of 3D, I respond by informing them that eight out of the top ten highest grossing films in 2013 were 3D, with six of those films being converted from 2D-to-3D. The fact is 3D continues to bring in very attractive box office revenue numbers. Consequently, the major studios are not about to abandon 3D releases and leave incremental ticket sales on the table. With that in mind, this year the industry will be releasing more 3D films than ever and 3D will start to have a dramatic influence on other industries including advertising, television broadcasting and visual display markets. As a leader in 2D-to-3D conversion services, the following article leverages our expertise to take a closer look at the 3D industry and further examine IT challenges and solutions.
"With a 4K production pipeline plan in place VFX and conversion studios must be able to handle 4 times the amount of data as high definition"
2D-to-3D Conversion Industry
The conversion industry has matured to the point where the short list of major players have it figured out and have developed advanced pipelines. While each new film brings with it unique technical challenges, there are few—if any—surprises that negatively influence a converted film. One of the most significant features of any conversion pipeline is flexibility. Pipelines need to be sufficiently adaptable to take on smaller projects such as commercials or movie trailers. The less pipeline stages and related specialist hands that smaller projects require, the greater the profit margin. Each feature film project typically goes through the same series of processes, requiring robust asset and production management tracking and control.

The increased bandwidth and storage required for Ultra High Definition (UHD) and in the future, High Dynamic Range (HDR) will create new technical challenges for visual effects facilities and 3D conversion houses. UHD was again a buzzword throughout the 2014 Consumer Electronics Show. However, it was HDR TVs that really won attendees over. HDR displays a wider gamut of color and it’s considerably brighter than a normal TV.  It comes much closer to the way our eyes see color.  The whole image is enhanced to the point where it looks “photo real.” Over the next year, we’ll see these technologies develop and start to infiltrate all aspects of the media sector.
A general mandate in the industry is to have a 4K production pipeline in place. That means that VFX studios must be able to handle 4 times the amount of data as high definition, and when you add another image representing a second eye in 3D conversion, it creates an 8K challenge for servers and render farms. Many of the large industry players, like Legend3D, have already developed or are in the process of developing 4K pipelines in anticipation of future studio demands.
The Global Workflow
Today all of the major conversion companies are consolidating operations in territories like Canada, the UK, Australia and New Zealand, where they are finding the most attractive government subsidies. With the price-per-minute cost of conversion continuing to go down, many of the major conversion vendors are also finding it necessary to outsource work, primarily to India, as a means of remaining competitive by supplementing their operations in tax incentive havens. This global approach is essential for budget management but has placed significant emphasis on the logistics of moving huge amounts of data in the most secure fashion. According to Legend3D Director of IT Anthony Lopez, “While there are clear advantages to outsourcing work worldwide, you always need to be vigilant and committed to the efficiency of how you move data to ensure that your global teams have timely access to their work.”

The Art of Selecting Partners
Partnerships with storage providers and software vendors are essential to the success and viability of visual effects companies serving entertainment studios. The conversion process requires state-of- the-art software and complex storage facilities to keep high-profile operations running smoothly and accessible 24/7.
To achieve this, it’s recommended to research skilled vendors who offer the flexibility to customize services to fit your exact needs. Ultimately, you want a partner that introduces added efficiencies, understands your business and can match your exact needs with minimal changes required to your current IT processes.

Parting Thoughts
Given the rapidly changing technology industry and fast-paced demands of the media sector, it’s pivotal that IT departments in this business stay one step ahead of the curve and are always dialed into the latest trends and upcoming technologies.
The bottom line is that 2D-to-3D conversion is continuing to thrive within the Hollywood community. The playing field has changed considerably and strategic outsourcing for cheaper labor and tax incentives is playing a significant role in bringing the cost of conversion down. Conversion companies must continue to advance their technologies to stay ahead of the curve and remain a market leader. One cannot minimize the importance of a strong Director of IT and a cohesive IT department, along with strategic partners, to both accommodate challenges and handle the logistics of moving data around the globe in a timely and secure way.

Wednesday, June 18, 2014

The Amazon 3D Smartphone Could Revolutionize Cell Phone Display Screens

Amazon Smartphone: 3D Retina Technology Could Revolutionize Cell Phone Display Screens

on June 18 2014 1:33 PM Originally Published in International Business Times
Audience wearing 3D glasses watches a Kraftwerk show

When rumors began to circulate in recent weeks that the June 18 Amazon event would in fact be a launch party unveiling the e-commerce company’s new 3D smartphone, the tech world responded with a collective shrug. Hollywood has spent decades working to convince TV and movie audiences that 3D is a thing of the future, if only people were willing to wear a pair of plastic blue-and-red glasses.
Developers and film veterans wondered out loud why the future-focused Amazon, which has teased drone delivery and sparred with struggling book publishers, would try to go down the same path along which so many others have failed. But a 3D phone might actually be crazy enough to work, and possibly be successful enough to change consumer expectations about new products.
Dr. Barry Sandrew – a technology patent owner and the founder, CTO, and COO of Legend3D, which converts 2D video into 3D format – said a smartphone, if developed the right way, could be the perfect entry point for a public ever hungry for a flashy new device.
“I believe that residual stigma attached to 3D came not from bad theatrical 3D conversions but from the consumer electronics industry, which introduced 3D TV too fast and with too much hype,” he said in an email to IBTimes. “Consumers were not ready because there was simply not enough content to make the purchase of a 3D TV worthwhile. In addition, I think it’s clear that the requirement of wearing 3D glasses became a negative issue – due primarily to influence from the media.”
To replicate “Avatar’s” success, and avoid the fate of a colossal flop like “Step Up 3D,” Sandrew said Amazon has no choice but to properly utilize autostereo (a concept that essentially boils down to glasses-free 3D viewing). This technology requires customers to “position themselves within a sweet spot in front of the image so that an overlaid sheet of corrugated ridges accurately refract reflected light to the appropriate eye,” Dr. Sandrew explained.
Advertisers have found success with this “sweet spot” when installing a 3D sign into a casino lobby, for instance, where someone approaching a sign would see it as blurry only to stare directly at it and find a bright, imaginative sign. An Amazon 3D phone could find the same success, with customers having little choice but to stare directly at their screen to use it.
“When applied to single users devices like smartphones, tablets and even laptops, the sweet spot limitation is not as much of an issue because the user typically positions their head in front of the display and can move the device backward and forward from their eyes to get an optimized stereo effect,” Dr. Sandrew went on.
As 3D TV developers have learned the hard way, the autostereo challenge could still be insurmountable for larger screens. Designers would need to conceive of some way to replicate that sweet spot for people sitting in different areas while looking at the same screen in a room.
“I’ve seen some decent quality autostereo video and images on smartphones such as the discontinued HTC 3D phone and I’ve seen adequate autostereo on a handful of tablets that are just making their debut in the market,” Sandrew said. “However it’s been clear that the technology has a long way to go before it can be considered a solid consumer product – that is, hopefully, until now.” 

Friday, June 13, 2014

The Fate Of Theatrical Exhibition in the Digital Age of TV

How Movie Theaters Will Get Their Mojo Back

                             Also published in Wired Innovation Insights - June 13, 2014

In a recent interview conducted by TheWrap I discussed how advances in home entertainment display technology will very likely trump today's movie going experience. Indeed, the next generation of home entertainment TVs will not only have greater spatial resolution and 3D, but they will also include High Dynamic Range (HDR) which significantly opens up the color gamut from 8 bits per color to 10 or 12 bits per color.  This increases the range and shades of emitted color from 36% of the CIE color space to almost 76%, creating a viewing experience that more closely mimics the color resolution of the human eye.

Advances in Digital TV Require New Standardization Specs

The latest advances in digital TV technology have spawned a new standard called Rec 2020 which is intended to replace the old Rec 709 High Definition (HD) TV standard set back in the '90's.  Rec 2020 establishes standards for 4K and 8K resolutions, which are 4 times and 8 times the resolution of our current HD TVs respectively, various high frame rates (HFR) up to 120 frames per second, as well as significantly greater brightness, up to 40 to 50 times the brightness of contemporary TVs along with an accompanying increase in contrast. Of course, most big screen TVs will also feature immersive 3D that requires passive glasses. As I stated in The Wrap interview, these dramatic innovations in home entertainment will most certainly pose a competitive advantage over our neighborhood multiplex and megaplex theater chains because our home TVs will eventually offer a far superior visual experience.

Standards In Screen Projection Will Become Essential To The Survival Of Theaters 

Xenon Bulb
I'm not including IMAX in this discussion of advanced digital TV's competitive advantage against theatrical exhibition because they are already going the direction of laser projection.  The value chain of IMAX has always been and continues to be unique to that particular format, providing an optimum standardized viewing and audio experience. But traditional theaters often do not adhere to a single standard for their exhibition screens. Entertainment professionals mandate that theatrical exhibition of first run feature films should always be projected at a minimum of 6 ft lamberts but all too often, the projector bulbs in movie theaters are either set considerably lower, often at 3.5 ft lamberts or less and/or the bulbs have not been adequately maintained or switched out at the end of their maintenance life cycle. Cost is most often the factore in suboptimal theatrical exhibition. There is often a desire by the exhibitors to maximize bulb life by turning them down in brightness. The problem with this practice is that it invariably affects the movie going experience in a negative way. This is most true in 3D theaters where the 3D glasses further reduce the brightness of the projected image.

Does This Signal The Death Nell For Theater Exhibition?

Despite future advances in TV technology, theatrical exhibition of feature films will not go the way of the dinosaurs. In reality nothing could be further from the truth.  Theatrical exhibition will continue to thrive, but only if theater owners recognize early on that advances in digital TV technology will eventually cut into their margins. This should force them to reexamine the quality of their feature film viewing experience rather than simply concentrating on ancillary income streams from online loyalty campaigns and food concessions. The value chain of theatrical exhibition has to be refocused on the essential and historical value proposition of movie going–creating a unique viewing experience that justifies traveling to a theater and paying the price of a ticket to see a first run feature film.

As a consequence, I believe that exhibitors will have no choice but to upgrade their theaters with the latest laser projectors that will serve as a solid foundation for future advances in theatrical display.

Toward that end, the dawn of the Laser Projector for theatrical exhibition is upon us. High resolution, high dynamic range, 3D laser projectors being developed by Christie, Barco, NEC and others increase the projected brightness of a feature film from 3.5 ft lamberts to 14+ ft lamberts and both gamut and saturation will be far superior to current exhibition standards. If anything, I believe traditional theaters equipped with laser projection should be able to offer IMAX a run for its money. 

Will We See A Further Consolidation of Theater Chains?

We've seen the theater industry in the U.S. change considerably over the past two decades.  The age of the downtown family run theater has long past. If they have survived at all, these single screen theaters have typically been transformed into art houses.

The neighborhood theaters found it impossible to compete with the suburban and mall multiplex and megaplex theater chains which tend to draw audiences from 4 to 5 times the distance depending on the population density. With the coming of the next generation of TVs threatening to cut into the margins of literally all theatrical exhibition, I believe the critical yet costly requirement to upgrade to laser projection will force a greater consolidation of theater chains where only the best capitalized will thrive.

Will Feature Film Release Schedules and Distribution Continue To Collapse?

Feature film release schedules might change considerably because of the advances both in home and theatrical display devices. There are really two populations that need to be served–audiences who have the resources to purchase the latest TV and prefer to watch first run feature films at home and those who, because of personal preference or various economic and/or social reasons prefer to watch feature film releases in the theater.
We're already seeing simultaneous releases where first run feature films released into the theater are also offered on TV as day and date on-demand services. As the distinction between home theaters and multiplexes eventually diminish, I believe that simultaneous day and date streaming of first run feature films to the home will become the norm.

In Conclusion

I see the exciting and dynamic future of home entertainment display devices a necessary wake up call to theatrical exhibition owners. There will come a time in the not too distant future where the survivability of theater chains, facing looming technological superiority of the home entertainment market, will depend on their timely move from antiquated xenon bulbs to the superior technology of laser projection. While the upfront installation costs of laser projection are significant, the long term economic gains from the draw of larger and newer audiences as well as the savings in terms of labor and maintenance will more than justify the expense over time. As a result everyone wins.