On Thursday, Lytro announced it has jumped into the video arena with Immerge, a video camera that uses light field tech to capture and blend realistic, immersive live action video and computer graphics.
“Immerge builds on the technical achievements that were unlocked by Illum and moves the technology further into live action video and a 360 virtual space,” Ariel Braunstein, chief product officer at Lytro, told Macworld.
This is a natural evolution for Lytro. Back last summer, the company released a massive update to Illum—a combination firmware and desktop upgrade that positioned the camera as a content creator for virtual reality headsets like the Oculus Rift, Sony PlayStation VR, Microsoft Hololens, and HTC Vive.
And now, with the help of a Lytro software plug-in, Immerge output can be edited on your Mac.
Immerge, currently still in the prototype stage, targets the pro movie-making crowd. It will be released to creative storytellers in the first quarter of 2016, the company said.
The main point of entry for editing is a cross-platform compositing and visual effects package from Foundry called Nuke for which Lytro has created a plug-in.
“We chose not to reinvent the editing environment,” Braunstein said. “Nuke is a post production visual effects standard. Editors can edit every pixel, mix it with computer graphics, and apply visual effects.”
While AR and VR have typically been used to create 3D gaming environments, one breakthrough for Immerge is its capability for realistic and immersive output for dramas and documentaries. This would significantly broaden the target viewing audience as well as creative possibilities for moviemakers.
Immerge is designed to capture once and play back on any device. While it’s optimized for high-end headsets, it also works with mobile phones as the screen for a VR device, such as Google’s Cardboard or other simple, low-cost headsets that include only optics.
Immerge was created to combine live action content with computer graphics, but first it had to solve the 2D problem. “A lot of the technologies available today are cobbled together from 2D and don’t work well in a 3D environment,” Braunstein said. “2D tech provides a compromised sense of presence—meaning you know where you are.”
For example, 2D images in VR must be stitched together, but when they are, the resulting seams and lines form a distracting discontinuous space. A talented video editor can hide seams, but not eliminate them.
Within a light field volume, Immerge captures data allowing for virtual views to be generated from any point, facing any direction, and with any field of view. This creates a much more realistic sense of presence, previously experienced only with computer graphics.
In addition, Immerge places viewers in the action by replicating natural light flow and corrects stereoscopic alignment to keep scenes consistent as users move their heads from side to side. The camera corrects flawed visual parallax, a phenomenon where moving objects at different distances appear to move at different speeds.
It allows for natural (“six degrees of freedom”) head movement in any direction with the ability to view the world in a natural way—side to side, up and down, and back and forth. Three additional axes of movement pivot on top of those three.
Immerge is designed not only as a camera, but as a complete system, providing all the necessary hardware, software and services required to capture, process, edit, and play back content.
It features a flexible, configurable dense light field camera array, a server for storage and processing, the Light Field Editor that integrates with Nuke, and the light field video playback engine for VR headsets and platforms.
The Immerge server stores up to one hour of capture, processes the light field data , and integrates with network and enterprise storage as well as VFX tools and even provides a portable render farm.
Lytro plans to make the Immerge system available for purchase or rent by storytellers during the first quarter of 2016, and to the general public later in the year.
Why this matters: Advances in the video realm make Lytro an important part of creating and combining live action and computer graphics for a new kind of immersive storytelling and VR-based presentations for mainstream audiences.