The improvements, announced at the Video@Scale conference Thursday at Facebook offices in Menlo Park, Calif., are being made to Facebook itself, but anyone can access the 360 transformation code on GitHub.
Managing the growth of video and scaling required retooling across the stack, ranging from new infrastructure to handle bigger loads to encoding techniques for better streaming to machine learning to better understand video content, Facebook said.
"Both 360 video and VR create immersive environments that engender a sense of connected-ness, detail, and intimacy," said Facebook's Evgeny Kuzayakov and David Pio, in a blog post. "Of course, all this richness creates a new and difficult set of engineering challenges. The file sizes are so large they can be an impediment to delivering 360 video or VR in a quality manner at scale."
To address this, Facebook created 360 video and VR encoding optimizations, moving from equirectangular layouts to a cube format to reduce file size. With 360-degree video for VR comes a unique set of challenges, Facebook said, including even bigger file sizes and streaming without waiting for buffering. Encoding 360 video with pyramid geometry reduces file size by 80 percent, the company said. Also, view-dependent adaptive bitrate streaming now allows Facebook to optimize the VR experience. Facebook introduced 360 video last September.
To improve streaming video, Facebook has developed a geometry for encoding 360 videos in VR to stream high-resolution videos without waiting for buffering. The company also built a SVE (streaming video engine) for transcoding video and migrated video traffic from its old system. Rather than treat a video as a single file, SVE splits video into segments, allowing parallel processing of videos and resulting in lower latency.
Facebook said it built SVE as the site was experiencing growth from 1 billion to 8 billion video views per day, without any service interruption. SVE offers a tenfold improvement in processing time between video uploading and playback, and it now serves all new Facebook video and videos on the Facebook Messenger service.
Facebook's Vision Understanding team, meanwhile, is investigating ways to understand video with unsupervised learning models. Instead of labeling objects, scenes, and actions, these networks label voxels, or individual video pixels over time, Facebook said. Using unlabeled data, the work presents an AI architecture that is a seat toward unsupervised understanding of what is happening in a video and predicting what happens next.