FORMAT



Videocodecs - the next generation

The race for the top position in next-generation video codecs.

Introduction

At the latest with the establishment of ultra-highresolution recordings, high colour depth and wider gamuts, the question of the industry leader in video codecs arises again.

On the one hand because the previous standard codec H.264 (up to a maximum of 4K) and its successor H.265/HEVC (up to a maximum of 8K) are protected by patents, and licensing is expensive and sometimes confusingly organized. 
On the other hand, there are now at least equal codecs that are freely available and can be used for example on YouTube for encoding or are in the development phase.

Nevertheless or just because of that, HEVC is currently cooperating with hardware and software manufacturers at short intervals. For example, HEVC has been implemented on Apple’s MacOS High Sierra, Final Cut Pro X and Premiere Pro.

But behind the emerging codecs of the VPx and AVx series there are powerful cooperation partners of the Alliance for Open Media (AO Media), which includes Amazon, Apple, Google, Cisco, Microsoft, Intel and Netflix .

The strategy supported by AO Media and the high cost of HEVC suggests that instead of upgrading to HEVC, users are increasingly expected to switch to the license-free alternative formats of the VPx and now AVx series, respectively.

This article gives an overview of the state of development and introduces the most important codecs in the race for the top position.

H.265/ HEVC

High Efficiency Video Coding (HEVC) is a data reduction technique. This codec is also known as H.265 or MPEG-H Part 2. HEVC is the successor to the widely used H.264 AVC codec, which has enabled mobile video applications, high-resolution video content on the web and the transmission of HDTV due to its compression capabilities.
 
H.265 enables the distribution of UHD signals due to its high compression performance. Compared to H.264 coded picture content, this codec is characterized by achieving twice the compression with the same picture quality. This makes H.265 particularly suitable as a distribution codec. Applications can be found in the transmission of UHD TV, Blu-ray discs with 4K (UHD) resolution or for streaming applications. For example, the video telephony of the iPhone 6 is based on HEVC encoding. Furthermore, the codec is also used for terrestrial distribution via DVB-T2.

HEVC achieves its compression performance by predicting the motion of the individual image elements to the current image from previous frames. Further processing is then based on these difference values. The more accurate this motion prediction is, the smaller are the differences that have to be transmitted. HEVC also uses flexible macroblock sizes. Macroblocks are the elements that a codec uses for viewing and evaluation. 

For H.264, the macro blocks are defined as 16x16 pixels. HEVC uses up to 64x64 pixels. This allows redundancies and movements to be found even between image areas or objects of different sizes, which is particularly advantageous with higher image resolutions such as HD or UHD. For this motion compensation and redundancy determination, however, a very high computing power is required. Therefore the HEVC encoding of the hardware requirements is still quite complex. The decoders, on the other hand, are not more complex than the H.264 decoders. 1080p50 signals, for example, can be decoded in real time on a standard tablet. TV receivers still require HEVC compatible receivers for decoding. Basically, depending on the application, dedicated hardware encoders, decoders or software solutions are available.

In addition to the enormous data reduction, which enables the bandwidth-saving transmission of UHD content, HEVC also offers further advantageous features. The codec scales from 320×240 pixels to 8192×4320 pixels. This makes the codec future-proof, at least until the next major standard change to 8K. Bit rates from 8 to 16 bits and colour sampling rates from 4:2:0 to 4:4:4, and frame rates up to 300fps are provided in the respective levels of the codec as well as multi-view and 3D transmissions. This codec also paves the way for new technologies such as HDR (High Dynamic Range). In addition to a high resolution, a much higher contrast than usual can also be transmitted.

VP9

The VP9 codec is a royalty-free, open format primarily for compression of video data. It was released in December 2012 and after VP8, it is the last official release of the TrueMotion video format series (VPx) purchased by Google and competes with H.265/HEVC. VP9 was initially mainly used on Google's video platform YouTube. Chromium, Chrome, Vivaldi, Firefox and Opera support the VP9 format for HTML5 video.

Although parts of the technique are subject to Google patents, the company grants a permanent and irrevocable right of free use on the condition of reciprocity (no cross-action of license claims).

VP9 is designed to encode material in resolutions beyond High Definition Video (UHD) and provides virtually lossless compression.
There are different versions of the VP9 format, the so-called coding profiles, which allow more and more features from the basic version, the profile 0 (binding minimum for hardware implementations), up to profile 3:
 
Profile 0

Colour depth: 8 bit, colour subsampling: 4:2:0

Profile 1

Colour depth: 8 bit, colour sub-sampling: 4:2:0, 4:2:2, 4:4:4, alpha + depth channel

Profile 2

Colour depth: 10-12 bit, colour subsampling: 4:2:0

Profile 3

Colour depth: 10-12 bit, colour subsampling: 4:2:0, 4:2:2, 4:4:4, alpha channel

VP9 supports the colour spaces Rec. 601, Rec. 709 , Rec. 2020 , SMPTE-170, SMPTE-240 and sRGB.
In a subjective quality comparison with the reference encoders, conducted in 2014, VP9 (still similar to H.264) needed about twice as much bitrate to achieve a comparable quality as an HEVC encoded reference video.
 
According to an algorithmic evaluation, in September 2015 VP9 achieved a 50% improvement in bit rate efficiency compared to the best H.264 encoder x264 in a test with 10 to 20 times shorter encoding time. According to the objective VQM metric, the VP9 reference encoder delivered the same video quality as the best HEVC implementations in early 2015 and generally encoded faster.
 
In practice, standard decoders require slightly more than 20 percent more computing than H.264 to decode, and newer decoders are expected to deliver comparable or faster decoding than H.264 and 25 to 50 percent much faster decoding.
Adobe Flash, which traditionally used VPx formats up to VP7, has never been converted to VP8 or VP9, but instead to H.264. Therefore, VP9 often only advanced into corresponding web video applications with the gradual conversion from Flash to the HTML5 technology that was still immature at the time of the VP9 market launch. 


Video Multi-Method Assessment Fusion (VMAF)

Daala

Daala is an open data format under development for lossy video compression. It is being developed under the direction of Timothy B. Terriberry of the Xiph Org Foundation and main sponsor Mozilla. Among three dozen other contributors outside these organizations are engineers from Google Inc. and Cisco Systems. Technologies from Daala and Opus are now included in AV1.
The name was taken from the female fantasy figure Admiral Natasi Daala from Star Wars.

For techniques used and developed in Daala, software patents are registered and freely made available for general use. However, they may be used to defend against patent actions brought by third parties. It is designed to be suitable as a future standard for use on the Internet and for real-time applications. It should be openly documented and free of software patent restrictions in order to be able to spread as unhindered as possible. Furthermore, it should be suitable for a wide range of usage scenarios.
 
The developers are also less concerned with the further development of proven design patterns for patent considerations. Instead, riskier investments in finding and testing basic coding techniques are expected to yield new and potentially more rewarding approaches. This is intended to reduce the ever stronger increase in coding effort, that is reflected in the development of other techniques, along with further improvements. All current techniques have shared the same basic design for two decades (H.261).
In addition, options for parallel processing are to be taken into account and hardware support is to be sought.
 
It is intended as a highly efficient video format for similar applications such as High Efficiency Video Coding (HEVC or H.265) and VP9. Its performance is declaredly aimed at the generation after HEVC and VP9
.
After several preliminary meetings since 1 October 2014, an appropriate IETF working team officially became active on 18 May  2015. Among other things, a number of Daala coding techniques were presented to them as a working basis.
In June 2016, Daala had approximately the same bitrate efficiency as HEVC.

AV1

The AV1 codec (AOMedia) is a freely available codec specifically developed for the transmission and distribution of video content over the Internet.
It was developed by the Alliance for Open Media (AOMedia), founded in 2015.

Knowledge gathered in the course of the development of the Daala codec are incorporated into AV1.

AV1 will replace its predecessor VP9 and is in direct competition with the H.265/HEVC codec of the Moving Picture Experts (MPEG) group.


VVC

In March 2018, the Joint Video Experts Team (JVET), an association of MPEG and the ITU-T Study Group 16's VCEG, officially started work on a new codec called VVC (Versatile Video Coding).

The aim is to outperform the current HEVC codec in terms of compression rate and image quality. VVC is also expected to deliver improvements in 360° applications and High Dynamic Range (HDR) workflows.

The new standard is intended to enable the distribution of UHD video material at data rates that are currently used for HDTV content. Alternatively, it is expected to store or stream twice as much HDTV video material on a data medium or server.

The development of the codec is expected to be completed by 2020.