FORMAT



Autor: Redaktion


ProRes RAW

ProRes RAW is a codec released by Apple in spring 2018.

It enables a new standard of video compression for RAW footage. The new algorithm produces much smaller video file sizes than before, while maintaining the high quality video data rates and editing capabilities of the original uncompressed RAW video.

This makes the video files smaller than the original ProRes formats while maintaining high quality.
Apple ProRes RAW is based on the same principles and technology as existing ProRes codecs, but is applied to the unaltered raw data of a camera sensor. Similar to the original ProRes codecs, ProRes RAW is an intraframe codec .

Initially, ProRes RAW was only licensed for a few products. But since Blackmagic push their own format Blackmagic RAW, an alliance seems to be forming that wants to promote ProRes RAW more strongly.

Time for an overview.

Retrospective

In 2007 Apple released the first ProRes variants. Even then there were many good quality codecs, e.g. H.264 , but there was no option that offered good quality at a reasonable data rate for FHD. This was particularly important because the HDV at the time was so highly compressed that conversion to another highly compressed format often exacerbated the problems.

ProRes 422 and ProRes 422 HQ solved this very elegantly by combining intra-frame encoding with 422 color sub-sampling . An amazing balance between usable data rate, excellent image quality and fast encoding and decoding has been achieved. And this at 10-bit color depth, which offers 1,024 instead of only 256 tone gradations at 8-bit.
ProRes 4444 was also introduced, using ProRes encoding with higher data rates, without color subsampling and the ability to have an alpha channel.

ProRes quickly conquered the post-production world and became a standard for mastering and providing a very wide range of materials.



ProRes also became important in the field of recording after the ARRI Alexa 2010 was offered with a built-in ProRes recorder. It became the ideal recording format, delivering images with low compression, high resolution and high bit depth directly to post production. Other manufacturers, such as Blackmagic Design, also started using cameras that could record ProRes internally.

RAW , on the other hand, had been an option for photographers for many years, but with the RED One, RAW also became important in cinematography.

Jim Jannard and his team at RED Digital Cinema announced in 2006 that they would build a camera with a super 35mm 4K image sensor and record in a new format called REDCODE RAW. REDCODE RAW is a compressed RAW file with a Bayer pattern.

It is fair to say that the resulting RED ONE, launched in 2007, is one of the few cameras that has really changed the evolution of film technology. In the following years, most professional camera manufacturers offered RAW recording options, mostly with additional recorders, to prevent RED's patents on compressed RAW recordings in a video camera from being infringed.

Finally, RED also offered ProRes as a recording option in its cameras and for some time now we have essentially had the choice between ProRes or RAW. Cameramen and colorists often argue for RAW because of its width and quality. Producers and post-production service providers often argue for ProRes because the files and data rates are manageable and therefore costs can be saved.

The thing about RAW: efficiency

With conventional 3-chip video cameras, including the high-end digital cinema cameras of the day such as the Panasonic Varicam and the Sony F900, the light coming through the lens is divided by a prism into red, green and blue rays. These rays fall on separate image sensors, each of which is tuned to the part of the visible spectrum. Each of these formats usually requires the same resolution as the recording format. For FHD, this means that a total of around 6.2 million pixels, with around 2 million red, green and blue pixels each, must be recorded.



The first steps to make this more practical in terms of data rate were most digital video formats from Digital Betacam to DV to HDCAM, which converted this RGB into a component video format with one luminance channel (Y) and two channels. There, the difference between the red and blue channels and the sum in the Y channel resulted in the overall image. From this information it is possible to mathematically calculate the exact red, green and blue values for each pixel. 

Since our eyes are much more sensitive to brightness than to color information in detail perception, it is possible to reduce the color difference channels to a lower resolution than the Y channel.

Most professional formats used the color difference channels at half the resolution of the luminance channel. This reduces the pixel count from around 6.2 million to around 4.2 million pixels and this process is digitally represented as a 4:2:2 scan structure (the source of the 422 in ProRes). This signal is often referred to as a YUV or technically more YPbPr signal.

However, this basic principle was only suitable for small and medium-sized sensors. For the increasingly large sensors, a different pattern was used, the Bayer pattern.



The Bayer pattern shows alternating lines of red & green, then blue and green. From this, single-sensor digital cameras can calculate with remarkable accuracy what the RGB values are for each pixel. Originally this was done in the still and video camera, but manufacturers realized that by maintaining the image in the Bayer sample pixels, they could reduce the data rate by two-thirds and use some of these savings to achieve much higher bit depth and lower compression. 

So to take our FHD image example: While YUV Component Video reduces the number of pixels from 6.2 million to 4.2 million, non-converting the Bayer pattern to only 2.1 million pixels requires the information of a 6.3 million pixel image to be obtained.

If you move to higher resolutions, it grows proportionally. A 4K DCI image has 26.5 million RGB pixels, equivalent to 17.6 million pixels in YUV. In contrast, there are only 8.8 million pixels in the Bayer pattern.

ProRes RAW - is it ProRes or is it RAW?

In fact, ProRes RAW is both. Apple has adopted the very clever compression basis of ProRes and applied it to pixels arranged in a Bayer pattern instead of a YPbPr component video pattern.
 
The real value of RAW is the high bit depth, which makes it possible to make massive corrections to white balance and ISO , among other things.

The process of converting a Bayer sample image into a conventional RGB image is known as de-mosaicing. With "conventional" formats, this is done directly in the camera and "burned" into the file. For playback, however, an application must now decode the signal. 

In contrast, ProRes RAW directly encodes the Bayer sample image. De-Mosaicing is shifted to the post-production or playback area. This requires more computing power, but allows greater flexibility.




Just as different image and video codecs have to compromise between compression rate, quality and complexity. By using ProRes RAW, you can move the de-mosaicing of the recorded Bayer pattern into post-production and take advantage of greater flexibility with high quality.

ProRes XQ can also be used instead of RAW. It is a very high-quality format that offers you the same flexibility in practice as ProRes RAW. However, ProRes XQ is a slightly compressed codec and the data rate remains almost as high as Lossless RAW.

ProRes RAW in practice

Let's take a moment to look at the different elements of what is usually thrown into a pot under the label "format". This refers to the image structure, codec and wrapper.

Image structure in this context means the number of target pixels, e.g. 4096×2160 combined with the channel arrangement like RGB, YUV or Bayer as well as the color pattern structure like 444, 422 or 420.
The codec is the way this information is encoded and decoded, such as MPEG-2, MPEG-4, REDCODE and JPEG-2000 .
The wrapper is the file format in which it is included.

This is where the actual practical meaning of ProRes RAW comes in. It takes the Bayer pattern image structure data from the camera, encodes it with a consistent and very efficient codec and then makes it available in a very practical MOV wrapper with many useful metadata.

This means that ProRes RAW can be a realistic option for many projects that might not otherwise have been able to use a RAW workflow. With some acceptance from major camera manufacturers, ProRes RAW could easily become the new de facto standard for a very wide range of productions, from commercials and documentaries to large features and high-end drama series.

Datarate

ProRes RAW is available in two compression levels: Apple ProRes RAW and Apple ProRes RAW HQ. Both achieve excellent preservation of the raw data. Video content. Visible artifacts due to compression are very unlikely in both quality levels.

As with existing ProRes codecs, the data rates of ProRes RAW are proportional to frame rate and resolution. The RAW data rates of ProRes also vary depending on the image content, but to a greater extent than ProRes.
Most video codecs, including the existing ProRes family, are a technique that dynamically adjusts compression to achieve a target data rate. This means that in practice the degree of compression varies from image to image depending on the content.

In contrast, ProRes RAW is designed to ensure consistent quality for all images. Therefore, images with more detail or higher sensor noise are also encoded at higher data rates.


Performance




Licensed products for ProRes RAW (as of October 2019, source: Apple):

Assimilate Scratch
Atomos SUMO 19
Atomos Shogun Inferno
DJI Zenmuse X7
Filmlight Baselight
Filmlight Daylight
MTI Film Cortex
Telestream Switch

Grass Valleys Edius starting with version 9.5 natively supports all RAW- versions (e.g. ProRes RAW, Blackmagic RAW, Canon Cinema RAW Light, Sony RAW).

Related Articles: