H.264 and Why is the AVC Intra codec so good!
AVC Intra – What you really need to know!
Although I now work in the field of HD cameras, I am first and foremost a mathematician, who specialised in non-linear differential equations at University, so it is perhaps understandable that I get somewhat excited by the technical side of video coding methods! However, having struggled through 3 years of a Physics and Maths degree, I also understand the need to keep things simple!
This is an introduction to the amazing technology of the Panasonic AVC Intra-based systems and why they enable pictures to look as good as they do.
Introduction
In any tape or solid-state based system, there is a need to compress the information recorded in an effort to keep the cost down and work within equipment limitations and simply, this system uses all of the lessons leant over the past 10 years to build a sophisticated compression scheme that compresses data more efficiently than ever before and this is called AVC Intra.
This means that without compromise, you can now record the full 1920x1080 image with full colour information (4:2:2) and significantly improve on HDCAM and DVCPro HD without making the data files any larger than they presently are.
To fit into regular production, Panasonic produced 2 versions of the AVC Intra codec:
- AVC Intra 50 – this was chosen to have the same physical data size as equivalent Standard Definition pictures but with currently accepted HD images
- AVC Intra 100 - this was chosen to use the existing physical data size as current HD images but improve the image quality to contain full 1920x1080 lines (unlike HDCAM) and to also record the full colour information (4:2:2 sampling) without resorting to clever economies like HDCAM does.
The result is a super-efficient video compression scheme that uses the latest developments in video compression to produce images comparable with HDCAM SR (in 4:2:2 mode) with a much lower bit rate. Figure 1 below shows 2 successive images of a pan which are compressed using the AVC Intra codec. This uses a sophisticated prediction system from which the residual image is the only information to actually be recorded. This compression system enables very high quality images to be captured using a very efficient coding system, which is entirely compatible with current AVID and Final Cut Pro editing systems.
Figure 1 - Movement image captured by AVC Intra to demonstrate efficient data codec
Original input image
Intra prediction image
Residual image
Movement Compensation Inter Frame Prediction – creates predictive pictures by detecting movement to create a ‘virtual’ frame. The clever software then compares the difference between the actual image frame with the predicted image frame and only records the difference.
Don’t worry about understanding how it works – I have explained this below for those of you who are interested! However, of most importance is that unlike other HD formats (and particularly HDCAM SR 4:2:2), AVC Intra has 4 x further advantages:
- Firstly, the system records to solid-state memory cards called P2. This is a low-cost proven format, which means that you shoot to memory card and then speedily and effectively transfer this to hard drive for post-production without incurring any stock costs. In its highest quality form, one hour of AVC Intra media takes only 60GB of hard drive space (1 minute per GB media, unlike uncompressed 4:2:2 which takes almost ½ TB storage space per hour of media).
- No expensive VTRs and no digitise cost. You have immediate access to the media and can begin editing immediately after shooting avoiding the digitise phase and don’t need to have an expensive VTR on location to record to tape.
- Secondly, compatible with AVID and FCP. In order to be at its most efficient, the media must be totally compatible with post production systems in its new compressed form so that the data can be accessed and edited without having to resort to converting or ‘rendering’ images from one data form into another compatible with the post-production system (unlike RED and XDCAM-EX with AVID for example). I am pleased to report that the AVC Intra codec now works seamlessly with the latest versions of AVID and Final Cut Pro (FCP), so that projects can be edited and graded wholly using AVC Intra.
- Can mix compressions schemes in Post. One further development of post production tools such as AVID and FCP is that they permit projects to work using multiple native compressed images on the same time-line on the same project. This means that you can now have one project which works in native P2, HDV, uncompressed and also AVC Intra – all without rendering and incurring any new unintended spurious artefacts (caused by mixing compression schemes).
To summarise then, AVC Intra:
- Records Full size HD 1920 x 1080 images – unlike HDCAM which records only 1440x1080 to tape.
- Full colour information recorded - 4:2:2 sampling like the best systems and also unlike HDCAM which records 3:1:1 colour sampling.
- No expensive VTR means no expensive deck on location and no digitise cost.
- Uses really solid P2 workflow that has been well tested in production.
-
Has 2 versions, both of which are unconditionally accepted for use in world-wideHD programme delivery
- AVC Intra 50 which records the equivalent quality of P2 1080 but only takes up half of the space (2mins/GB or 30 mins per hour of media).
- AVC Intra 100 which is comparable with HDCAM SR 4:2:2 and a significant improvement on P2 1080 without increasing the data size (1min/GB or 60 mins per hour of media)
The conclusion of all of the compression strategies of AVC Intra working together is that the system is not fooled by difficult conditions such as flash photography – in a situation like this, the previous frame does NOT act like the successive frame and this causes great problems with other systems like XDCAM-HD.
The tolerance of predictive frames also means that the system is more forgiving to practical camera movements with perspective etc which do not follow exact movement vectors compared with other system. I have seen multi-generation examples of images kept within the AVC Intra codec and even the 5th generation holds up very well.
Practically however, this system wins conclusively over other comparable systems if you have a film or other production with a large amount of chroma-key or require a heavy grade. In situations like these, you would generally use extremely expensive formats such as HDCAM SR, to faithfully record the full 1920x1080 image complete with the full 4:2:2 colour information. AVC Intra gives you this capability to allow extremely fine chroma-key in post-production and also allow heavy multi-channel compositing within the Intra codec.
The system also permits off-line and on-line to work with the native HD material in full HD without any rendering prior to post (unlike the RED workflow) and without the need for digitising (unlike HDCAM SR) and will intercut between seamlessly within the family of P2 and other AVC Intra products as well as codecs from other manufacturers and also uncompressed in post (post system allowing).
As you can doubtless tell, I am a great fan of AVC Intra and VMI stocks equipment from the majority of the product range.
How does it actually work? (The technical stuff)
Most compression schemes only use a single strategy, which means that they work well given certain conditions but the problem is that they cannot perform well under all situations. AVC Intra uses 6 separate strategies to ensure that the data is always reliably captured under all conditions. It does this so well that it is now possible to record a comparable quality to HDCAM SR without increasing the data rate from that of current DVCPro HD.
Traditional compression schemes (i.e. HDCAM) work on the principle that each image needs to be compressed individually – this takes up more space than is necessary but avoids situations where everything in the image is changing (for example pans, rain, flash photography) – this strategy is called INTRAFRAME compression.
Other schemes work on the principal that a succession of images will be sufficiently similar that you only need to record is what is actually changing – i.e. in an interview situation, the walls and floor remain the same, so a small amount of information about how the subject is moving enables a very low bit rate to record the changing pictures successfully. This has problems when everything changes – notably random elements (such as rain), pans and flash/strobe lighting. This strategy is called INTERFRAME compression.
AVC Intra uses a combination of these functions to look for common elements within successive frames to work out exactly what bits are moving and to understand the movement to create a predicted frame – the clever computer inside the camera then compares the expected frame from the actual frame and records the data which actually changes – see figure 2 below:
Figure 2 – Motion detection of AVC Intra codec
In the simple example above, the codec can interpret the pan of the main subject and creative a predictive frame which is correct, except for the moving mouth. As a result, to completely capture this second frame, only the mouth data and the vector of the movement need to be captured and then the second image can be extrapolated from the first image using this small amount of additional data recorded.
With enhanced movement detection, the image is segmented into small blocks – e.g. 16x16 pixels and the image then searches for the most similar blocks to detect the type, direction and speed of movement. The system is sophisticated enough to allow variable bock sizes to allow various sizes of subject movement to be optimised depending on the size of the object moving – see figure 3 below:
Figure 3 - Motion detection of AVC Intra codec
The system is programmed to understand 9 different kinds of movement and allows pans, tilts and combinations of these to dramatically reduce the recorded bit rate and is a greatly enhanced codec compared with the original MPEG4 codec on which it is based.
See Figure 4 (below) for a practical example of 2 successive images recoded from a pan, which has been compressed using the AVC Intra codec. Included is also the difference frame demonstrating a low amount of recorded information, as a result of the highly accurate intra-projection. This system is fundamentally different from other INTERFRAME coding methods which deteriorate image quality as a result of camera pan motion.
Figure 4 - Motion detection of AVC Intra codec
Original input image
Intra prediction image
Residual image
Entropy Coding
Another clever initiative which the AVC Intra codec adopts is a kind of ‘shorthand’ of frequently used data which the system sets up for each frame, to reduce the volume of recorded data.
e.g. in this essay if every time, instead of using the whole words AVC Intra, the codec decided to abbreviate this to !”£ and replace this with the full words “AVC Intra” on playback, then this would be fully reversible and would permit data to be efficiently compressed without losing any information and each incident of this has a 66% data saving.
DPCM (Differential Pulse Coding Modulation) technology
This is a predictive encoding scheme which only codes the difference between pixel samples of the current picture with pixel values surrounding the pixel concerned. In this way, similar coloured pixels of a common background can be efficiently coded without losing data. This compression scheme works within the current frame and does not rely on INTERFRAME predictive frames. The prediction accuracy has been improved with the additional of variable block sizes and some different prediction modes to ensure the most suitable settings for governing conditions.
Discrete Cosine Transform (DCT transform)
The eye is easier to trick by hiding high detail and this is the basis behind all DV coding methods. This is not new and in common with the DPCM method, does not rely on INTERFRAME predictive frames but has been refined to ensure that errors between transforms and inverse transforms have been reduced
Inter Frame Prediction
Uses variable block sizes to help predict movement
Tolerance of Movement Prediction
The system builds in a ¼ frame tolerance when calculating movement prediction to ensure that subtle changes of movement from predicted frames do not fool the system.
Plural Reference Frames
In the transition of one scene to another, more past and future frames can be selected by allowing multiple reference frames (up to 16) to enable the intra-frame predictive function to work as reliably as possible.
Weighted Prediction
It is possible for inter-frame systems to become confused by effects like fading and for this reason, the prediction error of reference frames are weighted so as to take special effects into account – very clever!
HPX-3700 'Varicam 3'
The Panasonic HPX-3700 is the camera to go for if you want to use the AVC Intra codec, as it records onto AVC Intra and also DVCpro HD in 1920x1080 and is the long-awaited 1080-line Varicam, commonly known as the Varicam 3!. It shoots in all basic frame rates including 24p and 25p and all shoot slow motion frame rates up to 30fps but will not function in any of the 720p or SD frame rates – but if you are shooting with this camera, you really won’t want to shoot in anything but 1080!
This camera replaces the very well received first offering, the HPX-3000 released in mid-2008 and is further improved by more filmic gamma curves and a dual-link 4:4:4 output with true log10 files, the same as the ARRI D21, Panavision Genesis and Thomson Viper. This dual-link output though has to be recorded on an external format such as HDCAM SR, as the non-linear record capability cannot yet cope with this.
The rental price has come down, so, it is much less expensive than before but this camera punches way above its weight and offers way higher quality than even the top Sony HDW-F900R HDCAM Camcorder and competes directly with the Sony F-23, ARRI D-21 and Panavision Genesis!
Also, because there are no moving parts, makes this technology incredibly reliable and robust and in recent testing, these cameras were tested to -42°C after being frozen for 6 hours to be tested in hostile conditions. They have received universal acclaim after being used on the BBC frozen planet series as a testament to this.
Similar to the HPX-3000, it can shoot in either AVC Intra 50Mb, AVC Intra 100Mb or DVCPro HD 100Mb modes onto P2 cards and this camera contains 5 x P2 slots for a maximum recording duration of 10 hours of HD material when fully-loaded with 5 x 64GB P2 cards!
HPX-3000
This was Panasonic’s first native 1920x1080 AVC Intra Camcorder released in mid 2008. It shoots really lovely images has the same CCD block and thus the same quality as the HPX-3700 and works in 24p, 25p and 29.97p (plus interlaced modes as well) but has no slow motion, SD or 720 compatibility.
Barry Bassett, Managing Director, VMI Broadcast, September 08
