XDCAM-HD, What you really need to know
What is HDCAM-HD
Think of HDCAM and you will have a tape format and of P2 and you have a solid-state memory format – but think about a format which records onto ‘flat tapes’ with great non-linear post-production workflow and you have described XDCAM HD.
This format records onto a new format called XDCAM HD which are magneto-optical disks (technology on which Blu Ray is based) and promise to replace HDCAM for professional HD TV production.
In HD, this is being contested by the P2 range of Camcorders from Panasonic with the HPX-500 to the HPX-3700 and by Sony with the well established HDW-F900R and also now with inclusion of the new Sony PDW-700 XDCAM Camcorder released in late 2008 and PDW-800 released in mid 2009.
Interestingly, XDCAM has existed for several years as a standard definition format which didn’t become well established due to the earlier success of Digi Beta and just existed for a while not doing very much. Then a couple of years ago, a High Definition version called XDCAM HD was released and this too didn’t make much impact on the market, mainly due to the early cameras not offering a standard 2/3” lens mount, which severely limited the choice of HD lenses. The latest camera model, the PDW-700, solves all of the previous problems with a stunning camera head and 1080 recording medium and looks set to become a very popular acquisition format.
What is Hot about XDCAM-HD?
Compared with other tapeless technologies, this format has the benefit of being an archival medium, where you actually have something to put onto your shelf after shooting and which can use conventional post-production techniques (off-line, conform, on-line) the same as HDCAM (and then have something to put on your shelf), or instead offer improved tapeless workflow in common with P2 and XDCAM-EX.
As well as recording images in high quality HD, the format also records a simultaneous low low-resolution version of all media, called ‘proxy’ files, which facilitates rapid editing off-line in low quality (15:1). One you have built your ‘off-line’ version, AVID can automatically rebuild the programme by reloading the same media clips in HD – the equivalent of an off-line followed by an auto-conform, only you replace the re-digitise process with loading in other XDCAM cards into your reader or camera. The advantage in editing in off-line quality first is that you have speedier transfer speeds of media and reduced storage capacity for the best of every world. This is possible because both full res and low-res version of all images are recorded simultaneously on the discs and you choose which you use for ingesting and editing.
Another plus is that the discs are half the price of HDCAM tapes and roughly 1/3 of the size and weight and offer improved quality as well; Take into account that Broadcasters are finally unconditionally releasing this format for general production and it is easy to see that you really can’t lose by using it!
You can also shoot in full resolution 1920x1080 resolution using a full 4:2:2 colour space which is actually an improvement on HDCAM and the high reliability of the format (the only moving part is the revolving disc and head for example) means that there are additional benefits to shooting onto tape.
You can also shoot slow motion images of 50fps or 60fps by shooting in 1280x720 mode and slow these images down without any rendering during post production in common with XDCAM EX and P2. (The new PDW-800 permits variable frame rate slow motion in contrast with the PDW-700 which only allows a single 50 or 60 fps slow motion record).
The format is an ideal archive medium as a tapeless archive. Software has been developed to allow XDCAM-EX media to be archived to XDCAM HD and it might be possible in time for RED and Panasonic codecs to be incorporated into this functionality as well, since it is an efficient and convenient backup medium. Important to mention that this function is actually available now, should you choose to ‘re-wrap’ your multi-format images from XDCAM-EX and RED etc into an to MXF format – XDCAM HD only currently allows to backup MXF files, so any ‘rewrapped’ or rendered MXF images from any format can be archived to XDCAM HD discs using the Sony software and the PDW-U1 XDCAM HD reader unit.
What is Not?
This system records about 1/3 of the data that HDCAM records to tape by adopting a very efficient compression system called MPEG2. The cliché states that you never get something for nothing, so what is the cost?
The real limiting factor to using XDCAM HD is the limited read-write speed of the discs which are not much faster than real-time with the relatively low-cost XDCAM HD reader unit PDW-U1. The Sony PDW-U1 XDCAM HD player only has one set of read heads and only a USB-2 data connectivity, both of which severely limits read/write speed. The more expensive studio deck PDWHD-1500 has two pairs of heads and allows dual speed read speed (but is also 6 times the cost!).
Although the compression codec is not new, this format is still relatively young, so you will need to have the latest version of FCP and AVID Media Composer version 3 for it to work. Even then, the interface to AVID is not yet seamless and there are still bugs being reported with various software Non-linear Edit applications (NLEs), as the workflow is being steadily refined.
There is no XDCAM HD proxy functionality for Final Cut Pro, although FCP uses its own built-in proxy edit application.
Also XDCAM HD uses a native MXF format, so the post production workflow when using FCP needs to be ‘re-wrapped’ when working with FCP but these stages are automated using the well-designed Sony software plug-ins when working from a combination of XDCAM-EX and XDCAM-HD – so you never really need to think about this. However integration of large and small camera in post from multiple capture mediums is a problem which simply does not exist with the Panasonic P2 format….
This system uses the same long-GOP compression scheme as HDV with much less compression, so it can struggle with increased digital noise when recording strobe lighting. This is because the compression scheme relies on the previous frames to build future frames causes a problem if a previous frame breaks the predicted pattern (like a single bright frame illuminated by a solitary flash gun). This can make the compression scheme unsuitable for certain types of programming. However, MPEG has passed the very stringent tests imposed by very fussy broadcasters, so there is little need to worry about this one.
This name shares the XDCAM name with the smaller XDCAM-EX format but these formats are quite different.
The XDCAM HD which shoots onto Magneto Optical disks (think of flat tapes…) and XDCAM EX uses the same compression scheme as XDCAM but records instead onto low cost SxS Express Card media cards and does so at a lower data rate than the XDCAM-HD format usually shoots at.
There are 3 x ‘flavours’ of the format in its HD variety which record at various HD sizes, colour space and data bit rate, which means that you can shoot at a quality which suits your production – shooting at a lower bit rate will reduce your quality but it will also increase the recording capacity and reduce the transfer speed, so a programme produced exclusively for internet use for example can shoot at the lowest 25Mb/s data rate without compromise.
On the technical side, the 25MB and 35MB XDCAM HD data rates record images in a compromised colour space (4:2:0 and inferior to HDCAM) and crop the images recording 1440x1080 in common with HDCAM. The 50Mb version actually shoots and records images at full 1920x1080 resolution and also in 4:2:2 which is greatly superior to HDCAM. However, it is based on a long GOP compression codec which means that if you shoot in an environment where all of the image changes from frame to frame (for example strobe lighting at a pop concert), then image quality can be affected.
Recording at lower bit rates to 50Mb benefit the record durations of the XDCAM HD discs accordingly as the table below shows.
The XDCAM discs are also available in single layer (23GB) and dual layer (50GB) variety. The maximum recording durations at 50Mb is 95 minutes.
|HD Data Bitrate||Image Size||Colour Space||23GB Rec Duration||46GB Rec Duration|
|25Mb/s||1440x1080||4:2:0||85 mins||190 mins|
|35Mb/s||1440x1080||4:2:0||65 mins||145 mins|
|50Mb/s||1920x1080||4:2:2||43 mins||95 mins|
Our experience is that everyone wants to shoot in ‘full raster’ 1920x1080 mode in the full colour space by using the highest 50Mb/s data rate, since the quality can be reduced if needed from a higher resolution image but not the other way around!
The PDW-700 XDCAM HD camera can also shoot in 1920x1080 mode in either 25p progressive mode or 50i interlaced mode and is also able to shoot in 1280x720 mode up to 50 fps with the latest V1.2 firmware update.
It also supports record and playback of XDCAM HD* (4:2:0) at 35Mb/s, 25Mb/s and 18Mb/s**, and has chargeable options to support XDCAM standard definition MPEG IMX and DVCAM material (SD operation requires CBKZ-MD01 option).
The idea behind the XDCAM HD format is that you shoot in the full colour space (4:2:2) so that colour accuracy is improved on the HDCAM format whilst at the same time, enjoy editing in off-line using the full HD quality, so avoiding any conform charges, faster than real time digitise of full res. images in common with the P2 workflow and even faster transfers using proxy files etc.
One problem is that Sony did not share the codec with all of the post-production manufacturers until relatively recently, since this is only a relatively recent format in its HD flavour anyway. Consequently, there are still a few niggles which need to be ironed out.
Sony’s idea is that you can edit in off-line using low-res. ‘proxy files’ which are really small and then re-version the programme automatically by pointing the edit system at the full-resolution files. This works very well with AVID but it currently dislikes any slow motion media as AVID detects time code discontinuities and aborts batch capture of slow motion files.
Also Final Cut Pro (FCP) does not yet recognise the file format directly and you need to use the Sony browser software to convert it by changing the software ‘wrapper’. This is very quick and the process is automated as the Sony software plug in works within FCP. FCP currently does not work from the Sony proxy files either.
However, if you are working in a traditional acquisition environment with traditional AVID/FCP workflow, then there is not really a lot of benefit to shooting on these camcorders that you cannot already have with HDCAM, although HDCAM records with a higher data rate. Sometimes simpler is better.
The first XDCAM HD camera, the PDW-F350 Camera permitted variable speed recording between 4 and 60fps and only a data rate up to 35 Mb/s but was not been popular, so Sony released the PDW-530 to compete with HDCAM, including a 2/3” lens mount and increased the date rate to 50Mb to improve the HD quality to compete with HDCAM and DVCPro HD, though both of these formats have a higher recording data rate to tape. The problem was that this camera did not have a 2/3” CCD block, so images shot using 2/3” HD lenses were heavily cropped, so everyone was waiting on the release of the 2/3” version, the PDW-700.
Released in late 2008, the Sony PDW-700 offers a similar camera specification to the Sony HDW-790P incorporating for the first time on an XDCAM-HD Camcorder, a 2/3” CCD block allowing conventional HD Broadcast lenses to be fitted without cropping the image.
This camcorder also allows all XDCAM HD data rates including 50Mb XDCAM HD, and also support slow motion acquisition of 1280x720 material.
Frustratingly, although a 24P (23.98P) facility for this camera has been announced, it will also be a chargeable option requiring the CBKZ-FC02 software planned to be available for Summer 2009.
Thanks to the 14-bit A/D converter, pre-knee signal compression in highlighted areas can be eliminated, and the camera can clearly reproduce a high-luminance subject at a 600% dynamic range.
It remains to be seen how much this format is accepted in mainstream production but it is clearly the aim of Sony that XDCAM should completely take the place of DVCAM and make HD acquisition possible at current DVCAM rates and at the same time provide a non-linear version of HDCAM as well.
Other XDCAM-HD Benefits
XDCAM HD is based around a standard industry format called MXF (Material eXchange Format) file format. This allows material to be handled with great flexibility in an IT-based environment - easily available for copying, transferring, sharing and archiving. All these operations are accomplished without the need for a digitizing process.
File-based data copying allows for degradation-free dubbing of AV content, which can be performed easily on a PC. The file-based recording system also allows for material to be viewed directly on a PC, simply by linking it to the XDCAM unit via an i.LINK connection. This works in just the same way as a PC reading files on an external drive.
High-quality 4 Channel 24-bit Audio Recording
The PDW-700 records uncompressed four-channel, 24-bit audio. It is also equipped with a range of audio interfaces.
Robust and Reliable
Only having one moving part (the spinning disc) and having the disk drive entered by 2 x spinning lids to prevent dust from entering the drive, the XDCAM HD format is designed to be rebust and the drive to be resistant to shock treatment. As a testament to this, Two Four took a PDW-700 XDCAM HD Camcorder to the South Pole and back working in conditions of minus 30 degrees C and lower without any reliability issue.
Two types of optional viewfinders are available for users: the HDVF-20A and HDVF-200 2.0-inch** monochrome viewfinders and the HDVF-C35W 3.5-inch** colour viewfinder. The camera includes a small colour flip-out LCD panel on the side of the PDW-700 camcorder. This allows operators to instantly review recorded footage, as well as access the camera's set-up menus and view status indications such as four-channel audio meters, and the remaining time available on the disc and battery. It also enables advanced operations such as Thumbnail Search and Scene Selection.
The shutter speed of the PDW-700 is selectable down to a 16-frame period (in 2-, 3-, 4-, 5-, 6-, 7-, 8- and 16-frame periods). During such a long frame period, electrical charges accumulate on the CCDs which dramatically increase sensitivity. This helps camera operators to shoot in extremely dark environments. The Slow Shutter function also allows operators to use shutter speeds longer than the frame rate, and to intentionally blur images when shooting a moving object, for increased shooting creativity.
*Only the even numbers of frame settings are available in 720 mode.
The PDW-700 offers an Interval Recording function which intermittently records signals at pre-determined intervals. This is convenient for shooting over long periods of time, and also when creating pictures with special effects of extremely quick motion.
Picture Cache Recording
The PDW-700 offers a Picture Cache Recording function that is especially useful during ENG applications. Up to 30 seconds of audio and video signals are buffered into the camcorder's memory before the Rec start button is even pressed (when in Standby mode). This means that everything that happened 30 seconds before the Rec start button was pressed will still be recorded onto the disc.
What's more, this function works even before the disc is inserted in the drive - thereby helping to prevent the loss of any unexpected, yet important events. The caching period can be adjusted by menu setting. This camcorder cache memory also allows users to exchange the discs while recording. By removing a disc from the drive and inserting a new disc within 30 seconds, video, audio, and time code can be recorded seamlessly onto the new disc.
Live & Play Function
The PDW-700 camcorder has a Live & Play function that can simultaneously output both playback signals (images already recorded) and incoming camera signals (images seen through the viewfinder). Both signals are fed to their respective output and viewfinder connectors independently, and can be viewed at the same time. This allows users to frame the next shot, adjust the exposure, and even focus the lens while the camcorder is playing back recordings from the disc.
The Digital Extender function of the PDW-700 enables images to be digitally doubled in size. Unlike lens extenders, the Digital Extender function performs this capability without any loss of image sensitivity, which is often referred to as the F-drop phenomenon.
*Use of the Digital Extender function reduces image resolution by half. Requires a software upgrade planned to be available in autumn 2008.
At the touch of a button, the centre of the screen on the viewfinder of the PDW-700 camcorder can be magnified to about twice the size, making it easier to confirm focus settings during manual focusing.
*Requires a software upgrade planned to be available in autumn 2008.
Trigger REC Function
The PDW-700 camcorder has the Trigger REC function that enables synchronized recording with PDW-HD1500 and PDW-F75 XDCAM decks or HDCAM™ portable decks connected via the HD-SDI interface - a convenient feature for backup recording.
Post Production Workflow
XDCAM EX and XDCAM HD both are MPEG 2 based compression system which are completely compatible with all current post production tools including AVID, Final Cut Pro (FCP), Canopus, Sony Vegas etc.
You can import (ingest) directly into AVID, as XDCAM HD is MXF native, however, this differs slightly from XDCAM EX as this is MP4 native. These are actually the same data file system but ‘re-wrapped’ with different file extensions. Cleverly, Sonly have developed plug-ins which run the translation applications within AVID and FCP, so that the import (ingest) does any necessary translation/conversion without any user input! This is great, as it means that it doesn’t matter if you are using AVID or FCP with any combination of XDCAM EX or XDCAM HD in any data format, you can simply import and edit!
In order to use this application, it is necessary to download and install the necessary Sony software, which is available for free download at: http://support.sonybiz.net/software/
Once you download the log and transfer programme on your PC or Mac, then importing XDCAM media to your Mac will open the xd-cam software program within final cut as soon as you import xd-cam and when you set your destination folder in the in your xd-cam software your clips will be automatically imported into final cut.
Very easy no need for log and transfer so it is cross platform with the xd-cam transfer software. The only trouble is again these solid state cards all work on a fat32 platform so it means everything over 4gigabyte would be split into segments of 4 gigabyte and as the p2 with log and transfer do not have an issue with it seeing it is inter-frame and just intermarries them the xd-cam software would not allow you to ingest any clip over 4 gigbyte because it suddenly changes from mp4 to a .smi file with segments of 4 gigabyte mp4 files which has to be opened in the logged window and clip checked or ticked before it can be transferred.
An important technical consideration worth of comment concerns 4GB file sizes. All tapeless formats (including XDCAM EX & XDCAM HD) work using FAT 32 file formats which means that all long clips are broken down into 4GB file segments. The ramifications of this are that you must enable all files relating to one clip prior to importing it, otherwise the import is blanked out – this will save some head-scratching and grey hairs!! It is a standard consideration with all system which work using this file system and works long-GOP compression.
XDCAM HD Proxy Editing Functionality
The XDCAM HD acquisition system always records 2 x versions of media simultaneously without any user input. There is the high quality 50Mb (or lower res 35Mb) media and also the very low resolution proxy files which are compatible with AVID (not FCP).
Sony’s idea is that you can edit in off-line using low-res. ‘proxy files’ which are really small and then re-version the programme automatically by pointing the edit system at the full-resolution files. This works very well with AVID but it currently dislikes any slow motion media as AVID detects time code discontinuities and aborts batch capture of slow motion files. It is also not compatible with multi-user Lan-share applications.
However, these issues aside, Proxy post production is a useful application if you have a LOT of recorded acquisition material and are working to a high shooting ratio (i.e. observational documentary with 30:1+ rushes!). In situations like this, the small file sizes mean that data transfer is very rapid (10x normal speed) and this compensates for the low picture and audio quality. Mog Solutions make a plug-in which permits low res pictures to be edited with high quality audio to overcome problems associated with editing poor quality sound.
Once you edit the off-line, it is then a fairly quick process to re-digitise the full-resolution HD images off the XDCAM HD disks in order to re-version at high quality.
Please note that this is only compatible currently with AVID and not FCP.
XDCAM EX Application Browser Software
The XDCAM EX products come with two application software packages that provide powerful yet easy and intuitive management of recorded content.
The difficulty which this software overcomes is that XDCAM EX records using an MP4 version codec which is not natively compatible with AVID and other professional editing applications on the timelines (FCP is an exception to this). Clip Browser allows you to open native MP4 files and view footage, transfer to other codecs and rewrap the media to be compatible with other edit systems.
There are two versions of Clip Browser for Mac and PC as well as the XDCAM Transfer application software for Apple Final Cut Pro non-linear editing systems.
All free Sony applications are available for download at: http://support.sonybiz.net/software/
Clip Browser Version 2
The Clip Browser software for the XDCAM EX products is a simple-to-use PC application software that allows users to easily browse and copy video clips recorded by the XDCAM EX camcorder or deck to other devices such as hard disk drives.
It also serves as a bridge tool between a variety of formats - converting XDCAM EX clips to other file formats. The Clip Browser software is available for both Windows-based PCs and Macintosh computers.
- Browsing of video clips recorded by the XDCAM EX products
- Copy XDCAM EX clip files from the SxS PRO memory card to hard disk drive
- Combine segmented clips recorded across two SxS PRO memory cards
File format conversion from MP4 to other file formats:
- MXF format for export to XDCAM HD discs or MXF-based non-linear editing systems* (option)
- DV format for export to DV-based non-linear editing systems
- AAF format for export to Avid non-linear editing systems
- H.264/AVC format for field viewing on Sony PSP™, Apple iPod/iPhone as well as WMV format* (option)
- Create sub clips with Mark IN/OUT operation
- Registration of metadata such as “title”, “creator” and “comments” for a clip
- Registration of “Shot Mark” metadata for instant cue-up to desired scenes
- Capture and create a still image file (BMP) for a desired scene
- To download the Clip Browser Software: http://www.sony.co.uk/res/attachment/file/10/1193315621810.zip
PDZK-P1* Ver. 2.7 XDCAM Transfer
The Log and Transfer plug-in software utility for XDCAM EX allows any resolution of current XDCAM EX file to be easily imported into Apple's Final Cut Pro using its standard Log and Transfer interface.
Requires Mac OS X 10.4.11 or later and Final Cut Pro 6.03 or later.
- Create a new folder to copy dmg file to.
- Copy file to your Mac.
- Double click on the icon in the folder you have created.
- Installer will launch. Follow the installation instructions.
- Use the Log and transfer window to ingest EX files into FCP.
Note: XDCAM Transfer has the following advantages over Log and Transfer:
- Supports both XDCAM EX & XDCAM Disc
- Handles XDCAM EX and XDCAM Disc clips
- Can mount SxS & Professional Disc media at the same time
- MXF export capability
- Can write edited material back to Professional Disc in MXF files
- Standalone browser/viewer
- Browsing/viewing are possible without running Final Cut Pro
- Download: XDCAM EX Log and Transfer software V1.00 - http://www.sony.co.uk/res/attachment/file/00/1214313427700.zip
Sony PDW-U1 XDCAM-HD Reader
The PDW-U1 XDCAM U1 is an XDCAM HD reader unit compatible with PCs and Macs which do not have XDCAM HD slots (all!).
It has a USB2 in/out interface and is used for ingest to Mac and PC. Note that the latest version of the Sony software permits MXF data to be re-written back to XDCAM HD, so effectively you can now shoot onto XDCAM EX, cross convert to XDCAM HD MXF data format (using the Sony clip browser software) and then archive the data to XDCAM HD disk and store this on your shelf for 50 years!
Sony PDW-1500 XDCAM-HD VTR
The PDW-1500 Compact Deck is a half-rack sized recorder optimized for use with non-linear editing systems. Despite its compact size, this deck offers high-speed data transfers between compatible non-linear devices to create a powerful editing tool for news production.
One advantage is that it allows faster data transfer from XDCAM HD to computer hard drive for transfer applications as it has twice the number of record and playback heads (though it is quite expensive).
XDCAM HD is an excellent acquisition format which seems to have it all – it has a non-linear ‘tapeless’ workflow which records superior images to HDCAM, yet you have an archival media ‘product’ which can sit on the shelf.
However, the data read/write speed is only as fast as real-time, so the advantages of working in a tapeless workflow aspect currently don’t offer any time saving benefits, which is a shame.
On a positive note though, the latest cameras offer stunning quality, good functionality and once you have the data copied to disk, the easy post-production workflow of this format makes it a joy to use.
For those people currently happy with working in HDCAM and used to working in a traditional off-line/on-line workflow, the benefits of this well-tested format may not be enough to migrate the majority of production away from HDCAM to XDCAM HD. Some Broadcasters have aging non-upgradable post-production architecture which simply does not lend itself to non-linear workflow so their motivation for upgrading production to XDCAM may take some time yet.
However, given time and once the few bugs of XDCAM HD are ironed-out, I am sure that it will be a sure-fire winner, so expect XDCAM HD to be around for quite a while.
Barry Bassett, Managing Director, VMI June 2009