Standards, such as MPEG, are implemented in various ways, all of which vary in levels of quality and efficiency. Broadcom has developed MPEG-4, JPEG, H.264 and many other codecs that provide the highest quality at very low power, which is essential in mobile video applications.
File Formats
Broadcom offers a range of libraries for handling different
file formats.
Codecs are licensed in bundles that correspond to typical mobile multimedia applications. Bundling by application simplifies the choice of required software and licensing arrangements while retaining flexibility for different kinds of end products. This bundling approach, together with pre-integration and the absence of any porting fees, offers out-of-the box functionality at the best possible value.
For more details on bundling, see the section on software licensing.
MPEG Series
MPEG4 is an ISO/IEC standard developed by MPEG (Moving Picture Experts Group), the committee that also developed MPEG-1 and MPEG-2.
MPEG-4 is the result of another international effort involving hundreds of researchers and engineers from all over the world. MPEG-4, (also known as ISO/IEC 14496) was finalized in October 1998 and became an International Standard early 1999. The fully backward compatible extensions under the title of MPEG-4 Version 2 were frozen at the end of 1999 and received International Standard Status in early 2000. Several extensions have been added since then and some specific work-items are still in progress.
MPEG4 builds on the proven success of three fields:
- Digital television
- Interactive graphics applications (synthetic content)
- Interactive multimedia (World Wide Web, distribution of and access to content)
Broadcom provides a highly optimized MPEG4 codec, with excellent performance and video quality:
- Extremely low processing latency
- Fully compliant with MPEG-4 ISO/IEC 14496-2
- Simple Visual Profile Level 0 - Level 3, H.263 (baseline)
- Encoded Bit Rates: 15 kbps to 2 Mbps
- Very low memory footprint for efficiency
- Ultra low code footprint for small code size
- Encoded Frame Rates: up to 30 fps
Broadcom offers a number of benefits to the above applications as well as other applications. These benefits include:
- High coding efficiency: Low bit rate video coding applications require that video sequences be encoded with the least possible number of bits while maintaining good picture quality. MPEG-4 provides the tools to achieve this capability.
- Error resilience options: These options provide the means necessary to recover from errors that usually occur during transmission over error prone channels such as wireless channels.
- Scalability: The scalability feature allows one to provide, for the same content, video streams with different spatial and temporal resolutions.
MPEG-1 is the first of the MPEG series of standards. It is used in video CD applications. This compression technology enables approximately one hour and a quarter of VHS-quality video and audio to be stored on a 680 Mbyte CD.
MPEG1 runs efficiently on the VideoCore® platform.
Aimed primarily at compression of D1 sized video for television, MPEG-2 is the chosen format for the majority of digital television broadcasting, including DVB and DVD.
Although MPEG2 is not aimed at the mobile market, it runs efficiently on the VideoCore platform.
H Series
H.264 is the culmination of both the ITU-T's H.26x series and ISO/IEC's MPEG standards. The two standards bodies collaborated to form the ultimate in video coding standards. H.264 draws upon the technology of both parties to offer unprecedented bit rate efficiency and high quality video.
H.264 is suitable for all of the latest video streaming applications, as well as video storage and services requiring low delay or low bit rate. The improvement in compression over MPEG4 is around fifty percent.
- Variety of motion compensation modes and block sizes
- Quarter pixel motion compensation accuracy
- Advanced arithmetic coding
- Bi-directional pictures
- Error resilience, provided by such features as independent macroblock coding and multiple reference pictures
- Rate control algorithms, for added flexibility and robustness
- Encoder and decoder scalability
- Additional layer of packetization to suit networking applications
- No decoder mismatches due to replacement of DCT by integer transform
- Options for low latency coding
The H.263 standard is published by the International Telecommunications Union (ITU). It supports video compression and coding for video-conferencing and video-telephony applications.
Broadcom provides a highly optimized H.263+ codec, with excellent performance and video quality.
JPEG Series
JPEG is an industry-standard compression scheme for lossy compression of still images. Broadcom has implemented an extremely fast JPEG encoder written specifically for VideoCore.
Broadcom's main objective was to develop an encoder that was fast and flexible enough to support all features and JPEG options. The encoder works with large images. This entails streaming with data being written as the image is being converted so that neither the entire image nor the entire JPEG stream needs to be kept in memory.
The JPEG encoding algorithm
A JPEG stream begins with some headers that provide information about the components of the image and the Huffman and quantization tables used. The JPEG encoding algorithm is relatively straightforward. Most of the computational work occurs during discrete cosine transformation (DCT), which transforms 8x8 blocks into frequency coefficients. The resulting coefficients are first quantized according to perceptual weights, and then, coded. The remainder of the algorithm involves coding of the coefficients. This involves run-length coding of zeros, and some Huffman coding.
Image structure
A JPEG stream contains a sequence of components. It is the JFIF (JPEG File Interchange Format) that specifies these components as luminance, Y, and chrominance, Cb and Cr. Components may be subsampled. The standard allows for a reasonable variation in the subsampling, but the codec assumes that the chrominance components are subsampled by a factor of two in both directions (4:2:0 format). So, for every pixel, there is 1.5 samples to transform and code.
The components can be fully transmitted, one after another, or interleaved. Because the aim is to be able to stream the data, the operation uses the interleaved mode. Each component is split into 8x8 blocks of pixels called data units. In the interleaved mode of operation, the data units from the different components are grouped into Minimal Coding Units (MCUs). In 4:2:0 format, there are 6 data units in an MCU: four Y data units, one Cb and one Cr unit. These MCUs are coded sequentially in the data stream. JPEG supports 8-bit and 12-bit sample values. The codec assumes 8-bit samples. Supporting 8 bit samples would only involve changes to the first pass and control code.
JPEG file format
The JPEG standard does not specify a file format. The codec uses the popular JFIF file format. There are various JPEG headers that need to be placed at the beginning of the file that describe the components and the required quantization and Huffman tables. The remainder of the file apart from the end marker consists of the encoded MCUs.
DCT and quantization
Before applying the DCT, the sample values are level shifted (by 128). The DCT is a straightforward 8-point DCT, applied in each direction on each 8x8 data unit. The resulting coefficients are quantized. The quantization factors may be different for each component. Broadcm applies different factors for the luminance and the chrominance components.
There is one DC coefficient that indicates the average sample value, and 63 AC coefficients, which are the size of the various two-dimensional frequencies components. The DC coefficients of the 2D transform are level predicted: each DC coefficient has the DC coefficient from the previous data unit (in the same component) subtracted from it before coding.
Run-length Coding
Many coefficients, particularly the high-frequency coefficients, are zero after quantization. The lengths of runs of zero AC coefficients are counted. Only non-zero AC coefficient are coded, but each with the number of zero AC components that precede it. These are later referred to as B-values and R-values. The R-value is the length of the preceding run of zeros. The B-value, is simply the quantized value that is being coded. The DC components are always coded whether zero or not, and always have an R-value of 0. The R-values are 4 bits. If there is a run of more than 15 zeros, the first 16 zeros are coded as (R,B)=(15,0). To improve the effectiveness of the run-length coding, the coefficients are treated in zigzag order.
The zigzag ordering of DCT coefficients are:
00 01 05 06 14 15 27 28
02 04 07 13 16 26 29 42
03 08 12 17 25 30 41 43
09 11 18 24 31 40 44 53
10 19 23 32 39 45 52 54
20 22 33 38 46 51 55 60
21 34 37 47 50 56 59 61
35 36 48 49 57 58 62 63
Huffman Coding
The (R,B) values generated in the run-length coding are encoded using a method based on Huffman coding. First the significant bits, S, in the B-values are counted. The R and S values are Huffman coded as a pair, giving a code of no more than 16-bits. R and S are each 4 bits long. These combine to give an 8-bit table index. After each Huffman code comes the S raw significant bits of the B-value. In coding, the raw bits of B, which is the sign of the B-value, replaces the most significant bit. Each sample, therefore, produces two variable-length strings of bits. Neither string is more than 16 bits in length. As with quantization tables, the Huffman tables can vary between the components. The codec uses separate tables for chrominance and luminance.
Bit and Byte Stuffing
The bits from the Huffman codes and the B-values are packed into bytes with the most significant bit first. As 0xff bytes are used as stream markers, any byte with value 0xff which naturally occurs in the byte stream has to be escaped by inserting a following 0x00 byte.
JPEG2000 is a pure, still-image compression based on the wavelet transform. It offers a better compression rate than the old JPEG, plus these additional features:
- Preview capability: you can decode an image at half or lower resolution using less computation time
- Various possibilities of stream ordering, for example, PSNR-scaled or resolution-scaled
- Partitioning of big images in tiles
- Regions of interest: you can encode an interesting part of the image in better quality
- Error resilience capabilities
- Random access: you can decode only an interesting part of the image
- Optional lossless compression
Broadcom has ported JPEG2000 to VideoCore
Microsoft Series
Windows Media 9 Series delivers the same quality audio and video quality at about 1/3 the bit rate of MPEG2 and about 1/2 the bitrate of MPEG4 Simple Profile.
Broadcom has licensed Microsoft WMV 9.0, and is able to supply WMV for the VideoCore architecture.
