Printer Friendly

Telestream: perfecting your image: extra processing--it's bad in food, but good in file-based HEVC.

In recent years, consumer electronics manufacturers have released a flood of video consumption devices, ranging from large, high resolution televisions to powerful second screen viewing devices in every person's palm or pocket. Modern video consumption is ultimately the product not only of spectacular viewing devices, but of a continuous drive for ever more efficient file formats that can deliver diverse and high-quality video services for those devices to consume.

Thus MPEG-1 yielded to MPEG-2, which became the foundation for digital television delivered to millions of cable, DBS, and terrestrial viewers. During the last decade, MPEG-2 has had to share the stage with MPEG-4--also known as H.264 or AVC--which enabled greater diversity of services to be delivered to the home and provided the efficiency and quality needed to light up tablets, phones, and PCs.

H.264 has been an enabling technology that has allowed for the birth and development of new viewing habits. It has served as an effective bridge between the relatively demanding requirements of traditional TV and has supported the emergence of high-definition television. It has also provided the flexibility and efficiency for current distribution models.

Now another evolutionary step forward is taking center stage--High Efficiency Video Codec (otherwise known as HEVC or H.265). Like AVC before it, HEVC promises improvements in both bitrate and picture quality (see Figure 1). HEVC is the first codec wholly designed in the modern video environment where file movement and delivery to the home is becoming the standard. As such, HEVC lends itself to new methods of moving content for which previous codecs were not optimized. These benefits have the potential to ripple throughout the entire ecosystem.

H.265 was developed in response to the growing need for higher compression of moving pictures for various applications such as Internet streaming, communication, videoconferencing, digital media storage, and television broadcasting. It is also designed to enable the use of the coded video representation in a flexible manner for a wide variety of network environments. The HEVC standard was accepted by ITU-T in April 2013 and products supporting the standard are already becoming available.


On the consumer-facing side, the objective of HEVC is to reach a point where current content requirements can be met using less bandwidth, and new formats for high-resolution content can be made available as pay services for consumers. Among the specific consumer drivers behind the need for HEVC are the following:

1. Changing viewing habits of the consumer. While AVC allowed us to get to today's changing landscape, it wasn't built from the ground up for the purpose of supporting the distribution environment we currently face and the additional services we envision for the near future.

2. The rise in prepared content as a percentage of total viewed content. As viewing has become increasingly nonlinear, opportunistic, and multiplatform dependent, providers need to deliver valuable content without spending more for costly bandwidth infrastructure. That means taking full advantage of existing bandwidth.

3. At the high end, the first signs of 4k and 8k resolution video services. This points to the need for a solution for the delivery of these high-bitrate formats within the existing bandwidth.

While the industry works toward the last-mile distribution, there are important opportunities on the operational side of the equation that can deliver significant cost-savings and optimization while the consumer monetization angle develops.


While revenue from consumers is the most widely publicized goal, there are equally and potentially more immediately addressable applications of HEVC in the file-based operations on which media companies rely. HEVC is not simply a benefit for last-mile delivery--it has important and pressing advantages in the back-end movement of video.

One example can be seen in the flow of moving video material between an organization's facilities. In a report released in August 2012, the average cost of a satellite transponder (worldwide) was $1.62 million per year for 36 megahertz of capacity. At that price, it is incumbent on operators to make the most of their existing capacity. The ability to get more from a single transponder translates into a significant operational cost savings. So, when a network production center in Los Angeles needs to send material over a satellite to a distribution center in New York, HEVC provides the advantages of moving better quality video, more video, or simply the same video at a lower bitrate--all in the same transponder bandwidth.

Additional savings may be found in storage capacity for VOD and web distribution. HEVC provides the benefits of storing content in less space than current formats and/or storing higher-quality content in storage capacity that is equivalent to what is used today. And while the benefits of monetized VOD services are predicated on consumer-level devices being able to take that HEVC stream and decode it, such devices are already coming to market.


While there are potential uses for live HEVC encoding--and it will certainly be adopted for that purpose--there are constraints on a live encoder that prohibit making use of the entire HEVC toolkit to best effect. We will expand on this topic in more detail later, but there are a number of tools in the HEVC encoding toolkit that cannot be used to full effect in live encoders because they are restricted by the latency constraint that is mandatory in live work, which-- by definition--limits the number of frames which are available at any point in time for the encoder to use in its predictive activities.

Fortunately, an increasing amount of what people view today is pre-prepared content. In truth, with the exception of live sports events or news, traditional broadcast relies on large quantities of pre-prepared material, and VOD and "catch-up TV" operations are basically 100% pre-prepared. When repurposed for re-use on websites, some of these live use cases also transform into pre-prepared content. For example, a news broadcast will be cut up into its constituent stories before posting to the website, and sporting events may well be reduced to highlight packages. Freed of the constraints of live encoding, these use cases can take advantage of the advanced features of HEVC, use the full toolkit, and extract the maximum image quality (and revenue) from the source material.

The benefits of HEVC's enhanced compression capabilities can be significant in multiscreen and VOD applications in areas other than inter-facility transfer and media storage. Consider the following real-world example: Delivery of a single 120-minute movie at Wi-Fi level bit rates (8.5Mbps video, 64Kbps audio) via a well-known CDN using AVC would cost approximately $0.441 per delivery In this case, delivery to one million households would cost $441,000! HEVC offers the means to significantly reduce those delivery costs. A 50% reduction in bitrate, applied over many different titles for many different markets, soon adds up to a substantial reduction in cost of delivery

Using less bandwidth is not the only opportunity, though. A more efficiently delivered video stream is less susceptible to stutters, stalls, and buffering issues, which have all been demonstrated to improve the overall viewer experience, with clearly documented improvements in consumer retention rates, which again translates into increased profitability for the media enterprise.

It is easy to fall into the trap of oversimplifying the benefits of HEVC. HEVC is not only a solution for higher quality or greater efficiency. Rather HEVC provides the flexibility to strike an optimal balance between the two, finding a "sweet spot" that makes the most sense for a particular business model--increasing the variety of services, enabling the introduction of new services, and managing capital and operating expenses.

The benefits of HEVC are not binary, and an operator is not forced to choose greater efficiency or higher quality. It is overly simplistic to reduce the benefits to the extremes when in reality it is a sliding scale. Operators can adjust the encoding to produce the video quality and bitrate that they believe fulfills their business needs.


All encoders work by removing repetitious (redundant) bits of data. There are two approaches to this. The first is lossless compression, in which only the truly redundant information is removed (and is therefore generally limited to compression ratios in the 2:1-3:1 range). The second approach (and the one most commonly used) is lossy compression, in which not only is truly redundant information removed, but some fine detail is also removed--the idea being that the human visual system will largely be unable to tell the difference between the original and the modified versions. This approach can yield much larger compression ratios. Determining the material that can be removed requires considerable computational power, and it is in this analysis that HEVC produces its performance gain. However, in a lossy scenario, as the codec approaches its maximum compression ratio, it is inevitable that greater loss of picture information must be accepted. Managing and masking the appearance of the resulting artifacts is the arena where encoding manufacturers differentiate themselves, and this is where the secret tools of pre-processing enable those errors to be hidden from view.


Like many of its predecessors, HEVC is an asymmetrical operation. In other words, encoding is much more computationally expensive than decoding. The effects of Moore's law means that considerably more processing power (and memory) are now economically available. This in turn allows manufacturers to insert additional analysis and processing into the encoder, which is where much of the coding gain comes from. Here are some examples of the processor-intensive enhancements over H.264 that are available in the HEVC toolkit:

* An increase in the number of directions for intra-frame (within a frame) encoding. The intra encoding portion of HEVC looks for groups of pixels in a frame which are repeated elsewhere in that same frame (see Figure 2).

* The use of 16-bit motion vectors. This allows for larger search areas in inter-frame (across frames) encoding.

* The availability of larger coding tree units (macroblocks) when compared to H.264,This can improve prediction efficiency.

* Support for wider color gamut (broader representation of visible colors). This requires not only more processing, but also more memory.

* Higher frame rates. HEVC allows for frame rates of up to 300 fps. The results of an increased frame rate are quite dramatic. This is borne out by several recent studies by the BBC, Fox, and others. This again requires more processing and memory.

Clearly, there are advantages to all encoding activities in utilizing as many of the above advantages as possible when creating HEVC content to the extent that the available processing power and time/ latency constraints will allow.


One approach that has been used to maximize image quality from today's encoders is through "look ahead"-- seeing further in time down the video stream to determine how often you can re-use the same data. This enables the encoder to effectively determine if there is an opportunity to re-use image information, which can then be turned into bitrate savings by not repeatedly re-resending that data. The efficiency of this approach is directly proportional to the number of "future frames" available to the encoder. In live applications, the number of "future" frames is limited by the number of frames available in the encoder's video buffer, and the buffer size is limited by the latency acceptable to the live application. In the encoding of prepared material, however, that limitation is removed--we have all of the frames that make up the entire clip (or section thereof), and can theoretically analyze the actual image content of all of them--although in practice there will still be limits to the total size of the lookahead.

This approach was used extensively in Telestream's implementation of the x264 codec. This advanced H.264 codec used a concept called a "macroblock tree" to significantly improve efficiency and image quality (see Figure 3).The macroblock tree is effectively a map for each macroblock in the current frame, which indicates where that macroblock can be used again. This may be in the next frame, or it may not be required for a second, but then be repeatedly used in the next two seconds worth of frames. By analyzing the reuse of the macroblocks and ranking them in reuse, the encoder can then apply more of the available bits to the encoding of that macroblock and less to macroblocks that may only be re-used once (or potentially not at all). This will result in a proportional increase in the video quality of the encoded picture, and an associated reduction in error propagation.

While this technology was developed for the x264 codec, the same technology and techniques are being carried forward into the x265 encoder detailed below (although to be accurate, the term "macroblock" is replaced by the term "coding tree unit" in x265). Looking seconds ahead on the video unlocks the full capability of the encoder's capacity for efficiency and image quality.

It is important to note that these benefits are not processor dependent. These benefits accrue regardless of the hardware, because the only barrier is the availability of subsequent frames of video for macroblock evaluation. The method's success is not a function of processing but simply one of data availability.

The non-transient nature of prepared material also allows for multi-pass encoding. The details of this technique are well known and beyond the scope of this article, but in a nutshell, multi-pass encoding involves performing a first pass at encoding the material, then going back and examining the results of that encode. Some frames or groups of frames are very difficult to encode, and may show some artifacts from the applied compression. Others are much easier to encode, and may produce quality measurements far in excess of the target value. By analyzing this, it is possible to "borrow" some of the bit budget from the easy sections, and give it to the difficult sections, which you can now re-encode with a lower compression ratio, improving the quality of the overall clip.


Developing an encoder is a very complex piece of work and requires a significant investment in research and development. As a result, three distinct methodologies have emerged amongst encoder manufacturers:

1. Many manufacturers outsource development of their codecs to a third party that specializes in that technology.

In fact, the majority of encoders rely on codecs created by a single vendor. While this approach can be cost-effective, it necessarily entails ceding control over development to a third party and accepting that the codec will be fundamentally similar to many competing products.

2. A second approach is for the developer to design the codec entirely in-house. While this approach gives the manufacturer full control, it is costly and limits the capabilities to the expertise of the staff on hand.

3. A third and highly compelling approach is to adopt the open source model. This is the approach that has been successfully followed by Telestream, Instead of limiting itself to an R&D department that fits into one floor of an office building, open source development utilizes the expertise of a global community of experts that span the world. Leading experts in universities and private labs bring incredible creativity and knowledge to bear on the challenge, In the case of HEVC, this approach has given rise to the x265 organization, of which Telestream is privileged to be a founding, funding member.

This approach makes it possible to build on the tremendous power of the best of the x264 technology, also developed by an open source team exceeding 110 in number, and apply these techniques to the x265 encoder design.

More information on the x265 project can be found at


HEVC is a highly advanced encoding standard, offering the possibility of dramatic improvements in compression efficiency and deliverable image quality, and providing significant benefits to consumers and media enterprises alike. HEVC has an extensive set of tools in its arsenal, but some aspects require that the encoder have access to significant numbers of frames in the asset in order to make the best use of the available bits; that requirement means that live encoders may not be able to make full use of all of the tools available to them. An analysis of the media consumption models shows that most material being consumed has a high proportion of prepared material, however, and this material does allow the encoding process to make use of the advanced processing options available in both x264 and x265. The algorithms developed as part of the open source x264 project are being brought to bear in the x265 implementation, and Telestream will be offering this best of breed implementation in all of its enterprise products--both CPU-and GPU-based--during the coming year.

By Paul Turner, VP of Enterprise Product Management, Telestream
COPYRIGHT 2014 Information Today, Inc.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2014 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Turner, Paul
Publication:Streaming Media
Date:Jan 1, 2014
Previous Article:DIVX[R]: HEVC accelerating adoption with DivX's end-to-end solution.
Next Article:Haivision: more efficient encoding, more effective transport.

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters