Offering data savings of up to 25%, Apple has joined the Alliance for Open Media and thus added its name to the long list of companies championing the HEVC rival video codec AV1.
Without any great fanfare, certainly no press announcement, the webpage that lists the founding members of the Alliance for Open Media (AOM) quietly updated in the first week of January to include Apple.
For the AOM and for the codec it has been developing for a couple of years now, AV1, this is a highly significant event. Apple is the last of the tech giants to join the Alliance and, of course, publicly threw its weight behind the rival HEVC/H.265 codec last summer (see our blog post, Video Streaming Services: All Change as Apple Chooses HEVC, for more details).
So, why now publicly (if quietly) back another codec? At least part of the reason is financial. As TV operators will all be too aware, encoding HEVC content costs and, if you are serving enough of it up to your customers, it costs a lot.
AV1: the history and why it exists
AV1 was first announced back in September 2015 by the Alliance for Open Media (AOM) which featured a stellar line-up of charter members including Amazon, Cisco, Google, Intel, Microsoft, Mozilla, and Netflix (the full list now includes Apple, ARM, Facebook ,IBM ,and Nvidia). It was positioned as a replacement for Google’s own open source video codec VP9 and as a direct competitor against HEVC.
One of the most impressive things it did was to bring together three, competing open source codecs — Cisco's Thor, Google's VP10, and Mozilla's Daala — into one development effort. Amongst other aims, the resulting codec was designed to be interoperable and open:
- Scalable to any modern device at any bandwidth
- Have a low computational footprint
- Be flexible for both commercial and non-commercial content
- Be capable of consistent, highest-quality, realtime video delivery
The key word in all of this is ‘open’. The AOM license for AV1 states that it is “perpetual, worldwide, non-exclusive, no-charge, royalty-free, and irrevocable”, whereas license fees are payable to deploy HEVC on devices and content royalties are incurred with HEVC-encoded videos.
Individually, as the table of royalty rates maintained at HEVC Advance shows, these don’t appear too onerous. A mobile phone incurs a $0.40 fee if manufactured in a Region 1 country; a 4K TV set will cost its manufacturer $1.20, with the addition of Advanced Profiles adding another approximate 60%. It’s when you realize the device caps per manufacturer for both are $30 million and $20 million respectively that you start to understand the scale of the costs involved.
Content is the same story. In fact, it’s worse, as although the cap is lower and the expense is limited at $2.5m, the fee levied per individual SVOD subscriber is increasing. In 2016-17 it was $0.005, in 2018-2019 that will rise to $0.15, and from 2020 on that will be $0.25.
It’s against this costs background that AV1 was born. Indeed, it had an effect fairly immediately too, with HEVC Advance announcing that it was waiving royalties for public/non-profit TV broadcasting, introducing the royalty caps, and offering “substantially reduced pricing”.
The potential benefits (and potential pitfalls) of AV1
So how does AV1 perform? One of the main reasons Apple switched from H.264 to H.265 for its new iOS11 was for the 40% saving in data sizes that it conferred. How does AV1 measure up against that?
The answer seems to be rather well. Detailed information on exactly how AV1 performs is scarce at the moment, though we can expect more to surface over the next few months. But Mozilla has been incorporating it into the Nightly Firefox test build since November and claims a 25% to 35% efficiency over and above HEVC capabilities.
For a codec whose bitstream is yet to be finalized, that is an impressive performance. However, there is a long way to go yet. As the same site reports: “The current state of the AV1 encoder is still far away from delivering reasonable encoding times without extensive tuning to the code base: e.g. it takes about 150 seconds on an off-the-shelf desktop computer to encode one second of video.”
This is a perennial problem for new generation codecs. H.265 was far more difficult to encode as a result of its complexity over H.264 and took up to 10 times the compute power to encode at the same speed as its predecessor. AV1 can already be encoded much faster than one frame per 150 seconds, but it takes a significant amount of resources to accomplish that, such as Bitmovin’s cloud native architecture. At NAB2017 this was showing 1080p live broadcast quality at just 1.5 Mbps.
AV1: where are we now?
Of course, by the time AV1 hits the mainstream the encoding will be far more optimized. As to when that will be, initial plans were for AV1 to ship in December 2016, but the project was delayed by technical complexities. While some of the features have now been frozen, work is still underway on some of the components — the previously mentioned bitstream format, in particular — but that is expected to be nearing fairly imminent completion. [Update: the bitstream was frozen in late March 2018.]
The timeline after that is not fixed, but AV1 chipsets will probably take at least a year to be developed and then productization will take a further chunk of time. As of January 2018, the best guess is that it won’t be until late 2019 that we will see AV1 as a commercial reality.
Apple’s announcement gives the whole project added impetus though and suggests that it could be either planning to put it at the center of its next generation of devices, using it as a key plank in a forthcoming SVOD business to rival Netflix and Amazon, or both. In the meantime, soon after everything is finalized expect swift browser support to be rolled out and YouTube to start supporting the format.
HEVC has already shrunk the amount of data on the web. Given the internet’s infrastructure, this provides a greater chance of dealing with the explosion of video content crossing it every day, but it does so at a financial cost. If that volume of traffic, and especially video traffic, can be further reduced, and for free, then that’s something very worth keeping an eye on. It will take a while for all this to happen — it’s certainly not time for operators to start thinking about AV1 encoding just yet — but as Streaming Media’s excellent analysis concludes: “Apple's joining AOM sends a clear message to HEVC IP owners that unlike what they believed after Apple added HEVC to HLS, HEVC isn't the only game in town.”