We had a chance to talk to Marta Karczewicz, vice president of technology Qualcomm Technologies, who helped shape what will be known as VVC or H 266. Compared with a raw data file, VVC H 266 offers a 1000x reduction in file size.
Marta’s team helped to compress textual content in the video, something that brought an additional 30 to 40 percent of bandwidth saving on top of close to 40 to 50 percent general reduction compared to H.265.
The heavy lifting work on the codec contribution started in 2018 when Marta Karczewicz and the rest of the Qualcomm team started working as one of the main contributors to Versatile Video Coding (VVC) known as H266. The High-Efficiency Video Coding, known as H 265, is the second most popular codec in the world that was used by 43 percent of video developers. The H 266 is the next generation codec set to replace H 265 in the coming years. The most popular codec today is its predecessor, H 264.
The H 266 is the successor of the H265, and the final standard finalized on 6 July 2020.
Text compression saves up to 40 percent bandwidth
Qualcomm found a way to compress the text inside of the video. A very sophisticated algorithm is identifying letters inside of the video, and codec is being able to identify and classify letters. Letters inside of the text and video presentation are very repetitive, and finding a way to identify and reuse some of the blocks and pixels can result in significant savings.
Marta described this as an incredibly hard task, but Qualcomm achieved an additional 30 to 40 percent of bandwidth saving on top of close to 40 to 50 percent general reduction compared to H.265.
Post-COVID 19 Video conference call world
The timing could not be better as since March 2020 and COVID 19 pandemic, most businesses are moving to online meetings instead of face to face. The Qualcomm contribution helps tremendously as it will reduce the bit rate necessary for a good quality video conference call.
Text and graphics are difficult to code, but the now added tool uses intra block copy and basically assumes that you have repetition in the frame. Letters have a high probability of repetition, and the algorithm designed by Qualcomm is looking to identify and reuse repeated letters. The algorithm searches for a given letter, for example, A in the previous frame, fetch it, code it with almost no bits and bandwidth, and draws it.
Being able to optimize the text inside of the presentation and video would bring a better quality at lower bandwidth and save millions to the streaming platforms.
The VVC H 266 focuses on 4K 60 FPS content, and companies like Netflix today bring 25 Gbps as a desirable bandwidth width to stream high-quality HDR 4K 60 FPS. In reality, the content bit rate varies, and it can e anywhere between a few Mbps and 25 Mbps. Just think about a dark scene with very repetitive blocks, will likely use much less bandwidth then high complex scenes with lots of small moving objects, for example, confetti flying.
From 25 Mbps to below 10 Mbps 4K 60
Instead of 25 Mbps streaming movies rate can drop to close to 15 Mbps with H 266, and in presentation and heavy graphics and text modes, the necessary bandwidth can go down to below nine to10 Mbps. Above is a very crude basic calculation of the best-case scenario, but in reality, one should expect slightly higher bandwidth rates.
In addition, some elements of optical flow, some estimation of motion, have been improved, and these new improvements work well on a high frame rate. The important task is to find an optical frame flow without reasonable complexity and follow the object moving in the following frames. In the new codec, not everything has to be a rectangle. Now the blocks can be represented as diagonal.
If 4K 60 sounds like a lot very soon 8K, 120 FPS might become a new normal, resulting in eight times more pixels to process.
The future Qualcomm chipsets will support this codec first on the decoding side and later on the encoding, but this is nothing unusual for the hardening codec in any of the chipsets.
Beyond phones and PCs, it is expected that this codec will find its way to the future automotive, smart cities, industrial IoT platforms as saving bandwidth will reduce the cost of infrastructure.
“By 2022, 82% of internet traffic will be video. In our connected world of Netflix, Zoom calls, and video sharing, video compression is an extraordinarily important technology area that people around the world rely on every day”, said Marta Karczewicz, vice president of technology, Qualcomm Technologies, Inc. and nominee for the European Inventor Award’s Lifetime Achievement laurel for her seminal work in video coding. “Qualcomm Technologies’ worldclass engineers have driven the standard to completion with our foundational technology contributions and unparalleled leadership.
“Qualcomm Technologies is extremely proud of the great work our inventors and engineers have done in creating the core technologies that went into the VVC standard and driving the standard to completion in the middle of a global pandemic. Qualcomm is an IP leader across so many different fields of technologies that facilitate people’s connected experiences on a daily basis."
"We change the way the world connects, computes, and communicates, and we’re excited to enable the new possibilities of video streaming VVC offers the world”, said Jim Thompson,executive vice president, engineering and chief technology officer, Qualcomm Technologies, Inc.
With the recent development and the major legal victory for Qualcomm’s FTC appeal case, not only does Qualcomm’s business model gets validated, it looks like that even high court executives recognized the importance of licensing and standard contribution.
VVC H266 would not be as good without Qualcomm’s major contribution, and neither would 5G. VVC, like the 5G cellular standard, is the result of a collaborative effort between dozens of companies offering technologies and driving consensus. Qualcomm has decades of contribution to technology standards, developing and contributing the foundational technological innovations that form the heart of how these technologies work.