Exporting Video Encoding Statistics in Android 13

Video encoding has been a long-time feature present in the Android system, it only keeps getting better with newer hardware and now, it even goes further in Android 13. This update brings us, even better video encoding and a new feature of exporting the video encoding statistics. In this article, we will look at how to implement this, and what could be challenges we might face while implementing this. By the end of this article, you will be able to successfully export the statistics and then cross-work with them, so without further ado, let's dive deep into this.

Android 13 Introduces Video Encoding Statistics

App clients can ask the video encoder to export encoding information for each encoded video frame as of Android version 13. Apps can optimize their video encodings activities, such as multipass encoding and frame preprocessing before encoding, with the statistics obtained from the video encoder. SoC vendors must update the video encoder driver as outlined in Updates to the video encoder driver in order to export video encoding statistics.

Use Cases of Video Encoding Statistics:

Various apps we use every day have a great usage scope of this API when they implement it, if you own an editing app, you can now tell the user about the granular changes to the file types when the codecs change, it could also help by:

1. Providing an extra layer of information to tell users when they use the app
2. Determining the estimated time to complete more properly than ever.
3. Predetermining if the project will comply and export fully on the running hardware.

What all Data Can Be Exported:

With this API, the developer can reference two values as of now, as this API has just been released in Android 13, there would be further improvements in the future to make it even more feature rich.

The current methods are:

Adding Video Drivers to Encoding

SoC suppliers must update the video encoder driver to facilitate the export of video encoding statistics. In Android 13, the following keys are added to the Codec 2.0 base:

1. The per-frame average block QP is described by the key KEY_VIDEO_QP_AVERAGE.
2. The KEY_VIDEO_QP_AVERAGE key must be implemented in accordance with the guidelines listed below:
3. Before broadcasting to Codec 2.0, the SoC encoder must round the average of the block QPs to the nearest integer.
4. Only a luma plane is used to determine the average value.

GeekTip: If every block in the current frame is in the skipped mode, the SoC encoder must output INT MAX. This situation arises when the frame has no important QP information and no coefficients.

Values that Determine Video Encoding

Prior to Android 13, there was a lot of chaos, as there were different values that were proposed by using different functions for Video Encoding, but now we have a safe, dedicated method to work and analyze the proportion. We would need to understand that the following functions are available for us to use now:

1. Using floor(), the average value is rounded down to an integer value.
2. This is a portion of the export function for the video "Encoding Statistics."
3. For a video frame, the video encoder emits this value.

Implementing the new standards in applications

All of this knowledge is determined by a really simple API, which we can use in applications to help determine the exported video statistics, there would be a huge chunk of apps that would work with this setup. Use the code below to store the video statistics in an integer, and then you can continue to work on it to get the desired result:

Verifying if the Addition Works

To confirm that the average QP portion of the video encoding statistics functions properly, run the VideoEncodingStatisticsTest CTS test: The same input video is encoded twice by CTS, once at a higher bitrate and once at a lower bitrate. The two exported average QP values from the two encodings are then compared by CTS. CTS passes if the average QP from the lower bitrate encoding is higher than the average QP from the higher bitrate encoding. Use the VIDEO_ENCODING_STATISTICS LEVEL_NONE flag to confirm disabling the exporting of video encoding statistics when it is enabled, like in the example shown above.

Conclusion

Hope this article helped you understand how to add the Video Encoding Stats, and helped you learn that for all formats that are listed as unsupported, compatible media transcoding is enabled; for all formats listed as supported, it is disabled. The platform chooses whether to transcode for any other formats that have not been stated. For all undeclared formats, transcoding is disabled in Android 13. Future formats may cause this behavior to alter.