How to Use FFmpeg in Android with Example?

FFmpeg, short for Fast-forward MPEG, is a free and open-source multimedia framework, which is able to decode, encode, transcode, mux, demux, stream, filter and play fairly all kinds of multimedia files that have been created to date. It also supports some of the eldest formats. FFmpeg compiles and runs, across various operating systems like Linux, Mac OS X, Microsoft Windows, the BSDs, Solaris, etc. among a large range of build environments, machine architectures, and configurations. Programming languages used in FFmpeg are C and Assembly language. We can do many fun kinds of stuff using Ffmpeg like, Video Compress, Audio Compress, Trim Video, Rotate Video, Crop Video, Adding filters to videos, Reverse a Video, Creating fast and slow-motion video, Fade in fade out, Merge audio and video, Creating a video from images, Convert video from one format into another, Extract Picture from Video or Sound from Video, Gifs overlay, and many more. FFmpeg is part of the workflow of hundreds of other media-related software projects, and it's often used behind the scenes. Also, It is an internal part of software such as VLC media player, YouTube, Plex, iTunes, Shortcut, Blender, Kodi, HandBrake, it handles video and audio playback in Google Chrome, and Linux version of Firefox. FFmpeg comprises an enormous set-up of libraries and projects for dealing with video, sound, and other multimedia files and streams.

FFmpeg Libraries 

• libavutil is a utility library to help versatile media programming. It contains portable string functions, arbitrary number generators, extra arithmetic capacities, data structures, cryptography, and core multimedia utilities.
• libavcodec is a library that provides encoders and decoders for video/audio codecs, subtitle streams, and several bitstream channels.
• libavformat is a library that provides multiplexing and demultiplexing framework for video/audio codecs, subtitle streams
• libavdevice is a library containing I/O devices for getting from and delivering to numerous multimedia I/O programming systems, including Video4Linux, ALSA, and VfW.
• libavfilter library provides a media filtering framework that contains several filters and sinks.
• libswscale library performs exceptionally enhanced picture scaling and pixel format transformation tasks.
• libswresample is a library that performs highly optimized but a lossy change in audio rate, change in channel layout, for example from stereo to mono, and sample format conversion operations.

Android doesn’t have efficient and robust APIs for multimedia which could provide functionalities like FFmpeg. The only API which android has is  MediaCodec API, but it is faster than FFmpeg because it uses the device hardware for video processing.

Create a Small Video Editor App in Android Studio using FFmpeg

Prerequisites: 

Before we start, we need to set up an environment to run our FFmpeg commands. There are two options to do so:

• By building our own library
• By using any compiled source provided by the community. There are many libraries that can be used to perform FFmpeg operations in Android. For example,
  • WritingMinds
  • Bravobit
  • tanersener/mobile-ffmpeg
  • yangjie10930/EpMedia: There are many inbuilt functions present in this library from which you can clip, crop, rotate, add a logo, add a custom filter, merge different videos.

Although it is highly recommended to build your library, because that will reduce your apk size, you can add a third-party library and can update the library with time as you want. But, this process is very time consuming and requires extra skills. So, as a beginner, you can use some above-mentioned libraries and if you face some issue you can raise that issue on their respective GitHub repository. In the below example I will be using tanersener/mobile-ffmpeg, as it has support for Android 10 scoped storage, and also it is the best library available on the internet for FFmpeg mobile. A sample GIF is given below to get an idea about what we are going to do in this article. Note that we are going to implement this project using the Java language. 

Step by Step Implementation

Step 1: Create a New Project

To create a new project in Android Studio please refer to How to Create/Start a New Project in Android Studio. Note that select Java as the programming language.

Step 2: Adding a dependency to the build.gradle file

We will be using tanersener/mobile-ffmpeg library to implement FFmpeg functionalities in our app. And we will also require a rangeseekbar to select the particular length of the video. So add these dependencies in build.gradle file.

implementation 'com.arthenica:mobile-ffmpeg-full:4.4'
implementation 'org.florescu.android.rangeseekbar:rangeseekbar-library:0.3.0'

Step 3: Working with the colors.xml file

Below is the code for the colors.xml file.

Step 4: Working with the activity_main.xml file

Go to the activity_main.xml file and refer to the following code. Below is the code for the activity_main.xml file. 

Step 5: Working with the MainActivity.java file

Go to the MainActivity.java file and refer to the following code. Below is the code for the MainActivity.java file. Comments are added inside the code to understand the code in more detail.

Step 6: Creating a new Java Class FileUtils.java

Refer to How to Create Classes in Android Studio to create a new java class in Android Studio. This is a Utility file that will help in retrieving the File from a Uri. Below is the code for the FileUtils.java file. Comments are added inside the code to understand the code in more detail.

Output:

Github Project Link: https://github.com/raghavtilak/VideoEditor

Few FFmpeg commands that you can play with

• concat video of different framerates in .mkv format:
  • -i input1.mp4 -i input2.mp4 -filter_complex [0:v:0][0:a:0][1:v:0][1:a:0]concat=n=2:v=1:a=1[outv][outa] -map [outv] -map [outa] output.mkv
• concat video with no sound/audio:
  • -y -i input.mp4 -filter_complex [0:v]trim=0:0,setpts=PTS-STARTPTS[v1];[0:v]trim=0:5,setpts=0.5*(PTS-STARTPTS)[v2];[0:v]trim=5,setpts=PTS-STARTPTS[v3];[v1][v2][v3]concat=n=3:v=1:a=0 -b:v 2097k -vcodec mpeg4 -crf 0 -preset superfast output.mp4
• Textoverlay:
  • -y -i input.mp4 -vf drawtext="fontsize=30:fontfile=cute.ttf:text='GFG'":x=w-tw-10:y=h-th-10 -c:v libx264 -preset ultrafast outputmp4
• gif/png/jpeg overlay
  • -i input.mp4 -i inputimage.png -filter_complex [1:v]scale=320:394[ovr1],[0:v][ovr1]overlay=0:0:enable='between(t,0,16)' -c:a copy output.mp4, (or)
  • -i input.mp4 -i inputimage.png -filter_complex overlay=(main_w-overlay_w)/2:(main_h-overlay_h)/2:enable='between(t,0,7)' -c:a copy output.mp4
• Add subtitles to a video file
  • -i input.mp4 -i subtitle.srt -map 0 -map 1 -c copy -c:v libx264 -crf 23 -preset superfast output.mp4
• Converting video files to audio files
  • -i input.mp4 -vn output.mp3
• Cropping videos
  • -i input.mp4 -filter:v "crop=w:h:x:y" output.mp4
    • w - width of the rectangle which we are intended to crop from the source video.
    • h - the height of that rectangle.
    • x - the x coordinate of that rectangle.
    • y - the y coordinate of the rectangle.
• Adding aposter image to audio files
  • -loop 1 -i inputimage.jpg -i inputaudio.mp3 -c:v libx264 -c:a aac -strict experimental -b:a 192k -shortest output.mp4

Pros of using FFmpeg

1. It is also highly portable.
2. It is profoundly valuable for the transcoding of all kinds of multimedia files into a single common format.
3. You don’t need heavy Third-party VideoEditors like Adobe Premiere Pro, Filmora for small editing tasks.

Cons of using FFmpeg

1. It’s difficult for beginners to use and implement.
2. It takes some time to process. We don’t get results in a second or two.
3. The official documentation is quite confusing and it's not beginner-friendly.
4. APK size becomes very large. The FFmpeg libraries alone will use 30-70MB depending upon the libraries you are including.

Alternatives of FFmpeg

• MediaCodec Android
• LiTr
• Gstreamer
• MP4Parser
• Intel INDE Media for Mobile

Notes:

1. If you set -preset to a higher value say, ultrafast then the video processing will be fast but the quality of the video will be compromised. The lower the -preset higher the quality of the video.

2. You can change the -crf value to change the quality of the output video. Lower the crf value higher the quality of the video.

3. If you use -y in starting of command then this means that if a file is present with the same name as that of the output file name that FFmpeg will overwrite the existing file.

4. In the case of video, to slow down the video set -PTS value larger than 1. The larger the value slower the video, Lower the value Faster the video. But in the case of Audio this is just the opposite, i.e. Larger the value faster the Audio, the Lower the value slower the audio.

5. The atempo(audio) filter is limited to using values between 0.5 and 2.0 (so it can slow it down to no less than half the original speed, and speed up to no more than double the input)

6. FFmpeg takes too much time working with audio. If the video file doesn't contain the audio we need not to command FFmpeg to work with audio, and hence this will reduce the workload and we will get the processed video fast/in less time.