Although I know my way around shooting photos on my Sony A7 II, the art of videos has always eluded me. Still, with the fall colors soon to grace Ontario, I thought I’d take up shooting videos for once. This means I’ll need a new set of editing tools besides Adobe Lightroom. While the clear default options are Adobe Premier and Davinci Resolve, I wanted to learn FFmpeg as a complementary post-production for its incredible versatility.

Calling FFmpeg useful is an understatement. This open-source universal media converter is an important part of many popular media software and services like VLC Media Player, Blender, Google Chrome, Plex, and more. It has even been used by NASA to compress videos and images on the Perseverance Mars rover in 2020.

Being a command-line tool, FFmpeg’s strengths lie in its granularity and speed. Moreover, it’s infinitely customizable, so when my project eventually becomes complex enough, I’ll be able to write custom scripts that are suited to what I want to do.

Convert videos

One of FFmpeg's most basic commands

Converting from one media container format to another is FFmpeg's bread and butter, but its functionalities go way beyond just that.

Converting between media container formats is the most foundational and frequently used FFmpeg function. The basic syntax is simply:

ffmpeg -i input.format outputfile.format

Where -i denotes the input file path and space separating the output file path.

The command for converting examplevideo.avi to an .mp4 container would look like:

ffmpeg -i examplevideo.avi examplevideo.mp4

Where inputvideo.avi and outputvideo.mp4 specify the input and output filenames.

Video compression

Reduce video sizes using almost any codec you can think of

Compression is by far the most time-consuming task FFmpeg performs. To speed up the process, it's best to manually select an ecoder that can be hardware accelerated. 

The basic syntax for compressing video is:

ffmpeg -i inputvideo.format -c:v codecname outputvideo.format

Where -c indicates that a compression, :v specifies that it’s a video, and codec name is the compression codec you want to use.

If I want to compress examplevideo.mp4 using AV1, the command would look like:

ffmpeg -i examplevideo.mp4 -c:v libaom-av1 -crf 20 examplevideocompressed.mp4.

The -crf flag sets the constant rate factor that FFmpeg automatically optimizes the output’s bitrate to reach. The lower the number, the higher the quality. It isn’t a requirement, but it’s highly recommended to maintain the output’s consistency.

Extract audio from video

Useful for editing audio independently or for transcriptions

Here's the audio extraction command in action. Because the stock video I used for this article did not contain audio, FFmpeg threw me a warning. 

To only extract the audio of a video, the basic command is:

ffmpeg -i inputvideo.format -vn outputaudio.format

The -vn flag disables the video, so only the audio gets processed. You can also specify an audio codec to encode the audio with the -ac flag behind the -vn flag.

This command isn’t only useful for video editing. It’s also useful for pulling the audio from a meeting or an interview to reduce the upload size into a transcription tool.

Convert audio format

Helpful for resolving audio compatibility issues

Here's FFmpeg converting an M4A file to MP3. Easy peasy.

FFmpeg can also convert audio formats. The basic syntax is similar to converting a video:

ffmepg -i inputaudio.mp3 outputaudio.wav

Remove audio from videos

Mute the background audio track to avoid copyright issues

Need to remove the audio from a video for an analysis or commentary? This command does that:

ffmpeg -i inputvideo.format -an outputvideo.format

The -an flag just mutes all audio streams. Like in the previous example, it’s a good idea to add -c copy to prevent the video from being encoded again.

Add subtitles to videos

Make videos watchable for a broader audience

For adding different language subtitles to videos, the command is:

ffmpeg -i inputvideo.format -vf subtitles=subtitlefile.srt outputvideo.format

Resizing videos

Optimize videos for specific viewports

Here, I resized a 1080p video to 720p.

For changing video to a specific resolution, the basic syntax is:

ffmpeg - i inputvideo.format -vf scale=horizontalresolution:verticalresolution outputvideo.format

The -vf flag introduces the concept of filters (as indicated by the “f” in the flag), a core idea in FFmpeg. Similar to the cat face filter on Instagram that applies to my face, a filter in FFmpeg means a process applied to an input, such as cutting, resizing, and compression. Sequentially connecting multiple filters to the previous output is called a filter chain, while a filter graph visually shows how data flows through different filters.

To change examplevideo.mp4 to 1920x1080p, the command would look like:

ffmpeg -i examplevideo.mp4 -vf scale=1920:1080 exampleouput.mp4

Cut videos

Leave only the good parts

There are more than one way to cut videos using FFmpeg. Using the -to flag also works, but specifying the duration with -t is easier for me.

This command trims videos to the specified timestamp. The basic syntax is:

ffmpeg -ss start time -i inputvideo.format -c copy -t cutvideoduration outputvideo.format

The -ss flag tells FFmpeg to seek a specific timestamp.The copy following the -c (codec) flag means it’s a direct copy without encoding. The -t flag specifies how long to cut the video.

For grabbing a video between minutes 1 to 5, the command would look like:

ffmpeg -ss 00:01:00 -i examplevideo.mp4 -c copy -t 00:04:00 exampleoutput.mp4

Why use FFmpeg when Handbrake (and other tools) exist?

Given its many commands and customization options, FFmpeg grants near total control over the workflow, codec, and media formats. As the above commands show, it’s also handy for extracting different information from files, especially when paired with the ffprobe tool, a part of the FFmpeg framework. It supports almost all media and codecs, and shedding a graphical user interface makes it more lightweight and direct.

With that said, tools like Handbrake more than suffice for basic post-edits. However, I’d imagine that most aspiring content creators will eventually reach for FFmpeg if their work becomes advanced enough. Might as well familiarize ourselves with the tool from the get-go.

What's next?

I've spent a good amount of time messing about in FFmpeg, and I've only scratched the surface. So far, I've only edited a few stock videos and short snippets captured on my phone's camera. Even given my cursory dabbling, FFmpeg has already proven to be extremely handy (and seeing the command tossing up all the code makes me feel like a wizard). Its commands are extremely flexible and adaptable, and I enjoyed modifying them to be more efficient. In the future, I plan on properly creating scripts and maybe creating a cheat sheet, too.

Also, as a note, LLMs like ChatGPT are now excellent at spitting out custom FFmpeg commands for specific use cases based on descriptions. However, it’s always prudent to test out the command first before applying it in production.