March 18, 2024
By Jan Ozer Contributing Editor
Featured Articles

如何为FFmpeg封装和ABR分发部署GPAC

As much as we love FFmpeg for transcoding operations, it can get frustrating when packaging your content for ABR delivery. By packaging, 我的意思是格式化和分割你的媒体文件, 为HLS和DASH创建清单文件, formatting for CMAF，以及管理多个音频和字幕流. Fortunately, there are easier-to-use solutions that are equally open source and equally free. In this article, I’ll focus on GPAC, which is a great packaging alternative.

你可以在它的网站上看到， gpac.io, GPAC is an open source multimedia framework used in many media production chains. It’s an adjunct to FFmpeg, and it can use FFmpeg for multiple processes. In this article, which focuses on VOD, I’ll use FFmpeg and MP4Box, one of the GPAC executables, separately.

In terms of credentials, 许多大公司都在使用GPAC, including Netflix, which is packaging some of its live-streaming offerings via GPAC. 您可以阅读其他一些GPAC用户 here.

Getting Started

让我们从一些初步的东西开始. GPAC适用于Windows 32位(Vista+), Windows 64-bit (Vista+), Mac OS X, Linux 32 bits (Ubuntu 18.04*)， Linux 64位(Ubuntu 20.4*)， iOS(需要为您的设备重新签名)和Android. 安装后，有三套工具:

MP4Box, the multimedia packager that I will use in this article
gpac, a generic media pipeline orchestrator not used in this article
Bindings for Python and Node.Js，本文中也没有使用

您需要在系统上安装Python和FFmpeg来运行MP4Box. I used Python 3.10.7 and FFmpeg 6.

我将在所有场景中使用一组通用文件, and you can download these, the batch commands, and all output files here. As you can see in Figure 1, 一个编码阶梯中有三个文件, 以及英语和法语音频和字幕音轨. 字幕由罗曼·布奎提供, one of GPAC’s principal architects since 2007 and my technical contact throughout this process. The French audio translation is computer-generated and not particularly synchronized, but it is French, so it probably won’t care.

gpac source files
Figure 1. Source files for all projects

我将创建四组打包文件:

hls -每梯级一个文件(MPEG-2传输流格式)
DASH—One file per ladder rung (fragmented MP4 format; fMP4)
cmaf -每梯级一个文件(fMP4)
CMAF—One file per ladder rung with English/ French audio and subtitles (fMP4)

Example 1: HLS—One File per Ladder Rung (MPEG-2 Transport Stream)

The initial project is to create HLS output in segmented transport stream files, HLS最常见的用例. 下面是命令字符串和解释:

MP4Box -dash 2000 -profile onDemand jan_1080p.mp4 jan_720p.mp4 jan_540p.mp4 jan_audio.mp4 jan.vtt -out hls.m3u8 --muxtype=ts

MP4Box—Calls MP4Box
–dash 2000—Creates 2,000 ms segments
–profile onDemand—Creates a single .ts file for each media asset. 使用-profile live分隔 .ts files.
梯子中所有媒体文件的列表(包括标题)
-out hls.m3u8 --muxtype=ts-命名HLS清单文件并将其格式为 .ts files, not fMP4

Note that the subtitles didn’t appear in the VLC media player until I upgraded to version 3.0.20. 如果你在这些过程中有任何问题, be sure to upgrade to the most recent copy of GPAC and the most recent version of VLC (go2sm.com/vlc).

The outputs are (see Figure 2):

hls.m3u8—Master manifest
hls_n.m3u8-每个媒体资产的清单文件
.ts/vtt-每个媒体资产的媒体文件

Figure 2. 由MP4Box创建的媒体和清单文件

将所有这些复制到你网站上的同一个文件夹中, and link to the master, 你就有了一个HLS的线人.

注意，媒体文件包含单词“破折号”, because architecturally, HLS由创建DASH输出的GPAC模块生成. You can remove the dash designation via commands documented in the help file, 但我决定在本教程中保持简单.

Moving on, Figure 3 shows the master manifest file, which you would link to or click to play the HLS presentation. As you can see, the master contains file details that allow the player to choose the correct file at the start and during playback. Typically, playback begins with the first file in the master but can switch to any other video based on available bandwidth or compatibility. So, 如果一个很旧的iPhone点击了这个链接, it might have to skip to the 540p file if it can’t play the 1080p file.

media manifest
Figure 3. The master manifest file identifies the location and relevant details of the media files.

If you open any of the other .m3u8 files, you’ll see that they call 2-second byte range requests in each media file to play the presentation.

例2:dash -每梯级一个文件(fMP4)

现在，让我们从相同的源文件创建DASH输出. Here’s the command string:

MP4Box -dash 2000 jan_1080p.mp4 jan_720p. mp4 jan_540p.mp4 jan_audio.mp4 jan.vtt -out jan_DASH.mpd

MP4Box—Calls program
–dash 2000-DASH输出，2秒字节范围请求
所有媒体文件(包括标题)的列表
-out jan_DASH.mpd-将MPD文件标记为jan_DASH

MP4Box创建如下所示的文件 Figure 4. 你可以链接到警局文件, which contains byte range requests for all of the 2-second segments in the media files.

Figure 4. DASH output

Figure 5 显示字幕轨道和字幕播放的视频. Since I’ve only included a single subtitle track, that’s all you see.

Dash演示在VLC中播放
Figure 5. 这是在VLC中播放的DASH演示.

示例3:cmaf -每梯级一个文件(fMP4)

CMAF演示包含一组音频, video, and caption files that play on HLS and DASH and manifest files for HLS and DASH. Note that the output format must be fragmented MPEG-4 files for both standards to work and can’t be transport streams.

Here’s the GPAC command:

MP4Box -dash 2000 jan_1080p.mp4 jan_720p.mp4 jan_540p.mp4 jan_audio.mp4 jan.vtt -out jan_CMF.mpd:dual

新命令执行以下操作:

jan_CMAF.mpd:dual-强制CMAF输出与HLS和名称清单文件JAN_CMF

Figure 6 包含所有输出文件. 共享的媒体文件均为fMP4格式, which won’t play on the already tiny and rapidly shrinking range of legacy HLS devices. 主HLS清单是jan_CMF.m3u8, which links to the byte range requests in the other media manifest files. 单个MPD文件包含所有DASH链接.

CMAF output

Figure 6. CMAF output with a single set of content files and DASH and HLS manifest files

Again, you can create individual segmented fMP4 files for the content using the profile live switch. 如果您播放m3u8文件或MPD, you can load the captions file by clicking Subtitle > Sub Track and choosing the only available subtitle track.

Example 4: CMAF—One File per Ladder Rung With English/French Audio and Subtitles (fMP4)

我们的最后一个项目是告别练习, 就像前面的练习一样, but I’ll add the French audio and subtitle track and will label both sets of subtitles and audio tracks:

MP4Box -dash 2000 jan_540p.mp4 jan_1080p.mp4 jan_720p.mp4 jan_audio.mp4:#udta_name= " English ":#HLSMExt=DEFAULT=YES .mp4:#udta_name=“French” jan.vtt:#Language= en:#Representation= " English.vtt: #Language=fr:#Representation= " French " -out JanCMAF_ML.mpd:dual

Here are the new commands:

jan_audio.mp4: # udta_name = "英语":# HLSM Ext =默认= YES—Identifies the audio file as English and makes it the default audio track
Jan_audio_fr.mp4:#udta_name=“French”— 将音频文件标识为法语
jan.vtt: #语言= en: #表示=“英语”-识别并标记字幕为英语
jan_fr.vtt: #语言= fr: #表示=“法国”-识别并标记字幕为法语

这会生成与中所示相同的文件 Figure 7，有一个额外的音频和字幕轨道. Playing the HLS file in VLC enabled both audio tracks, as shown in Figure 7. 法语轨道应该是一个相对准确的翻译, although I didn’t take the time to synchronize the spoken word to the video.

properly labeled audio tracks

Figure 7. Properly labeled audio tracks

你可以看到字幕 Figure 8法国人出现在视频中. This is playing the DASH MPD file; you will see a very similar, but not identical, presentation in HLS.

subtitle tracks in vlc

Figure 8. 摇摆英语和法语字幕轨道在VLC

Summing Up

I’ve found that packaging media for ABR distribution in FFmpeg can be challenging, frustrating, 有时是完全不可能的. GPAC simplifies most common use cases into command strings that even a total novice could master in less than a few minutes.

Free

for qualified subscribers

Subscribe Now Current Issue Past Issues

如何用FFmpeg制作VVC

任何时候你开始使用新的编解码器, there are some fundamental tests that you should run to achieve optimal performance/quality optimization. In this article, I'll take you through those tests while encoding VVC using a version of FFmpeg that includes the Fraunhofer VVC codec.

21 Mar 2023

如何使用PowerShell和BASH为FFmpeg编写脚本

FFmpeg was designed as a cross-platform solution for video and audio recording, conversion, 以及使用简单的静态命令行进行流式传输. Using variables and "for loops" in a command string simplifies the reuse of existing scripts and helps automate their operation. 虽然你不能在Windows命令窗口中使用这些脚本, you can use Microsoft PowerShell in Windows and Bash on Linux and the Mac. In this tutorial, you'll learn how to create and run such scripts with PowerShell and Bash.

14 Mar 2023