VOOZH about

URL: https://bugzilla.mozilla.org/show_bug.cgi?id=2021722

⇱ 2021722 - Add Vulkan Video path to FFmpegVideoDecoder in Firefox


Closed Bug 2021722 Opened 3 months ago Closed 11 days ago

Add Vulkan Video path to FFmpegVideoDecoder in Firefox

Add Vulkan Video path to FFmpegVideoDecoder in Firefox
Core
Audio/Video: Playback
Firefox 150
Unspecified
Unspecified
enhancement
Points:
---
RESOLVED FIXED
RESOLVED
FIXED
153 Branch
Iteration:
---
a11y-review
Accessibility Severity
Performance Impact
Size Estimate
Webcompat Priority
Webcompat Score
Tracking Status
firefox153 --- fixed
Tracking Status
relnote-firefox
thunderbird_esr115
thunderbird_esr140
firefox-esr115
firefox-esr140
firefox-esr153
firefox152
firefox153
firefox154
---
QA Whiteboard:
[qa-triage-done-c154/b153]
Has STR:
---
Change Request:
---
Bug Flags:
Signature:
None
This bug is publicly visible.

 
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
4.53 KB, text/plain
Details
48 bytes, text/x-phabricator-request
Details | Review
Assignee

Description

3 months ago

Steps to reproduce:

Play any video in the Firefox browser on (Aarch64 on Linux)

Actual results:

No hardware acceleration for video decoding on Aarch64 (on Linux)

Expected results:

Browser should utilize GPU HW decoder when video decoding if any available on Aarch64 (on Linux)

Assignee

Comment 1

3 months ago

Added Vulkan Video path to FFmpegVideoDecoder in Firefox:

  • Vulkan video decode: Added Vulkan video decode with DRM modifier handling using FFmpeg Vulkan API to FFmpegVideoDecoder class;
  • Fixes Aarch64 HW detection and decoding (MOZ_FFVPX_AUDIOONLY so it didn't not disable HW decode on Aarch64);
  • Fixed EGL texture creation on DMA-BUF import on UV plane (renderer device may not support RG88 or GR88) (by querying supported plame 1 DRM format with queryDmaBufModifierExt);
  • Compositor–decoder DRM formats: Add IPC negotiation of supported DRM formats, opaque fd semaphore (lists intersection).
  • Synchronization: decode–copy image sync was done via opaque or sync fd binary vk semaphore;
  • Added DRM modifier decoder output: use direct DMA-BUF export when FFmpeg supports VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT, otherwise use copy path; pass DRM modifier list via hw_device_ctx->user_opaque.
  • Added distinction between direct/copy on signle/cross gpu scenarios;
  • Added poll fallback if android_native_fence_fd is not supported;
  • Enabled HW decoding tests for Linux.
Assignee: nobody → tboiko
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

The Bugbug bot thinks this bug should belong to the 'Core::Graphics' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → Graphics
Product: Firefox → Core
Component: Graphics → Audio/Video: Playback

Hello Tymur, thanks for the patch!

It looks great but we can't land it 'as is' as it may break existing decoding workflow. I suggest to split it to series of smaller patches and land them independently to integrate the whole solution. I'll go through the patch and suggest which parts should be separated to patches. There are also some parts I'd like to clarify how it's supposed to work and what's the intention.

Also, is that solution supposed to work with other hardware too? I don't have any NVIDIA hardware available, only Intel/AMD (and I can't make it work there - looks like dmabuf frames are not copied properly from vkImage - but that can be investigated later).
Thanks.

Flags: needinfo?(tboiko)
Assignee

Comment 5

3 months ago

Hi Martin,
Thank you for the comments - I will reply on them.
Yes, this change should work with other hardware too If you have Vulkan SDK and FFmpeg 6.1.1 installed. However, I have only checked it on NVIDIA and Intel GPUs.
What is the exact error log from you configuration?

Flags: needinfo?(tboiko)

(In reply to Tymur Boiko from comment #5)

What is the exact error log from you configuration?

I think I have all Vulkan bits installed - tested mpv vulkan video playback and WebGPU in Firefox. The bug may not be related to Vulkan but it's GL related - it fails to create EGLImage over the exported dmabuf. But tested on AMD only, will check Intel too. But that may not matter for now - it's fine to land it for NVIDIA first and and adjust for other setup.

Assignee

Comment 7

3 months ago

I need to test this again on other hardware - after that I'll confirm if it works (or I will fix it).

(In reply to Tymur Boiko from comment #7)

I need to test this again on other hardware - after that I'll confirm if it works (or I will fix it).

Well, I think it's okay to just make it work on NVIDIA first as far is doesn't break other arches. I added comments about the EGLImage creation which should be adjusted to make sure the recent VA-API setup works.

Assignee

Comment 9

3 months ago

Added hwcontext_vulkan.h from FFmpeg tag n6.1.1, n7.1
.1, n8.1.0, version.h from tag n8.1.0

Assignee

Comment 13

2 months ago

Accidentally created 2 revisions of the vulkan-headers patch, prefer the latter

Assignee

Comment 14

2 months ago

(D290317)

Attachment #9559010 - Attachment is obsolete: true

Thanks, landing the latter and obsoleted the former.

Comment 17

2 months ago
Pushed by stransky@redhat.com: https://github.com/mozilla-firefox/firefox/commit/e13534a525fb https://hg.mozilla.org/integration/autoland/rev/4f01fc22a0d4 Added (vendored) Khronos Vulkan-Headers (vulkan-sdk-1.4.341.0) under third_party. r=stransky

Comment 18

2 months ago
Pushed by asilaghi@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/340b752d07ea https://hg.mozilla.org/integration/autoland/rev/ccfc3c544041 Revert "Bug 2021722 - Added (vendored) Khronos Vulkan-Headers (vulkan-sdk-1.4.341.0) under third_party. r=stransky" for causing bugzilla lint failures at khronos/vulkan-headers

Backout for causing bugzilla lint failures at khronos/vulkan-headers
Backout link
Push with failure
Failure log
Failure line FileToBugzillaMappingError: Missing Bugzilla component: third_party/khronos/vulkan-headers/LICENSE.md - Set the BUG_COMPONENT in the moz.build file to fix the issue.

Flags: needinfo?(tboiko)

(In reply to Silaghi Andreea from comment #19)

Backout for causing bugzilla lint failures at khronos/vulkan-headers
Backout link
Push with failure
Failure log
Failure line FileToBugzillaMappingError: Missing Bugzilla component: third_party/khronos/vulkan-headers/LICENSE.md - Set the BUG_COMPONENT in the moz.build file to fix the issue.

will look at it.

Flags: needinfo?(tboiko) → needinfo?(stransky)
Assignee

Comment 21

2 months ago

Add Mozilla-side enablement for Vulkan video on Linux/GTK:

  • Static prefs and package-manifest / build wiring needed for the feature.
  • toolkit/moz.configure and related knobs (including AArch64 hardware decode
    vs MOZ_FFVPX_AUDIOONLY so HW decode is not incorrectly disabled on aarch64).
  • Widget/GTK: GfxInfo / DMA-BUF paths and local vulkantest where applicable
    for capability reporting and validation.
Assignee

Comment 22

2 months ago

Graphics stack support for Vulkan decode to compositor:

  • EGL / DMA-BUF import fixes for the UV plane when the renderer device may not
    support RG88/GR88 (e.g. query supported plane-1 DRM format via
    queryDmaBufModifierExt).
  • Compositor-decoder DRM format negotiation over IPC (supported DRM formats,
    opaque / sync-fd semaphore; intersection of capability lists).
  • Synchronization between decode and copy paths using opaque or sync-fd
    Vulkan binary semaphores where applicable (CompositorTypes / layers / WR
    bridge).
Assignee

Comment 23

2 months ago

Decoder and tests:

  • Vulkan video decode in FFmpegVideoDecoder using FFmpeg Vulkan API, with
    DRM modifier handling.
  • DRM modifier output: direct DMA-BUF export when FFmpeg supports
    VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT, otherwise copy path; pass modifier
    lists via hw_device_ctx->user_opaque as needed.
  • Direct vs copy paths for same-GPU vs cross-GPU scenarios.
  • Poll fallback when android_native_fence_fd sync is not available.
  • Enable / extend hardware decoding tests on Linux (mochitest updates).
Assignee

Comment 24

2 months ago

Sandbox:

  • Allow required syscalls and broker paths for Vulkan video in the RDD process
    (seccomp and filesystem broker policy updates for drivers, sockets, and
    related resources).
Assignee

Comment 25

2 months ago

Although the revisions are stacked across 4 patches for review and CI convenience, but each commit is independent, applies on its own, and targets a distinct area: RDD sandbox policy, DOM/Media FFmpeg Vulkan decode, GFX compositor/EGL/IPC, and MOZ/GTK enablement (prefs, build, GfxInfo/vulkantest).

D290521 is actually needed for the dmabuf patch as we do EGLimage copy and modifier extraction from GL.

Comment 27

2 months ago
Pushed by stransky@redhat.com: https://github.com/mozilla-firefox/firefox/commit/2c8a2018a02f https://hg.mozilla.org/integration/autoland/rev/b62491de1ada Added (vendored) Khronos Vulkan-Headers (vulkan-sdk-1.4.341.0) under third_party. r=stransky
Flags: needinfo?(stransky)

Isn't this a duplicate of bug#1753129 ?

Duplicate of this bug: 1753129

Tymur, if you update the changeset to latest trunk we can land the patches up to D290521 which means we'll have the support in dmabuf. (I tried to apply it but it fails - better if you do the rebase).

Assignee

Comment 32

1 month ago

Hi Martin,
Done, rebased all patches onto latest mozilla-central and updated the Phabricator revisions.

Assignee

Comment 33

1 month ago

These errors are false positives from the bot building each differential in isolation without the prerequisite patches in the stack. The patches are ordered by review domain (prefs/GTK, GFX, DOM media, sandbox) rather than strict dependency order — for example, D290520 (MOZ/GTK) uses GL bindings and IPC fields that are added by D290521 (GFX), so none of the patches can build standalone. FFmpegVulkanVideoDecoder.cpp is additionally a body-include file intentionally not compiled standalone. All errors disappear when the full stack is applied.

The separation into 4 patches was done for review clarity, but the cross-domain dependencies mean independent builds are not possible by design.

Assignee

Comment 34

1 month ago

However, D290520+D290521 together might build.

Pushed by stransky@redhat.com: https://github.com/mozilla-firefox/firefox/commit/50eed584edf8 https://hg.mozilla.org/integration/autoland/rev/cdd49c4d1c81 MOZ/GTK - Prefs, packaging, configure, and GTK platform hooks for Vulkan video. r=stransky https://github.com/mozilla-firefox/firefox/commit/e930cad7fa2a https://hg.mozilla.org/integration/autoland/rev/2e325646cf3f GFX - GL/EGL, layers, and compositor IPC for Vulkan video. r=stransky,lsalzman
Pushed by rperta@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/c72fae6f46d3 https://hg.mozilla.org/integration/autoland/rev/1743ac0be9bb Revert "Bug 2021722 - GFX - GL/EGL, layers, and compositor IPC for Vulkan video. r=stransky,lsalzman" for causing build bustages
Assignee

Comment 38

1 month ago

Added #undef DRM_FORMAT_MOD_INVALID before #include <libdrm/drm_fourcc.h> in
DMABufFormats.cpp to avoid a macro redefinition error in unified builds.
Matches the existing pattern in DMABufSurface.cpp.

Assignee

Comment 39

1 month ago

A potential fix has been submitted https://phabricator.services.mozilla.com/D296451

Tymur, please merge D296451 (the build fix) to D290520 so we can land it (right now there's a fork).

You can use 'hg rebase' tool to rebase D290521 on top of D296451 which makes one patch chain and then use 'hg histedit' to merge D296451 and D290520 and then abandon the orginal 'potential fix'.

If you wish I can also do that.
Thanks.

Flags: needinfo?(stransky) → needinfo?(tboiko)
Assignee

Comment 41

1 month ago

Hi Martin,
Done — merged D296451 into D290520 and rebased D290521 on top. Updated both revisions in Phabricator. Please double check it

Flags: needinfo?(tboiko)

Thanks, looks good, landing again.

Tymur, looks like the D290520 revision needs an update. Applying to latest trunk I'm getting:

$moz-phab patch D290521 --apply-to bf6944807b55
Revision D290521 has child commits. Would you like to patch the full stack?. (YES/No/Always)? No
Patching revisions: D290520 D290521
Checked out bf6944807b55
Bookmark set to phab-D290521
patching file widget/gtk/DMABufFormats.cpp
Hunk #1 FAILED at 17
1 out of 3 hunks FAILED -- saving rejects to file widget/gtk/DMABufFormats.cpp.rej
patching file widget/GfxInfoFeatureDefs.inc
Hunk #1 FAILED at 31
1 out of 1 hunks FAILED -- saving rejects to file widget/GfxInfoFeatureDefs.inc.rej
abort: patch failed to apply
CommandError: command 'import' failed to complete successfully
Run moz-phab again with '--trace' to show debugging output

Do you mind to check?
Thanks.

Flags: needinfo?(tboiko)

Sorry, looks like D290520 is outdated.

Assignee

Comment 45

1 month ago

Sorry, I have updated the branches, please double check again

Flags: needinfo?(tboiko)

D290520 works, D290521 needs update:

$moz-phab patch D290521 --apply-to 081c265a6023
Local VCS (hg) is different from the one defined in the repository (git).
Revision D290521 has child commits. Would you like to patch the full stack?. (YES/No/Always)? No
Patching revisions: D290520 D290521
Checked out 081c265a6023
Bookmark set to phab-D290521_1
file widget/gtk/vulkantest/vulkantest.cpp already exists
1 out of 1 hunks FAILED -- saving rejects to file widget/gtk/vulkantest/vulkantest.cpp.rej
file widget/gtk/vulkantest/moz.build already exists
1 out of 1 hunks FAILED -- saving rejects to file widget/gtk/vulkantest/moz.build.rej
abort: patch failed to apply
CommandError: command 'import' failed to complete successfully
Run moz-phab again with '--trace' to show debugging output
Flags: needinfo?(tboiko)
Assignee

Comment 47

1 month ago

Hi Martin, isn't D290520 already landed? If so, we only need D290521 (I submitted 2 of them today, changing that).
Updated D290521 on top of the main Fabricator.

Flags: needinfo?(tboiko)
Assignee

Comment 48

1 month ago

Sorry, I see they were reverted. I'll reapply both again on the rebased main branch.

Assignee

Comment 49

1 month ago

I have updated/rebase the patches again, please try again

Assignee

Comment 50

1 month ago

Note that I'm using the official Mozilla git mirror, not hg

Pushed by stransky@redhat.com: https://github.com/mozilla-firefox/firefox/commit/adeda526b1d5 https://hg.mozilla.org/integration/autoland/rev/f55edc75fd9c MOZ/GTK - Prefs, packaging, configure, and GTK platform hooks for Vulkan video. r=stransky https://github.com/mozilla-firefox/firefox/commit/157217280755 https://hg.mozilla.org/integration/autoland/rev/54abb3e4fbd0 GFX - GL/EGL, layers, and compositor IPC for Vulkan video. r=stransky,lsalzman
Pushed by csabou@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/b1d82a575362 https://hg.mozilla.org/integration/autoland/rev/0deb117694a7 Revert "Bug 2021722 - GFX - GL/EGL, layers, and compositor IPC for Vulkan video. r=stransky,lsalzman" for causing build bustages on TestDMABufSurface

Backed out for causing build bustages on TestDMABufSurface

Push with failures

Failure log

Backout link

Flags: needinfo?(tboiko)
Assignee

Comment 54

1 month ago

origin/main has 22 fields (including nullable FileHandleWrapper semaphoreFd) -> TestDMABufSurface.cpp calls with 22 args -> works
D290521 adds bool semaphoreFdIsSyncFd as a 23rd field -> test still calls with 22 args -> build fails
Updated (resubmitted) D290520, D290521

Flags: needinfo?(tboiko)
Assignee

Comment 55

1 month ago

Hi Martin, can we try submitting again. Sorry for the build bustages - I didn't catch the error on my side during common firefox build/run

Pushed by stransky@redhat.com: https://github.com/mozilla-firefox/firefox/commit/c1f5aae79695 https://hg.mozilla.org/integration/autoland/rev/009ece72f8eb MOZ/GTK - Prefs, packaging, configure, and GTK platform hooks for Vulkan video. r=stransky https://github.com/mozilla-firefox/firefox/commit/fc65b037d367 https://hg.mozilla.org/integration/autoland/rev/f6affd94efbc GFX - GL/EGL, layers, and compositor IPC for Vulkan video. r=stransky,lsalzman
Regressions: 2035749
Assignee

Comment 57

1 month ago

I duplicate a message I already posted in the above regression:

"Looks like Build A has vulkantest (line 54), Build B (baseline) doesn't. That's it — no binary content difference, just a new file.
This is comparing the patched build against the base commit which doesn't have vulkantest. The "failure" is simply that we added a new binary — it's expected and not a real reproducibility issue.

Is that correct?"

Regressions: 2036839

Thanks Tymur and Martin.

I had a chance to look into why this doesn't work on AMD. radeonsi (EGL) does not accept the DMABUF frames because the pitch that EGLCreateImage is called with is not correct.

The pitch comes from FFmpegDescToVA, which sets it to uncrop_width for both planes. AMD surfaces have alignment requirements -- it should pull the pitch from aDesc.layers[0].planes[i].pitch. This path seems to not be used by VAAPI before, so that's why it was never hit. However, I'm not sure if my suggestion may regress V4L2?

Assignee

Comment 60

1 month ago

Hi Benjamin,
Thanks for the investigation. We can make this fix Vulkan-specific only - uvPitch is now taken from planes[1].pitch only when the HW device type is AV_HWDEVICE_TYPE_VULKAN, leaving VA-API and V4L2 paths unchanged with the prior behavior. Please see the updated commit.

Yes, it works, thanks for the quick fix.

I'm not a Firefox maintainer nor am I familiar with V4L2, but I feel making the fix Vulkan only is a bit hacky. IMO the AVDRMFrameDescriptor coming from V4L2 should be outputting the correct pitch also. I tried confirming this by reading the ffmpeg code, but it seems like there's some ffmpeg patches required for the V4L2 path in Firefox to work, because I don't see how upstream ffmpeg can output a AVDRMFrameDescriptor for V4L2.

Regressions: 2037864
Pushed by amarc@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/bd3583d7c8db https://hg.mozilla.org/integration/autoland/rev/c0c4359d53e0 Revert "Bug 2021722 - DOM Media - FFmpeg Vulkan Video decode in RDD. r=stransky,alwu" for causing build bustages @ FFmpegVideoDecoder

Comment 64

1 month ago

Backed out for causing build bustages

Flags: needinfo?(tboiko)
Assignee

Comment 65

1 month ago

Fixed. Updated the patch.
dom/media: remove unused and non-portable includes from FFmpegVideoDecoder.cpp

Flags: needinfo?(tboiko)
Pushed by amarc@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/1e656c0b3946 https://hg.mozilla.org/integration/autoland/rev/42fd3a8ec164 Revert "Bug 2021722 - DOM Media - FFmpeg Vulkan Video decode in RDD. r=stransky,alwu" for causing build bustages @ FFmpegVulkanVideoDecoder

Comment 68

1 month ago

Backed out for causing build bustages

Flags: needinfo?(tboiko)
Assignee

Comment 69

1 month ago

Fixed again. Updated the patch.
The reason is MOZ_RELEASE_CONSTINIT is defined in mozilla/Attributes.h. FFmpegVulkanVideoDecoder.cpp never included that header, so in the unified build the macro was not defined
Sorry, haven't caught it on my side (there is no such an error on Ubuntu)

Flags: needinfo?(tboiko)
Pushed by sstanca@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/5eafa4d4761e https://hg.mozilla.org/integration/autoland/rev/fd4c75ad2802 Revert "Bug 2021722 - DOM Media - FFmpeg Vulkan Video decode in RDD. r=stransky,alwu" for causing build bustages in FFmpegVulkanVideoDecoder.cpp.

Reverted this because it was causing build bustages in FFmpegVulkanVideoDecoder.cpp.

  • Revert link
  • Push with failures
  • Failure Log
  • Failure line: ./../../../../../../../checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegVulkanVideoDecoder.cpp:X:10: fatal error: 'libdrm/drm_fourcc.h' file not found

Please also check this.

Flags: needinfo?(tboiko)
Assignee

Comment 73

1 month ago

OK, updated
Please, try again

Flags: needinfo?(tboiko)
Pushed by sstanca@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/ffa01844ca17 https://hg.mozilla.org/integration/autoland/rev/b4e341065155 Revert "Bug 2021722 - DOM Media - FFmpeg Vulkan Video decode in RDD. r=stransky,alwu" for causing build bustages and linting opt failures.
Assignee

Comment 76

1 month ago

Sorry for the bustages (there are no errors at my side locally)
I need to make it buildable inside the DOM patch first, then I'll try.

Reverted this because it was causing build bustages and linting opt failures.

Assignee

Comment 78

1 month ago

I have identified the root cause:
MOZ_RELEASE_CONSTINIT static StaticDataMutex<InstanceFunctionCache> (but it works with MOZ_RUNINIT)
I will submit a fix soon

Assignee

Comment 79

1 month ago

In the latest change constinit is used directly instead of MOZ_RELEASE_CONSTINIT because MOZ_RELEASE_CONSTINIT is only defined in mfbt/Attributes.h's non-plugin branch (#else of #if defined(MOZ_CLANG_PLUGIN)). When clang-tidy runs with --extra-arg=-DMOZ_CLANG_PLUGIN (as mach static-analysis does), it takes the plugin branch where the macro is undefined, causing an unknown type name error at namespace scope.
checked with ./mach static-analysis check --outgoing

Assignee

Comment 80

1 month ago

No it is buildable: Harbormaster completed remote builds in B966122: Diff 1271563.
https://phabricator.services.mozilla.com/D290522
We can retry

Assignee

Comment 81

1 month ago

It's now built: Harbormaster completed remote builds in B966122: Diff 1271563.
https://phabricator.services.mozilla.com/D290522
We can try to submit it again

Pushed by asilaghi@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/d263becf5471 https://hg.mozilla.org/integration/autoland/rev/14483e73c6bf Revert "Bug 2021722 - DOM Media - FFmpeg Vulkan Video decode in RDD. r=stransky,alwu" for causing build bustages at FFmpegVideoDecoder.cpp

Backed out for causing build bustages at FFmpegVideoDecoder.cpp
Backout Link
Push with failures
Failure Log
Failure line ./../../../../../../../checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp(30,10): fatal error: 'unistd.h' file not found

Flags: needinfo?(tboiko)
Assignee

Comment 85

1 month ago

Cause: FFmpegVideoDecoder.cpp had an unconditional #include <unistd.h>. That header is POSIX-only; Windows toolchains don’t ship it, so the Win32 /opt build failed while compiling the unified ffvpx TU.

The patch updated, please try again.
Sorry, this is something I didn't see on either the remote or local build.

Flags: needinfo?(tboiko)
Assignee

Comment 89

1 month ago

Cause: FFmpegVideoDecoder.h pulls in FFmpegVideoFramePool.h -> DMABufDevice.h -> DMABufFormats.h, which defines DRM_FORMAT_MOD_INVALID when libdrm has not been included yet. Later, #include <libdrm/drm_fourcc.h> defines the same macro again with a different spelling, which triggers -Werror,-Wmacro-redefined. We #undef that one macro immediately before including drm_fourcc.h so the full header is still processed (we need all other DRM_FORMAT_* symbols), then optionally re-define the fallback if an old libdrm omits it - matching DMABufSurface.cpp / FFmpegVideoFramePool.cpp.

Let's retry, please.

Flags: needinfo?(tboiko)
Assignee

Comment 91

1 month ago

Thanks, I will ll try ./mach fuzzy if the build fails (or with the -full option).
The above build seems to have passed the above test.

(In reply to Tymur Boiko from comment #91)

Thanks, I will ll try ./mach fuzzy if the build fails (or with the -full option).
The above build seems to have passed the above test.

Will do that. You'd need L1 access for it (https://firefox-source-docs.mozilla.org/tools/try/configuration.html).

Pushed by smolnar@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/1f1c87d20bb9 https://hg.mozilla.org/integration/autoland/rev/d7b4321c0d2e Revert "Bug 2021722 - DOM Media - FFmpeg Vulkan Video decode in RDD. r=stransky,alwu" for causing build bustages
Assignee

Comment 96

1 month ago

Fixed the above failure. Updated the patch. Please, try pushing it again

Flags: needinfo?(tboiko)
Assignee

Comment 97

1 month ago

Right, it seems I can't authorize to trigger the jobs myself remotely.

Assignee

Comment 99

1 month ago

The next two tests failed, but were they really affected by this change?

  1. i1821693 Intermittent SimpleTimerTest.RepeatingPrecise | Value of: delay.isSome() (bug 1821693)
  2. i1869612 Intermittent /intersection-observer/scroll-margin-no-intersect.html | single tracking bug (bug 1869612)
    Other than that, no errors.

So the last missing piece is the sandbox patch - D290523.

Jed, do you mind the check the sandbox patch? It's last one missing. Thanks.

Flags: needinfo?(jld)
Regressions: 2039309
See Also: → 2028402

FYI: This ("DOM Media - FFmpeg Vulkan Video decode in RDD. r=stransky,alwu") seems to be breaking the build on 32-bit systems, as a bunch of Vulkan handles are being set to nullptr, when they're not pointer-sized. Logically, they aren't (CPU) pointers, they're handles, so nullptr really isn't correct, and we get a bunch of errors like:

 8:56.08 E ./../../../../../../dom/media/platforms/ffmpeg/FFmpegVulkanVideoDecoder.cpp:120:21: error: assigning to 'VkImage' (aka 'unsigned long long') from incompatible type 'std::nullptr_t'
 8:56.09 E 120 | mNv12Image[i] = nullptr;
 8:56.09 E | ^~~~~~~
 8:56.09 E ./../../../../../../dom/media/platforms/ffmpeg/FFmpegVulkanVideoDecoder.cpp:121:19: error: assigning to 'VkDeviceMemory' (aka 'unsigned long long') from incompatible type 'std::nullptr_t'
 8:56.10 E 121 | mNv12Mem[i] = nullptr;
 8:56.10 E | ^~~~~~~

Setting these to 0 instead of nullptr seems to work (patch attached).

Assignee

Comment 106

1 month ago

Thank you, David. Will you submit the patch or should I?

Assignee

Comment 107

1 month ago

Will VK_NULL_HANDLE work for you?

Assignee

Comment 108

1 month ago

I have created a bug, will submit the patch from there
https://bugzilla.mozilla.org/show_bug.cgi?id=2039878

Assignee

Comment 109

1 month ago

David, please check if the above patch works for you.

Thanks — the linked patch (with VK_NULL_HANDLE) works fine here.

(In reply to Martin Stránský [:stransky] (ni? me) from comment #103)

Jed, do you mind the check the sandbox patch? It's last one missing. Thanks.

Sorry about the long delay here; I've had a number of high-priority tasks come in over the past few weeks. I should be able to take a look tomorrow or Wednesday.

Assignee

Comment 112

13 days ago

Hi Martin,
I see that the latest patch from the stack has been accepted, can we try landing it?
https://phabricator.services.mozilla.com/D290523

(In reply to Tymur Boiko from comment #112)

Hi Martin,
I see that the latest patch from the stack has been accepted, can we try landing it?
https://phabricator.services.mozilla.com/D290523

Sure, will do so.

Assignee

Comment 115

13 days ago

Martin, can we also add this to StaticPrefList.yaml (after media.hardware-video-decoding-vulkan.enabled)?

  • name: media.hardware-video-decoding-vulkan.direct-export.enabled
    type: bool
    value: false
    mirror: once
    In FFmpegVideoDecoder.cpp, use this pref instead of MOZ_VULKAN_ENABLE_DIRECT_DECODE_EXPORT. Only attempt the av_hwframe_map -> DRM PRIME path when both media.hardware-video-decoding-vulkan.enabled and media.hardware-video-decoding-vulkan.direct-export.enabled are true (same as today: the env var only mattered once Vulkan decode was already in use).
    This lets us set a zero-copy path via mozilla.cfg (default is off). It works with FFmpeg master today and will matter for the future FFmpeg releases.
Attachment #9550795 - Attachment is obsolete: true

(In reply to Tymur Boiko from comment #115)

Martin, can we also add this to StaticPrefList.yaml (after media.hardware-video-decoding-vulkan.enabled)?

  • name: media.hardware-video-decoding-vulkan.direct-export.enabled
    type: bool
    value: false
    mirror: once
    In FFmpegVideoDecoder.cpp, use this pref instead of MOZ_VULKAN_ENABLE_DIRECT_DECODE_EXPORT. Only attempt the av_hwframe_map -> DRM PRIME path when both media.hardware-video-decoding-vulkan.enabled and media.hardware-video-decoding-vulkan.direct-export.enabled are true (same as today: the env var only mattered once Vulkan decode was already in use).
    This lets us set a zero-copy path via mozilla.cfg (default is off). It works with FFmpeg master today and will matter for the future FFmpeg releases.

Please create a patch for it and will review it.
Thanks.

Flags: needinfo?(jld) → needinfo?(tboiko)
Pushed by stransky@redhat.com: https://github.com/mozilla-firefox/firefox/commit/adf919afb29f https://hg.mozilla.org/integration/autoland/rev/6b1114cb4a97 DOM Media - Replace MOZ_VULKAN_ENABLE_DIRECT_DECODE_EXPORT with pref. r=stransky,alwu

Comment 120

11 days ago
bugherder
Status: ASSIGNED → RESOLVED
Closed: 11 days ago
Resolution: --- → FIXED
Target Milestone: --- → 153 Branch

With the Vulkan support landed we may consider to enable it in Nightly for qualified devices (known working driver version & GFX hardware) to receive more testing from users and get on track to enable it by default.
Tymur, do you know hich drivers/HW is supposed to work seamlessly there?
Thanks.

No longer blocks: 2045970
Depends on: 2045970
Blocks: 2045970
No longer depends on: 2045970

(In reply to Martin Stránský [:stransky] (ni? me) from comment #121)

With the Vulkan support landed we may consider to enable it in Nightly for qualified devices (known working driver version & GFX hardware) to receive more testing from users and get on track to enable it by default.
Tymur, do you know hich drivers/HW is supposed to work seamlessly there?
Thanks.

Filed as Bug 2045970.

Filed Bug 2045971 to add Vulkan HW decode support to bundled ffvpx for open formats.

Regressions: 2046425
QA Whiteboard: [qa-triage-done-c154/b153]
You need to log in before you can comment on or make changes to this bug.