Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature request: Add hardware acceleration for FFmpeg #190

Open
Klench opened this issue Dec 3, 2024 · 10 comments · May be fixed by #192
Open

Feature request: Add hardware acceleration for FFmpeg #190

Klench opened this issue Dec 3, 2024 · 10 comments · May be fixed by #192
Labels
enhancement New feature or request

Comments

@Klench
Copy link

Klench commented Dec 3, 2024

I have just started using this software and it seems great. However, using FFmpeg for conversion of large video files can take a lot of time if using CPU only for transcoding. Using e.g. Intel Quick Sync for hardware acceleration would probably speed things up a lot.
@C4illin
Copy link
Owner

C4illin commented Dec 4, 2024

What is missing to make it possible?

@C10udburst
Copy link
Contributor

Isn't passing the /dev/dri to the container enough?

@Klench
Copy link
Author

Klench commented Dec 9, 2024 •
edited
Loading

I am not an expert but I assume some hardware acceleration related arguments need to be added to the ffmpeg command:

https://trac.ffmpeg.org/wiki/Hardware/QuickSync

(In addition to passing the device to the container)

C4illin added a commit that referenced this issue Dec 11, 2024
@C4illin
fix: add FFMPEG_ARGS env variable
f537c81 
issue #190
@C4illin C4illin linked a pull request Dec 11, 2024 that will close this issue
Open
@C4illin
Copy link
Owner

C4illin commented Dec 11, 2024

I added FFMPEG_ARGS as en env variable, set it to "-hwaccel qsv" and pass the gpu to the container. Hopefully it works, I dont have an intel cpu but I tested with vulkan.

@C4illin C4illin added the enhancement New feature or request label Dec 11, 2024
@broizter
Copy link

broizter commented Dec 11, 2024 •
edited
Loading

I added FFMPEG_ARGS as en env variable, set it to "-hwaccel qsv" and pass the gpu to the container. Hopefully it works, I dont have an intel cpu but I tested with vulkan.

When trying to convert something I'm getting this error:

convertx  | ConvertX v0.9.0
convertx  | 🦊 Elysia is running at http://localhost:3000
convertx  | Alpine Linux v3.20
convertx  | GraphicsMagick 1.3.43 2024-03-23 Q16 http://www.GraphicsMagick.org/
convertx  | resvg v0.44.0
convertx  | Bun v1.1.38
convertx  | XeTeX 3.141592653-2.6-0.999996 (TeX Live 2024/Alpine Linux)
convertx  | djxl v0.10.2 [AVX2,SSE4,SSE2]
convertx  | assimp vVersion 5.4 -shared -st (GIT commit 0)
convertx  | pandoc 3.1.13
convertx  | vips-8.15.2
convertx  | ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
convertx  | No write access to /.config/calibre using a temporary dir instead
convertx  | Inkscape 1.3.2 (091e20ef0f, 2023-11-25)
convertx  | jobId set to: 8
convertx  | stderr: ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
convertx  |   built with gcc 13.2.1 (Alpine 13.2.1_git20240309) 20240309
convertx  |   configuration: --prefix=/usr --disable-librtmp --disable-lzma --disable-static --disable-stripping --enable-avfilter --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libmp3lame --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librist --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-lto=auto --enable-lv2 --enable-openssl --enable-pic --enable-postproc --enable-pthreads --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --optflags=-O3 --enable-libjxl --enable-libsvtav1 --enable-libvpl
convertx  |   libavutil      58. 29.100 / 58. 29.100
convertx  |   libavcodec     60. 31.102 / 60. 31.102
convertx  |   libavformat    60. 16.100 / 60. 16.100
convertx  |   libavdevice    60.  3.100 / 60.  3.100
convertx  |   libavfilter     9. 12.100 /  9. 12.100
convertx  |   libswscale      7.  5.100 /  7.  5.100
convertx  |   libswresample   4. 12.100 /  4. 12.100
convertx  |   libpostproc    57.  3.100 / 57.  3.100
convertx  | [vist#0:1/h264 @ 0x15087a1f9680] WARNING: defaulting hwaccel_output_format to qsv for compatibility with old commandlines. This behaviour is DEPRECATED and will be removed in the future. Please explicitly set "-hwaccel_output_format qsv".
convertx  | Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './data/uploads/1/8/BigBuckBunny.mp4':
convertx  |   Metadata:
convertx  |     major_brand     : mp42
convertx  |     minor_version   : 0
convertx  |     compatible_brands: isomavc1mp42
convertx  |     creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |   Duration: 00:09:56.47, start: 0.000000, bitrate: 2119 kb/s
convertx  |   Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
convertx  |     Metadata:
convertx  |       creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |       handler_name    : (C) 2007 Google Inc. v08.13.2007.
convertx  |       vendor_id       : [0][0][0][0]
convertx  |   Stream #0:1[0x2](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 1991 kb/s, 24 fps, 24 tbr, 24k tbn (default)
convertx  |     Metadata:
convertx  |       creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |       handler_name    : (C) 2007 Google Inc. v08.13.2007.
convertx  |       vendor_id       : [0][0][0][0]
convertx  | [AVHWDeviceContext @ 0x15087be6fd80] Failed to initialise VAAPI connection: -1 (unknown libva error).
convertx  | Device creation failed: -5.
convertx  | No device available for decoder: device type qsv needed for codec h264_qsv.
convertx  | [vist#0:1/h264 @ 0x15087a1f9680] Hardware device setup failed for decoder: I/O error
convertx  | [vost#0:0/libx265 @ 0x15087a1fb800] Error initializing a simple filtergraph
convertx  | Error opening output file ./data/output/1/8/BigBuckBunny.h265.
convertx  | Error opening output files: I/O error
convertx  | 
convertx  | Failed to convert ./data/uploads/1/8/BigBuckBunny.mp4 from mp4 to h265 using ffmpeg. error: Error: Command failed: ffmpeg -hwaccel qsv -i "./data/uploads/1/8/BigBuckBunny.mp4"  "./data/output/1/8/BigBuckBunny.h265"
convertx  | ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
convertx  |   built with gcc 13.2.1 (Alpine 13.2.1_git20240309) 20240309
convertx  |   configuration: --prefix=/usr --disable-librtmp --disable-lzma --disable-static --disable-stripping --enable-avfilter --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libmp3lame --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librist --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-lto=auto --enable-lv2 --enable-openssl --enable-pic --enable-postproc --enable-pthreads --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --optflags=-O3 --enable-libjxl --enable-libsvtav1 --enable-libvpl
convertx  |   libavutil      58. 29.100 / 58. 29.100
convertx  |   libavcodec     60. 31.102 / 60. 31.102
convertx  |   libavformat    60. 16.100 / 60. 16.100
convertx  |   libavdevice    60.  3.100 / 60.  3.100
convertx  |   libavfilter     9. 12.100 /  9. 12.100
convertx  |   libswscale      7.  5.100 /  7.  5.100
convertx  |   libswresample   4. 12.100 /  4. 12.100
convertx  |   libpostproc    57.  3.100 / 57.  3.100
convertx  | [vist#0:1/h264 @ 0x15087a1f9680] WARNING: defaulting hwaccel_output_format to qsv for compatibility with old commandlines. This behaviour is DEPRECATED and will be removed in the future. Please explicitly set "-hwaccel_output_format qsv".
convertx  | Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './data/uploads/1/8/BigBuckBunny.mp4':
convertx  |   Metadata:
convertx  |     major_brand     : mp42
convertx  |     minor_version   : 0
convertx  |     compatible_brands: isomavc1mp42
convertx  |     creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |   Duration: 00:09:56.47, start: 0.000000, bitrate: 2119 kb/s
convertx  |   Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
convertx  |     Metadata:
convertx  |       creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |       handler_name    : (C) 2007 Google Inc. v08.13.2007.
convertx  |       vendor_id       : [0][0][0][0]
convertx  |   Stream #0:1[0x2](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 1991 kb/s, 24 fps, 24 tbr, 24k tbn (default)
convertx  |     Metadata:
convertx  |       creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |       handler_name    : (C) 2007 Google Inc. v08.13.2007.
convertx  |       vendor_id       : [0][0][0][0]
convertx  | [AVHWDeviceContext @ 0x15087be6fd80] Failed to initialise VAAPI connection: -1 (unknown libva error).
convertx  | Device creation failed: -5.
convertx  | No device available for decoder: device type qsv needed for codec h264_qsv.
convertx  | [vist#0:1/h264 @ 0x15087a1f9680] Hardware device setup failed for decoder: I/O error
convertx  | [vost#0:0/libx265 @ 0x15087a1fb800] Error initializing a simple filtergraph
convertx  | Error opening output file ./data/output/1/8/BigBuckBunny.h265.
convertx  | Error opening output files: I/O error
convertx  | 
convertx  | jobId set to: 9
convertx  | stderr: ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
convertx  |   built with gcc 13.2.1 (Alpine 13.2.1_git20240309) 20240309
convertx  |   configuration: --prefix=/usr --disable-librtmp --disable-lzma --disable-static --disable-stripping --enable-avfilter --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libmp3lame --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librist --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-lto=auto --enable-lv2 --enable-openssl --enable-pic --enable-postproc --enable-pthreads --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --optflags=-O3 --enable-libjxl --enable-libsvtav1 --enable-libvpl
convertx  |   libavutil      58. 29.100 / 58. 29.100
convertx  |   libavcodec     60. 31.102 / 60. 31.102
convertx  |   libavformat    60. 16.100 / 60. 16.100
convertx  |   libavdevice    60.  3.100 / 60.  3.100
convertx  |   libavfilter     9. 12.100 /  9. 12.100
convertx  |   libswscale      7.  5.100 /  7.  5.100
convertx  |   libswresample   4. 12.100 /  4. 12.100
convertx  |   libpostproc    57.  3.100 / 57.  3.100
convertx  | [vist#0:1/h264 @ 0x1491cbee8680] WARNING: defaulting hwaccel_output_format to qsv for compatibility with old commandlines. This behaviour is DEPRECATED and will be removed in the future. Please explicitly set "-hwaccel_output_format qsv".
convertx  | Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './data/uploads/1/9/BigBuckBunny.mp4':
convertx  |   Metadata:
convertx  |     major_brand     : mp42
convertx  |     minor_version   : 0
convertx  |     compatible_brands: isomavc1mp42
convertx  |     creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |   Duration: 00:09:56.47, start: 0.000000, bitrate: 2119 kb/s
convertx  |   Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
convertx  |     Metadata:
convertx  |       creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |       handler_name    : (C) 2007 Google Inc. v08.13.2007.
convertx  |       vendor_id       : [0][0][0][0]
convertx  |   Stream #0:1[0x2](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 1991 kb/s, 24 fps, 24 tbr, 24k tbn (default)
convertx  |     Metadata:
convertx  |       creation_time   : 2010-01-10T08:29:06.000000Z
convertx  |       handler_name    : (C) 2007 Google Inc. v08.13.2007.
convertx  |       vendor_id       : [0][0][0][0]
convertx  | [AVHWDeviceContext @ 0x1491cc22bd00] Failed to initialise VAAPI connection: -1 (unknown libva error).
convertx  | Device creation failed: -5.
convertx  | No device available for decoder: device type qsv needed for codec h264_qsv.
convertx  | [vist#0:1/h264 @ 0x1491cbee8680] Hardware device setup failed for decoder: I/O error
convertx  | [vost#0:0/mpeg1video @ 0x1491cbeea800] Error initializing a simple filtergraph
convertx  | Error opening output file ./data/output/1/9/BigBuckBunny.mpeg.
convertx  | Error opening output files: I/O error
convertx  |

Here is my docker compose:

  convertx:
    user: "701:701"
    image: "ghcr.io/c4illin/convertx:main"
    container_name: "convertx"
    restart: "unless-stopped"
    devices:
      - "/dev/dri:/dev/dri"
    environment:
      - "FFMPEG_ARGS=-hwaccel qsv"
      - "TZ=*****"
      - "JWT_SECRET=*******"
    volumes:
      - "./convertx:/app/data"
    networks:
      - external

Output of ls -l /dev/dri outside the container

crw-rw---- 1 root 701 226,   1 dec  9 11:40 card1
crw-rw-rw- 1 root 701 226, 128 dec  9 11:40 renderD128

And inside the container

crw-rw----    1 root     701       226,   1 Dec  9 11:40 card1
crw-rw-rw-    1 root     701       226, 128 Dec  9 11:40 renderD128

Hardware acceleration is working fine in other containers with the same setup.

@C4illin
Copy link
Owner

C4illin commented Dec 11, 2024

Hardware acceleration is working fine in other containers with the same setup.

Does ffmpeg work with hardware acceleration outside the container? Maybe some more args is needed

@broizter
Copy link

Hardware acceleration is working fine in other containers with the same setup.

Does ffmpeg work with hardware acceleration outside the container? Maybe some more args is needed

It should be enough I think. This works outside the container ffmpeg -hwaccel qsv -c:v h264_qsv -i BigBuckBunny.mp4 -c:v h264_qsv -b:v 5M test.mp4

@C4illin
Copy link
Owner

C4illin commented Dec 15, 2024

Ok, if the command works by itself but not in the container with the same input there is only two possibilities: you added the wrong device or the alpine version of ffmpeg doesn't inclue qsv

@C4illin
Copy link
Owner

C4illin commented Dec 15, 2024

Maybe try again and add -loglevel verbose -hwaccel_device /dev/dri/renderD128

@C4illin
Copy link
Owner

C4illin commented Dec 15, 2024

You could also test -hwaccel vaapi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

chore(main): release 0.10.0 C4illin/ConvertX
4 participants
@C10udburst @C4illin @broizter @Klench