Thanks. I learned something new. Yes, as you discovered I would only expect VLC hardware decoding to work in a "Ubuntu on Xorg" session with the VLC option "VA-API video decoder via X11". Because that's the only combination of session and VLC option that's even theoretically compatible at all. Although performance in this mode is still relatively poor compared to MPV or even Totem. So I'd recommend those first.
Using "VA-API video decoder via DRM" I would not expect to work under Wayland. And it doesn't. It sounds like someone needs to log an enhancement/bug with VLC upstream to add a new option "VA-API video decoder via Wayland". Because that's the missing option (where the possible options are https://github.com/01org/libva/blob/master/va/va_backend.h#L39 ). ** No longer affects: vlc (Ubuntu) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to gst-plugins-bad1.0 in Ubuntu. https://bugs.launchpad.net/bugs/1698287 Title: VA-API fails to initialize in a Gnome Shell Wayland session Status in Libva: New Status in Mutter: Confirmed Status in Totem: Confirmed Status in VLC media player: New Status in chromium-browser package in Ubuntu: Confirmed Status in gst-plugins-bad1.0 package in Ubuntu: Fix Released Status in gstreamer-vaapi package in Ubuntu: Won't Fix Status in libva package in Ubuntu: Triaged Status in mpv package in Ubuntu: Fix Released Status in mutter package in Ubuntu: Triaged Bug description: The Intel i965 VA-API driver works well in Xorg sessions (Unity7 and Gnome Shell). But it refuses to load when in a Wayland session: $ totem bbb_sunflower_1080p_60fps_normal.mp4 libva error: va_getDriverName() failed with unknown libva error,driver_name=(null) $ env LIBVA_DRIVER_NAME=i965 totem bbb_sunflower_1080p_60fps_normal.mp4 totem: intel_driver.c:112: intel_driver_init: Assertion `VA_CHECK_DRM_AUTH_TYPE(ctx, VA_DRM_AUTH_DRI1) || VA_CHECK_DRM_AUTH_TYPE(ctx, VA_DRM_AUTH_DRI2) || VA_CHECK_DRM_AUTH_TYPE(ctx, VA_DRM_AUTH_CUSTOM)' failed. Aborted (core dumped) $ gst-play-1.0 bbb_sunflower_1080p_60fps_normal.mp4 Press 'k' to see a list of keyboard shortcuts. Now playing /home/dan/Videos/bbb_sunflower_1080p_60fps_normal.mp4 Redistribute latency... Redistribute latency... ERROR Internal error: could not render surface for file:///home/dan/Videos/bbb_sunflower_1080p_60fps_normal.mp4 ERROR debug information: ../../../gst/vaapi/gstvaapisink.c(1482): gst_vaapisink_show_frame_unlocked (): /GstPlayBin:playbin/GstPlaySink:playsink/GstBin:vbin/GstVaapiSink:vaapisink0 WORKAROUNDS: * Use weston instead of gnome-shell; or * $ env -uDISPLAY totem ... # but creates corruption -> bug 1701463 * $ env GST_GL_WINDOW=wayland totem ... # but creates corruption -> bug 1701463 * $ env GST_GL_WINDOW=wayland gst-play-1.0 --videosink glimagesink ... * $ env -uDISPLAY mpv --hwdec --opengl-backend=wayland ... ProblemType: Bug DistroRelease: Ubuntu 17.10 Package: i965-va-driver 1.8.1-1 ProcVersionSignature: Ubuntu 4.10.0-22.24-generic 4.10.15 Uname: Linux 4.10.0-22-generic x86_64 ApportVersion: 2.20.5-0ubuntu4 Architecture: amd64 Date: Fri Jun 16 13:40:38 2017 InstallationDate: Installed on 2017-05-03 (44 days ago) InstallationMedia: Ubuntu 17.10 "Artful Aardvark" - Alpha amd64 (20170502) SourcePackage: intel-vaapi-driver UpgradeStatus: No upgrade log present (probably fresh install) To manage notifications about this bug go to: https://bugs.launchpad.net/libva/+bug/1698287/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp