Hi Marton/ffmpeg devs/all,
Haven't seen any response to the proposed XComposite windows capture.
As per below the reasoning for adding this code to libav*/ffmpeg is that
seems to be better in quality than x11grab, although it uses more CPU in
its current implementation (uses OpenGL and PBO - optional - to fetch
XComposite windows).
Please let me know if it's still of interest and/or you have further
technical feedback.
I've posted some performance tests in the thread below.
Here links of two captures (one with x11grab, the other with xcompgrab)
and the sources:
https://send.firefox.com/download/df859689f4cc6429/#EN6w9upyIMHqO7n_1hMTaQ
https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c
Thanks in advance for your time and feedback,
Emanuele
Ps. Apologies if I've missed reposes on this list
On 10/05/2020 19:01, Emanuele Oriani wrote:
Hi Marton/all,
I've re-uploaded the firefox send link (expired) with some sample
capture at 60 FPS:
https://send.firefox.com/download/51b45decae720c08/#r5o4J2SgCJZndRdMOxxRBg
Please note I have integrated usage of OpenGL's Pixel Buffer Objects -
now the memory management of the captured framed can be managed by the
drivers and hopefully should be DMA and less memory hops.
This new code works on both Nvidia and AMD.
CPU usage is still higher than x11grab (similar levels as below).
Let me know your thoughts.
Thanks,
Emanuele
Ps. quick link to demuxer:
https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c
On 08/05/2020 11:43, Emanuele Oriani wrote:
Hi Marton,
Capturing at 60 FPS 1920x1080, the difference is even more noticeable:
x11grab is choppy, xcompgrab is smooth (this time I used H264
ultrafast to encode).
https://send.firefox.com/download/17001ef60837a5ec/#FaQTa-dP4MB28YfPNMXxuw
CPU performance:
real user sys
- xcompgrab: 10.32s, 18.2s, 0.38s
- x11grab: 10.20s, 11.6s, 0.22s
Definitely much more CPU usage, but the output quality can't be compared.
Please note no frames were lost/skipped, everything was processed on
time for both tests.
Thanks,
Emanuele
On 08/05/2020 11:05, Emanuele Oriani wrote:
Hi Marton,
TL;DR
xcompgrab uses more CPU but produces much better streams than x11grab.
I have the following CPU usage performance report (on my i7-8700k,
Nvidia 2080 Ti RTX, governor set to 'performance', on Ubuntu 18.04
using Gnome shell).
The capture has been a surface area of 1720x1376 for 10 seconds at
30 FPS, only getting the frames, not trying to encode them into a file:
real user sys
- xcompgrab: 10.13s, 2.06s, 0.085s
- x11grab: 10.05s, 0.20s, 0.035s
from a first outlook x11grab is much more CPU efficient, almost 90%
less CPU usage, but upon starting to record the video, it feels like
it's 'choppier' and loses frames.
Below videos have been captured at 1920x1080, the encoder is exactly
the same with same settings and the total CPU time is greater for
xcompgrab (as expected).
But if you look at both, the xcompgrab doesn't lose frames/is not
choppier, instead x11grab seems to be much worse.
I've uploaded both 10 secs segments at:
https://send.firefox.com/download/0cbf01f9126519c7/#DPKzIfjemAFm263ZSdMOBw
The code used to grab and encode is
https://github.com/Emanem/replayer again very simple, one thread gets
the packets/frames, the other picks the frames and encodes them.
Performance of this run has been:
real user sys
- xcompgrab: 10.42s, 12.5s, 0.22s
- x11grab: 10.21s, 8.90s, 0.31s
As expected x11grab yields better CPU usage but the quality of output
is somehow lacking.
Let me know your thoughts.
Thanks,
Emanuele
On 07/05/2020 19:52, Marton Balint wrote:
On Thu, 7 May 2020, Emanuele Oriani wrote:
Hi FFMPEG devel,
I have been writing a simple XComposite window capture demuxer,
heavily inspired from x11grab sources and OBS Window capture
logic/code.
Have you compared performance to x11grab for various resolutions and
frame rates? Do you have some numbers?
Thanks,
Marton
I would like to give back to the community and share the sources;
before I would formally submit for a review/patch, I would really
appreciate if someone could have a high level pass at the code
(it's relatively simple actually) and point out any major
issues/problems/...
The code is available as part of a simple test application on
github, at:
https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c
Being the author of the code I would be happy to re-license it to
be used in ffmpeg/libav* mainline once I get the OK from you.
In order to compile/execute this code, one needs the following dev
packages:
libav*-dev, libx11-dev, libxcomposite-dev, OpenGL headers/libs.
Libraries to be linked are:
-lavcodec -lavformat -lavdevice -lavutil -lswscale -lX11
-lXcomposite -lGL
I've tested it and it's working fine on both AMD and Nvidia hardware.
Let me know if this is of interest, otherwise no worries and thanks
again for the fantastic library/software.
Emanuele
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
