On 25/01/2021 23:20, Nicolas George wrote:
Mark Thompson (12021-01-25):
Merging the libraries (in source form, orthogonal to merging the
binaries) only makes sense if we are going to continue using the
libavformat internals, and that is exactly the thing we are trying to
get rid of.
"We"?
It seems to me that many people here have an agenda about libavdevice
while they have no reason to.
A number of the people in the project have expressed this opinion, hence I believe it is something which
"we" want to do. ("we" may also not want to do it; "we" do not need to be
consistent.)
If we replace the libavdevice API with a proper one oriented towards
devices rather than being hacked on to the libavformat API then we
actively don't want that merger.
So, what do people actually want here? The situation as I see it is
(please correct me on anything you feel is in error here):
* The features which libavdevice offers are useful to a lot of people.
Thank you for acknowledging it.
* Many of those are using it via the ffmpeg utility, but not all.
* The libavdevice API is the libavformat API because it was originally
split out from libavformat, and it has the nice property that devices
and files end up being interchangable in some contexts.
* The libavdevice API, being the libavformat API for files, is not
particularly well-suited in other contexts, because devices may not
have the same properties as files.
* Some odd things like the completely-unused capabilities API and the
almost-never-used message API are hacked on top of that to try to
avoid some libavformat issues, but are not actually useful to anyone
(hence the lack of use).
* To implement devices as AVInputFormat/AVOutputFormat instances,
libavdevice currently needs access to the internals of libavformat.
* Many developers want to get rid of that dependency on libavformat
internals, because it creates a corresponding ugliness on the
libavformat side which has to leave those parts exposed in an
ABI-constrained way.
What might we do about this:
* Delete libavdevice, with a statement that its capabilities are
outside the scope of ffmpeg. Obviously this isn't sensible, the
ffmpeg utility would then need to depend on some external library to
offer the same features it does now.
* Merge libavdevice into the ffmpeg utility. Since it is the primary
user this would neatly sidestep the previous objection, but now
external users are in the same boat so again unacceptable.
One of the main reasons the fork failed while original ffmpeg kept going
was that while ffmpeg has a policy of being useful, the fork had an
unspoken rule about code purity that lead to removing many useful
features. IIRC, Anton was one of the most vocal proponent of this
harmful unspoken rule.
I find worrisome to see the same attitude start to appear here. We need
to be very careful.
Also, if the project actually starts removing features for purity, like
some people discussed here, I'll stop bothering about this dying
project.
* Merge libavdevice into libavformat. This works to eliminate the
dependency problem, but it only really makes sense if we accept that
libavdevice is finished and will never want new API which isn't
repurposed libavformat.
* Make a new libavdevice-specific API and switch to that. The
eliminates the dependency problem, but there is a nasty transition
period and it would break the interchangability of the two libraries.
It also would be the most work, but leave libavdevice in the best
state for doing more things with in future.
One of the important features of libavdevice, as you mentioned earlier,
is that devices can be a drop-in replacement for demuxers and muxers. An
application that is not designed for devices can be made to use them,
and that is immensely convenient. It does not work always, but it work
often enough to be, I repeat, immensely useful.
I'm afraid I disagree with this sentiment. Each library should be designed
around an API which makes sense for the abstraction it is trying to create.
Forcing one library into the ill-suited API of another to work makes it less
useful, not more. (OpenMAX is a less useful API than libavcodec/libavfilter,
despite covering more cases.)
If a solution does not have this feature, we can just skip it.
Nothing here excludes the possibility of making a libavdevice psuedo-format
which covers the device use-cases possible within the libavformat API (compare
the lavfi pseudo-demuxer in the ffmpeg utility).
* Nothing. People will continue to be annoyed about the problems above.
Duh. People often are annoyed at the wrong things, let them be annoyed.
I have had proof in the past that some of the annoyance on this subject
was actually a deep misunderstanding of how linking works.
There is another option:
* Stop imposing ourselves constraints that bring absolutely no benefit.
Having internals is useful for a library because we can change them
without bothering about compatibility: if we change the internals, we
can change all places that use them at the same time, and everything
keeps working.
It is a good hygiene that the fftools are written to use no internals,
because that proves the public API is complete enough to build powerful
tools.
But for the libraries, it makes no sense to have this constraint. What
benefit is there in it? Nobody has been able to answer this question,
because there are none. Libraries should be able to use each other
internals (typical example: sharing header decoding and format-specific
utility functions between lavc and lavf).
Even after such a merge, the libavdevice API is still a problem.
Opinions?
Personally, I think the right option here is to make a new
libavdevice-specific API and cut the connection to libavformat
entirely, and I would be prepared to do some work towards that goal.
Exactly what form it would take is unclear, but I would be happy to
talk about that further if people agree that it is the right approach.
One of the very important things you missed:
A lot of devices would also be very useful as libavfilter sources and
sinks. It allows automatic format negotiation, direct rendering and many
other benefits.
Direct rendering with devices is not possible without a proper libavdevice API.
Making an internal special case allowing it in libavfilter only does not seem
like the most useful step to take in this direction to me.
But it cannot work for now, because the API to run a filter graph only
works with buffer sinks, no other kinds of sinks.
I agree that this change makes sense for the use-cases you describe, but I'm
unsure why it has to be related (though I suspect I may be missing some
subtlety).
I'm also unclear why it would need to mess with libavdevice internals - a
pseudo-filter which acts as a source is already possible if you want to write
one.
Therefore, before considering enhancing the API of libavdevice, we need
a proper API tu run a filter graph.
Why is this a dependency? libavdevice is useful in a context with no
libavfilter at all.
And since we want inter-filters parallelism, a feature that has been
wanted for a long time, for that we need a proper event loop and
parallel task scheduler. Which brings us back to this mail, on which I
intend to follow-up soon:
https://ffmpeg.org/pipermail/ffmpeg-devel/2020-December/274172.html
I agree that careful movements towards an event loop are a good idea, but again
not sure why it has to be related.
Yes, libavdevice's API is far from perfect, but it is part of a much
wider problem: everything in FFmpeg that is not directly codecs and
formats, and secondarily filters, has been let to slowly rot, because
"we don't need this", "it's not our core mission", "it's unusual",
whatever.
Or lack of developer interest, which in the playback case which prompted this discussion
originally is I think driven by "other projects do it better, so why should we
include it here".
Continue like that, and FFmpeg will become irrelevant. Or change
completely, embrace originality, create the necessary infrastructure,
and FFmpeg can be again a leader in efficient audio-video programming.
- Mark
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
