On Tue, Dec 20, 2016 at 07:24:44PM +0100, Nicolas George wrote: > Le nonidi 29 frimaire, an CCXXV, Michael Niedermayer a écrit : > > though i wonder what mistake we did to end here, the original design > > was intended to be very simple ... > > That was the mistake: trying for a simple design. > > Ha ha! Only serious. Linear filter chains are simple, and could have > worked easily with a simple design. But non-linear filter graphs are > inherently more complex: the system needs to decide what filter to > activate first from several parallel branches.
it shouldnt really be complex, not in concept, maybe in (efficient) implementation For example as a concept one could imagine each filter runs as its own thread and waits on its inputs availability and output space. if it gets input and has output space its woken up and works and produces output and then wakes its surroundings up. no difference between a linear chain or a complex graph here and its simple as we use and need just a local view centered on a filter and its neighbors IIUC you already implemented the gory details of filters handing stuff around and that working and not getting stuck nor bloating up too bad. But as a concept ignoring the implementation details i would call it simple. iam not sure its usfull but Another view from a different direction would be to see the filter graph as a network of pipes with flowing water and interconnected pumps. This too is a simple concept and intuition would suggest that this would not easily end up with water being stuck nor too much accumulating. It does not 1:1 match real filters though which work on discrete frames, i wonder if this view is usefull It would allow awnsering global questions about a graph, like what inputs are useless when some output is welded shut or vice versa > > Graphs with several inputs and/or outputs are even more difficult, > because the application needs to decide which input needs a frame most > urgently. > > Both lavfi's original design (API and implementation) and its use in > ffmpeg (actually, avconv) for complex filter graphs neglect partially or > completely this issue. i think the original lavfi design didnt really had any issue with graphs with multiple inputs or outputs. A user app could decide were to in and out put. but FFmpeg didnt support this at the time IIRC so the people working on the original lavfi had nothing to implement. the problems came when this support was added much later [...] > > You complain that noone else is working on libavfilter > > how many people know what _NEEDS_ to be done to make the API good > > enough for a first public and stable API ? > > > > while i have a long todo and maybe no time for this, as it is i cant > > even work on making libavfilter public+stable. Its not a technical > > issue, its one of peoples requirements on what they require to be > > changed first. > > This is true, but there is another side to the human problem: explaining > the plans and requirements of the API takes time; if nobody will read it > carefully, that time is better spent actually coding (I am writing this > mail by snippets while FATE is running). > > Here is an outline, in approximate order (there are dependencies, but > not everything): > > - Add a callback AVFilter.activate() to replace filter_frame() on all > inputs and request_frame() on all outputs. Most non-trivial filters > are written that way in the first place. ok thats mostly cosmetic id say. I think we possibly even had something somewhat in that direction in the original lavfi discussions MANY years ago this makes more sense with non recursive stuff > > - Change buffersink to implement that callback and peek directly in the > FIFO. ok, "cosmetic" > > - Write a generic activate callback for 1-to-1 filters and use it. there were plans for simplifying generic filters 1 in 1 out frame filters apply something to every pixel filters some more cases i forgot while different now with activate() the idea is old and IIRC was supposed to be done long long ago. I think its a good idea if it does happen > > - Rewrite framesync (the utility system for filters with several video > inputs that need synchroized frames) to implement activate and use the > FIFO directly. cosmetic :) > > - Allow to give buffersrc a timestamp on EOF, make sure the timestamp is > forwarded by most filters and allow to retrieve it from buffersink. > > (If somebody has a suggestion of a good data structure for that...) AVFrame.duration This possibly is a big subject for discussion on its own, but maybe not i dont know. > > - Allow to set a callback on buffersinks and buffersrcs to be notified > when a frame arrives or is needed. It is much more convenient than > walking the buffers to check one by one. agree that walking is bad. cannot ATM argue on what is better as i dont feel that i have a clear enough view of this and the surroundings. > > - Allow to merge several filter graphs into one. This may be useful when > applications have several graphs to run simultaneously, so that they > do not need to decide which one to activate. Another option would be > to have the functions in the next step work on possibly several > graphs. This may be orthogonal but i think a filter graph should be a filter (maybe not at struct level litterally but it should be possible to have a AVFilter that is backed by a arbitrary user specified filter graph) > > - Keep a priority queue of ready filters and use it instead of walking > the graph. something better than O(n) yes, agree > > - Add a function to run the graph until "something" happens; "something" > meaning a stop instruction called by the callbacks. dont understand > > - Add utility functions to easily run the graph. The most useful IMHO: > run until a frame comes out from any sink (we get the frame and > something useful identifying the sink) or one is needed from a source > (we get something identifying the source) or EOF. if the API itself is simpler with utility functions then this sounds like a good idea thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB There will always be a question for which you do not know the correct answer.
signature.asc
Description: Digital signature
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
