On 2022-07-05 09:45 pm, Anton Khirnov wrote:
Quoting Gyan Doshi (2022-07-04 10:20:22)
On 2022-07-04 11:51 am, Anton Khirnov wrote:
Quoting Gyan Doshi (2022-07-02 11:51:53)
On 2022-07-02 02:12 pm, Anton Khirnov wrote:
Quoting Gyan Doshi (2022-07-01 13:03:04)
On 2022-07-01 03:33 pm, Anton Khirnov wrote:
Quoting Gyan Doshi (2022-06-25 10:29:51)
This is a per-file input option that adjusts an input's timestamps
with reference to another input, so that emitted packet timestamps
account for the difference between the start times of the two inputs.
Typical use case is to sync two or more live inputs such as from capture
devices. Both the target and reference input source timestamps should be
based on the same clock source.
If both streams are using the same clock, then why is any extra
synchronization needed?
Because ffmpeg.c normalizes timestamps by default. We can keep
timestamps using -copyts, but these inputs are usually preprocessed
using single-input filters which won't have access to the reference
inputs,
No idea what you mean by "reference inputs" here.
The reference input is the one the target is being synced against. e.g.
in a karaoke session - the music track from a DAW would be ref and the
user's voice via mic is the target.
or the merge filters like e.g. amix don't sync by timestamp.
amix does seem to look at timestamps.
amix does not *sync* by timestamp. If one input starts at 4 and the
other at 7, the 2nd isn't aligned by timestamp.
So maybe it should?
My concern generally with this patchset is that it seems like you're
changing things where it's easier to do rather than where it's correct.
There are many multi=input filters which may be used. amix is just one
example.
The basic 'deficiency' here is that filters operate upon frames and only
look at single frames for the most part, even though frames are part of
streams. These streams may have companion streams (which may be part of
programs) which are part of a single input. These inputs may have
companion inputs. Anything in this tree may be relevant for a
particular operation as a reference, e.g. we have a bespoke filter
scale2ref so that we can look at another stream's frames. But we don't
have pad2ref, crop2ref ..etc. So, the absolutely correct thing to do
would be to supply a global context to processing modules like
filtergraphs , maybe an array of dicts, containing attributes of all
inputs like starting time stamps, resolution, string metadata..etc. That
would obviate need for these bespoke fields and even filters.
I don't see how the second paragraph relates to the first one. scale,
pad, or crop are not multi-input filters, so why are you comparing them
scale is a singe-input filter but scale2ref is a multi-input filter
which is needed solely because there is no means at present to convey
info about other streams to a single input filter.
Similarly, we would need a crop2ref, pad2ref..etc to achieve the same
attribute transfer. If we had a global context, these counterpart
filters wouldn't be necessary.
to amix? I don't think there are so many multi-input filters in lavfi,
and the issue should be solvable using the same code for all of them.
Because reference about other streams isn't helpful only at the point of
multi-filter use. One of the streams may want to be resampled to a
specific rate or sample format based on some user's logic instead of
letting amix choose one. That's where a global context would help.
Actually, this functionality sounds like it sort of existed earlier in
the form of map sync (i.e. -map 1:a,0:a:1). Although the assignment
syntax still remains (and doesn't warn/error out), it's a no-op now
since the application code was removed in 2012 by Michael, who said he
based it off an idea from one of your commits, presumably in Libav.
So why are you not restoring that functionality and adding a new option
instead?
I said 'sort of'. That adjustment was implemented in do_video/audio_out,
so it won't help in filtering, or streamcopying.
This current option adjusts just after demux, so it doesn't have those
limitations.
Regards,
Gyan
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
