Follow up: I appear to have solved this and have some further
information which may be useful.
TL;DR: Setting AVStream's avg_frame_rate, which corresponds to mkvinfo's
"Default duration" field, appears to have fixed this.
From my investigation, it appears VLC will use a few different sources
of information to determine the timecode. The first and foremost is the
aforementioned "default duration", corresponding to avg_frame_rate.
However if it's missing, it appears to use the MKV cluster size
(corresponding to the codec's gop_size). I noticed from mkvinfo that
each cluster started at the times VLC's playhead would appear to jump to
(0:00, 0:03, 0:07, etc.) and that changing the gop_size would shift
these around.
Following from this, it was somewhat inaccurate to say all of the
examples exhibit this issue. The "muxing" example, which sets the
gop_size to 12, creates an MKV that plays reasonably in VLC. With far
more frequent MKV clusters, VLC's timecode doesn't hang for seconds at a
time. However the downside of decreasing the gop_size is that naturally
it makes the file larger due to more intra frames and it presumably
means VLC's playhead is still only accurate to the nearest 12th frame.
When FFmpeg remuxes, it obviously doesn't change the gop_size, but it
does set the "default duration"/avg_frame_rate. Output from my program,
with avg_frame_rate set, works as expected without the file size
increases that result from changing the gop_size, so that appears to be
the solution.
As for the "remuxing" and "transcoding" examples:
* The output from "transcoding" will always exhibit this issue since
it doesn't set gop_size nor avg_frame_rate.
* The "remuxing" example depends on the input. If the MKV has a low
gop_size, it will obviously preserve that. However, it doesn't set
nor preserve "avg_frame_rate", so the output will exhibit this issue
with larger gop_size inputs (including non-MKV files such as MP4s).
* Disregard my comment about "encode_video", I forgot it didn't mux
anything.
It may be worth updating the relevant tests to fix this, but it depends
on whether this is considered a priority or not.
PS Andreas: You were correct about my subtitle packet duration being set
wrong, I have fixed this now.
Matt
On 8/7/2021 10:52 PM, MattKC wrote:
Andreas,
(A quick look showed that the transcoding example does not use
pkt_timebase for decoding, as does your code;
This is certainly the first time I've heard of using pkt_timebase.
Almost all examples and tutorials I've seen use AVStream::time_base.
What's the usage of this though? Documentation says "encoding unused"
and it seems to be null (0/1) when I tried using it just now.
furthermore, your code
sets the duration of subtitle packets wrong, as the end_display_time is
in a different timebase than the subtitle's pts.)
Is that incorrect? The AVSubtitle struct definition says
start_display_time and end_display_time are in ms not in time base
units. Also, the subtitles, as encoded by my program, appear to be
correct when played back.
I can open an issue on trac at some point, but the issue can be fairly
easily recreated just by using "remuxing <input-file> output.mkv" or
"encode-video output.mkv libx264" or "transcoding <input-file>
output.mkv". It would seem most if not all of the examples produce a
broken file when outputting to MKV.
I've attached the output of "mkvinfo -a" on both a broken MKV exported
from my program and a remuxed MKV from FFmpeg. They're largely
similar; the "Default duration" section in the remux and slightly
different order of packets are the only parts that stands out to me.
Let me know if you can get any meaningful information out of them.
Matt
On 8/7/2021 9:58 PM, Andreas Rheinhardt wrote:
MattKC:
==Summary==
I've been implementing ffmpeg/libav into an application, and have run
into timing-related issues when playing back Matroska/MKV videos created
by my application in VLC. I tried asking about this on the libav-user
mailing list and received no response so I thought I'd try here. While
it's true that there may be issues with VLC's implementation too, I've
noticed specifically that videos generated by FFmpeg do not cause these
issues, while my application and the examples in "doc/examples" (which
my code is modeled from) do, which makes me think this code is missing
something crucial.
==Symptoms==
During playback, VLC's playback time will appear to stay frozen at 0:00
while the video and audio plays normally. It will then jump only in
increments of a few seconds (0:03, then 0:07, then 0:11, etc.) This
behavior occurs with videos produced both by my application and by the
FFmpeg examples programs "transcoding.c" and "remuxing.c".
The fact that this occurs with "remuxing.c" is particularly interesting
since that would seem to imply it's an issue with the format/container
writing specifically. If "remuxing.c" is given a correct MKV file (i.e.
one that does not exhibit this issue) produced by FFmpeg as its input,
and outputs to a second MKV file, that second MKV will demonstrate this
issue even though the first one didn't, indicating that something is
breaking during the remuxing process despite no decoding/encoding
occurring.
The worst demonstration of this issue that I've run into is if my
application writes any audio packets before writing the first video
packet (with "av_interleaved_write_frame"), the first few seconds of
video (until that first 0:03) will be considered completely blank by
VLC. The fact that it will pick up after 0:03, the same time the timer
will make its first update on other MKV files, leads me to believe it's
the exact same root cause, just a different presentation of it.
This issue only occurs on MKV files. MP4/MOV appear to have no issues
whatsoever.
==Conclusion==
Considering FFmpeg produces correct MKV files, I don't believe this to
be a bug in libav (though it could be considered a bug in the examples).
I'm assuming the examples and my program are missing something or doing
something incorrectly.
My code can be seen here, however the examples will almost certainly be
easier to read and test:
https://github.com/olive-editor/olive/blob/master/app/codec/ffmpeg/ffmpegencoder.cpp
Thanks in advance for any help, I'm a little confused about this whole
issue.
If it happens with our examples, too (as you said), then you should open
an issue on trac (for the examples). Be sure to add samples that allow
to reproduce it.
It would probably be enlightening to use mkvinfo to look at the
generated files. The files are probably badly interleaved (your
description makes me suspect that the generated files have lots of audio
packets at the beginning before the first video packet).
(A quick look showed that the transcoding example does not use
pkt_timebase for decoding, as does your code; furthermore, your code
sets the duration of subtitle packets wrong, as the end_display_time is
in a different timebase than the subtitle's pts.)
- Andreas
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
