Re: [FFmpeg-devel] [PATCH] avformat/dv: fix timestamps of audio packets in case of dropped corrupt audio frames

Sat, 31 Oct 2020 12:48:40 -0700 


On Sat, 31 Oct 2020, Dave Rice wrote:


Hi Marton,


On Oct 31, 2020, at 12:56 PM, Marton Balint <c...@passwd.hu> wrote:

Fixes out of sync timestamps in ticket #8762.



Although Michael’s recent patch does address the issue documented in 
8762, I haven’t found this patch to fix the issue. I tried with -c:a 
copy and with -c:a pcm_s16le with some sample files that exhibit this 
issue but each output was out of sync. I put an output at 
https://gist.github.com/dericed/659bd843bd38b6f24a60198b5e345795. That 
output notes that 3597 packages of video are read and 3586 packets of 
audio. In the resulting file, at the end of the timeline the audio is 9 
frames out of sync and my output video stream is 00:02:00.020 and output 
audio stream is 00:01:59.653.



Beyond copying or encoding the audio, are there other options I should 
use to test this?



Well, it depends on what you want. After this patch you should get a file 
which has audio packets synced to video, but the audio stream is sparse, 
not every video packet has a corresponding audio packet. (It looks like 
our MOV muxer does not support muxing of sparse audio therefore does not 
produce proper timestamps. But MKV does, please try that.)



You can also make ffmpeg generate the missing audio based on packet 
timestamps. Swresample has an async=1 option, so something like this 
should get you synced audio with continous audio packets:

ffmpeg -y -i 1670520000_12.dv -c:v copy \
-af aresample=async=1:min_hard_comp=0.01 -c:a pcm_s16le 1670520000_12.mov

Regards,
Marton





Signed-off-by: Marton Balint <c...@passwd.hu>
---
libavformat/dv.c       | 16 ++--------------
tests/ref/seek/lavf-dv | 18 +++++++++---------
2 files changed, 11 insertions(+), 23 deletions(-)

diff --git a/libavformat/dv.c b/libavformat/dv.c
index 3e0d12c0e3..26a78139f5 100644
--- a/libavformat/dv.c
+++ b/libavformat/dv.c
@@ -49,7 +49,6 @@ struct DVDemuxContext {
    uint8_t           audio_buf[4][8192];
    int               ach;
    int               frames;
-    uint64_t          abytes;
};

static inline uint16_t dv_audio_12to16(uint16_t sample)
@@ -258,7 +257,7 @@ static int dv_extract_audio_info(DVDemuxContext *c, const 
uint8_t *frame)
            c->ast[i] = avformat_new_stream(c->fctx, NULL);
            if (!c->ast[i])
                break;
-            avpriv_set_pts_info(c->ast[i], 64, 1, 30000);
+            avpriv_set_pts_info(c->ast[i], 64, c->sys->time_base.num, 
c->sys->time_base.den);
            c->ast[i]->codecpar->codec_type = AVMEDIA_TYPE_AUDIO;
            c->ast[i]->codecpar->codec_id   = AV_CODEC_ID_PCM_S16LE;

@@ -387,8 +386,7 @@ int avpriv_dv_produce_packet(DVDemuxContext *c, AVPacket 
*pkt,
    for (i = 0; i < c->ach; i++) {
        c->audio_pkt[i].pos  = pos;
        c->audio_pkt[i].size = size;
-        c->audio_pkt[i].pts  = c->abytes * 30000 * 8 /
-                               c->ast[i]->codecpar->bit_rate;
+        c->audio_pkt[i].pts  = (c->sys->height == 720) ? (c->frames & ~1) : 
c->frames;
        ppcm[i] = c->audio_buf[i];
    }
    if (c->ach)
@@ -401,10 +399,7 @@ int avpriv_dv_produce_packet(DVDemuxContext *c, AVPacket 
*pkt,
            c->audio_pkt[2].size = c->audio_pkt[3].size = 0;
        } else {
            c->audio_pkt[0].size = c->audio_pkt[1].size = 0;
-            c->abytes           += size;
        }
-    } else {
-        c->abytes += size;
    }

    /* Now it's time to return video packet */
@@ -444,13 +439,6 @@ static int64_t dv_frame_offset(AVFormatContext *s, 
DVDemuxContext *c,
void ff_dv_offset_reset(DVDemuxContext *c, int64_t frame_offset)
{
    c->frames = frame_offset;
-    if (c->ach) {
-        if (c->sys) {
-        c->abytes = av_rescale_q(c->frames, c->sys->time_base,
-                                 (AVRational) { 8, 
c->ast[0]->codecpar->bit_rate });
-        } else
-            av_log(c->fctx, AV_LOG_ERROR, "cannot adjust audio bytes\n");
-    }
    c->audio_pkt[0].size = c->audio_pkt[1].size = 0;
    c->audio_pkt[2].size = c->audio_pkt[3].size = 0;
}
diff --git a/tests/ref/seek/lavf-dv b/tests/ref/seek/lavf-dv
index 0000ff5abe..f63e4460be 100644
--- a/tests/ref/seek/lavf-dv
+++ b/tests/ref/seek/lavf-dv
@@ -7,9 +7,9 @@ ret: 0         st: 0 flags:0  ts: 0.800000
ret: 0         st: 0 flags:1 dts: 0.800000 pts: 0.800000 pos:2880000 size:144000
ret: 0         st: 0 flags:1  ts:-0.320000
ret: 0         st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos:      0 size:144000
-ret: 0         st: 1 flags:0  ts: 2.576667
+ret: 0         st: 1 flags:0  ts: 2.560000
ret: 0         st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000
-ret: 0         st: 1 flags:1  ts: 1.470833
+ret: 0         st: 1 flags:1  ts: 1.480000
ret: 0         st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000
ret: 0         st:-1 flags:0  ts: 0.365002
ret: 0         st: 0 flags:1 dts: 0.360000 pts: 0.360000 pos:1296000 size:144000
@@ -19,9 +19,9 @@ ret: 0         st: 0 flags:0  ts: 2.160000
ret: 0         st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000
ret: 0         st: 0 flags:1  ts: 1.040000
ret: 0         st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000
-ret: 0         st: 1 flags:0  ts:-0.058333
+ret: 0         st: 1 flags:0  ts:-0.040000
ret: 0         st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos:      0 size:144000
-ret: 0         st: 1 flags:1  ts: 2.835833
+ret: 0         st: 1 flags:1  ts: 2.840000
ret: 0         st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000
ret: 0         st:-1 flags:0  ts: 1.730004
ret: 0         st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000
@@ -31,10 +31,10 @@ ret: 0         st: 0 flags:0  ts:-0.480000
ret: 0         st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos:      0 size:144000
ret: 0         st: 0 flags:1  ts: 2.400000
ret: 0         st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000
-ret: 0         st: 1 flags:0  ts: 1.306667
-ret: 0         st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 
size:144000
-ret: 0         st: 1 flags:1  ts: 0.200833
+ret: 0         st: 1 flags:0  ts: 1.320000
ret: 0         st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000
+ret: 0         st: 1 flags:1  ts: 0.200000
+ret: 0         st: 0 flags:1 dts: 0.200000 pts: 0.200000 pos: 720000 
size:144000
ret: 0         st:-1 flags:0  ts:-0.904994
ret: 0         st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos:      0 size:144000
ret: 0         st:-1 flags:1  ts: 1.989173
@@ -43,9 +43,9 @@ ret: 0         st: 0 flags:0  ts: 0.880000
ret: 0         st: 0 flags:1 dts: 0.880000 pts: 0.880000 pos:3168000 size:144000
ret: 0         st: 0 flags:1  ts:-0.240000
ret: 0         st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos:      0 size:144000
-ret: 0         st: 1 flags:0  ts: 2.671667
+ret: 0         st: 1 flags:0  ts: 2.680000
ret: 0         st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000
-ret: 0         st: 1 flags:1  ts: 1.565833
+ret: 0         st: 1 flags:1  ts: 1.560000
ret: 0         st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000
ret: 0         st:-1 flags:0  ts: 0.460008
ret: 0         st: 0 flags:1 dts: 0.480000 pts: 0.480000 pos:1728000 size:144000

— 
2.26.2


Best Regards,
Dave Rice

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".





















					Reply via email to