PR #22779 opened by LTe URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/22779 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/22779.patch
Fix stream-copied audio being dropped when using output `-ss` with `-output_ts_offset` and mixed encoded/stream-copied streams from separate inputs. Fixes https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/22765 ## Problem When transcoding video from one input and stream-copying audio from a separate input (e.g., external MKA file) with output `-ss` and `-output_ts_offset`, HLS fMP4 segments contain no audio packets. Video is present and plays fine. The bug affects both `-copyts` and non-`-copyts` modes. ## Root cause The trim logic in `of_streamcopy()` compares packet timestamps against `ts_copy_start` and `of->start_time`, but does not account for `output_ts_offset` which is applied later in `libavformat/mux.c`. Packets that would be in range after the offset are rejected before it is applied. With `-copyts`, `of_streamcopy()` also unconditionally subtracts `start_time` from stream-copied timestamps while the encoder filter adds it via `ts_offset`, putting the two streams on different scales. The bug only manifests with HLS output because HLS writes packets directly via `ff_write_chained(..., 0)` without interleaving. Other muxers use `av_interleaved_write_frame()` which buffers packets and masks the issue. ## Patch series 1. **fftools/ffmpeg_mux: fix stream-copied audio with -copyts and output -ss** -- Gate the `start_time` subtraction on `!copy_ts` so stream-copied packets preserve their original timestamps. Set the encoder filter `ts_offset` to 0 with `-copyts` since the encoder output already has correct timestamps. 2. **fftools/ffmpeg_mux: fix stream-copied audio with -output_ts_offset** -- Reduce `ts_copy_start`, the dts trim threshold, and the `start_time` subtraction by `output_ts_offset` (clamped to 0) so the trim accounts for the shift that `libavformat` will apply later. 3. **tests/fate/hlsenc: add tests for stream-copied external audio with seek** -- Two FATE tests (`fate-streamcopy-extaudio-copyts`, `fate-streamcopy-extaudio-nocopyts`) using built-in encoders only. From 5ea6c453c9be1118b817402bc745dfd20530ea3e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Piotr=20Nie=C5=82acny?= <[email protected]> Date: Fri, 10 Apr 2026 10:07:42 +0200 Subject: [PATCH 1/3] fftools/ffmpeg_mux: fix stream-copied audio with -copyts and output -ss MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit With -copyts, of_streamcopy() unconditionally subtracts start_time from stream-copied packet timestamps, shifting them out of the valid range. Meanwhile the encoder filter applies start_time as ts_offset, so encoded video keeps its original timestamps while stream-copied audio is shifted to a different scale. Fix by gating the start_time subtraction on !copy_ts. With -copyts, stream-copied packets already carry the correct original timestamps from the demuxer, so no adjustment is needed -- only the fallback dts estimation for packets without a dts value is preserved. The encoder filter ts_offset is also set to 0 with -copyts, since the encoder output already has the correct timestamps via -copyts and does not need the start_time shift. Signed-off-by: Piotr Niełacny <[email protected]> --- fftools/ffmpeg_mux.c | 25 +++++++++++++++---------- fftools/ffmpeg_mux_init.c | 5 +++-- 2 files changed, 18 insertions(+), 12 deletions(-) diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c index 25f66dd185..59765432bd 100644 --- a/fftools/ffmpeg_mux.c +++ b/fftools/ffmpeg_mux.c @@ -463,7 +463,6 @@ static int of_streamcopy(OutputFile *of, OutputStream *ost, AVPacket *pkt) FrameData *fd = pkt->opaque_ref ? (FrameData*)pkt->opaque_ref->data : NULL; int64_t dts = fd ? fd->dts_est : AV_NOPTS_VALUE; int64_t start_time = (of->start_time == AV_NOPTS_VALUE) ? 0 : of->start_time; - int64_t ts_offset; if (of->recording_time != INT64_MAX && dts >= of->recording_time + start_time) @@ -484,19 +483,25 @@ static int of_streamcopy(OutputFile *of, OutputStream *ost, AVPacket *pkt) return AVERROR(EAGAIN); } - ts_offset = av_rescale_q(start_time, AV_TIME_BASE_Q, pkt->time_base); + if (!copy_ts) { + int64_t ts_offset = av_rescale_q(start_time, AV_TIME_BASE_Q, + pkt->time_base); - if (pkt->pts != AV_NOPTS_VALUE) - pkt->pts -= ts_offset; + if (pkt->pts != AV_NOPTS_VALUE) + pkt->pts -= ts_offset; - if (pkt->dts == AV_NOPTS_VALUE) { - pkt->dts = av_rescale_q(dts, AV_TIME_BASE_Q, pkt->time_base); - } else if (ost->st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) { - pkt->pts = pkt->dts - ts_offset; + if (pkt->dts == AV_NOPTS_VALUE) { + pkt->dts = av_rescale_q(dts, AV_TIME_BASE_Q, pkt->time_base); + } else if (ost->st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) { + pkt->pts = pkt->dts - ts_offset; + } + + pkt->dts -= ts_offset; + } else { + if (pkt->dts == AV_NOPTS_VALUE) + pkt->dts = av_rescale_q(dts, AV_TIME_BASE_Q, pkt->time_base); } - pkt->dts -= ts_offset; - ms->streamcopy_started = 1; return 0; diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index e0eecf78f2..0d146db368 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -933,8 +933,9 @@ ost_bind_filter(const Muxer *mux, MuxStream *ms, OutputFilter *ofilter, .output_tb = enc_tb, .trim_start_us = mux->of.start_time, .trim_duration_us = mux->of.recording_time, - .ts_offset = mux->of.start_time == AV_NOPTS_VALUE ? - 0 : mux->of.start_time, + .ts_offset = copy_ts ? 0 : + (mux->of.start_time == AV_NOPTS_VALUE ? + 0 : mux->of.start_time), .vs = vs, .nb_threads = -1, -- 2.52.0 From 1b291490c318218ed323fa6cf03425c57875d52a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Piotr=20Nie=C5=82acny?= <[email protected]> Date: Fri, 10 Apr 2026 10:08:28 +0200 Subject: [PATCH 2/3] fftools/ffmpeg_mux: fix stream-copied audio with -output_ts_offset MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Without -copyts, input -ss resets stream-copied timestamps to ~0. The trim checks in of_streamcopy() compare against ts_copy_start and of->start_time which are set to the output -ss value, discarding all packets. The output_ts_offset that would shift them into range is applied later in libavformat/mux.c, after the packets are already dropped. Additionally, of_streamcopy() subtracts start_time from stream-copied timestamps while the encoder filter adds it via ts_offset, putting audio and video on different scales when output_ts_offset is also set. Fix by reducing ts_copy_start, the dts trim threshold, and the start_time subtraction by output_ts_offset (clamped to 0), so the trim and timestamp adjustment account for the shift that libavformat will apply later. The dts trim adjustment applies regardless of -copyts since output_ts_offset affects both modes equally; the ts_copy_start and start_time adjustments are guarded by !copy_ts since -copyts preserves original timestamps and does not need them. Signed-off-by: Piotr Niełacny <[email protected]> --- fftools/ffmpeg_mux.c | 14 +++++++++++--- fftools/ffmpeg_mux_init.c | 3 +++ 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c index 59765432bd..73ae139955 100644 --- a/fftools/ffmpeg_mux.c +++ b/fftools/ffmpeg_mux.c @@ -459,6 +459,7 @@ finish: static int of_streamcopy(OutputFile *of, OutputStream *ost, AVPacket *pkt) { + Muxer *mux = mux_from_of(of); MuxStream *ms = ms_from_ost(ost); FrameData *fd = pkt->opaque_ref ? (FrameData*)pkt->opaque_ref->data : NULL; int64_t dts = fd ? fd->dts_est : AV_NOPTS_VALUE; @@ -479,12 +480,19 @@ static int of_streamcopy(OutputFile *of, OutputStream *ost, AVPacket *pkt) pkt->pts < av_rescale_q(ms->ts_copy_start, AV_TIME_BASE_Q, pkt->time_base))) return AVERROR(EAGAIN); - if (of->start_time != AV_NOPTS_VALUE && dts < of->start_time) - return AVERROR(EAGAIN); + if (of->start_time != AV_NOPTS_VALUE) { + int64_t dts_start = of->start_time; + if (!copy_ts && mux->fc->output_ts_offset) + dts_start = FFMAX(0, dts_start - mux->fc->output_ts_offset); + if (dts < dts_start) + return AVERROR(EAGAIN); + } } if (!copy_ts) { - int64_t ts_offset = av_rescale_q(start_time, AV_TIME_BASE_Q, + int64_t adj_start = FFMAX(0, start_time - + mux->fc->output_ts_offset); + int64_t ts_offset = av_rescale_q(adj_start, AV_TIME_BASE_Q, pkt->time_base); if (pkt->pts != AV_NOPTS_VALUE) diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index 0d146db368..e75a53c28d 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -1119,6 +1119,9 @@ static int streamcopy_init(const OptionsContext *o, const Muxer *mux, ms->ts_copy_start = FFMAX(ms->ts_copy_start, ifile->start_time + ifile->ts_offset); } + if (!copy_ts && mux->fc->output_ts_offset) + ms->ts_copy_start = FFMAX(0, + ms->ts_copy_start - mux->fc->output_ts_offset); } for (int i = 0; i < ist->st->codecpar->nb_coded_side_data; i++) { -- 2.52.0 From 9f993abd3f66f33aecefe4b1e5c14f244d056482 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Piotr=20Nie=C5=82acny?= <[email protected]> Date: Fri, 10 Apr 2026 10:09:25 +0200 Subject: [PATCH 3/3] tests/fate/hlsenc: add tests for stream-copied external audio with seek MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add FATE tests verifying that stream-copied audio from a separate input is present in HLS fMP4 segments when using output -ss with -output_ts_offset, both with and without -copyts. Uses built-in encoders (mpeg2video, mpeg4, mp2fixed) so the tests run on all FATE configurations without requiring external libraries. Both tests currently produce the same packet counts because the first segment size is determined by the HLS muxer's 3-second target, which is independent of the timestamp mode. The counts may diverge if the timestamp handling changes. Signed-off-by: Piotr Niełacny <[email protected]> --- tests/fate/hlsenc.mak | 56 +++++++++++++++++++++ tests/ref/fate/streamcopy-extaudio-copyts | 2 + tests/ref/fate/streamcopy-extaudio-nocopyts | 2 + 3 files changed, 60 insertions(+) create mode 100644 tests/ref/fate/streamcopy-extaudio-copyts create mode 100644 tests/ref/fate/streamcopy-extaudio-nocopyts diff --git a/tests/fate/hlsenc.mak b/tests/fate/hlsenc.mak index 66f23f4d6e..b945ba7042 100644 --- a/tests/fate/hlsenc.mak +++ b/tests/fate/hlsenc.mak @@ -211,5 +211,61 @@ fate-hls-start-number: CMP = diff FATE_HLSENC_LAVFI-yes := $(if $(call FRAMECRC), $(FATE_HLSENC_LAVFI-yes)) +tests/data/streamcopy_seek_video.mkv: TAG = GEN +tests/data/streamcopy_seek_video.mkv: ffmpeg$(PROGSSUF)$(EXESUF) | tests/data + $(M)$(TARGET_EXEC) $(TARGET_PATH)/$< -nostdin \ + -f lavfi -i "testsrc2=size=128x72:rate=24:d=30" \ + -c:v mpeg2video -g 1 -qscale 5 -an \ + -y $(TARGET_PATH)/$@ 2>/dev/null + +tests/data/streamcopy_seek_audio.mka: TAG = GEN +tests/data/streamcopy_seek_audio.mka: ffmpeg$(PROGSSUF)$(EXESUF) | tests/data + $(M)$(TARGET_EXEC) $(TARGET_PATH)/$< -nostdin \ + -f lavfi -i "sine=frequency=440:d=30" -c:a mp2fixed \ + -y $(TARGET_PATH)/$@ 2>/dev/null + +tests/data/streamcopy_seek_copyts_seg0.mp4: TAG = GEN +tests/data/streamcopy_seek_copyts_seg0.mp4: ffmpeg$(PROGSSUF)$(EXESUF) tests/data/streamcopy_seek_video.mkv tests/data/streamcopy_seek_audio.mka | tests/data + $(M)$(TARGET_EXEC) $(TARGET_PATH)/$< -nostdin \ + -ss 10 -noaccurate_seek -i $(TARGET_PATH)/tests/data/streamcopy_seek_video.mkv \ + -ss 10 -noaccurate_seek -i $(TARGET_PATH)/tests/data/streamcopy_seek_audio.mka \ + -ss 10 -output_ts_offset 10 \ + -map 0:v -map 1:a -c:v mpeg4 -qscale 5 -c:a copy \ + -copyts -avoid_negative_ts disabled \ + -f hls -hls_time 3 -hls_segment_type fmp4 \ + -hls_fmp4_init_filename streamcopy_seek_copyts_init.mp4 -start_number 0 \ + -hls_segment_filename $(TARGET_PATH)/tests/data/streamcopy_seek_copyts_%d.mp4 \ + -hls_list_size 0 \ + -y $(TARGET_PATH)/tests/data/streamcopy_seek_copyts.m3u8 2>/dev/null + $(M)cat $(TARGET_PATH)/tests/data/streamcopy_seek_copyts_init.mp4 \ + $(TARGET_PATH)/tests/data/streamcopy_seek_copyts_0.mp4 \ + > $(TARGET_PATH)/$@ + +tests/data/streamcopy_seek_nocopyts_seg0.mp4: TAG = GEN +tests/data/streamcopy_seek_nocopyts_seg0.mp4: ffmpeg$(PROGSSUF)$(EXESUF) tests/data/streamcopy_seek_video.mkv tests/data/streamcopy_seek_audio.mka | tests/data + $(M)$(TARGET_EXEC) $(TARGET_PATH)/$< -nostdin \ + -ss 10 -noaccurate_seek -i $(TARGET_PATH)/tests/data/streamcopy_seek_video.mkv \ + -ss 10 -noaccurate_seek -i $(TARGET_PATH)/tests/data/streamcopy_seek_audio.mka \ + -ss 10 -output_ts_offset 10 \ + -map 0:v -map 1:a -c:v mpeg4 -qscale 5 -c:a copy \ + -f hls -hls_time 3 -hls_segment_type fmp4 \ + -hls_fmp4_init_filename streamcopy_seek_nocopyts_init.mp4 -start_number 0 \ + -hls_segment_filename $(TARGET_PATH)/tests/data/streamcopy_seek_nocopyts_%d.mp4 \ + -hls_list_size 0 \ + -y $(TARGET_PATH)/tests/data/streamcopy_seek_nocopyts.m3u8 2>/dev/null + $(M)cat $(TARGET_PATH)/tests/data/streamcopy_seek_nocopyts_init.mp4 \ + $(TARGET_PATH)/tests/data/streamcopy_seek_nocopyts_0.mp4 \ + > $(TARGET_PATH)/$@ + +FATE_STREAMCOPY_SEEK_PREREQS = TESTSRC2_FILTER SINE_FILTER LAVFI_INDEV MPEG2VIDEO_ENCODER MPEG4_ENCODER MP2FIXED_ENCODER MP2_DECODER HLS_MUXER MOV_MUXER MATROSKA_MUXER FILE_PROTOCOL + +FATE_HLSENC_LAVFI-$(call ALLYES, $(FATE_STREAMCOPY_SEEK_PREREQS)) += fate-streamcopy-extaudio-copyts +fate-streamcopy-extaudio-copyts: tests/data/streamcopy_seek_copyts_seg0.mp4 +fate-streamcopy-extaudio-copyts: CMD = run ffprobe$(PROGSSUF)$(EXESUF) -bitexact -v error -count_packets -show_entries stream=codec_type,nb_read_packets -of csv=p=0 $(TARGET_PATH)/tests/data/streamcopy_seek_copyts_seg0.mp4 + +FATE_HLSENC_LAVFI-$(call ALLYES, $(FATE_STREAMCOPY_SEEK_PREREQS)) += fate-streamcopy-extaudio-nocopyts +fate-streamcopy-extaudio-nocopyts: tests/data/streamcopy_seek_nocopyts_seg0.mp4 +fate-streamcopy-extaudio-nocopyts: CMD = run ffprobe$(PROGSSUF)$(EXESUF) -bitexact -v error -count_packets -show_entries stream=codec_type,nb_read_packets -of csv=p=0 $(TARGET_PATH)/tests/data/streamcopy_seek_nocopyts_seg0.mp4 + FATE_FFMPEG += $(FATE_HLSENC_LAVFI-yes) fate-hlsenc: $(FATE_HLSENC-yes) $(FATE_HLSENC_PROBE-yes) $(FATE_HLSENC_LAVFI-yes) diff --git a/tests/ref/fate/streamcopy-extaudio-copyts b/tests/ref/fate/streamcopy-extaudio-copyts new file mode 100644 index 0000000000..0072f4b449 --- /dev/null +++ b/tests/ref/fate/streamcopy-extaudio-copyts @@ -0,0 +1,2 @@ +video,72 +audio,115 diff --git a/tests/ref/fate/streamcopy-extaudio-nocopyts b/tests/ref/fate/streamcopy-extaudio-nocopyts new file mode 100644 index 0000000000..0072f4b449 --- /dev/null +++ b/tests/ref/fate/streamcopy-extaudio-nocopyts @@ -0,0 +1,2 @@ +video,72 +audio,115 -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
