[FFmpeg-devel] [PATCH] libavcodec/qsvenc: Enable fixed QP configure in qsv CQP runtime
From: Yue Heng Enable dynamic QP configuration in runtime on qsv encoder. Through AVFrame->metadata, we can set key "qsv_config_qp" to change QP configuration when we encode video in CQP mode. Signed-off-by: Yue Heng Signed-off-by: Wenbin Chen --- doc/encoders.texi | 9 +++ libavcodec/qsvenc.c | 59 + 2 files changed, 68 insertions(+) diff --git a/doc/encoders.texi b/doc/encoders.texi index 9796a606fa..1eb75e199a 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -,6 +,15 @@ Forcing I frames as IDR frames. For encoders set this flag to ON to reduce power consumption and GPU usage. @end table +@subsection Runtime Options +Following options can be used durning qsv encoding. + +@table @option +@item @var{qsv_config_qp} +This option can be set in per-frame metadata. QP parameter can be dynamically +changed when encode in CQP mode. +@end table + @subsection H264 options These options are used by h264_qsv diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 2b3b06767d..52462950be 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -146,6 +146,14 @@ static const struct { { MFX_RATECONTROL_QVBR,"QVBR" }, }; +#define UPDATE_PARAM(a, b) \ +do {\ +if ((a) != (b)) { \ +a = b; \ +updated = 1;\ +} \ +} while (0) \ + static const char *print_ratecontrol(mfxU16 rc_mode) { int i; @@ -1520,6 +1528,53 @@ static void print_interlace_msg(AVCodecContext *avctx, QSVEncContext *q) } } +static int update_qp(AVCodecContext *avctx, QSVEncContext *q, + const AVFrame *frame) +{ +int updated = 0, qp = 0, new_qp; +AVDictionaryEntry *entry = av_dict_get(frame->metadata, "qsv_config_qp", + NULL, 0); +if (entry && q->param.mfx.RateControlMethod == MFX_RATECONTROL_CQP) { +qp = atoi(entry->value); +av_log(avctx, AV_LOG_DEBUG, "Configure qp: %d\n",qp); +if (qp < 0 || qp > 51) +av_log(avctx, AV_LOG_WARNING, "Invalid qp, clip to 0 ~ 51\n"); + +qp = av_clip(qp, 0, 51); +UPDATE_PARAM(q->param.mfx.QPP, qp); +new_qp = av_clip(qp * fabs(avctx->i_quant_factor) + +avctx->i_quant_offset, 0, 51); +UPDATE_PARAM(q->param.mfx.QPI, new_qp); +new_qp = av_clip(qp * fabs(avctx->b_quant_factor) + +avctx->b_quant_offset, 0, 51); +UPDATE_PARAM(q->param.mfx.QPB, new_qp); +av_log(avctx, AV_LOG_DEBUG, +"using fixed qp = %d/%d/%d for idr/p/b frames\n", +q->param.mfx.QPI, q->param.mfx.QPP, q->param.mfx.QPB); +} +return updated; +} + +static int update_parameters(AVCodecContext *avctx, QSVEncContext *q, + const AVFrame *frame) +{ +int needReset = 0, ret = 0; + +if (!frame) +return 0; + +needReset = update_qp(avctx, q, frame); +if (needReset) { +q->param.ExtParam= q->extparam_internal; +q->param.NumExtParam = q->nb_extparam_internal; +av_log(avctx, AV_LOG_DEBUG, "Parameter change, call msdk reset.\n"); +ret = MFXVideoENCODE_Reset(q->session, &q->param); +if (ret < 0) +return ff_qsv_print_error(avctx, ret, "Error during resetting"); +} +return 0; +} + static int encode_frame(AVCodecContext *avctx, QSVEncContext *q, const AVFrame *frame) { @@ -1630,6 +1685,10 @@ int ff_qsv_encode(AVCodecContext *avctx, QSVEncContext *q, { int ret; +ret = update_parameters(avctx, q, frame); +if (ret < 0) +return ret; + ret = encode_frame(avctx, q, frame); if (ret < 0) return ret; -- 2.32.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] doc: describe QOI image format
--- doc/general_contents.texi | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/general_contents.texi b/doc/general_contents.texi index 93a90a5e52..987a2f82fb 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -785,6 +785,8 @@ following image formats are supported: @tab Photoshop @item PTX @tab @tab X @tab V.Flash PTX format +@item QOI @tab X @tab X +@tab Quite OK Image format @item SGI @tab X @tab X @tab SGI RGB image format @item Sun Rasterfile @tab X @tab X -- 2.35.1 -- Peter (A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B) signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] doc/APIchanges: add missing marker for release 5.0
Will push later today. On 2022-06-14 03:11 pm, Gyan Doshi wrote: --- doc/APIchanges | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/APIchanges b/doc/APIchanges index 5857e67ae6..20b944933a 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -96,6 +96,8 @@ API changes, most recent first: 2022-01-26 - af94ab7c7c0 - lavu 57.19.100 - tx.h Add AV_TX_FLOAT_RDFT, AV_TX_DOUBLE_RDFT and AV_TX_INT32_RDFT. + 8< - FFmpeg 5.0 was cut here 8< - + 2022-01-04 - 78dc21b123e - lavu 57.16.100 - frame.h Add AV_FRAME_DATA_DOVI_METADATA. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 06/13] lavu/mem: Add ff_fast_recalloc()
tis 2022-06-14 klockan 22:26 +0200 skrev Michael Niedermayer: > On Tue, Jun 14, 2022 at 04:42:06PM +0200, Tomas Härdin wrote: > > Left this as an ff_ funtion for now since it's only used by the j2k > > code > > > > /Tomas > > > mem.c | 24 > > mem.h | 55 > > +++ > > 2 files changed, 79 insertions(+) > > 21be65bd06e3260f9f36598d5d574ee32e7131a6 0006-lavu-mem-Add- > > ff_fast_recalloc.patch > > From 5d36d431ffe4c8ba0f698d0c288ebc16b83f0bbc Mon Sep 17 00:00:00 > > 2001 > > From: =?UTF-8?q?Tomas=20H=C3=A4rdin?= > > Date: Tue, 14 Jun 2022 13:35:18 +0200 > > Subject: [PATCH 06/13] lavu/mem: Add ff_fast_recalloc() > > You cannot call a ff_* function thats in libavutil from outside > libavutil > this will fail with shared libs as the ff* stuff is not exported Ah, I suspected as much. Would there be much opposition to a public function like this in lavu? I could just keep it local to the j2k code /Tomas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 07/13] lavc/jpeg2000*: Use ff_fast_recalloc() to eliminate lots of allocations
tis 2022-06-14 klockan 17:23 +0200 skrev Andreas Rheinhardt: > Tomas Härdin: > > > > > > @@ -2166,12 +2163,13 @@ static int > > jpeg2000_mct_write_frame(AVCodecContext *avctx, void *td, > > return 0; > > } > > > > -static void jpeg2000_dec_cleanup(Jpeg2000DecoderContext *s) > > +static void jpeg2000_dec_cleanup(Jpeg2000DecoderContext *s, int > > close) > > { > > int tileno, compno; > > - for (tileno = 0; tileno < s->numXtiles * s->numYtiles; > > tileno++) { > > + if (close) { > > + for (tileno = 0; tileno < s->tile_size/sizeof(*s->tile); > > tileno++) { > > if (s->tile[tileno].comp) { > > - for (compno = 0; compno < s->ncomponents; compno++) { > > + for (compno = 0; compno < s- > > >tile[tileno].comp_size/sizeof(*s->tile[tileno].comp); compno++) { > > Jpeg2000Component *comp = s- > > >tile[tileno].comp + compno; > > Jpeg2000CodingStyle *codsty = s- > > >tile[tileno].codsty + compno; > > > > @@ -2182,10 +2180,11 @@ static void > > jpeg2000_dec_cleanup(Jpeg2000DecoderContext *s) > > s->tile[tileno].packed_headers_size = 0; > > } > > } > > + av_freep(&s->tile); > > + } > > av_freep(&s->packed_headers); > > s->packed_headers_size = 0; > > memset(&s->packed_headers_stream, 0, sizeof(s- > > >packed_headers_stream)); > > - av_freep(&s->tile); > > memset(s->codsty, 0, sizeof(s->codsty)); > > memset(s->qntsty, 0, sizeof(s->qntsty)); > > memset(s->properties, 0, sizeof(s->properties)); > > @@ -2689,7 +2688,7 @@ static int > > jpeg2000_decode_frame(AVCodecContext *avctx, AVFrame *picture, > > > > avctx->execute2(avctx, jpeg2000_mct_write_frame, picture, > > NULL, s->numXtiles * s->numYtiles); > > > > - jpeg2000_dec_cleanup(s); > > + jpeg2000_dec_cleanup(s, 0); > > > > *got_frame = 1; > > > > @@ -2702,7 +2701,7 @@ static int > > jpeg2000_decode_frame(AVCodecContext *avctx, AVFrame *picture, > > return bytestream2_tell(&s->g); > > > > end: > > - jpeg2000_dec_cleanup(s); > > + jpeg2000_dec_cleanup(s, 0); > > return ret; > > } > > > > @@ -2712,6 +2711,7 @@ static av_cold int > > jpeg2000_decode_close(AVCodecContext *avctx) > > > > av_freep(&s->idwt); > > av_freep(&s->cb); > > + jpeg2000_dec_cleanup(s, 1); > > > > return 0; > > } > > Why don't you just move the part of jpeg2000_dec_cleanup() that you > intend to be only executed in jpeg2000_decode_close() to > jpeg2000_decode_close()? I had in mind to do just that but forgot. Will do! /Tomas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 11/13] lavc/jpeg2000: Minimize calls to av_codec_is_encoder()
tis 2022-06-14 klockan 17:04 +0200 skrev Andreas Rheinhardt: > Tomas Härdin: > > > > > > Why call it at all? Why not just add a new parameter to > ff_jpeg2000_init_component that is always set to 1 when called from > the > encoder and 0 when called from the decoder? Oh yeah that's even simpler > (And is this really a bottleneck?) Callgrind certainly thinks so. It's called hundreds of thousands of times per frame. Remember that this is in the serial part of the code so any savings there get amplified -threads fold fps-wise. init_tile() accounts for a mere 0.8 seconds out of 59.8 partly thanks to this. Here's a rough breakdown for the curious with -threads 64: 0.8 everything up to and including init_tiles() 23.0 jpeg2000_read_bitstream_packets() 0.1 jpeg2000_setup_cbs() 24.1 jpeg2000_decode_cb() 9.8 jpeg2000_idwt() 2.0 jpeg2000_mct_write_frame() 0.0 jpeg2000_dec_cleanup() jpeg2000_read_bitstream_packets() is obviously the main thing to focus on for anyone wanting to bump the speed up even more. But it's nasty. Maybe it could be tile-threaded, but it takes some doing.. /Tomas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/4] avformat/matroskaenc: Don't waste bytes to Write Tag length fields
This is possible by using a dynamic buffer to write them; said dynamic buffer is (re)used and reset as appropriate. Signed-off-by: Andreas Rheinhardt --- libavformat/matroskaenc.c | 113 ++ tests/ref/fate/aac-autobsf-adtstoasc | 4 +- tests/ref/fate/matroska-avoid-negative-ts | 4 +- tests/ref/fate/matroska-dovi-write-config7| 4 +- tests/ref/fate/matroska-dovi-write-config8| 4 +- tests/ref/fate/matroska-dvbsub-remux | 4 +- tests/ref/fate/matroska-flac-extradata-update | 4 +- tests/ref/fate/matroska-h264-remux| 4 +- .../fate/matroska-mastering-display-metadata | 4 +- tests/ref/fate/matroska-move-cues-to-front| 4 +- tests/ref/fate/matroska-mpegts-remux | 4 +- tests/ref/fate/matroska-ms-mode | 4 +- tests/ref/fate/matroska-pgs-remux | 4 +- tests/ref/fate/matroska-pgs-remux-durations | 4 +- tests/ref/fate/matroska-qt-mode | 4 +- tests/ref/fate/matroska-spherical-mono-remux | 4 +- tests/ref/fate/matroska-vp8-alpha-remux | 4 +- tests/ref/fate/matroska-zero-length-block | 4 +- tests/ref/fate/rgb24-mkv | 4 +- tests/ref/fate/webm-dash-chapters | 4 +- tests/ref/fate/webm-webvtt-remux | 4 +- tests/ref/lavf-fate/av1.mkv | 4 +- tests/ref/lavf/mka| 4 +- tests/ref/lavf/mkv| 4 +- tests/ref/lavf/mkv_attachment | 4 +- tests/ref/seek/lavf-mkv | 44 +++ 26 files changed, 131 insertions(+), 122 deletions(-) diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index 2211d99ae8..404fbdf579 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -216,6 +216,13 @@ typedef struct MatroskaMuxContext { BlockContextcur_block; +/* Used as temporary buffer to use the minimal amount of bytes + * to write the length field of EBML Masters. + * Every user has to reset the buffer after using it and + * different uses may not overlap. It is currently used in + * mkv_write_tag(). */ +AVIOContext*tmp_bc; + AVPacket *cur_audio_pkt; unsignednb_attachments; @@ -247,6 +254,9 @@ typedef struct MatroskaMuxContext { /** 4 * (1-byte EBML ID, 1-byte EBML size, 8-byte uint max) */ #define MAX_CUETRACKPOS_SIZE 40 +/** 2 + 1 Simpletag header, 2 + 1 + 8 Name "DURATION", 23B for TagString */ +#define DURATION_SIMPLETAG_SIZE (2 + 1 + (2 + 1 + 8) + 23) + /** Seek preroll value for opus */ #define OPUS_SEEK_PREROLL 8000 @@ -814,6 +824,7 @@ static void mkv_deinit(AVFormatContext *s) ffio_free_dyn_buf(&mkv->info.bc); ffio_free_dyn_buf(&mkv->track.bc); ffio_free_dyn_buf(&mkv->tags.bc); +ffio_free_dyn_buf(&mkv->tmp_bc); av_freep(&mkv->cur_block.h2645_nalu_list.nalus); av_freep(&mkv->cues.entries); @@ -1911,24 +1922,14 @@ static int mkv_write_simpletag(AVIOContext *pb, const AVDictionaryEntry *t) return ret; } -static int mkv_write_tag_targets(MatroskaMuxContext *mkv, AVIOContext **pb, - ebml_master *tag, uint32_t elementid, uint64_t uid) +static void mkv_write_tag_targets(MatroskaMuxContext *mkv, AVIOContext *pb, + uint32_t elementid, uint64_t uid) { -ebml_master targets; -int ret; - -if (!*pb) { -ret = start_ebml_master_crc32(pb, mkv); -if (ret < 0) -return ret; -} - -*tag= start_ebml_master(*pb, MATROSKA_ID_TAG,0); -targets = start_ebml_master(*pb, MATROSKA_ID_TAGTARGETS, 4 + 1 + 8); +ebml_master targets = start_ebml_master(pb, MATROSKA_ID_TAGTARGETS, +4 + 1 + 8); if (elementid) -put_ebml_uid(*pb, elementid, uid); -end_ebml_master(*pb, targets); -return 0; +put_ebml_uid(pb, elementid, uid); +end_ebml_master(pb, targets); } static int mkv_check_tag_name(const char *name, uint32_t elementid) @@ -1946,29 +1947,41 @@ static int mkv_check_tag_name(const char *name, uint32_t elementid) } static int mkv_write_tag(MatroskaMuxContext *mkv, const AVDictionary *m, - AVIOContext **pb, ebml_master *tag, + AVIOContext **pb, unsigned reserved_size, uint32_t elementid, uint64_t uid) { const AVDictionaryEntry *t = NULL; -ebml_master tag2; -int ret; +AVIOContext *const tmp_bc = mkv->tmp_bc; +uint8_t *buf; +int ret, size; -ret = mkv_write_tag_targets(mkv, pb, tag ? tag : &tag2, elementid, uid); -if (ret < 0) -return ret; +mkv_write_tag_targets(mkv, tmp_bc, elementid, uid); while ((t = av_dict_get(m, "", t, AV_DICT_IGNORE_SUFFIX))) { if (mkv_check_tag_name(t->key, elementid)) { -re
[FFmpeg-devel] [PATCH 3/4] avformat/matroskaenc: Don't check twice whether to write tags
Because not all metadata is written as tags, the Matroska muxer filters out the tags that are not written as tags. Therefore the code first checks whether a Tag master element needs to be opened for a given stream/chapter/attachment/global metadata. If the answer turns out to be yes, it is checked again whether a given AVDictionaryEntry is written as a tag. This commit changes this: The Tag element is opened unconditionally and in case it turns out that it was unneeded, it is discarded again. This is possible because the Tag element is written into its own dynamic buffer. Signed-off-by: Andreas Rheinhardt --- libavformat/matroskaenc.c | 40 +++ 1 file changed, 11 insertions(+), 29 deletions(-) diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index 404fbdf579..f1385c6b21 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -1953,7 +1953,7 @@ static int mkv_write_tag(MatroskaMuxContext *mkv, const AVDictionary *m, const AVDictionaryEntry *t = NULL; AVIOContext *const tmp_bc = mkv->tmp_bc; uint8_t *buf; -int ret, size; +int ret = 0, size, tag_written = 0; mkv_write_tag_targets(mkv, tmp_bc, elementid, uid); @@ -1962,10 +1962,13 @@ static int mkv_write_tag(MatroskaMuxContext *mkv, const AVDictionary *m, ret = mkv_write_simpletag(tmp_bc, t); if (ret < 0) goto end; +tag_written = 1; } } if (reserved_size) put_ebml_void(tmp_bc, reserved_size); +else if (!tag_written) +goto end; size = avio_get_dyn_buf(tmp_bc, &buf); if (tmp_bc->error) { @@ -1984,17 +1987,6 @@ end: return ret; } -static int mkv_check_tag(const AVDictionary *m, uint32_t elementid) -{ -const AVDictionaryEntry *t = NULL; - -while ((t = av_dict_get(m, "", t, AV_DICT_IGNORE_SUFFIX))) -if (mkv_check_tag_name(t->key, elementid)) -return 1; - -return 0; -} - static int mkv_write_tags(AVFormatContext *s) { MatroskaMuxContext *mkv = s->priv_data; @@ -2004,11 +1996,9 @@ static int mkv_write_tags(AVFormatContext *s) ff_metadata_conv_ctx(s, ff_mkv_metadata_conv, NULL); -if (mkv_check_tag(s->metadata, 0)) { -ret = mkv_write_tag(mkv, s->metadata, &mkv->tags.bc, 0, 0, 0); -if (ret < 0) -return ret; -} +ret = mkv_write_tag(mkv, s->metadata, &mkv->tags.bc, 0, 0, 0); +if (ret < 0) +return ret; for (i = 0; i < s->nb_streams; i++) { const AVStream *st = s->streams[i]; @@ -2017,9 +2007,6 @@ static int mkv_write_tags(AVFormatContext *s) if (st->codecpar->codec_type == AVMEDIA_TYPE_ATTACHMENT) continue; -if (!seekable && !mkv_check_tag(st->metadata, MATROSKA_ID_TAGTARGETS_TRACKUID)) -continue; - ret = mkv_write_tag(mkv, st->metadata, &mkv->tags.bc, seekable ? DURATION_SIMPLETAG_SIZE : 0, MATROSKA_ID_TAGTARGETS_TRACKUID, track->uid); @@ -2037,9 +2024,6 @@ static int mkv_write_tags(AVFormatContext *s) if (st->codecpar->codec_type != AVMEDIA_TYPE_ATTACHMENT) continue; -if (!mkv_check_tag(st->metadata, MATROSKA_ID_TAGTARGETS_ATTACHUID)) -continue; - ret = mkv_write_tag(mkv, st->metadata, &mkv->tags.bc, 0, MATROSKA_ID_TAGTARGETS_ATTACHUID, track->uid); if (ret < 0) @@ -2124,12 +2108,10 @@ static int mkv_write_chapters(AVFormatContext *s) if (tags) { ff_metadata_conv(&c->metadata, ff_mkv_metadata_conv, NULL); -if (mkv_check_tag(c->metadata, MATROSKA_ID_TAGTARGETS_CHAPTERUID)) { -ret = mkv_write_tag(mkv, c->metadata, tags, 0, -MATROSKA_ID_TAGTARGETS_CHAPTERUID, uid); -if (ret < 0) -goto fail; -} +ret = mkv_write_tag(mkv, c->metadata, tags, 0, +MATROSKA_ID_TAGTARGETS_CHAPTERUID, uid); +if (ret < 0) +goto fail; } } end_ebml_master(dyn_cp, editionentry); -- 2.34.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 4/4] avformat/matroskaenc: Reuse dynamic buffer
Avoids some allocations. Signed-off-by: Andreas Rheinhardt --- libavformat/matroskaenc.c | 30 ++ 1 file changed, 10 insertions(+), 20 deletions(-) diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index f1385c6b21..60647869ca 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -220,7 +220,8 @@ typedef struct MatroskaMuxContext { * to write the length field of EBML Masters. * Every user has to reset the buffer after using it and * different uses may not overlap. It is currently used in - * mkv_write_tag(). */ + * mkv_write_tag(), in mkv_assemble_cues() as well as in + * mkv_check_new_extra_data(). */ AVIOContext*tmp_bc; AVPacket *cur_audio_pkt; @@ -929,17 +930,10 @@ static int mkv_add_cuepoint(MatroskaMuxContext *mkv, int stream, int64_t ts, return 0; } -static int mkv_assemble_cues(AVStream **streams, AVIOContext *dyn_cp, +static int mkv_assemble_cues(AVStream **streams, AVIOContext *dyn_cp, AVIOContext *cuepoint, const mkv_cues *cues, mkv_track *tracks, int num_tracks, uint64_t offset) { -AVIOContext *cuepoint; -int ret; - -ret = avio_open_dyn_buf(&cuepoint); -if (ret < 0) -return ret; - for (mkv_cuepoint *entry = cues->entries, *end = entry + cues->num_entries; entry < end;) { uint64_t pts = entry->pts; @@ -969,14 +963,13 @@ static int mkv_assemble_cues(AVStream **streams, AVIOContext *dyn_cp, end_ebml_master(cuepoint, track_positions); } while (++entry < end && entry->pts == pts); size = avio_get_dyn_buf(cuepoint, &buf); -if ((ret = cuepoint->error) < 0) -break; +if (cuepoint->error < 0) +return cuepoint->error; put_ebml_binary(dyn_cp, MATROSKA_ID_POINTENTRY, buf, size); ffio_reset_dyn_buf(cuepoint); } -ffio_free_dyn_buf(&cuepoint); -return ret; +return 0; } static int put_xiph_codecpriv(AVFormatContext *s, AVIOContext *pb, @@ -2648,23 +2641,20 @@ static int mkv_check_new_extra_data(AVFormatContext *s, const AVPacket *pkt) //See https://bugs.chromium.org/p/aomedia/issues/detail?id=2012 case AV_CODEC_ID_AV1: if (side_data_size && mkv->track.bc && !par->extradata_size) { -AVIOContext *dyn_cp; +AVIOContext *const dyn_cp = mkv->tmp_bc; uint8_t *codecpriv; int codecpriv_size; -ret = avio_open_dyn_buf(&dyn_cp); -if (ret < 0) -return ret; ff_isom_write_av1c(dyn_cp, side_data, side_data_size, 1); codecpriv_size = avio_get_dyn_buf(dyn_cp, &codecpriv); if ((ret = dyn_cp->error) < 0 || !codecpriv_size && (ret = AVERROR_INVALIDDATA)) { -ffio_free_dyn_buf(&dyn_cp); +ffio_reset_dyn_buf(dyn_cp); return ret; } avio_seek(mkv->track.bc, track->codecpriv_offset, SEEK_SET); // Do not write the OBUs as we don't have space saved for them put_ebml_binary(mkv->track.bc, MATROSKA_ID_CODECPRIVATE, codecpriv, 4); -ffio_free_dyn_buf(&dyn_cp); +ffio_reset_dyn_buf(dyn_cp); ret = ff_alloc_extradata(par, side_data_size); if (ret < 0) return ret; @@ -2891,7 +2881,7 @@ redo_cues: if (ret < 0) return ret; -ret = mkv_assemble_cues(s->streams, cues, &mkv->cues, +ret = mkv_assemble_cues(s->streams, cues, mkv->tmp_bc, &mkv->cues, mkv->tracks, s->nb_streams, offset); if (ret < 0) { ffio_free_dyn_buf(&cues); -- 2.34.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 06/13] lavu/mem: Add ff_fast_recalloc()
On 6/15/2022 6:59 AM, Tomas Härdin wrote: tis 2022-06-14 klockan 22:26 +0200 skrev Michael Niedermayer: On Tue, Jun 14, 2022 at 04:42:06PM +0200, Tomas Härdin wrote: Left this as an ff_ funtion for now since it's only used by the j2k code /Tomas mem.c | 24 mem.h | 55 +++ 2 files changed, 79 insertions(+) 21be65bd06e3260f9f36598d5d574ee32e7131a6 0006-lavu-mem-Add- ff_fast_recalloc.patch From 5d36d431ffe4c8ba0f698d0c288ebc16b83f0bbc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tomas=20H=C3=A4rdin?= Date: Tue, 14 Jun 2022 13:35:18 +0200 Subject: [PATCH 06/13] lavu/mem: Add ff_fast_recalloc() You cannot call a ff_* function thats in libavutil from outside libavutil this will fail with shared libs as the ff* stuff is not exported Ah, I suspected as much. Would there be much opposition to a public function like this in lavu? I could just keep it local to the j2k code Just make it public by using the av_ prefix (You in fact added it to mem.h, which is installed. You'd need to add it to mem_internal.h if you wanted to avoid exposing it). Don't forget to add an APIChanges entry and minor lavu version bump before you push if you do. /Tomas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 05/13] lavc/jpeg2000dec: Thread init_tile()
tis 2022-06-14 klockan 23:11 +0200 skrev Michael Niedermayer: > On Tue, Jun 14, 2022 at 04:41:14PM +0200, Tomas Härdin wrote: > > > > > jpeg2000dec.c | 30 +++--- > > 1 file changed, 15 insertions(+), 15 deletions(-) > > 6fa2fbf99afee36ee73459863df0527a72663f43 0005-lavc-jpeg2000dec- > > Thread-init_tile.patch > > From 080ebdc9bad130098bff575f9ce690b8a522c9f7 Mon Sep 17 00:00:00 > > 2001 > > From: =?UTF-8?q?Tomas=20H=C3=A4rdin?= > > Date: Mon, 13 Jun 2022 15:09:17 +0200 > > Subject: [PATCH 05/13] lavc/jpeg2000dec: Thread init_tile() > > Causes segfaults > > [jpeg2000 @ 0x2cf53380] End mismatch 149 > [jpeg2000 @ 0x2cf53380] ==1439== Thread 6: > ==1439== Invalid read of size 4 > ==1439== at 0x9771F0: jpeg2000_mct_write_frame (in ffmpeg_g) > ==1439== by 0x78BA6F: avcodec_default_execute2 (in ffmpeg_g) > ==1439== by 0x97C0BB: jpeg2000_decode_frame (in ffmpeg_g) > ==1439== by 0xA90F72: frame_worker_thread (in ffmpeg_g) > ==1439== by 0x54046DA: start_thread (pthread_create.c:463) > ==1439== by 0xF8F261E: clone (clone.S:95) > > i will send you the sample privatly This is because init_tile() fails. I had assumed errors were handled in some way like longjmp since the function already called execute2() but it seems the threading doesn't do any kind of magic for this. Can we have execute2() return some kind of error code when one or more jobs fail? Either say FFMIN() of all errors or negative jobnr that failed? This would save on having to allocate an array for errors when we don't really care which exact jobs failed.. /Tomas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v8 0/2] libjxl Colorspace Fixes
On 6/9/22 07:31, Leo Izen wrote: On 6/1/22 22:14, Leo Izen wrote: Changes in v8: - Use avutil/csp for both encoding and decoding - Handle the non-XYB case with an attached ICC Profile on decoding - clean up some code and segment it out to static functions Leo Izen (2): avcodec/libjxldec: properly tag output colorspace avcodec/libjxlenc: properly read input colorspace libavcodec/libjxldec.c | 142 +++--- libavcodec/libjxlenc.c | 153 + 2 files changed, 256 insertions(+), 39 deletions(-) I believe this should make it into 5.1 as it fixes a known bug and improves existing behavior. - Leo Izen (thebombzen) Bumping again for review. - Leo Izen (thebombzen) ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avfilter: add virtualbass filter
On Fri, Jun 10, 2022 at 12:09 PM Paul B Mahol wrote: > Hi, > > Patch attached. > Will apply in next 5 minutes. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v15 5/5] libavfilter/vf_frei0r.c: Use UTF-8 version of getenv()
--- libavfilter/vf_frei0r.c | 14 ++ 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/libavfilter/vf_frei0r.c b/libavfilter/vf_frei0r.c index f11ae6e55c..727e96561a 100644 --- a/libavfilter/vf_frei0r.c +++ b/libavfilter/vf_frei0r.c @@ -31,6 +31,7 @@ #include "libavutil/avstring.h" #include "libavutil/common.h" #include "libavutil/eval.h" +#include "libavutil/getenv_utf8.h" #include "libavutil/imgutils.h" #include "libavutil/internal.h" #include "libavutil/mathematics.h" @@ -204,7 +205,7 @@ static av_cold int frei0r_init(AVFilterContext *ctx, } /* see: http://frei0r.dyne.org/codedoc/html/group__pluglocations.html */ -if ((path = av_strdup(getenv("FREI0R_PATH" { +if ((path = getenv_utf8("FREI0R_PATH"))) { #ifdef _WIN32 const char *separator = ";"; #else @@ -231,12 +232,17 @@ static av_cold int frei0r_init(AVFilterContext *ctx, if (ret < 0) return ret; } -if (!s->dl_handle && (path = getenv("HOME"))) { +if (!s->dl_handle && (path = getenv_utf8("HOME"))) { char *prefix = av_asprintf("%s/.frei0r-1/lib/", path); -if (!prefix) -return AVERROR(ENOMEM); +if (!prefix) { +ret = AVERROR(ENOMEM); +goto home_path_end; +} ret = load_path(ctx, &s->dl_handle, prefix, dl_name); av_free(prefix); + +home_path_end: +av_free(path); if (ret < 0) return ret; } -- 2.34.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v15 1/5] libavutil: Add wchartoutf8(), wchartoansi(), utf8toansi() and getenv_utf8()
wchartoutf8() converts strings returned by WinAPI into UTF-8, which is FFmpeg's preffered encoding. Some external dependencies, such as AviSynth, are still not Unicode-enabled. utf8toansi() converts UTF-8 strings into ANSI in two steps: UTF-8 -> wchar_t -> ANSI. wchartoansi() is responsible for the second step of the conversion. Conversion in just one step is not supported by WinAPI. Since these character converting functions allocate the buffer of necessary size, they also facilitate the removal of MAX_PATH limit in places where fixed-size ANSI/WCHAR strings were used as filename buffers. getenv_utf8() wraps _wgetenv() converting its input from and its output to UTF-8. Compared to plain getenv(), getenv_utf8() requires a cleanup. Because of that, in places that only test the existence of an environment variable or compare its value with a string consisting entirely of ASCII characters, the use of plain getenv() is still preferred. (libavutil/log.c check_color_terminal() is an example of such a place.) Plain getenv() is also preffered in UNIX-only code, such as bktr.c, fbdev_common.c, oss.c in libavdevice or af_ladspa.c in libavfilter. --- .vscode/settings.json | 11 ++ configure | 1 + libavutil/getenv_utf8.h| 71 ++ libavutil/wchar_filename.h | 51 +++ 4 files changed, 134 insertions(+) create mode 100644 .vscode/settings.json create mode 100644 libavutil/getenv_utf8.h diff --git a/.vscode/settings.json b/.vscode/settings.json new file mode 100644 index 00..e866d57743 --- /dev/null +++ b/.vscode/settings.json @@ -0,0 +1,11 @@ +{ +"files.associations": { +"w32dlfcn.h": "c", +"mem.h": "c", +"internal.h": "c", +"os_support.h": "c", +"packet_internal.h": "c", +"stdlib.h": "c", +"mathematics.h": "c" +} +} \ No newline at end of file diff --git a/configure b/configure index 3dca1c4bd3..fa37a74531 100755 --- a/configure +++ b/configure @@ -2272,6 +2272,7 @@ SYSTEM_FUNCS=" fcntl getaddrinfo getauxval +getenv gethrtime getopt GetModuleHandle diff --git a/libavutil/getenv_utf8.h b/libavutil/getenv_utf8.h new file mode 100644 index 00..161e3e6202 --- /dev/null +++ b/libavutil/getenv_utf8.h @@ -0,0 +1,71 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_GETENV_UTF8_H +#define AVUTIL_GETENV_UTF8_H + +#include + +#include "mem.h" + +#ifdef HAVE_GETENV + +#ifdef _WIN32 + +#include "libavutil/wchar_filename.h" + +static inline char *getenv_utf8(const char *varname) +{ +wchar_t *varname_w, *var_w; +char *var; + +if (utf8towchar(varname, &varname_w)) +return NULL; +if (!varname_w) +return NULL; + +var_w = _wgetenv(varname_w); +av_free(varname_w); + +if (!var_w) +return NULL; +if (wchartoutf8(var_w, &var)) +return NULL; + +return var; + +// No CP_ACP fallback compared to other *_utf8() functions: +// non UTF-8 strings must not be returned. +} + +#else + +static inline char *getenv_utf8(const char *varname) +{ +return av_strdup(getenv(varname)); +} + +#endif // _WIN32 + +#else + +#define getenv_utf8(x) NULL + +#endif // HAVE_GETENV + +#endif // AVUTIL_GETENV_UTF8_H diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h index f36d9dfea3..a6d71e52e5 100644 --- a/libavutil/wchar_filename.h +++ b/libavutil/wchar_filename.h @@ -41,6 +41,57 @@ static inline int utf8towchar(const char *filename_utf8, wchar_t **filename_w) return 0; } +av_warn_unused_result +static inline int wchartocp(unsigned int code_page, const wchar_t *filename_w, +char **filename) +{ +DWORD flags = code_page == CP_UTF8 ? WC_ERR_INVALID_CHARS : 0; +int num_chars = WideCharToMultiByte(code_page, flags, filename_w, -1, +NULL, 0, NULL, NULL); +if (num_chars <= 0) { +*filename = NULL; +return 0; +} +*filename = av_malloc_array(num_chars, sizeof *filename); +if (!*filename) { +errno = ENOMEM; +return -1; +} +WideCharToMultiByte(code_page, flags, filename_w, -1, +*
[FFmpeg-devel] [PATCH v15 2/5] compat/w32dlfcn.h: Remove MAX_PATH limit and replace LoadLibraryExA with LoadLibraryExW
--- compat/w32dlfcn.h | 95 +-- libavcodec/mf_utils.h | 1 + 2 files changed, 74 insertions(+), 22 deletions(-) diff --git a/compat/w32dlfcn.h b/compat/w32dlfcn.h index 52a94efafb..e4f46c488c 100644 --- a/compat/w32dlfcn.h +++ b/compat/w32dlfcn.h @@ -20,11 +20,35 @@ #define COMPAT_W32DLFCN_H #ifdef _WIN32 +#include #include #include "config.h" -#if (_WIN32_WINNT < 0x0602) || HAVE_WINRT #include "libavutil/wchar_filename.h" -#endif + +static inline wchar_t *get_module_filename(HMODULE module) +{ +wchar_t *path = NULL, *new_path; +const DWORD max_path_size = INT16_MAX + 1; +DWORD path_size = 0, path_len; + +do { +path_size = path_size ? 2 * path_size : MAX_PATH; +new_path = av_realloc_array(path, path_size, sizeof *path); +if (!new_path) { +av_free(path); +return NULL; +} +path = new_path; +path_len = GetModuleFileNameW(module, path, path_size); +} while (path_len && path_size <= max_path_size && path_size <= path_len); + +if (!path_len) { +av_free(path); +return NULL; +} +return path; +} + /** * Safe function used to open dynamic libs. This attempts to improve program security * by removing the current directory from the dll search path. Only dll's found in the @@ -34,29 +58,53 @@ */ static inline HMODULE win32_dlopen(const char *name) { +wchar_t *name_w; +HMODULE module = NULL; +if (utf8towchar(name, &name_w)) +name_w = NULL; #if _WIN32_WINNT < 0x0602 -// Need to check if KB2533623 is available +// On Win7 and earlier we check if KB2533623 is available if (!GetProcAddress(GetModuleHandleW(L"kernel32.dll"), "SetDefaultDllDirectories")) { -HMODULE module = NULL; -wchar_t *path = NULL, *name_w = NULL; -DWORD pathlen; -if (utf8towchar(name, &name_w)) +wchar_t *path = NULL, *new_path; +DWORD pathlen, pathsize, namelen; +if (!name_w) goto exit; -path = (wchar_t *)av_calloc(MAX_PATH, sizeof(wchar_t)); +namelen = wcslen(name_w); // Try local directory first -pathlen = GetModuleFileNameW(NULL, path, MAX_PATH); -pathlen = wcsrchr(path, '\\') - path; -if (pathlen == 0 || pathlen + wcslen(name_w) + 2 > MAX_PATH) +path = get_module_filename(NULL); +if (!path) +goto exit; +new_path = wcsrchr(path, '\\'); +if (!new_path) goto exit; -path[pathlen] = '\\'; +pathlen = new_path - path; +pathsize = pathlen + namelen + 2; +new_path = av_realloc_array(path, pathsize, sizeof *path); +if (!new_path) +goto exit; +path = new_path; wcscpy(path + pathlen + 1, name_w); module = LoadLibraryExW(path, NULL, LOAD_WITH_ALTERED_SEARCH_PATH); if (module == NULL) { // Next try System32 directory -pathlen = GetSystemDirectoryW(path, MAX_PATH); -if (pathlen == 0 || pathlen + wcslen(name_w) + 2 > MAX_PATH) +pathlen = GetSystemDirectoryW(path, pathsize); +if (!pathlen) goto exit; -path[pathlen] = '\\'; +// Buffer is not enough in two cases: +// 1. system directory + \ + module name +// 2. system directory even without the module name. +if (pathlen + namelen + 2 > pathsize) { +pathsize = pathlen + namelen + 2; +new_path = av_realloc_array(path, pathsize, sizeof *path); +if (!new_path) +goto exit; +path = new_path; +// Query again to handle the case #2. +pathlen = GetSystemDirectoryW(path, pathsize); +if (!pathlen) +goto exit; +} +path[pathlen] = L'\\'; wcscpy(path + pathlen + 1, name_w); module = LoadLibraryExW(path, NULL, LOAD_WITH_ALTERED_SEARCH_PATH); } @@ -73,16 +121,19 @@ exit: # define LOAD_LIBRARY_SEARCH_SYSTEM320x0800 #endif #if HAVE_WINRT -wchar_t *name_w = NULL; -int ret; -if (utf8towchar(name, &name_w)) +if (!name_w) return NULL; -ret = LoadPackagedLibrary(name_w, 0); -av_free(name_w); -return ret; +module = LoadPackagedLibrary(name_w, 0); #else -return LoadLibraryExA(name, NULL, LOAD_LIBRARY_SEARCH_APPLICATION_DIR | LOAD_LIBRARY_SEARCH_SYSTEM32); +#define LOAD_FLAGS (LOAD_LIBRARY_SEARCH_APPLICATION_DIR | LOAD_LIBRARY_SEARCH_SYSTEM32) +/* filename may be be in CP_ACP */ +if (!name_w) +return LoadLibraryExA(name, NULL, LOAD_FLAGS); +module = LoadLibraryExW(name_w, NULL, LOAD_FLAGS); +#undef LOAD_FLAGS #endif +av_free(name_w); +return module; } #define dlopen(name, flags) win32_dlopen(name) #define dlclose FreeLibrary diff --g
[FFmpeg-devel] [PATCH v15 3/5] fftools: Remove MAX_PATH limit and switch to UTF-8 versions of fopen() and getenv()
--- fftools/cmdutils.c | 53 +--- fftools/ffmpeg_opt.c | 9 ++-- 2 files changed, 47 insertions(+), 15 deletions(-) diff --git a/fftools/cmdutils.c b/fftools/cmdutils.c index 5d7cdc3e10..5e7fbbe2ee 100644 --- a/fftools/cmdutils.c +++ b/fftools/cmdutils.c @@ -39,6 +39,7 @@ #include "libavutil/avstring.h" #include "libavutil/channel_layout.h" #include "libavutil/display.h" +#include "libavutil/getenv_utf8.h" #include "libavutil/mathematics.h" #include "libavutil/imgutils.h" #include "libavutil/libm.h" @@ -47,9 +48,11 @@ #include "libavutil/dict.h" #include "libavutil/opt.h" #include "cmdutils.h" +#include "fopen_utf8.h" #include "opt_common.h" #ifdef _WIN32 #include +#include "compat/w32dlfcn.h" #endif AVDictionary *sws_dict; @@ -465,7 +468,7 @@ static void check_options(const OptionDef *po) void parse_loglevel(int argc, char **argv, const OptionDef *options) { int idx = locate_option(argc, argv, options, "loglevel"); -const char *env; +char *env; check_options(options); @@ -474,7 +477,8 @@ void parse_loglevel(int argc, char **argv, const OptionDef *options) if (idx && argv[idx + 1]) opt_loglevel(NULL, "loglevel", argv[idx + 1]); idx = locate_option(argc, argv, options, "report"); -if ((env = getenv("FFREPORT")) || idx) { +env = getenv_utf8("FFREPORT"); +if (env || idx) { FILE *report_file = NULL; init_report(env, &report_file); if (report_file) { @@ -487,6 +491,7 @@ void parse_loglevel(int argc, char **argv, const OptionDef *options) fflush(report_file); } } +av_free(env); idx = locate_option(argc, argv, options, "hide_banner"); if (idx) hide_banner = 1; @@ -812,28 +817,45 @@ FILE *get_preset_file(char *filename, size_t filename_size, { FILE *f = NULL; int i; -const char *base[3] = { getenv("FFMPEG_DATADIR"), -getenv("HOME"), +#if HAVE_GETMODULEHANDLE && defined(_WIN32) +char *datadir = NULL; +#endif +char *env_home = getenv_utf8("HOME"); +char *env_ffmpeg_datadir = getenv_utf8("FFMPEG_DATADIR"); +const char *base[3] = { env_home, +env_ffmpeg_datadir, FFMPEG_DATADIR, }; if (is_path) { av_strlcpy(filename, preset_name, filename_size); -f = fopen(filename, "r"); +f = fopen_utf8(filename, "r"); } else { #if HAVE_GETMODULEHANDLE && defined(_WIN32) -char datadir[MAX_PATH], *ls; +wchar_t *datadir_w = get_module_filename(NULL); base[2] = NULL; -if (GetModuleFileNameA(GetModuleHandleA(NULL), datadir, sizeof(datadir) - 1)) +if (wchartoutf8(datadir_w, &datadir)) +datadir = NULL; +av_free(datadir_w); + +if (datadir) { -for (ls = datadir; ls < datadir + strlen(datadir); ls++) +char *ls; +for (ls = datadir; *ls; ls++) if (*ls == '\\') *ls = '/'; if (ls = strrchr(datadir, '/')) { -*ls = 0; -strncat(datadir, "/ffpresets", sizeof(datadir) - 1 - strlen(datadir)); -base[2] = datadir; +ptrdiff_t datadir_len = ls - datadir; +size_t desired_size = datadir_len + strlen("/ffpresets") + 1; +char *new_datadir = av_realloc_array( +datadir, desired_size, sizeof *datadir); +if (new_datadir) { +datadir = new_datadir; +datadir[datadir_len] = 0; +strncat(datadir, "/ffpresets", desired_size - 1 - datadir_len); +base[2] = datadir; +} } } #endif @@ -842,17 +864,22 @@ FILE *get_preset_file(char *filename, size_t filename_size, continue; snprintf(filename, filename_size, "%s%s/%s.ffpreset", base[i], i != 1 ? "" : "/.ffmpeg", preset_name); -f = fopen(filename, "r"); +f = fopen_utf8(filename, "r"); if (!f && codec_name) { snprintf(filename, filename_size, "%s%s/%s-%s.ffpreset", base[i], i != 1 ? "" : "/.ffmpeg", codec_name, preset_name); -f = fopen(filename, "r"); +f = fopen_utf8(filename, "r"); } } } +#if HAVE_GETMODULEHANDLE && defined(_WIN32) +av_free(datadir); +#endif +av_free(env_ffmpeg_datadir); +av_free(env_home); return f; } diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 398067da96..f49acf6ad0 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -44,6 +44,7 @@ #include "libavutil/avutil.h" #include "libavutil/bprint.h" #include "libavutil/channel_layout.h" +#include "libavutil/getenv_utf8.
[FFmpeg-devel] [PATCH v15 4/5] libavformat: Remove MAX_PATH limit and use UTF-8 version of getenv()
1. getenv() is replaced with getenv_utf8() across libavformat. 2. New versions of AviSynth+ are now called with UTF-8 filenames. 3. Old versions of AviSynth are still using ANSI strings, but MAX_PATH limit on filename is removed. --- libavformat/avisynth.c| 39 +++ libavformat/http.c| 20 +--- libavformat/ipfsgateway.c | 35 +++ libavformat/tls.c | 11 +-- 4 files changed, 72 insertions(+), 33 deletions(-) diff --git a/libavformat/avisynth.c b/libavformat/avisynth.c index 8ba2bdead2..a97d12b6b6 100644 --- a/libavformat/avisynth.c +++ b/libavformat/avisynth.c @@ -34,6 +34,7 @@ /* Platform-specific directives. */ #ifdef _WIN32 #include "compat/w32dlfcn.h" + #include "libavutil/wchar_filename.h" #undef EXTERN_C #define AVISYNTH_LIB "avisynth" #else @@ -56,6 +57,7 @@ typedef struct AviSynthLibrary { #define AVSC_DECLARE_FUNC(name) name ## _func name AVSC_DECLARE_FUNC(avs_bit_blt); AVSC_DECLARE_FUNC(avs_clip_get_error); +AVSC_DECLARE_FUNC(avs_check_version); AVSC_DECLARE_FUNC(avs_create_script_environment); AVSC_DECLARE_FUNC(avs_delete_script_environment); AVSC_DECLARE_FUNC(avs_get_audio); @@ -137,6 +139,7 @@ static av_cold int avisynth_load_library(void) LOAD_AVS_FUNC(avs_bit_blt, 0); LOAD_AVS_FUNC(avs_clip_get_error, 0); +LOAD_AVS_FUNC(avs_check_version, 0); LOAD_AVS_FUNC(avs_create_script_environment, 0); LOAD_AVS_FUNC(avs_delete_script_environment, 0); LOAD_AVS_FUNC(avs_get_audio, 0); @@ -807,26 +810,38 @@ static int avisynth_create_stream(AVFormatContext *s) static int avisynth_open_file(AVFormatContext *s) { AviSynthContext *avs = s->priv_data; -AVS_Value arg, val; +AVS_Value val; int ret; -#ifdef _WIN32 -char filename_ansi[MAX_PATH * 4]; -wchar_t filename_wc[MAX_PATH * 4]; -#endif if (ret = avisynth_context_create(s)) return ret; +if (!avs_library.avs_check_version(avs->env, 7)) { +AVS_Value args[] = { +avs_new_value_string(s->url), +avs_new_value_bool(1) // filename is in UTF-8 +}; +val = avs_library.avs_invoke(avs->env, "Import", + avs_new_value_array(args, 2), 0); +} else { +AVS_Value arg; #ifdef _WIN32 -/* Convert UTF-8 to ANSI code page */ -MultiByteToWideChar(CP_UTF8, 0, s->url, -1, filename_wc, MAX_PATH * 4); -WideCharToMultiByte(CP_THREAD_ACP, 0, filename_wc, -1, filename_ansi, -MAX_PATH * 4, NULL, NULL); -arg = avs_new_value_string(filename_ansi); +char *filename_ansi; +/* Convert UTF-8 to ANSI code page */ +if (utf8toansi(s->url, &filename_ansi)) { +ret = AVERROR_UNKNOWN; +goto fail; +} +arg = avs_new_value_string(filename_ansi); #else -arg = avs_new_value_string(s->url); +arg = avs_new_value_string(s->url); #endif -val = avs_library.avs_invoke(avs->env, "Import", arg, 0); +val = avs_library.avs_invoke(avs->env, "Import", arg, 0); +#ifdef _WIN32 +av_free(filename_ansi); +#endif +} + if (avs_is_error(val)) { av_log(s, AV_LOG_ERROR, "%s\n", avs_as_error(val)); ret = AVERROR_UNKNOWN; diff --git a/libavformat/http.c b/libavformat/http.c index c8f3f4b6a3..d90117e422 100644 --- a/libavformat/http.c +++ b/libavformat/http.c @@ -29,6 +29,7 @@ #include "libavutil/avassert.h" #include "libavutil/avstring.h" #include "libavutil/bprint.h" +#include "libavutil/getenv_utf8.h" #include "libavutil/opt.h" #include "libavutil/time.h" #include "libavutil/parseutils.h" @@ -198,6 +199,7 @@ void ff_http_init_auth_state(URLContext *dest, const URLContext *src) static int http_open_cnx_internal(URLContext *h, AVDictionary **options) { const char *path, *proxy_path, *lower_proto = "tcp", *local_path; +char *env_http_proxy, *env_no_proxy; char *hashmark; char hostname[1024], hoststr[1024], proto[10]; char auth[1024], proxyauth[1024] = ""; @@ -211,9 +213,13 @@ static int http_open_cnx_internal(URLContext *h, AVDictionary **options) path1, sizeof(path1), s->location); ff_url_join(hoststr, sizeof(hoststr), NULL, NULL, hostname, port, NULL); -proxy_path = s->http_proxy ? s->http_proxy : getenv("http_proxy"); -use_proxy = !ff_http_match_no_proxy(getenv("no_proxy"), hostname) && +env_http_proxy = getenv_utf8("http_proxy"); +proxy_path = s->http_proxy ? s->http_proxy : env_http_proxy; + +env_no_proxy = getenv_utf8("no_proxy"); +use_proxy = !ff_http_match_no_proxy(env_no_proxy, hostname) && proxy_path && av_strstart(proxy_path, "http://";, NULL); +av_freep(&env_no_proxy); if (!strcmp(proto, "https")) { lower_proto = "tls"; @@ -224,7 +230,7 @@ static int http_open_cnx_internal(URLContext *h, AVDictionary **options
Re: [FFmpeg-devel] [PATCH v14 2/5] compat/w32dlfcn.h: Remove MAX_PATH limit and replace LoadLibraryExA with LoadLibraryExW
> path_size <= INT16_MAX > > (the edge case is already covered by the equals) Done: https://ffmpeg.org/pipermail/ffmpeg-devel/2022-June/297590.html. Don't quite understand what edge case you've meant: INT16_MAX is 32767, which is the maximal path length allowed, + 1 is needed for the terminating null. > I know this line existed before your path, but it would be nice > to clarify check and condition, like: > > // On Win7 an earlier we check if KB2533623 is available Changed the comment. > name_w leaks here Fixed. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v16 2/5] compat/w32dlfcn.h: Remove MAX_PATH limit and replace LoadLibraryExA with LoadLibraryExW
--- compat/w32dlfcn.h | 95 +-- libavcodec/mf_utils.h | 1 + 2 files changed, 74 insertions(+), 22 deletions(-) diff --git a/compat/w32dlfcn.h b/compat/w32dlfcn.h index 52a94efafb..e4f46c488c 100644 --- a/compat/w32dlfcn.h +++ b/compat/w32dlfcn.h @@ -20,11 +20,35 @@ #define COMPAT_W32DLFCN_H #ifdef _WIN32 +#include #include #include "config.h" -#if (_WIN32_WINNT < 0x0602) || HAVE_WINRT #include "libavutil/wchar_filename.h" -#endif + +static inline wchar_t *get_module_filename(HMODULE module) +{ +wchar_t *path = NULL, *new_path; +const DWORD max_path_size = INT16_MAX + 1; +DWORD path_size = 0, path_len; + +do { +path_size = path_size ? 2 * path_size : MAX_PATH; +new_path = av_realloc_array(path, path_size, sizeof *path); +if (!new_path) { +av_free(path); +return NULL; +} +path = new_path; +path_len = GetModuleFileNameW(module, path, path_size); +} while (path_len && path_size <= max_path_size && path_size <= path_len); + +if (!path_len) { +av_free(path); +return NULL; +} +return path; +} + /** * Safe function used to open dynamic libs. This attempts to improve program security * by removing the current directory from the dll search path. Only dll's found in the @@ -34,29 +58,53 @@ */ static inline HMODULE win32_dlopen(const char *name) { +wchar_t *name_w; +HMODULE module = NULL; +if (utf8towchar(name, &name_w)) +name_w = NULL; #if _WIN32_WINNT < 0x0602 -// Need to check if KB2533623 is available +// On Win7 and earlier we check if KB2533623 is available if (!GetProcAddress(GetModuleHandleW(L"kernel32.dll"), "SetDefaultDllDirectories")) { -HMODULE module = NULL; -wchar_t *path = NULL, *name_w = NULL; -DWORD pathlen; -if (utf8towchar(name, &name_w)) +wchar_t *path = NULL, *new_path; +DWORD pathlen, pathsize, namelen; +if (!name_w) goto exit; -path = (wchar_t *)av_calloc(MAX_PATH, sizeof(wchar_t)); +namelen = wcslen(name_w); // Try local directory first -pathlen = GetModuleFileNameW(NULL, path, MAX_PATH); -pathlen = wcsrchr(path, '\\') - path; -if (pathlen == 0 || pathlen + wcslen(name_w) + 2 > MAX_PATH) +path = get_module_filename(NULL); +if (!path) +goto exit; +new_path = wcsrchr(path, '\\'); +if (!new_path) goto exit; -path[pathlen] = '\\'; +pathlen = new_path - path; +pathsize = pathlen + namelen + 2; +new_path = av_realloc_array(path, pathsize, sizeof *path); +if (!new_path) +goto exit; +path = new_path; wcscpy(path + pathlen + 1, name_w); module = LoadLibraryExW(path, NULL, LOAD_WITH_ALTERED_SEARCH_PATH); if (module == NULL) { // Next try System32 directory -pathlen = GetSystemDirectoryW(path, MAX_PATH); -if (pathlen == 0 || pathlen + wcslen(name_w) + 2 > MAX_PATH) +pathlen = GetSystemDirectoryW(path, pathsize); +if (!pathlen) goto exit; -path[pathlen] = '\\'; +// Buffer is not enough in two cases: +// 1. system directory + \ + module name +// 2. system directory even without the module name. +if (pathlen + namelen + 2 > pathsize) { +pathsize = pathlen + namelen + 2; +new_path = av_realloc_array(path, pathsize, sizeof *path); +if (!new_path) +goto exit; +path = new_path; +// Query again to handle the case #2. +pathlen = GetSystemDirectoryW(path, pathsize); +if (!pathlen) +goto exit; +} +path[pathlen] = L'\\'; wcscpy(path + pathlen + 1, name_w); module = LoadLibraryExW(path, NULL, LOAD_WITH_ALTERED_SEARCH_PATH); } @@ -73,16 +121,19 @@ exit: # define LOAD_LIBRARY_SEARCH_SYSTEM320x0800 #endif #if HAVE_WINRT -wchar_t *name_w = NULL; -int ret; -if (utf8towchar(name, &name_w)) +if (!name_w) return NULL; -ret = LoadPackagedLibrary(name_w, 0); -av_free(name_w); -return ret; +module = LoadPackagedLibrary(name_w, 0); #else -return LoadLibraryExA(name, NULL, LOAD_LIBRARY_SEARCH_APPLICATION_DIR | LOAD_LIBRARY_SEARCH_SYSTEM32); +#define LOAD_FLAGS (LOAD_LIBRARY_SEARCH_APPLICATION_DIR | LOAD_LIBRARY_SEARCH_SYSTEM32) +/* filename may be be in CP_ACP */ +if (!name_w) +return LoadLibraryExA(name, NULL, LOAD_FLAGS); +module = LoadLibraryExW(name_w, NULL, LOAD_FLAGS); +#undef LOAD_FLAGS #endif +av_free(name_w); +return module; } #define dlopen(name, flags) win32_dlopen(name) #define dlclose FreeLibrary diff --g
[FFmpeg-devel] [PATCH v16 5/5] libavfilter/vf_frei0r.c: Use UTF-8 version of getenv()
--- libavfilter/vf_frei0r.c | 14 ++ 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/libavfilter/vf_frei0r.c b/libavfilter/vf_frei0r.c index f11ae6e55c..727e96561a 100644 --- a/libavfilter/vf_frei0r.c +++ b/libavfilter/vf_frei0r.c @@ -31,6 +31,7 @@ #include "libavutil/avstring.h" #include "libavutil/common.h" #include "libavutil/eval.h" +#include "libavutil/getenv_utf8.h" #include "libavutil/imgutils.h" #include "libavutil/internal.h" #include "libavutil/mathematics.h" @@ -204,7 +205,7 @@ static av_cold int frei0r_init(AVFilterContext *ctx, } /* see: http://frei0r.dyne.org/codedoc/html/group__pluglocations.html */ -if ((path = av_strdup(getenv("FREI0R_PATH" { +if ((path = getenv_utf8("FREI0R_PATH"))) { #ifdef _WIN32 const char *separator = ";"; #else @@ -231,12 +232,17 @@ static av_cold int frei0r_init(AVFilterContext *ctx, if (ret < 0) return ret; } -if (!s->dl_handle && (path = getenv("HOME"))) { +if (!s->dl_handle && (path = getenv_utf8("HOME"))) { char *prefix = av_asprintf("%s/.frei0r-1/lib/", path); -if (!prefix) -return AVERROR(ENOMEM); +if (!prefix) { +ret = AVERROR(ENOMEM); +goto home_path_end; +} ret = load_path(ctx, &s->dl_handle, prefix, dl_name); av_free(prefix); + +home_path_end: +av_free(path); if (ret < 0) return ret; } -- 2.34.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v16 1/5] libavutil: Add wchartoutf8(), wchartoansi(), utf8toansi() and getenv_utf8()
wchartoutf8() converts strings returned by WinAPI into UTF-8, which is FFmpeg's preffered encoding. Some external dependencies, such as AviSynth, are still not Unicode-enabled. utf8toansi() converts UTF-8 strings into ANSI in two steps: UTF-8 -> wchar_t -> ANSI. wchartoansi() is responsible for the second step of the conversion. Conversion in just one step is not supported by WinAPI. Since these character converting functions allocate the buffer of necessary size, they also facilitate the removal of MAX_PATH limit in places where fixed-size ANSI/WCHAR strings were used as filename buffers. getenv_utf8() wraps _wgetenv() converting its input from and its output to UTF-8. Compared to plain getenv(), getenv_utf8() requires a cleanup. Because of that, in places that only test the existence of an environment variable or compare its value with a string consisting entirely of ASCII characters, the use of plain getenv() is still preferred. (libavutil/log.c check_color_terminal() is an example of such a place.) Plain getenv() is also preffered in UNIX-only code, such as bktr.c, fbdev_common.c, oss.c in libavdevice or af_ladspa.c in libavfilter. --- configure | 1 + libavutil/getenv_utf8.h| 71 ++ libavutil/wchar_filename.h | 51 +++ 3 files changed, 123 insertions(+) create mode 100644 libavutil/getenv_utf8.h diff --git a/configure b/configure index 3dca1c4bd3..fa37a74531 100755 --- a/configure +++ b/configure @@ -2272,6 +2272,7 @@ SYSTEM_FUNCS=" fcntl getaddrinfo getauxval +getenv gethrtime getopt GetModuleHandle diff --git a/libavutil/getenv_utf8.h b/libavutil/getenv_utf8.h new file mode 100644 index 00..161e3e6202 --- /dev/null +++ b/libavutil/getenv_utf8.h @@ -0,0 +1,71 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_GETENV_UTF8_H +#define AVUTIL_GETENV_UTF8_H + +#include + +#include "mem.h" + +#ifdef HAVE_GETENV + +#ifdef _WIN32 + +#include "libavutil/wchar_filename.h" + +static inline char *getenv_utf8(const char *varname) +{ +wchar_t *varname_w, *var_w; +char *var; + +if (utf8towchar(varname, &varname_w)) +return NULL; +if (!varname_w) +return NULL; + +var_w = _wgetenv(varname_w); +av_free(varname_w); + +if (!var_w) +return NULL; +if (wchartoutf8(var_w, &var)) +return NULL; + +return var; + +// No CP_ACP fallback compared to other *_utf8() functions: +// non UTF-8 strings must not be returned. +} + +#else + +static inline char *getenv_utf8(const char *varname) +{ +return av_strdup(getenv(varname)); +} + +#endif // _WIN32 + +#else + +#define getenv_utf8(x) NULL + +#endif // HAVE_GETENV + +#endif // AVUTIL_GETENV_UTF8_H diff --git a/libavutil/wchar_filename.h b/libavutil/wchar_filename.h index f36d9dfea3..a6d71e52e5 100644 --- a/libavutil/wchar_filename.h +++ b/libavutil/wchar_filename.h @@ -41,6 +41,57 @@ static inline int utf8towchar(const char *filename_utf8, wchar_t **filename_w) return 0; } +av_warn_unused_result +static inline int wchartocp(unsigned int code_page, const wchar_t *filename_w, +char **filename) +{ +DWORD flags = code_page == CP_UTF8 ? WC_ERR_INVALID_CHARS : 0; +int num_chars = WideCharToMultiByte(code_page, flags, filename_w, -1, +NULL, 0, NULL, NULL); +if (num_chars <= 0) { +*filename = NULL; +return 0; +} +*filename = av_malloc_array(num_chars, sizeof *filename); +if (!*filename) { +errno = ENOMEM; +return -1; +} +WideCharToMultiByte(code_page, flags, filename_w, -1, +*filename, num_chars, NULL, NULL); +return 0; +} + +av_warn_unused_result +static inline int wchartoutf8(const wchar_t *filename_w, char **filename) +{ +return wchartocp(CP_UTF8, filename_w, filename); +} + +av_warn_unused_result +static inline int wchartoansi(const wchar_t *filename_w, char **filename) +{ +return wchartocp(CP_ACP, filename_w, filename); +} + +av_warn_unused_result +static inline int utf8toansi(const char *filename_utf8, char **filename) +{ +wchar_t *filename_w = NULL; +int ret =
[FFmpeg-devel] [PATCH v16 3/5] fftools: Remove MAX_PATH limit and switch to UTF-8 versions of fopen() and getenv()
--- fftools/cmdutils.c | 53 +--- fftools/ffmpeg_opt.c | 9 ++-- 2 files changed, 47 insertions(+), 15 deletions(-) diff --git a/fftools/cmdutils.c b/fftools/cmdutils.c index 5d7cdc3e10..5e7fbbe2ee 100644 --- a/fftools/cmdutils.c +++ b/fftools/cmdutils.c @@ -39,6 +39,7 @@ #include "libavutil/avstring.h" #include "libavutil/channel_layout.h" #include "libavutil/display.h" +#include "libavutil/getenv_utf8.h" #include "libavutil/mathematics.h" #include "libavutil/imgutils.h" #include "libavutil/libm.h" @@ -47,9 +48,11 @@ #include "libavutil/dict.h" #include "libavutil/opt.h" #include "cmdutils.h" +#include "fopen_utf8.h" #include "opt_common.h" #ifdef _WIN32 #include +#include "compat/w32dlfcn.h" #endif AVDictionary *sws_dict; @@ -465,7 +468,7 @@ static void check_options(const OptionDef *po) void parse_loglevel(int argc, char **argv, const OptionDef *options) { int idx = locate_option(argc, argv, options, "loglevel"); -const char *env; +char *env; check_options(options); @@ -474,7 +477,8 @@ void parse_loglevel(int argc, char **argv, const OptionDef *options) if (idx && argv[idx + 1]) opt_loglevel(NULL, "loglevel", argv[idx + 1]); idx = locate_option(argc, argv, options, "report"); -if ((env = getenv("FFREPORT")) || idx) { +env = getenv_utf8("FFREPORT"); +if (env || idx) { FILE *report_file = NULL; init_report(env, &report_file); if (report_file) { @@ -487,6 +491,7 @@ void parse_loglevel(int argc, char **argv, const OptionDef *options) fflush(report_file); } } +av_free(env); idx = locate_option(argc, argv, options, "hide_banner"); if (idx) hide_banner = 1; @@ -812,28 +817,45 @@ FILE *get_preset_file(char *filename, size_t filename_size, { FILE *f = NULL; int i; -const char *base[3] = { getenv("FFMPEG_DATADIR"), -getenv("HOME"), +#if HAVE_GETMODULEHANDLE && defined(_WIN32) +char *datadir = NULL; +#endif +char *env_home = getenv_utf8("HOME"); +char *env_ffmpeg_datadir = getenv_utf8("FFMPEG_DATADIR"); +const char *base[3] = { env_home, +env_ffmpeg_datadir, FFMPEG_DATADIR, }; if (is_path) { av_strlcpy(filename, preset_name, filename_size); -f = fopen(filename, "r"); +f = fopen_utf8(filename, "r"); } else { #if HAVE_GETMODULEHANDLE && defined(_WIN32) -char datadir[MAX_PATH], *ls; +wchar_t *datadir_w = get_module_filename(NULL); base[2] = NULL; -if (GetModuleFileNameA(GetModuleHandleA(NULL), datadir, sizeof(datadir) - 1)) +if (wchartoutf8(datadir_w, &datadir)) +datadir = NULL; +av_free(datadir_w); + +if (datadir) { -for (ls = datadir; ls < datadir + strlen(datadir); ls++) +char *ls; +for (ls = datadir; *ls; ls++) if (*ls == '\\') *ls = '/'; if (ls = strrchr(datadir, '/')) { -*ls = 0; -strncat(datadir, "/ffpresets", sizeof(datadir) - 1 - strlen(datadir)); -base[2] = datadir; +ptrdiff_t datadir_len = ls - datadir; +size_t desired_size = datadir_len + strlen("/ffpresets") + 1; +char *new_datadir = av_realloc_array( +datadir, desired_size, sizeof *datadir); +if (new_datadir) { +datadir = new_datadir; +datadir[datadir_len] = 0; +strncat(datadir, "/ffpresets", desired_size - 1 - datadir_len); +base[2] = datadir; +} } } #endif @@ -842,17 +864,22 @@ FILE *get_preset_file(char *filename, size_t filename_size, continue; snprintf(filename, filename_size, "%s%s/%s.ffpreset", base[i], i != 1 ? "" : "/.ffmpeg", preset_name); -f = fopen(filename, "r"); +f = fopen_utf8(filename, "r"); if (!f && codec_name) { snprintf(filename, filename_size, "%s%s/%s-%s.ffpreset", base[i], i != 1 ? "" : "/.ffmpeg", codec_name, preset_name); -f = fopen(filename, "r"); +f = fopen_utf8(filename, "r"); } } } +#if HAVE_GETMODULEHANDLE && defined(_WIN32) +av_free(datadir); +#endif +av_free(env_ffmpeg_datadir); +av_free(env_home); return f; } diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 398067da96..f49acf6ad0 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -44,6 +44,7 @@ #include "libavutil/avutil.h" #include "libavutil/bprint.h" #include "libavutil/channel_layout.h" +#include "libavutil/getenv_utf8.
[FFmpeg-devel] [PATCH v16 4/5] libavformat: Remove MAX_PATH limit and use UTF-8 version of getenv()
1. getenv() is replaced with getenv_utf8() across libavformat. 2. New versions of AviSynth+ are now called with UTF-8 filenames. 3. Old versions of AviSynth are still using ANSI strings, but MAX_PATH limit on filename is removed. --- libavformat/avisynth.c| 39 +++ libavformat/http.c| 20 +--- libavformat/ipfsgateway.c | 35 +++ libavformat/tls.c | 11 +-- 4 files changed, 72 insertions(+), 33 deletions(-) diff --git a/libavformat/avisynth.c b/libavformat/avisynth.c index 8ba2bdead2..a97d12b6b6 100644 --- a/libavformat/avisynth.c +++ b/libavformat/avisynth.c @@ -34,6 +34,7 @@ /* Platform-specific directives. */ #ifdef _WIN32 #include "compat/w32dlfcn.h" + #include "libavutil/wchar_filename.h" #undef EXTERN_C #define AVISYNTH_LIB "avisynth" #else @@ -56,6 +57,7 @@ typedef struct AviSynthLibrary { #define AVSC_DECLARE_FUNC(name) name ## _func name AVSC_DECLARE_FUNC(avs_bit_blt); AVSC_DECLARE_FUNC(avs_clip_get_error); +AVSC_DECLARE_FUNC(avs_check_version); AVSC_DECLARE_FUNC(avs_create_script_environment); AVSC_DECLARE_FUNC(avs_delete_script_environment); AVSC_DECLARE_FUNC(avs_get_audio); @@ -137,6 +139,7 @@ static av_cold int avisynth_load_library(void) LOAD_AVS_FUNC(avs_bit_blt, 0); LOAD_AVS_FUNC(avs_clip_get_error, 0); +LOAD_AVS_FUNC(avs_check_version, 0); LOAD_AVS_FUNC(avs_create_script_environment, 0); LOAD_AVS_FUNC(avs_delete_script_environment, 0); LOAD_AVS_FUNC(avs_get_audio, 0); @@ -807,26 +810,38 @@ static int avisynth_create_stream(AVFormatContext *s) static int avisynth_open_file(AVFormatContext *s) { AviSynthContext *avs = s->priv_data; -AVS_Value arg, val; +AVS_Value val; int ret; -#ifdef _WIN32 -char filename_ansi[MAX_PATH * 4]; -wchar_t filename_wc[MAX_PATH * 4]; -#endif if (ret = avisynth_context_create(s)) return ret; +if (!avs_library.avs_check_version(avs->env, 7)) { +AVS_Value args[] = { +avs_new_value_string(s->url), +avs_new_value_bool(1) // filename is in UTF-8 +}; +val = avs_library.avs_invoke(avs->env, "Import", + avs_new_value_array(args, 2), 0); +} else { +AVS_Value arg; #ifdef _WIN32 -/* Convert UTF-8 to ANSI code page */ -MultiByteToWideChar(CP_UTF8, 0, s->url, -1, filename_wc, MAX_PATH * 4); -WideCharToMultiByte(CP_THREAD_ACP, 0, filename_wc, -1, filename_ansi, -MAX_PATH * 4, NULL, NULL); -arg = avs_new_value_string(filename_ansi); +char *filename_ansi; +/* Convert UTF-8 to ANSI code page */ +if (utf8toansi(s->url, &filename_ansi)) { +ret = AVERROR_UNKNOWN; +goto fail; +} +arg = avs_new_value_string(filename_ansi); #else -arg = avs_new_value_string(s->url); +arg = avs_new_value_string(s->url); #endif -val = avs_library.avs_invoke(avs->env, "Import", arg, 0); +val = avs_library.avs_invoke(avs->env, "Import", arg, 0); +#ifdef _WIN32 +av_free(filename_ansi); +#endif +} + if (avs_is_error(val)) { av_log(s, AV_LOG_ERROR, "%s\n", avs_as_error(val)); ret = AVERROR_UNKNOWN; diff --git a/libavformat/http.c b/libavformat/http.c index c8f3f4b6a3..d90117e422 100644 --- a/libavformat/http.c +++ b/libavformat/http.c @@ -29,6 +29,7 @@ #include "libavutil/avassert.h" #include "libavutil/avstring.h" #include "libavutil/bprint.h" +#include "libavutil/getenv_utf8.h" #include "libavutil/opt.h" #include "libavutil/time.h" #include "libavutil/parseutils.h" @@ -198,6 +199,7 @@ void ff_http_init_auth_state(URLContext *dest, const URLContext *src) static int http_open_cnx_internal(URLContext *h, AVDictionary **options) { const char *path, *proxy_path, *lower_proto = "tcp", *local_path; +char *env_http_proxy, *env_no_proxy; char *hashmark; char hostname[1024], hoststr[1024], proto[10]; char auth[1024], proxyauth[1024] = ""; @@ -211,9 +213,13 @@ static int http_open_cnx_internal(URLContext *h, AVDictionary **options) path1, sizeof(path1), s->location); ff_url_join(hoststr, sizeof(hoststr), NULL, NULL, hostname, port, NULL); -proxy_path = s->http_proxy ? s->http_proxy : getenv("http_proxy"); -use_proxy = !ff_http_match_no_proxy(getenv("no_proxy"), hostname) && +env_http_proxy = getenv_utf8("http_proxy"); +proxy_path = s->http_proxy ? s->http_proxy : env_http_proxy; + +env_no_proxy = getenv_utf8("no_proxy"); +use_proxy = !ff_http_match_no_proxy(env_no_proxy, hostname) && proxy_path && av_strstart(proxy_path, "http://";, NULL); +av_freep(&env_no_proxy); if (!strcmp(proto, "https")) { lower_proto = "tls"; @@ -224,7 +230,7 @@ static int http_open_cnx_internal(URLContext *h, AVDictionary **options
Re: [FFmpeg-devel] [PATCH v14 1/5] libavutil: Add wchartoutf8(), wchartoansi(), utf8toansi() and getenv_utf8()
> I guess we'd might have to add getenv to e.g. the SYSTEM_FUNCS list, so > we'd get a HAVE_GETENV in config.h - then we could make getenv_utf8 a > no-op if HAVE_GETENV is 0. Done: https://ffmpeg.org/pipermail/ffmpeg-devel/2022-June/297596.html ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] The case for a good string API
On date Wednesday 2021-12-22 17:12:41 +0100, Nicolas George wrote: > Hi. > > I will try again proposing a good string API for the project. But this > time, instead of showing the implementation code, I will ask about the > principle. > > So, please, read the argument, and give your opinion on the principle. > If we agree on the principle, we can discuss the code then. [...] > * What string API we should use > > We have AVBprint, but the API is ugly, and it only brings a few of the > benefits I promised. I have a proposal: AVWriter. > > I posted it last spring, here is an introduction on how to use it (with > the implementation in the same thread): > > https://ffmpeg.org/pipermail/ffmpeg-devel/2021-April/279383.html > > > * Conclusion > > Well, I am convinced, but what about you? > > 1. Do you think FFmpeg needs a good string API? If not, please explain > why? > > 2. Assuming we agree ‘yes’ on the previous question, do you think > AVWriter is a good candidate? If not, what else would you propose? Hi, and sorry for the long delay (I'll comment soon about the AVWriter API). Before jumping to the discussion, probably it's good to think a bit about the bprint.h API and its limitations (the ones which come to mind are: no errors in case of truncation, and possible inefficiency due to the realloc). So while it covers the case for small strings (and it's not that bad IMO from the API point of view), probably it's underkill for data serialization. Do you have more in mind about its limitations? Also, is the new API supposed to be a replacement for AVBprint or is it supposed to live in parallel with it (to serve different purposes)? ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 05/13] lavc/jpeg2000dec: Thread init_tile()
On Wed, Jun 15, 2022 at 03:11:34PM +0200, Tomas Härdin wrote: > tis 2022-06-14 klockan 23:11 +0200 skrev Michael Niedermayer: > > On Tue, Jun 14, 2022 at 04:41:14PM +0200, Tomas Härdin wrote: > > > > > > > > jpeg2000dec.c | 30 +++--- > > > 1 file changed, 15 insertions(+), 15 deletions(-) > > > 6fa2fbf99afee36ee73459863df0527a72663f43 0005-lavc-jpeg2000dec- > > > Thread-init_tile.patch > > > From 080ebdc9bad130098bff575f9ce690b8a522c9f7 Mon Sep 17 00:00:00 > > > 2001 > > > From: =?UTF-8?q?Tomas=20H=C3=A4rdin?= > > > Date: Mon, 13 Jun 2022 15:09:17 +0200 > > > Subject: [PATCH 05/13] lavc/jpeg2000dec: Thread init_tile() > > > > Causes segfaults > > > > [jpeg2000 @ 0x2cf53380] End mismatch 149 > > [jpeg2000 @ 0x2cf53380] ==1439== Thread 6: > > ==1439== Invalid read of size 4 > > ==1439== at 0x9771F0: jpeg2000_mct_write_frame (in ffmpeg_g) > > ==1439== by 0x78BA6F: avcodec_default_execute2 (in ffmpeg_g) > > ==1439== by 0x97C0BB: jpeg2000_decode_frame (in ffmpeg_g) > > ==1439== by 0xA90F72: frame_worker_thread (in ffmpeg_g) > > ==1439== by 0x54046DA: start_thread (pthread_create.c:463) > > ==1439== by 0xF8F261E: clone (clone.S:95) > > > > i will send you the sample privatly > > This is because init_tile() fails. I had assumed errors were handled in > some way like longjmp since the function already called execute2() but > it seems the threading doesn't do any kind of magic for this. > > Can we have execute2() return some kind of error code when one or more > jobs fail? Either say FFMIN() of all errors or negative jobnr that > failed? This would save on having to allocate an array for errors when > we don't really care which exact jobs failed.. one could return a struct with error code, index and number of failed ones or something. But then maybe just atomically setting some error flag and leaving the API would be fine too. Iam fine with either thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Republics decline into democracies and democracies degenerate into despotisms. -- Aristotle signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/7] lavu: new AVWriter API.
Preliminary quick review. On date Wednesday 2021-04-21 14:27:01 +0200, Nicolas George wrote: [...] > --- /dev/null > +++ b/libavutil/writer.c > @@ -0,0 +1,443 @@ > +/* > + * Copyright (c) 2021 Nicolas George > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 > USA > + */ > + > +#include "avassert.h" > +#include "log.h" > +#include "writer.h" > + > +/*** > + * Generic API > + ***/ > + > +#define FIELDOK(st, f) ((char *)(&(st)->f + 1) <= (char *)(st) + > (st)->self_size) > + > +#define methods_assert_abi(methods) av_assert1(FIELDOK(methods, flush)) "methods" is somehow confusing since it can contain also other things (name, size, etc.). What about using something as class instead? Or "implementation" to mean that the functionality is embedded there. > + > +static void printf_unchecked(AVWriter wr, const char *fmt, ...) > +{ > +va_list va; > + > +va_start(va, fmt); > +wr.methods->vprintf(wr, fmt, va); > +va_end(va); > +} > + > +static void write_or_discard(AVWriter wr, size_t buf_size, size_t write_size) > +{ > +av_assert1(wr.methods->advance_buffer); > +wr.methods->advance_buffer(wr, FFMIN(buf_size, write_size)); > +if (write_size > buf_size && wr.methods->notify_discard) > +wr.methods->notify_discard(wr, write_size - buf_size); > +} > + > +static void av_writer_impossible(AVWriter wr, const char *message) you can possibly discard the av_ and rename to something as notify_unsupported_operation() ("impossible" is confusing in this context). [...] > +/*** > + * AVBufWriter - write to pre-allocated memory > + ***/ > + > +#define buf_writer_assert_abi(bwr) av_assert1(FIELDOK(bwr, pos)) > + > +static size_t buf_writer_room(AVBufWriter *bwr) inline? > +{ > +return bwr->pos < bwr->size ? bwr->size - bwr->pos - 1 : 0; > +} > + > +static void buf_writer_write(AVWriter wr, const char *data, size_t size) > +{ > +AVBufWriter *bwr = wr.obj; > + > +av_assert1(av_buf_writer_check(wr)); > +buf_writer_assert_abi(bwr); maybe factorize with a macro? > +size = FFMIN(buf_writer_room(bwr), size); > +memcpy(bwr->buf + bwr->pos, data, size); > +bwr->pos += size; > +bwr->buf[bwr->pos] = 0; > +} > + > +static void buf_writer_vprintf(AVWriter wr, const char *fmt, va_list va) > +{ > +AVBufWriter *bwr = wr.obj; > +int ret; > + > +av_assert1(av_buf_writer_check(wr)); > +buf_writer_assert_abi(bwr); > +ret = vsnprintf(bwr->buf + bwr->pos, buf_writer_room(bwr) + 1, fmt, va); > +if (ret > 0) > +bwr->pos += ret; > +} > + > +static char *buf_writer_get_buffer(AVWriter wr, size_t min_size, size_t > *size) > +{ > +AVBufWriter *bwr = wr.obj; > + > +av_assert1(av_buf_writer_check(wr)); > +buf_writer_assert_abi(bwr); > +*size = bwr->size - 1 - bwr->pos; > +return bwr->buf + bwr->pos; > +} > + > +static void buf_writer_advance_buffer(AVWriter wr, size_t size) > +{ > +AVBufWriter *bwr = wr.obj; > + > +av_assert1(av_buf_writer_check(wr)); > +buf_writer_assert_abi(bwr); > +bwr->pos += size; > +bwr->buf[bwr->pos] = 0; buffer overflow, is this set needed? > +} > + > +AV_WRITER_DEFINE_METHODS(/*public*/, AVBufWriter, av_buf_writer) { > +.self_size= sizeof(AVWriterMethods), > +.name = "AVBufWriter", > +.write= buf_writer_write, > +.vprintf = buf_writer_vprintf, > +.get_buffer = buf_writer_get_buffer, > +.advance_buffer = buf_writer_advance_buffer, > +}; > + > +AVBufWriter *av_buf_writer_init(AVBufWriter *bwr, char *buf, size_t size) > +{ > +buf_writer_assert_abi(bwr); > +bwr->buf = buf; > +bwr->size = size; > +bwr->pos = 0; > +buf[0] = 0; > +return bwr; > +} > + > +AVWriter av_buf_writer_wrap(AVBufWriter *bwr) > +{ > +AVWriter r = { av_buf_writer_get_methods(), bwr }; nit+++: wr for consistency [...] > +AVWriter av_dynbuf_writer_wrap(AVDynbufWriter *dwr) > +{
Re: [FFmpeg-devel] [PATCH v14 2/5] compat/w32dlfcn.h: Remove MAX_PATH limit and replace LoadLibraryExA with LoadLibraryExW
> -Original Message- > From: ffmpeg-devel On Behalf Of > nil-admir...@mailo.com > Sent: Wednesday, June 15, 2022 10:00 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v14 2/5] compat/w32dlfcn.h: Remove > MAX_PATH limit and replace LoadLibraryExA with LoadLibraryExW > > > path_size <= INT16_MAX > > > > (the edge case is already covered by the equals) > > Done: https://ffmpeg.org/pipermail/ffmpeg-devel/2022- > June/297590.html. > Don't quite understand what edge case you've meant: > INT16_MAX is 32767, which is the maximal path length allowed, > + 1 is needed for the terminating null. With the +1 your while condition term is effectively path_size <= 32768 But when the path_size is 32768, you do not need to go for another loop with an increased buffer because this is already as large as it can get. There won't be any 32769 or 32770 (...) cases, I think. > > I know this line existed before your path, but it would be nice > > to clarify check and condition, like: > > > > // On Win7 an earlier we check if KB2533623 is available > > Changed the comment. Cool. Thanks. softworkz > > > name_w leaks here > > Fixed. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".