On Sat, Jul 2, 2022 at 7:32 PM Vignesh Venkatasubramanian <vigneshv-at-google....@ffmpeg.org> wrote: > > On Sat, Jul 2, 2022 at 2:35 AM Anton Khirnov <an...@khirnov.net> wrote: > > > > Quoting Vignesh Venkatasubramanian (2022-06-30 23:04:34) > > > Parse the alpha channel for still AVIF images and expose it as a > > > separate track. This is the simplest way of supporting AVIF alpha > > > channel in a codec independent manner (similar to how ffmpeg > > > supports animated AVIF with alpha channel). > > > > > > One can use the alphamerge filter to get a transparent image with > > > a single command. For example: > > > > > > ffmpeg -i image_with_alpha.avif -filter_complex alphamerge > > > image_with_alpha.png > > > > > > Signed-off-by: Vignesh Venkatasubramanian <vigne...@google.com> > > > --- > > > libavformat/isom.h | 1 + > > > libavformat/mov.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++ > > > 2 files changed, 67 insertions(+) > > > > I am against this patch, this is a digusting hack. > > > > These are not two streams, it is a single stream and should be exposed > > as such. > > > > Yes, while it is a hack, it is also the simplest way of supporting > AVIF images with alpha. Since AVIF alpha images require two separate > decoders, i could not think of any other solution that does not > involve modifying the underlying av1 codec itself. > > If there are any alternative solutions where we can expose a single > stream that needs two codecs, then i am open to implementing that. > > Otherwise, this solution improves the current situation where there is > no way for the user to extract the alpha channel (albeit in a hacky > way).
I have discussed this on IRC, and as far as I can see we have multiple cases where an additional image has to be decoded by a separate decoder context to get alpha, namely: - vp8/9 (currently handled by AV_PKT_DATA_MATROSKA_BLOCKADDITIONAL) - HEIF-like formats (such as AVIF), which are codec agnostic (you can put JPEG, HEVC, AV1 etc into HEIF) This means the logic (preferably) should not be codec-specific, and would allow to implement at least two separate My proposal - which Anton didn't really like - was to add a side data entry AV_PKT_DATA_AUXILIARY_IMAGE , which could have type (ALPHA, DEPTH etc) as well as the relevant packet(s) - as well as if any of specifications define codec initialization data, that. Then libavcodec/decode or so could initialize a secondary decoder context for alpha images and add the third plane if parameters match what's required. This would also make remux possible, since containers that do not support such auxiliary images would ignore the alpha plane, and those that do would properly pass it through. As for encoding, not fully sure how it should be integrated, if any encoders actually at this moment do proper alpha coding, or do all API clients have to separately encode with one context the primary image, and the alpha with another? That said, the multi-stream implementation is already merged on the muxer side, so that can be utilized in the mean time, even though the auxiliary images are not technically a separate stream within HEIF-likes. Jan _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
