Hi. Niklas Haas via ffmpeg-devel (HE12025-08-20): > I accidentally deleted a line too many here, the text was supposed to read: > > Do you expect filter authors to remember to enable support for > premultiplied alpha, even if they don't even touch the alpha plane?
Hi. It is a nice coincidence that these two lines summarize perfectly the mistake you make that is leading you to the wrong conclusion. Therefore, I will only reply to this, addressing other points of your mail as necessary. The filters that do not even touch the alpha plane are the most likely to produce invalid garbage with premultiplied alpha. They should NOT enable support. Let me explain. First, by garbage, I mean: (1) not what the user expects + (2) not what the documentation of the filters says + (3) not the same result as if the processing were done in full range and conversion to premultiplied alpha was done afterwards (and not just because of different rounding). And by invalid garbage I mean cases where the value of a component ends up greater than the value of alpha. Processing frame data is written based on several assumptions of linearity and independence: filters assume that the same (R,G,B) or (Y,U,V) triplet corresponds to the same color, even if not taking A into account; filters assume that averaging colors is done by averaging each component, etc. If we were talking about change in color primaries, then all would be fine because these assumptions still hold for different primaries: it is just a change of basis in the space of colors with the alpha direction a stable subspace. (The issue of gamma breaks all these assumptions, but the consensus on the issue of gamma seems to be to just ignore it and have the habit to compensate for it. If we were adding “float gamma” to AVFrame I would insist on the same caution.) But premultiplied alpha is not just a change of basis, it is not linear. Let us just take an example: a filter will average two pixels, one very dark red opaque and one bright red very transparent. Full range input: (0.2, 0, 0, 1) and (1, 0, 0, 0.2) Average component by component: (0.6, 0, 0, 0.6) Conversion to premultiplied: (0.2, 0, 0, 1) and (0.2, 0, 0, 0.2) Average component by component: (0.2, 0, 0, 0.6) As you cans see, this not the same result. The same result would be expressed as (0.36, 0, 0, 0.6) in premultiplied. This gave us 0.2/0.6 =~ 0.33 red instead of 0.6 red. That was for filters that do the average on all components, including alpha. Since the space of valid components value with premultiplied alpha is a pyramid, convex, the result is garbage but not invalid garbage. Now let us look at would produce a filter that does not take alpha into account at all. The computations are the same, the filter will get (0.2, 0, 0) for the average, and it will store it in a pixel, not touching the alpha value for that pixel. And this is where garbage becomes invalid: if the target pixel had alpha 0.1, then it now has (0.2, 0, 0, 0.1), which is invalid. I took average, but the issue arises for all kinds of operations: every time the values of multiple pixels are mixed together, the proper way to compute it in premultiplied is not the usual way. It also applies to filters that do not draw pixels but only compute statistics: they will assume a pixel is dark when it is just transparent. This is not theoretical. Remember: you had to update drawutils, because you could see, with your own eyes, that filters using drawutils produced garbage with premultiplied. Do you really think drawutils was the only code that produces garbage with premultiplied? Drawutils is just sharing code between many filters that have needs similar enough. If a filter has needs that are not covered by drawutils, it will use its own code, but the same kind of code as drawutils. Code that will need fixing too. The only filters we can be sure will work with premultiplied are (1) the filters that implement formulas specific to the premultiplied case, (2) filters that only copy pixel values, all four components, and never do any kind of computation on them and (3) filters that do not touch the pixels at all, only the frame properties and metadata. (Technically, (3) is a subset of (2).) Most filters are in none of these cases. This is why I insist support must be opt-in, white list in your own words. Also, please remember this: with the work you did to make it properly part of the negotiation, automatic conversion filters are inserted automatically. If we forget to flag a filter that works, the annoyance will be minimal. And we can always flag it later, whereas garbage already written cannot be fixed. (I want to emphasize that you did the work of making it part of the negotiation not on my demand. It is useful, there is no doubt about, and much better than the quick and dirty version I sent, and it is a good thing you did it. But it was not on my demand and it does not change the fact that support must be opt-in.) Regards, -- Nicolas George _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
