> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-boun...@ffmpeg.org> On Behalf Of > Clément Bœsch > Sent: Monday, October 31, 2022 1:30 AM > To: FFmpeg development discussions and patches <ffmpeg- > de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] Revert > "avfilter/vf_palette(gen|use): support palettes with alpha" > > On Sun, Oct 30, 2022 at 10:55:31PM +0000, Soft Works wrote: > [...] > > > I understand why. I know that it's not perfect. But it's the best > > > what's achievable within the way the filter is working. > > > > > > But I wouldn't go that far as saying it would be "broken". I > think > > > the result is quite acceptable with regards to the method being > > > using. > > It's broken because the alpha channel in the output is really > completely > random. If we blend the output of that "O" PNG somewhere, it's going > to be > a real mess. > > Here is a more concrete example: if your input has some red fully > transparent (00ff0000), and some red fully opaque (ffff0000) which > end up > in the same box, they will be averaged to a red with an alpha of > 0x80, and > I'm not even accounting for the weight and other colors with > different > transparency. That non-opaque average alpha might end up being used > in > area that are expected to be opaque, and in some area where it's > supposed > to be transparent. That's exactly what I showed with the "O" png. > > In addition to that problem, since you're also accounting the alpha > as a > weight to determine the proximity of 2 colors, you're going to select > the > wrong colors. For example if we want to find the closest color to an > opaque green (ff00ff00), and the palette has a slightly transparent > green > (fa00ff00) and an opaque blue (ff0000ff), then now the algorithm will > prefer the blue over the green. > > That explains why in addition to the alpha being random in the "O" > png, > the colors are also messed up. > > > > The patch I had submitted doesn't change the previous behavior > > > without the use_alpha parameter. > > Yes I noticed, but unfortunately I'm reworking the color distance to > work > in perceptual color space, and the way that alpha is mixed up in the > equation just doesn't make any sense at all and prevents me from > doing > these changes.
If you want to implement a new color distance algorithm, it should be either a new filter or a new (switchable) mode for the existing filter. Photoshop has these different modes as well and it would surely be useful, but I don't think it should be replacing the existing behavior. When it turns out that the use_alpha implementation doesn't fit with your new color distance calculation and you add it as an additional mode, then it would be fine IMO when the filter errors in case it would be attempted to use that mode in combination with use_alpha. > > Do you think it might make sense to put more weight on the > > alpha value by tripling it? So it would be weighted equally to the > > RGB value? > > You cannot mix alpha with colors at all, they are separate domains > and you > need to treat them as such. What's interesting is that I've followed the same (simplified) way for adding a use_alpha option to vf_elbg and it provides excellent results without treating alpha separately. > From paletteuse perspective what you need to do is first choose the > colors > in the palette that match exactly the alpha (or at least the closest > if > and only there is no exact match). Then within that set, and only > within > that one, you'd pick the closest color. > > From palettegen perspective, you need to split the colors in > different > transparency domain (a first dimensional quantization), then quantize > the > colors in each quantized alpha dimension. And when you have all your > quantized palettes for each level of alpha, you find an algorithm to > reduce the number of transparency dimensions or the number of colors > per > dimension to make it fit inside a single palette. But you can't just > do > the alpha and the colors at the same time, it cannot work, whatever > weights you choose. I would be curious to see how well that would work, especially in cases when the target palettes have just a few number of colors. But to return to the proposal of removal: If everything from ffmpeg would be removed which is not perfect, then it would be lacking quite a number of features I suppose :-) In the same way, one could say that palettegen/-use should be removed because its results are wrong and colors are randomly mixed and misplaced while the vf_elbg filter does it right. When you look at the result under the heading "Paletteuse/gen Regular (to 8-bit non-alpha palette; only single transparent color)" https://gist.github.com/softworkz/deef5c2a43d3d629c3e17f9e21544a8f?permalink_comment_id=3905155#gistcomment-3905155 Even without the alpha, many color pixels appear to be wrong and random like for example the light purple pixels on the darker purple at the bottom of the "O". That's not much different from irregularities in the alpha channel you've shown (https://imgur.com/a/50YyRGV). So, I agree to that it's not perfect, but the whole filter is not perfect (vf_elbg is close-to). Do we remove the filter because it's not perfect? As mentioned above, if you want to add an additional mode for calculating the color distance, it's fine when it doesn't work with use_alpha IMO. Thanks, softworkz _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
