Hello Ruiling, > Thanks for sharing your idea with me. I basically also noticed some poor > quality tone mapping result for some hdr stream. > I will try your suggestion to see whether I can make it in good state so I > can include it in next version. > In fact I have not thought detecting scene change quite well. A question for > your idea is: > is it possible that your scene detection heuristic may still failed to detect > some particular scene change and lead to poor tone mapping quality?
The way my scene change detection heuristic works like this: I trigger a scene change (and therefore discard the frame averaging buffer) if the distance between the current frame average brightness and the current running average exceeds a threshold value, that threshold being (by default) 20 cd/m². So we can divide the failures of this algorithm into two categories: 1. False negative (scene change without resetting the buffer): This can only happen if there was an actual scene change but the average brightness change did not exceed 20 cd/m², i.e. the scenes are similar in brightness. I consider this a fairly harmful failure because that also means there's no visual discontinuity since the scenes are so similar to begin with. 2. False positive (buffer got reset without a scene change). This is the more worrying failure of the algorithm, since it can happen in the middle of a scene (e.g. as the result of a bright flash of light on-screen), which will manifest itself as a sudden decrease in the total frame brightness coinciding with the new source of light. (Or vice versa, a sudden increase in brightness coinciding with the sudden absence of a light source). The scene change threshold is a trade-off. Lowering the value decreases the likelihood of #1 but increases the likelihood of #2. Increasing the value decreases the likelihood of #2, but increases the likelihood (and apparent effect) of #1. If you want to optimize or improve the algorithm, the case #2 is the one I would be most interested in, i.e. reducing the rate of false positives. This can surely be done in a smarter way, e.g. by comparing more than just the scene average but also other easily obtained metrics. If you have access to low-level frame information, you could do something like increasing the threshold for non-keyframes significantly, since keyframes are likely to coincide with scene changes or cuts this might help the algorithm out. > I just copy this piece of code from vf_tonemap.c. I think we need to fix it > there first if this is wrong. > I guess the code was like this because we think that all video stream that > truly use ST. 2084 should > Include the mastering display metadata, if it is absent, then transfer > function should be arib-std-b67. Yeah, good point. And ideally, maybe those two should share this logic to avoid code duplication. > Basically I am targeting full gpu pipeline transcoding(gpu decoding + gpu > filtering + gpu encoding), > Most video streams I have encountered are YUV stream. > Could you show me what kind of use-case that need RGB support? > So I will try to see whether adding it in this patch or do it later. One particular use case I have in mind that would be good to support is OpenEXR formats, which are typically RGB and high depth or floating point. > In fact, I have not learned about HLG deeply. I still need some time to add > HLG support. Well the one key thing that's interesting about HLG is that the OOTF (which forms part of the EOTF) is parametrized by the peak brightness of the display. So when tone mapping (which is done in display-referred space) a HLG signal to a lower brightness, the inverse OOTF you need to apply to go back to HLG afterwards needs to be done on the new peak brightness, to reflect the changes made to the signal. As said I'm not sure which component handles the OOTF in FFmpeg land, so if you implement it yourself (which may be a necessary) that would be a thing to keep in mind. > Well, this is very useful reference for me. I need some time to digest it. > Thanks a lot. No problem, Niklas _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
