The av_display_rotation_get function previously yielded inconsistent angles for certain reflected (mirrored) matrices. For instance, a matrix representing a -45 degree rotation followed by a horizontal flip resulted in 135 degrees, while the desired interpretation after canonicalizing the reflection is 45 degrees.
This patch revises av_display_rotation_get to: 1. Detect reflections by checking the determinant of the 2x2 transform. 2. If a reflection is detected, it canonicalizes the matrix by effectively negating the components derived from the first column (m0, m3) before calculating the atan2 arguments. This is equivalent to extracting the rotation from M * F_h (where F_h is a horizontal flip). 3. The final negation in the return path is preserved. This change ensures more predictable rotation angle extraction when reflections are involved. For example, a pure horizontal flip now correctly reports 0 degrees. The FATE test in libavutil/tests/display.c has been updated: - The original test sequence now has assertions checking against the new, correct behavior of the patched function. - An additional 8 comprehensive test cases have been added to cover 0, 90, 180, 270 degree rotations, both unflipped and when composed with a horizontal flip. - The FATE reference file for 'fate-display' has been updated to reflect these changes and new test outputs. All tests now pass with this revised logic. Signed-off-by: Syed AbuTalib <low...@google.com>
0001-libavutil-display-improve-av_display_rotation_get-fo.patch
Description: Binary data
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
