Re: [PATCH v3] drm/modes: Fix division by zero due to overflow

2023-08-03 Thread Jani Nikula
On Wed, 02 Aug 2023, Ziqi Zhao wrote: > In the bug reported by Syzbot, the variable `den == (1 << 22)` and > `mode->vscan == (1 << 10)`, causing the multiplication to overflow and > accidentally make `den == 0`. To prevent any chance of overflow, we > replace `num` and `den` with 64-bit unsigned i

[PATCH v3] drm/modes: Fix division by zero due to overflow

2023-08-02 Thread Ziqi Zhao
In the bug reported by Syzbot, the variable `den == (1 << 22)` and `mode->vscan == (1 << 10)`, causing the multiplication to overflow and accidentally make `den == 0`. To prevent any chance of overflow, we replace `num` and `den` with 64-bit unsigned integers, and explicitly check if the divisor `d