When bitpos equals sizeof(inflightbits) * 8 (i.e., 32), the shift
'v << s->bitpos' has undefined behavior per C11 6.5.7p3, which states
that if the shift amount is greater than or equal to the width of the
promoted left operand, the behavior is undefined.
The original code used a branchless mask '(!rem - 1)' to zero out the
result when rem == 0, but the undefined shift is still evaluated before
the mask is applied. Replace with a conditional to avoid the shift
entirely when rem is zero.
Found via UBSan (-fsanitize=undefined). Similar to commit 2cd5114
("lib: fix undefined behavior in zstd dict_size bit shift").
Signed-off-by: Utkal Singh <[email protected]>
---
lib/kite_deflate.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/kite_deflate.c b/lib/kite_deflate.c
index f9eb3fb..29e44b3 100644
--- a/lib/kite_deflate.c
+++ b/lib/kite_deflate.c
@@ -144,7 +144,7 @@ static void writebits(struct kite_deflate *s, unsigned int
v, u8 bits)
{
unsigned int rem = sizeof(s->inflightbits) * 8 - s->bitpos;
- s->inflightbits |= (v << s->bitpos) & (!rem - 1);
+ s->inflightbits |= rem ? (v << s->bitpos) : 0;
if (bits > rem) {
u8 *out = s->out + s->pos_out;
--
2.43.0
