Hello, I recently asked for some help as I got a problem when using md5_process_bytes (in libiberty/md5.c): http://gcc.gnu.org/ml/gcc-help/2011-09/msg00126.html, http://gcc.gnu.org/ml/gcc-help/2011-09/msg00127.html and it appears that there is a bug in md5_process_bytes.
The bug can conduct to a miscomputed md5 result. It tooks time to me to make the bug reproducible but I was finally able to do so. The fact is that it only appears in very particular situation. I have written a small gcc plugin, allowing to reproduce it (see attachment). The bad news is that the bug only appears when use libiberty compiled in -g -O0 (it works well with -O2). It is quite sad, because It could means another bug in an optimization function. I have attached a README which detail how to use the plugin and how to explain the bug. I have tried to explain as good as possible (and I apologize for my very bad english). The bug appears when: 1) We use libiberty compiled with -O0 2) We first call md5_process_bytes with a less than 64 bits buffer (we call his size len1). 3) We make a new call of md5_process_bytes with a buffer which has a size len2 such as: len2 > 127 + 65 (so test in line 228 of md5.C will be true) 128 -len1 != Mulint with Mulint % __alignof__ (md5_uint32) != 0 (so condition on line 238 is true) len2 - (128 - len1) = Mul64 and Mul64 such as Mul %64=0 (so the loop of line 239 is broken with len = 64, this leads to the bug as, line 249, (len & ~63) = 64 and we shift the buffer without processing the data). Please, can you reproduce the bug? Is there any useful informations I can add? Must I contact somebody from libiberty (I don't know the status of this library (is this part of gcc or from another project?)). I already sent a patch correcting this issue (it does not correct the fact that we don't get the bug with an optimised libiberty): http://gcc.gnu.org/ml/gcc-patches/2011-09/msg01098.html. It has not been reviewed, could someone reviews this? Thanks! Pierre Vittet
md5sum_plugin.tar.gz
Description: application/gzip