Hi Julien,
On Sun, 2012-04-29 at 19:05 +0200, Julien Cristau wrote:
> On Wed, Apr 11, 2012 at 08:07:54 +0200, Tobias Frost wrote:
> > seems that bogofilter can be fixed soon, it seems that Steven found an
> > workaround in the sqlite3 library. (See #665363)
> What's up with that?
The "bug" lies in SQLite3, in commit 2e8ab3cedf [1]. As src/mem1.c
adds malloc_usable_size() to sqlite3MemSize() to get the available
memory to use. On my amd64 system, malloc() calls are rounded up to n*24
bytes and that size may be usable. However as the manpage states:
"Returns the number of bytes available in the dynamically
allocated buffer ptr, which may be greater than the requested
size (but is guaranteed to be at least as large, if the request was
successful). Typically, you should store the requested allocation
size rather than use this function."
So in general nothing is wrong if you use the size reported by this
function. However when you set MALLOC_CHECK_ to 1 or 2, glibc enforces
the requested size. This is where the problem lies. SQLite3 use the
memory normally, a bit larger size than originally requested but not
more than the maximum available. This is normal and doesn't cause memory
corruption. But when asked via the MALLOC_CHECK_ setting, glibc detects
the difference and issue a warning only (=1) or aborts (=2).
Bogofilter asks for this check in src/tests/t.frame in line 173 and 174.
It may be debatable where to fix this. Do not set glibc malloc
enforcement in Bogofilter or disable this memory use in SQLite3 itself.
Let's go on with the latter.
By the way, attached a small example that demonstrates this problem on
64 bit archs. Compile with 'gcc -o check check.c' and run check with
MALLOC_CHECK_ set to 0 and later set to 2.
Regards,
Laszlo/GCS
[1] http://www.sqlite.org/src/info/2e8ab3cedf
#include <stdio.h>
#include <stdlib.h>
#include <malloc.h>
#include <string.h>
int main(void)
{
void *p = NULL;
size_t size = 7;
/* allocate a small size of memory and inform the user */
printf("Size to malloc(): %u\n", size);
p = malloc(size);
/* check how much memory we got */
size = malloc_usable_size(p);
printf("Size reported by malloc_usable_size(): %u\n", size);
/* use that memory */
memset(p, 0x0, size);
/* we don't need the memory anymore */
free(p);
/* just inform the user about the exit */
printf("Program ends normally.\n");
return 0;
}
