On Thu, Sep 13, 2007 at 05:25:32PM -0300, Alejandro Lozanoff wrote:
> Hello list,
re
> We recently updated a 3.7 machine running awstat(perl) to parse all our
> websites logs with the biggest being around 1GB.
> When parsing the big log it randomly segfaults on 4.1, 3.9 and 3.8, we
> tried new clean release installs and it still segfaults. On 3.7 it works
> flawlessly, on 3.8 which has the same perl version as 3.7 (5.8.6) it
> still segfaults. The problem is completely random but it tends to happen
> after its been running for a while as it doesnt happen on small logs (or
> the probability for it to happen on those files is too low )
>
> As the trace below (from the perl.core) shows, it's an out of bounds
> problem, then we remember about a change on 3.8:
>
> "malloc(3) has been rewritten to use the mmap(2) system call,
> introducing unpredictable allocation addresses and guard pages, which
> helps in detecting heap based buffer overflows and prevents various
> types of attacks."
yes. this increases memory fragmentation immensly resulting
in (practically) less virtual space available for data.
as an increased penalty (200-300%) for cpu consuption
on processes w/ lots (20M and more) malloc(3)ed memory...
as well increased demand for the physical memory that
on the overcommiting nature of it you perhaps observe.
a way around it is only to use perl malloc (sbrk-based)
cu
--
paranoic mickey (my employers have changed but, the name has remained)
