> The line in $evalstr that causes the stack fault is:
>
> if ($self->{conf}->{scores}->{q{WLS_URI_OPT_377}}) {
>
> Which is on line 21720 (out of ~30800 lines). From what I've gathered,
each
> rule in the .cf files results in 4 lines being strung together in
$evalstr:
>
> if ($self->{conf}->{scores}->{q{<RULE_NAME>}}) {
> <RULE_NAME>_uri_test($self, @_); # call procedurally for speed
> }
>
> It doesn't matter which rule is on line 21720 (as I've changed them around
> to make sure it wasn't specifically WLS_URI_OPT_377, from
> sa-blacklist.current.uri.cf), It seems when you get to line 21720 w/ a
64MB
> stack on FreeBSD 5.4 w/ perl 5.8.7 etc, etc. that you've ran out of stack
> space. This is part of the gdb bt from a perl5.8.7.core that I had:
>
> #0 0x280a2144 in Perl_malloc (nbytes=25) at malloc.c:1411
> #1 0x281075b5 in S_save_hek_flags (str=0x935b328 "WLS_URI_OPT_377",
len=15,
> hash=1020016264, flags=0) at hv.c:97
etc.
Ok. This sounds to me like a perl bug that can probably be worked around.
It is dying in malloc, apparently trying to malloc heap data on the stack,
which gcc tends to do a lot. Since it happens on the same line in the file
regardless of which rule is there, it is happening at the same 'call a rule'
number in the procedure containing the calls to the rules.
So the first useful question is to convert that line number to a rule call
number in the containing procedure. Subtract the line number for the first
one of those 'if ($self...' lines from the failing line, and then divide by
4 to convert line numbers into rule counts. (Assuming there are 4 lines
including a blank line, else divide by 3.)
Once that number is known, I suspect the trivial (but somewhat ugly) change
is to maintain a count in sa of the number of rules being generated, and
split the rule calling procedure into multiple procedures every N rules,
where N is some small number (16 or so) less than the number you compute
above. Then generate a final wrapper procedure that will call all of these
split procedures.
The end result will be one extra level of call to evaluate the rules, but
the procedure flipping will probably be fairly rare (I'm assuming there will
be several hundred rules per sub-procedure), so there won't be any noticible
change in performance. And the erroneous heap buildup will disappear each
time one of the sub-procedures is exited.
A possible second workaround is to try to figure out what hash it is that
Perl seems to be setting flags in as it is calling the rules. I don't see
it obviously in the code; but then I don't really read Perl, and it may be
obvious to someone else. If this is obvious, there may be some trivial code
formatting change that will unconvince Perl that it needs to be mucking with
the hash in the first place.
And of course I'd be inclined to supply the 1MB file as a bug attachment to
perl, explaining that it blows stack on BSD and works fine everywhere else.
However, that is likely a longer-term fix than changing the rule generating
code to build a second level of calling procedure and segment the rule calls
into multiple procedures.
Loren
