On Wed, Oct 13, 2010 at 04:41, Panda-X <exilepa...@gmail.com> wrote:
snip
> From my observation, the Out of Memory error could happen anytime, where
> the "MEM" size was never over 10K.
>
> Any clues ?
snip
Then the problem is unlikely to be that message. The two most common
things that will cause you to run out of memory are slurping a massive
file into memory and cyclic references. You can avoid the first by
always processing files line by line. The second is more complicated.
Perl 5 uses reference counting for garbage collection. When a
scalar, hash, or array has no references it is eligible for garbage
collection. Here is an example:
{
my $foo = 42; #creating and naming the variable causes the
refcnt to go to 1
}
#the scope ends destroying the name $foo, this causes the refcnt
#to go down 1 to 0, and the contents are garbage collected
my $ref;
{
my $foo = 42; #refcnt = 1
$ref = \$foo; #taking a reference bumps the refcnt by 1 to 2
} #the name $foo goes out of scope so refcnt goes down to 1
$ref = undef;
#the reference to the scalar goes away taking the
#the refcnt down to 0 and garbage collection occurs
Reference counting works really well, but there is possible to shoot
yourself in the foot:
{
my ($x, $y); # x's and y's refcnt is now 1
$x = \$y; #y's refcnt is 2
$y = \$x; #x's refcnt is 2
}
#the names $x and $y go out of scope bring the refcnt for each to 1
#but nothing can access those scalars anymore, so the memory for
#each will stay allocated until the end of the program
This is one form of cyclic reference. Finding cycles can be hard.
The first step is to look at all of your data structures with
[Devel::Cycle][0]. If you find a cycle you can weaken a reference in
the cycle with [weaken][1] from Scalar::Util. Weakened references
don't increase the reference count, for example:
use Scalar::Util qw/weaken/;
{
my ($x, $y); # $x and $y have a refcnt of 1
$x = \$y; # $y has a refcnt of 2
$y = \$x; # $x has a refcnt of 2
weaken $y; #$x has a refcnt of 1, but the reference still works
}
#the names $x and $y go out of scope taking the refcnts for the scalars
#associated with $x and $y to refcnts of 0 and 1 respectively. This makes
#the scalar associated with $x eligible for garbage collection. The
#garbage collection destroys the reference to the scalar associated with
#$y, so its refcnt goes down to 0, which makes it eligible for garbage
#collection as well.
[0]: http://search.cpan.org/dist/Devel-Cycle/lib/Devel/Cycle.pm
[1]: http://perldoc.perl.org/Scalar/Util.html#weaken-REF
--
Chas. Owens
wonkden.net
The most important skill a programmer can have is the ability to read.
--
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/
