Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-14 Thread Boysenberry Payne
On May 11, 2007, at 12:05 PM, Michael Peters wrote: This kind of thing happens all the time. Think of SAX XML parsers or mod_perl filters. It's not terribly difficult to parse something in chunks like that. I wasn't saying that it wouldn't be easier to have everything in memory. Heck I'

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-12 Thread Lionel MARTIN
On 5/12/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: Of course, I understand why $tmp value is reset during each call (the lexical variable goes out of scope) but, as we can see, the address used by the variable is changing as well during each request, and this fact doesn't seem consistent w

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-12 Thread Perrin Harkins
On 5/12/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: Of course, I understand why $tmp value is reset during each call (the lexical variable goes out of scope) but, as we can see, the address used by the variable is changing as well during each request, and this fact doesn't seem consistent with w

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Clinton Gormley
> #** > use strict; > package mypack; > > my $tmp; > $tmp .= 'a'; > print '$tmp value: ' . $tmp . ' - address: ' . \$tmp . "\n"; > #** > > I am running this script in a ModPerl::Registry environment. > > In this sit

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Lionel MARTIN
Hi all (and Perrin), I understand the concepts being discussed here. However, no on replied one of my questions: Let's conside this script: #** use strict; package mypack; my $tmp; $tmp .= 'a'; print '$tmp value: ' . $tmp . ' - address: ' . \$tmp . "\n"

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Joe Schaefer
<[EMAIL PROTECTED]> writes: > Why am I getting these emails even after unsubscribing? I unsubscribed this person. -- Joe Schaefer

RE: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Kiran.Nadgir
Why am I getting these emails even after unsubscribing? -Original Message- From: [EMAIL PROTECTED] on behalf of Perrin Harkins Sent: Fri 5/11/2007 8:53 PM To: James. L Cc: modperl@perl.apache.org Subject:Re: After retrieving data from DB, the memory doesn't se

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Perrin Harkins
On 5/11/07, James. L <[EMAIL PROTECTED]> wrote: sorry, i wasn't clear. I understand that memory allocated by a lexical variable(not reference) can be released to perl. as described by the practical modperl link you posted. I would expect that both behave the same however you are saying a referen

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread James. L
--- Perrin Harkins <[EMAIL PROTECTED]> wrote: > On 5/11/07, James. L <[EMAIL PROTECTED]> wrote: > On 5/10/07, Andy Armstrong <[EMAIL PROTECTED]> wrote: > > Unless I'm misunderstanding you that's not true. > When a value's > > reference count goes to zero the memory is freed - > at least to Perl

RE: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Kiran.Nadgir
bject: Re: After retrieving data from DB, the memory doesn't seem to be freed up On 5/11/07, James. L <[EMAIL PROTECTED]> wrote: > hm. lexical variable release memory to perl. but > reference doesn't.. No, that isn't what I was saying, and you're mixing separate c

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Perrin Harkins
On 5/11/07, James. L <[EMAIL PROTECTED]> wrote: hm. lexical variable release memory to perl. but reference doesn't.. No, that isn't what I was saying, and you're mixing separate concepts together. A lexical variable can be a reference. You can have a reference to a lexical variable. They are

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread James. L
--- Perrin Harkins <[EMAIL PROTECTED]> wrote: > On 5/10/07, Andy Armstrong <[EMAIL PROTECTED]> wrote: > > Unless I'm misunderstanding you that's not true. > When a value's > > reference count goes to zero the memory is freed - > at least to Perl > > if not to the OS. > > No, it's not. I know it'

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Andy Armstrong
On 11 May 2007, at 19:07, Lionel MARTIN wrote: -another example hat comes to my mind is a project (implemented in PHP) where I had to work. Part of the process was retrieving cached HTML template pages from the server and do BB tags parsing before serving the client. Of course, you would

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Lionel MARTIN
-another example hat comes to my mind is a project (implemented in PHP) where I had to work. Part of the process was retrieving cached HTML template pages from the server and do BB tags parsing before serving the client. Of course, you would tell me that I could retrieve the data chunk by c

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Lionel MARTIN
(sorry, I posted the message below from another email address hours ago, and it didn't get publish - see below if anyone could reply) On May 10, 2007, at 6:52 PM, Andy Armstrong wrote: On 10 May 2007, at 23:48, Jonathan Vanasco wrote: that also means that variables are sticky -- if you chang

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Clinton Gormley
> That's not really large data -- you're talking about dealing with > 10-300k per request ( it should never go beyond that, because you'd > be chunking stuff off the db for ease of download to the end user ). > > I've been under the impression ( and I'd imagine that others on this > list ar

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Jonathan Vanasco
On May 11, 2007, at 12:57 PM, Lionel MARTIN wrote: I don't have any clear situations right here in mind, but we could imagine many: -for example, a bulletin board system, where you are retrieving posted message from a DB. Each message could weigh several dozens of kilo. (especially if you

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Michael Peters
Lionel MARTIN wrote: > >> Lionel MARTIN wrote: >> - Don't load large amounts of data into scalars. >>> Fine. Now I know why. But sometimes, you don't have the choice. >> >> I'd like to know what situations you encounter where you are forced to >> load >> large amounts of data into scalars. I

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Andy Armstrong
On 11 May 2007, at 17:57, Lionel MARTIN wrote: Lionel MARTIN wrote: - Don't load large amounts of data into scalars. Fine. Now I know why. But sometimes, you don't have the choice. I'd like to know what situations you encounter where you are forced to load large amounts of data into scala

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Lionel MARTIN
Lionel MARTIN wrote: - Don't load large amounts of data into scalars. Fine. Now I know why. But sometimes, you don't have the choice. I'd like to know what situations you encounter where you are forced to load large amounts of data into scalars. I can't think of any. I don't have any cl

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Michael Peters
Lionel MARTIN wrote: > I'll see what I can do when the problem arises, but of course, I now > know I should better avoid loading too much data in RAM at once, because > of data persistance. Not to be too picky, but just so it's clear to everyone and to future readers of the archives: The issue w

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Lionel MARTIN
Hi again, If you have a lot of scripts that, for example, load large data files, make a module with a sub that you call to load the files and use it from all your scripts. Pass the data by reference to the calling scripts. Then you will isolate the large lexical in that one module. Better ye

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Perrin Harkins
On 5/11/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: For example, instead of using a "my $bigdata" in 50 different script, that could this result in big memory allocation, I would use a unique $mypack::bigdata in each script, so that each script would share the same memory address for this potent

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Michael Peters
Lionel MARTIN wrote: >> - Don't load large amounts of data into scalars. > Fine. Now I know why. But sometimes, you don't have the choice. I'd like to know what situations you encounter where you are forced to load large amounts of data into scalars. I can't think of any. -- Michael Peters Deve

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-11 Thread Lionel MARTIN
Hi Perrin, No, if the message you're getting is to use globals instead of lexicals then you've got it all wrong. - Use lexicals for everything, unless you actually want the value to persist across requests. I was thinking, for large amounts of data, about using globals. Indeed, imagine you hav

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Lionel MARTIN
I would even say PER THREAD or PER PERL INTERPRETER. Indeed, I'm running Per/mod perl under Windows, and there's one unique Apache process (except the parent one) hosting all perl interpreters. Something to ask about modperl and memory in Windows. I know modperl uses threads in Windows. But do

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Perrin Harkins
On 5/10/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: If my website is made of hundreds of different pages, then, I should better ensure that lexical variables are properly freed up. Even if that's not good programming practive, at least, with global variables, we are sure that the same memory lo

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Foo JH
I would even say PER THREAD or PER PERL INTERPRETER. Indeed, I'm running Per/mod perl under Windows, and there's one unique Apache process (except the parent one) hosting all perl interpreters. Something to ask about modperl and memory in Windows. I know modperl uses threads in Windows. But

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Jonathan Vanasco
On May 10, 2007, at 6:52 PM, Andy Armstrong wrote: On 10 May 2007, at 23:48, Jonathan Vanasco wrote: that also means that variables are sticky -- if you change something from $a= "b" to $a.= "b" , you'll see the var appending b on every iteration No you don't: sub frog { my $x;

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Lionel MARTIN
Thanks for this. well, to complicate things, this behavior is PER CHILD... meaning: if you run test1.pl 2x and they are both served by the same apache child, then the memory will be reused. if you run test1.pl 2x and they are both served by the different apache children, then the memory wi

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Andy Armstrong
On 10 May 2007, at 23:48, Jonathan Vanasco wrote: that also means that variables are sticky -- if you change something from $a= "b" to $a.= "b" , you'll see the var appending b on every iteration No you don't: sub frog { my $x; $x .= 'x'; return $x; } print frog(),

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Jonathan Vanasco
On May 10, 2007, at 6:25 PM, Lionel MARTIN wrote: So, to sum up, if I have got 10 different scripts in a mod perl environment (let's call them test1.pltest10.pl), and using lexical variables there. If I first run test1.pl and then, run test2.pl, the only way for test2.p to get access to

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Lionel MARTIN
- Original Message - From: "Andy Armstrong" <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTED]> Cc: Sent: Friday, May 11, 2007 12:34 AM Subject: Re: After retrieving data from DB, the memory doesn't seem to be freed up On 10 May 2007

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Lionel MARTIN
MAIL PROTECTED]> Cc: Sent: Friday, May 11, 2007 12:29 AM Subject: Re: After retrieving data from DB, the memory doesn't seem to be freed up On 5/10/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: So, to sum up, if I have got 10 different scripts in a mod perl environment (let&

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Andy Armstrong
On 10 May 2007, at 23:25, Lionel MARTIN wrote: OK, fine. So, to sum up, if I have got 10 different scripts in a mod perl environment (let's call them test1.pltest10.pl), and using lexical variables there. If I first run test1.pl and then, run test2.pl, the only way for test2.p to get

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Perrin Harkins
On 5/10/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: So, to sum up, if I have got 10 different scripts in a mod perl environment (let's call them test1.pltest10.pl), and using lexical variables there. If I first run test1.pl and then, run test2.pl, the only way for test2.p to get access to th

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Lionel MARTIN
unning the script twice in a mod perl environment)? Lionel. - Original Message - From: "Perrin Harkins" <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTED]> Cc: Sent: Friday, May 11, 2007 12:17 AM Subject: Re: After retrieving data from DB, the memor

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Perrin Harkins
On 5/10/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: > There's really no difference in the way globals behave under mod_perl. > You just don't notice it under CGI because the process quits right > after the request has been served. [...] So, this clearly shows that the global variables sticks i

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Andy Armstrong
On 10 May 2007, at 22:59, Lionel MARTIN wrote: I would have believed the same, and that's why I believed that $tmp = [0..10]; followed by $tmp = 1; would free memory (no more reference to the anonymous array), but Perrin is telling me this is not the case. Perl hangs on to the memory u

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Andy Armstrong
On 10 May 2007, at 23:00, Tom Schindl wrote: Right: http://www.nntp.perl.org/group/perl.perl5.porters/2006/03/ msg111095.html Yeah, that's lexicals - not things referred to by lexicals. The example was a reference to an array. -- Andy Armstrong, hexten.net

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Lionel MARTIN
-I'll try to use, in my Perl scripts, lexical variables instead of global variables, so that I'm sure the used memory can be resued for later requests (global variables, on the other hand, stick in memory, due to mod perl way of operating) Not really. In terms of memory, there's no differenc

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Lionel MARTIN
When the array goes out of scope its reference count is decremented. If the reference count goes to zero (implying there are no more references) the memory is released. I would have believed the same, and that's why I believed that $tmp = [0..10]; followed by $tmp = 1; would free memor

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Tom Schindl
Right: http://www.nntp.perl.org/group/perl.perl5.porters/2006/03/msg111095.html Tom Perrin Harkins schrieb: On 5/10/07, Andy Armstrong <[EMAIL PROTECTED]> wrote: Unless I'm misunderstanding you that's not true. When a value's reference count goes to zero the memory is freed - at least to Perl

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Perrin Harkins
On 5/10/07, Andy Armstrong <[EMAIL PROTECTED]> wrote: Unless I'm misunderstanding you that's not true. When a value's reference count goes to zero the memory is freed - at least to Perl if not to the OS. No, it's not. I know it's counter-intutive, but this is a feature of Perl, and the behavio

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Perrin Harkins
On 5/10/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: -I'll try avoid using large chunks of data so that interpreters memory footprint don't grow too big. Yes. -I'll try to use, in my Perl scripts, lexical variables instead of global variables, so that I'm sure the used memory can be resued fo

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Andy Armstrong
On 10 May 2007, at 22:30, Lionel MARTIN wrote: No. Those are two different things. If you explicitly undef it, the memory gets handed back to Perl: undef $foo; If it just goes out of scope, the memory stays allocated to that variable. Unless I'm misunderstanding you that's not true. When

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Lionel MARTIN
D]> Cc: Sent: Thursday, May 10, 2007 7:57 PM Subject: Re: After retrieving data from DB, the memory doesn't seem to be freed up On 5/10/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: On Windows XP, it seems to me that Perl never gves back memory to the OS, even after I undef variable

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Perrin Harkins
On 5/10/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: On Windows XP, it seems to me that Perl never gves back memory to the OS, even after I undef variables or after they go out of scope. That's pretty common. Perl will be able to use the memory for other variables though. But then, I'm wonde

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Lionel MARTIN
ng large amounts of data, I should undef if possible the variables before allocating others, so that processes don't grow too big? Thanks, Lionel. - Original Message - From: "Perrin Harkins" <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTE

Re: After retrieving data from DB, the memory doesn't seem to be freed up

2007-05-10 Thread Perrin Harkins
On 5/10/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: I suspect that this is coming from data retrieved from my DB not being well freed. That's right. There are two things you need to understand here. The first is that Perl doesn't free memory from variables unless you tell it to, even when th