Re: [GENERAL] plperl doesn't release memory

2005-04-02 Thread GIROIRE Nicolas (COFRAMI)
Title: RE: [GENERAL] plperl doesn't release memory The function is a little big, so i put it in an enclosure file. The function write regularly in a file to empty the variable theClob. The fact the written is done all 10 has no signification, it's just to empty theClob before

Re: [GENERAL] plperl doesn't release memory

2005-03-31 Thread Dan Sugalski
Title: Re: [GENERAL] plperl doesn't release memory At 8:38 AM +0200 3/31/05, GIROIRE Nicolas (COFRAMI) wrote: Hi, I work with William. In fact, we have already done the procedure in pl/pgsql but it is too slow and we use array which are native in perl. The procedure is recursive, an

Re: [GENERAL] plperl doesn't release memory

2005-03-31 Thread Harald Fuchs
In article <[EMAIL PROTECTED]>, Martijn van Oosterhout writes: > Perl uses reference counting, so as long as a string is visibile > anywhere (remember closures), it stays around and disappears as soon as > it's unreferenced. > If you have large strings or arrays you don't need, maybe you need to

Re: [GENERAL] plperl doesn't release memory

2005-03-31 Thread GIROIRE Nicolas (COFRAMI)
Title: RE: [GENERAL] plperl doesn't release memory Another solution would be to use pl/python, but i don't know anythig in this language.   Is a solution viable ? Can pl/python replace pl/perl without losing performance and use sort under an array ? Are the array native in python

Re: [GENERAL] plperl doesn't release memory

2005-03-31 Thread Martijn van Oosterhout
On Thu, Mar 31, 2005 at 08:38:09AM +0200, GIROIRE Nicolas (COFRAMI) wrote: > Can we oblige pl/perl to free memory for variable ? > Or can we configure postgresql to accept this rise in load ? > Or another idea ? Perl uses reference counting, so as long as a string is visibile anywhere (remember c

Re: [GENERAL] plperl doesn't release memory

2005-03-30 Thread GIROIRE Nicolas (COFRAMI)
Title: RE: [GENERAL] plperl doesn't release memory Hi, I work with William. In fact, we have already done the procedure in pl/pgsql but it is too slow and we use array which are native in perl. The procedure is recursive, and use request on postgreSQL. According to the evolution of m

Re: [GENERAL] plperl doesn't release memory

2005-03-30 Thread Sean Davis
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] la part de Dan Sugalski Envoyé : vendredi 25 mars 2005 19:34 À : Greg Stark; pgsql-general@postgresql.org Objet : Re: [GENERAL] plperl doesn't release memory At 6:58 PM -0500 3/24/05, Greg Stark wrote: >Dan Sugalski <[EMAIL PROTECTED]> writes

Re: [GENERAL] plperl doesn't release memory

2005-03-30 Thread FERREIRA William (COFRAMI)
Title: RE: [GENERAL] plperl doesn't release memory i have a similar problem i'm running PostgreSQL on a PIV with 1GO and Windows 2000 NT i have a large database and a big traitment taking more than 4 hours. during the first hour postgresql use as much memory as virtual memory and i

Re: [GENERAL] plperl doesn't release memory

2005-03-25 Thread Dan Sugalski
At 6:58 PM -0500 3/24/05, Greg Stark wrote: Dan Sugalski <[EMAIL PROTECTED]> writes: Anyway, if perl's using its own memory allocator you'll want to rebuild it to not do that. You would need to do that if you wanted to use a debugging malloc. But there's no particular reason to think that you sh

Re: [GENERAL] plperl doesn't release memory

2005-03-24 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > Two mallocs can work fine alongside each other. They each call mmap or > sbrk to allocate new pages and they each manage the pages they've > received. They won't have any idea why the allocator seems to be > skipping pages, but they should be careful not to

Re: [GENERAL] plperl doesn't release memory

2005-03-24 Thread Greg Stark
Dan Sugalski <[EMAIL PROTECTED]> writes: > Anyway, if perl's using its own memory allocator you'll want to rebuild it > to not do that. You would need to do that if you wanted to use a debugging malloc. But there's no particular reason to think that you should need to do this just to work properl

Re: [GENERAL] plperl doesn't release memory

2005-03-24 Thread Sven Willenberger
On Thu, 2005-03-24 at 15:52 -0500, Tom Lane wrote: > Sven Willenberger <[EMAIL PROTECTED]> writes: > > Yes, on these systems, perl was build with -Dusemymalloc (and > > concurrently -Ui_malloc) so there could very well be an issue with > > malloc pools going awry. Doing a quick dig reveals that Lin

Re: [GENERAL] plperl doesn't release memory

2005-03-24 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > (Come to think of it, doesn't Perl normally use its very own private malloc? > Maybe there's an issue right there ...) Perl can be built either way. It should work to have two different malloc's running side by side as long as the correct free() is always c

Re: [GENERAL] plperl doesn't release memory

2005-03-24 Thread Tom Lane
Sven Willenberger <[EMAIL PROTECTED]> writes: > Yes, on these systems, perl was build with -Dusemymalloc (and > concurrently -Ui_malloc) so there could very well be an issue with > malloc pools going awry. Doing a quick dig reveals that Linux tends to > build perl (by default) with the system mallo

Re: [GENERAL] plperl doesn't release memory

2005-03-24 Thread Dan Sugalski
At 3:14 PM -0500 3/24/05, Tom Lane wrote: Dan Sugalski <[EMAIL PROTECTED]> writes: ... I'm pretty sure for embedding use it uses whatever routines the embedder defines, but it's been a while since I've poked around in there. Hmm. plperl is definitely not doing anything to try to manipulate that

Re: [GENERAL] plperl doesn't release memory

2005-03-24 Thread Sven Willenberger
On Thu, 2005-03-24 at 13:51 -0500, Tom Lane wrote: > Sven Willenberger <[EMAIL PROTECTED]> writes: > > On Thu, 2005-03-24 at 11:34 -0500, Tom Lane wrote: > >> The first thing to figure out is whether the leak is inside Perl or in > >> Postgres proper. If I were trying to do this I'd run the functi

Re: [GENERAL] plperl doesn't release memory

2005-03-24 Thread Tom Lane
Dan Sugalski <[EMAIL PROTECTED]> writes: > ... I'm pretty sure for embedding use it uses whatever routines > the embedder defines, but it's been a while since I've poked around > in there. Hmm. plperl is definitely not doing anything to try to manipulate that behavior; maybe it should? Where c

Re: [GENERAL] plperl doesn't release memory

2005-03-24 Thread Dan Sugalski
At 1:51 PM -0500 3/24/05, Tom Lane wrote: What you probably ought to do next is build Postgres with a debugging malloc library to learn more about who's eating up what. I am not sure whether libperl will automatically use the malloc attached to the main executable or whether you need to whack it a

Re: [GENERAL] plperl doesn't release memory

2005-03-24 Thread Tom Lane
Sven Willenberger <[EMAIL PROTECTED]> writes: > On Thu, 2005-03-24 at 11:34 -0500, Tom Lane wrote: >> The first thing to figure out is whether the leak is inside Perl or in >> Postgres proper. If I were trying to do this I'd run the function a >> couple times, then attach to the (idle) backend wit

Re: [GENERAL] plperl doesn't release memory

2005-03-24 Thread Sven Willenberger
On Thu, 2005-03-24 at 11:34 -0500, Tom Lane wrote: > Sven Willenberger <[EMAIL PROTECTED]> writes: > > Any suggestions on how to trace what is going on? Debug output methods? > > The first thing to figure out is whether the leak is inside Perl or in > Postgres proper. If I were trying to do this

Re: [GENERAL] plperl doesn't release memory

2005-03-24 Thread Tom Lane
Sven Willenberger <[EMAIL PROTECTED]> writes: > Any suggestions on how to trace what is going on? Debug output methods? The first thing to figure out is whether the leak is inside Perl or in Postgres proper. If I were trying to do this I'd run the function a couple times, then attach to the (idle

Re: [GENERAL] plperl doesn't release memory

2005-03-24 Thread Sven Willenberger
On Wed, 2005-03-23 at 18:25 -0500, Tom Lane wrote: > Sven Willenberger <[EMAIL PROTECTED]> writes: > > I have been experiencing an issue with plperl and PostgreSQL 8.0.1 in > > that after calling a plperl function memory does not get released. > > AFAICT the result of spi_exec_query gets released

Re: [GENERAL] plperl doesn't release memory

2005-03-23 Thread Tom Lane
Sven Willenberger <[EMAIL PROTECTED]> writes: > I have been experiencing an issue with plperl and PostgreSQL 8.0.1 in > that after calling a plperl function memory does not get released. AFAICT the result of spi_exec_query gets released fine, as soon as it's no longer referenced within perl. Perh

[GENERAL] plperl doesn't release memory

2005-03-23 Thread Sven Willenberger
I have been experiencing an issue with plperl and PostgreSQL 8.0.1 in that after calling a plperl function memory does not get released. Two different systems and each show different symptoms: 1) system: FreeBSD 5.3-Stable i386 with 1 GB RAM, dual Xeon P4 processors. script: plperl issues an SP