[BUGS] BUG #1739: memory leak in pl/perl with spi_exec_query
The following bug has been logged online:
Bug reference: 1739
Logged by: Jean-Max Reymond
Email address: [EMAIL PROTECTED]
PostgreSQL version: 8.0.3
Operating system: Linux Ubuntu
Description:memory leak in pl/perl with spi_exec_query
Details:
whith this code 1, postmaster is growing and growing and eat a lot of
memory
CREATE FUNCTION jmax() RETURNS integer
AS $_$use strict;
for (my $i=0; $i<1000;$i++) {
spi_exec_query("select 'foo'");
}
my $j=1;$_$
LANGUAGE plperlu SECURITY DEFINER
With code 2, we can see that perl does correctly the job to unreference
unused memory
CREATE FUNCTION jmax() RETURNS integer
AS $_$use strict;
for (my $i=0; $i<1000;$i++) {
my $ch = "0123456789"x10;
}
my $j=1;$_$
LANGUAGE plperlu SECURITY DEFINER
With code 3, we try to help pl/perl to clean memory allocation
CREATE FUNCTION jmax() RETURNS integer
AS $_$use strict;
for (my $i=0; $i<1000;$i++) {
my ch=spi_exec_query("select 'foo'");
}
my $j=1;$_$
LANGUAGE plperlu SECURITY DEFINER
So, spi_exec_query allocates memory but this memory is never released until
the end of the stored procedure.
For my application, the need is to call millions of spy_exec and we use 1.1
Gb of not necessary memory.
A workaround could be to provide a routine to free the memory allocated by
spi_exec_query
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [BUGS] BUG #1739: memory leak in pl/perl with spi_exec_query
Jean-Max Reymond a écrit :
The following bug has been logged online:
Bug reference: 1739
Logged by: Jean-Max Reymond
Email address: [EMAIL PROTECTED]
PostgreSQL version: 8.0.3
Operating system: Linux Ubuntu
Description:memory leak in pl/perl with spi_exec_query
Details:
whith this code 1, postmaster is growing and growing and eat a lot of
memory
CREATE FUNCTION jmax() RETURNS integer
AS $_$use strict;
for (my $i=0; $i<1000;$i++) {
spi_exec_query("select 'foo'");
}
my $j=1;$_$
LANGUAGE plperlu SECURITY DEFINER
With code 2, we can see that perl does correctly the job to unreference
unused memory
CREATE FUNCTION jmax() RETURNS integer
AS $_$use strict;
for (my $i=0; $i<1000;$i++) {
my $ch = "0123456789"x10;
}
my $j=1;$_$
LANGUAGE plperlu SECURITY DEFINER
With code 3, we try to help pl/perl to clean memory allocation
CREATE FUNCTION jmax() RETURNS integer
AS $_$use strict;
for (my $i=0; $i<1000;$i++) {
my ch=spi_exec_query("select 'foo'");
}
my $j=1;$_$
LANGUAGE plperlu SECURITY DEFINER
So, spi_exec_query allocates memory but this memory is never released until
the end of the stored procedure.
For my application, the need is to call millions of spy_exec and we use 1.1
Gb of not necessary memory.
A workaround could be to provide a routine to free the memory allocated by
spi_exec_query
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
can anyone confirm this behavior ?
is it normal to have allocated memory in the pool even if this memory is
unreferenced ?
I am searching in the code but it is not very easy :-(
begin:vcard
fn:Jean-Max Reymond
n:Reymond;Jean-Max
org:CKR Solutions Open Source
adr;dom:;;67 bvd des roses;Mandelieu;;06210
email;internet:[EMAIL PROTECTED]
title:responsable technique
tel;work:08 71 03 25 11
tel;cell:06 16 94 25 11
x-mozilla-html:FALSE
url:http://www.ckr-solutions.com
version:2.1
end:vcard
---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings
Re: [BUGS] BUG #1739: memory leak in pl/perl with spi_exec_query
Jean-Max Reymond a écrit : Jean-Max Reymond a écrit : The following bug has been logged online: Bug reference: 1739 Logged by: Jean-Max Reymond Email address: [EMAIL PROTECTED] PostgreSQL version: 8.0.3 Operating system: Linux Ubuntu Description:memory leak in pl/perl with spi_exec_query I have run again my real plperl procedure and attach a gdb to the process. a call MemoryContextStats(TopMemoryContext) gives me the results as above. is it possible for a guru to check that all is ok ? A very interesting fact will be to prove that memory is not used by spi_exec but by perl and confirm that memory allocation and garbage collector in perl is pitifully. I have very carefully read the thread "plperl doesn't release memory" and I am not sure I am in the same context. TopMemoryContext: 32768 total in 3 blocks; 4264 free (7 chunks); 28504 used TopTransactionContext: 8192 total in 1 blocks; 6992 free (0 chunks); 1200 used CurTransactionContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used SPI Exec: 57344 total in 3 blocks; 7520 free (4 chunks); 49824 used ExecutorState: 8192 total in 1 blocks; 3368 free (4 chunks); 4824 used ExecutorState: 8192 total in 1 blocks; 2048 free (0 chunks); 6144 used ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used AggContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used SPI Proc: 1015013376 total in 130 blocks; 1893336 free (0 chunks); 1013120040 used SPI TupTable: 8192 total in 1 blocks; 6584 free (0 chunks); 1608 used MessageContext: 8192 total in 1 blocks; 5400 free (1 chunks); 2792 used PortalMemory: 8192 total in 1 blocks; 8040 free (0 chunks); 152 used PortalHeapMemory: 1024 total in 1 blocks; 880 free (0 chunks); 144 used ExecutorState: 8192 total in 1 blocks; 6864 free (1 chunks); 1328 used ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used CacheMemoryContext: 4186112 total in 9 blocks; 1214808 free (1 chunks); 2971304 used index_xdb_pi: 1024 total in 1 blocks; 568 free (0 chunks); 456 used index_xdb_str: 1024 total in 1 blocks; 712 free (0 chunks); 312 used index_xdb_ele: 1024 total in 1 blocks; 696 free (0 chunks); 328 used index_xdb_child: 1024 total in 1 blocks; 568 free (0 chunks); 456 used index_xdb_attr: 1024 total in 1 blocks; 696 free (0 chunks); 328 used pg_toast_1255_index: 1024 total in 1 blocks; 848 free (0 chunks); 176 used pg_index_indrelid_index: 1024 total in 1 blocks; 912 free (0 chunks); 112 used pg_user: 7168 total in 3 blocks; 3736 free (0 chunks); 3432 used pg_namespace_nspname_index: 1024 total in 1 blocks; 912 free (0 chunks); 112 used pg_attribute_relid_attnum_index: 1024 total in 1 blocks; 848 free (0 chunks); 176 used pg_shadow_usesysid_index: 1024 total in 1 blocks; 912 free (0 chunks); 112 used pg_aggregate_fnoid_index: 1024 total in 1 blocks; 912 free (0 chunks); 112 used pg_class_relname_nsp_index: 1024 total in 1 blocks; 848 free (0 chunks); 176 used pg_rewrite_rel_rulename_index: 1024 total in 1 blocks; 848 free (0 chunks); 176 used pg_inherits_relid_seqno_index: 1024 total in 1 blocks; 848 free (0 chunks); 176 used pg_cast_source_target_index: 1024 total in 1 blocks; 848 free (0 chunks); 176 used pg_type_oid_index: 1024 total in 1 blocks; 912 free (0 chunks); 112 used pg_language_name_index: 1024 total in 1 blocks; 912 free (0 chunks); 112 used pg_class_oid_index: 1024 total in 1 blocks; 912 free (0 chunks); 112 used pg_operator_oid_index: 1024 total in 1 blocks; 912 free (0 chunks); 112 used pg_operator_oprname_l_r_n_index: 1024 total in 1 blocks; 712 free (0 chunks); 312 used pg_statistic_relid_att_index: 1024 total in 1 blocks; 848 free (0 chunks); 176 used pg_proc_oid_index: 1024 total in 1 blocks; 912 free (0 chunks); 112 used pg_amop_opc_strat_index: 1024 total in 1 blocks; 776 free (0 chunks); 248 used pg_opclass_am_name_nsp_index: 1024 total in 1 blocks; 776 free (0 chunks); 248 used pg_index_indexrelid_index: 1024 total in 1 blocks; 912 free (0 chunks); 112 used pg_conversion_name_nsp_index: 1024 total in 1 blocks; 848 free (0 chunks); 176 used pg_conversion_oid_index: 1024 total in 1 blocks; 912 free (0 chunks); 112 used pg_trigger_tgrelid_tgname_index: 1024 total in 1 blocks; 848 free (0 chunks); 176 used pg_amproc_opc_proc_index: 1024 total in 1 blocks; 776 free (0 chunks); 248 used pg_language_oid_index: 1024 total in 1 blocks; 912 free (0 chunks); 112 used pg_conversion_default_index: 1024 total in 1 blocks; 712 free (0 chunks); 312 used pg_shadow_usename_index: 1024 total in 1 blocks; 912 free (0 chunks); 112 used pg_attribute_relid_attnam_index: 1024 total in 1 blocks; 848 free (0 chunks); 176 used pg_namespace_oid_index: 1024 total in 1 blocks; 912 free (0 chunks); 112 used pg_group_sysid_index: 1024 total in 1 blocks; 912 free (0 ch
Re: [BUGS] BUG #1739: memory leak in pl/perl with spi_exec_query
Tom Lane a écrit : "Jean-Max Reymond" <[EMAIL PROTECTED]> writes: So, spi_exec_query allocates memory but this memory is never released until the end of the stored procedure. Ah, found it. OK, it solves my problem. great job :-) thanks a lot Tom begin:vcard fn:Jean-Max Reymond n:Reymond;Jean-Max org:CKR Solutions Open Source adr;dom:;;67 bvd des roses;Mandelieu;;06210 email;internet:[EMAIL PROTECTED] title:responsable technique tel;work:08 71 03 25 11 tel;cell:06 16 94 25 11 x-mozilla-html:FALSE url:http://www.ckr-solutions.com version:2.1 end:vcard ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
