Re: [HACKERS] patch: tsearch - some memory diet

2010-10-06 Thread Tom Lane
Robert Haas writes: > Nice. What was the overall effect on memory consumption? Before: cspell: 31352608 total in 3814 blocks; 37432 free (6 chunks); 31315176 used After: cspell: 16214816 total in 1951 blocks; 13744 free (12 chunks); 16201072 used This is on a 32-bit machine that uses MAXALIGN

Re: [HACKERS] patch: tsearch - some memory diet

2010-10-06 Thread Robert Haas
On Wed, Oct 6, 2010 at 7:36 PM, Tom Lane wrote: > Teodor Sigaev writes: >>> on 32bit from 27MB (3399 blocks) to 13MB (1564 blocks) >>> on 64bit from 55MB to cca 27MB. > >> Good results. But, I think, there are more places in ispell to use >> hold_memory(): >> - affixes and affix tree >> - regis

Re: [HACKERS] patch: tsearch - some memory diet

2010-10-06 Thread Tom Lane
Teodor Sigaev writes: >> on 32bit from 27MB (3399 blocks) to 13MB (1564 blocks) >> on 64bit from 55MB to cca 27MB. > Good results. But, I think, there are more places in ispell to use > hold_memory(): > - affixes and affix tree > - regis (REGex for ISpell, regis.c) I fixed the affix stuff as mu

Re: [HACKERS] patch: tsearch - some memory diet

2010-10-06 Thread Tom Lane
Pavel Stehule writes: > this simple patch reduce a persistent allocated memory for tsearch > ispell dictionaries. > on 32bit from 27MB (3399 blocks) to 13MB (1564 blocks) > on 64bit from 55MB to cca 27MB. Applied with revisions --- I got rid of the risky static state as per discussion, and exten

Re: [HACKERS] patch: tsearch - some memory diet

2010-10-06 Thread Tom Lane
Robert Haas writes: > On Wed, Oct 6, 2010 at 1:40 PM, Tom Lane wrote: >> Robert Haas writes: >>> ...but I don't really see why that has to be done as part of this patch. >> >> Because patches that reduce maintainability seldom get cleaned up after. > I don't think you've made a convincing argu

Re: [HACKERS] patch: tsearch - some memory diet

2010-10-06 Thread Robert Haas
On Wed, Oct 6, 2010 at 1:40 PM, Tom Lane wrote: > Robert Haas writes: >> I think it would be cleaner to get rid of checkTmpCtx() and instead >> have dispell_init() set up and tear down the temporary context, > > What I was thinking of doing was getting rid of the static variable > altogether: we

Re: [HACKERS] patch: tsearch - some memory diet

2010-10-06 Thread Tom Lane
Robert Haas writes: > I think it would be cleaner to get rid of checkTmpCtx() and instead > have dispell_init() set up and tear down the temporary context, What I was thinking of doing was getting rid of the static variable altogether: we should do what you say above, but in the form of a state s

Re: [HACKERS] patch: tsearch - some memory diet

2010-10-06 Thread Robert Haas
On Mon, Oct 4, 2010 at 2:05 AM, Pavel Stehule wrote: > 2010/10/4 Robert Haas : >> On Oct 3, 2010, at 7:02 PM, Tom Lane wrote: >>> It's not at all apparent that the code is even >>> safe as-is, because it's depending on the unstated assumption that that >>> static variable will get reset once per

Re: [HACKERS] patch: tsearch - some memory diet

2010-10-04 Thread Pavel Stehule
Hello 2010/10/4 Robert Haas : > On Oct 3, 2010, at 7:02 PM, Tom Lane wrote: >> It's not at all apparent that the code is even >> safe as-is, because it's depending on the unstated assumption that that >> static variable will get reset once per dictionary.  The documentation >> is horrible: it doe

Re: [HACKERS] patch: tsearch - some memory diet

2010-10-03 Thread Robert Haas
On Oct 3, 2010, at 7:02 PM, Tom Lane wrote: > It's not at all apparent that the code is even > safe as-is, because it's depending on the unstated assumption that that > static variable will get reset once per dictionary. The documentation > is horrible: it doesn't really explain what the patch is

Re: [HACKERS] patch: tsearch - some memory diet

2010-10-03 Thread Tom Lane
Robert Haas writes: > On Tue, Sep 7, 2010 at 1:30 PM, Tom Lane wrote: >> In the particular case here, the dictionary structures could probably >> safely use such a context type, but I'm not sure it's worth bothering >> if the long-term plan is to implement a precompiler.  There would be >> no nee

Re: [HACKERS] patch: tsearch - some memory diet

2010-10-01 Thread Robert Haas
On Fri, Oct 1, 2010 at 2:34 PM, Pavel Stehule wrote: > Hello > > 2010/10/1 Robert Haas : >> On Mon, Sep 27, 2010 at 11:30 PM, Robert Haas wrote: >>> On Tue, Sep 7, 2010 at 1:30 PM, Tom Lane wrote: In the particular case here, the dictionary structures could probably safely use such a c

Re: [HACKERS] patch: tsearch - some memory diet

2010-10-01 Thread Pavel Stehule
Hello 2010/10/1 Robert Haas : > On Mon, Sep 27, 2010 at 11:30 PM, Robert Haas wrote: >> On Tue, Sep 7, 2010 at 1:30 PM, Tom Lane wrote: >>> In the particular case here, the dictionary structures could probably >>> safely use such a context type, but I'm not sure it's worth bothering >>> if the l

Re: [HACKERS] patch: tsearch - some memory diet

2010-10-01 Thread Robert Haas
On Mon, Sep 27, 2010 at 11:30 PM, Robert Haas wrote: > On Tue, Sep 7, 2010 at 1:30 PM, Tom Lane wrote: >> In the particular case here, the dictionary structures could probably >> safely use such a context type, but I'm not sure it's worth bothering >> if the long-term plan is to implement a preco

Re: [HACKERS] patch: tsearch - some memory diet

2010-09-27 Thread Robert Haas
On Tue, Sep 7, 2010 at 1:30 PM, Tom Lane wrote: > In the particular case here, the dictionary structures could probably > safely use such a context type, but I'm not sure it's worth bothering > if the long-term plan is to implement a precompiler.  There would be > no need for this after the precom

Re: [HACKERS] patch: tsearch - some memory diet

2010-09-07 Thread Tom Lane
Heikki Linnakangas writes: > A more general solution would be to have a new MemoryContext > implementation that does the same your patch does. Ie. instead of > tracking each allocation, just allocate a big chunk, and have palloc() > return the next n free bytes from it, like a stack. pfree() wo

Re: [HACKERS] patch: tsearch - some memory diet

2010-09-07 Thread Teodor Sigaev
A more general solution would be to have a new MemoryContext implementation that does the same your patch does. Ie. instead of tracking each allocation, just allocate a big chunk, and have palloc() return the next n free bytes from it, like a stack. pfree() would obviously not work, but wholesale

Re: [HACKERS] patch: tsearch - some memory diet

2010-09-07 Thread Pavel Stehule
2010/9/7 Heikki Linnakangas : > On 07/09/10 19:27, Teodor Sigaev wrote: >>> >>> on 32bit from 27MB (3399 blocks) to 13MB (1564 blocks) >>> on 64bit from 55MB to cca 27MB. >> >> Good results. But, I think, there are more places in ispell to use >> hold_memory(): >> - affixes and affix tree >> - regi

Re: [HACKERS] patch: tsearch - some memory diet

2010-09-07 Thread Pavel Stehule
2010/9/7 Teodor Sigaev : >> on 32bit from 27MB (3399 blocks) to 13MB (1564 blocks) >> on 64bit from 55MB to cca 27MB. > > Good results. But, I think, there are more places in ispell to use > hold_memory(): > - affixes and affix tree > - regis (REGex for ISpell, regis.c) yes, but minimally for Czec

Re: [HACKERS] patch: tsearch - some memory diet

2010-09-07 Thread Heikki Linnakangas
On 07/09/10 19:27, Teodor Sigaev wrote: on 32bit from 27MB (3399 blocks) to 13MB (1564 blocks) on 64bit from 55MB to cca 27MB. Good results. But, I think, there are more places in ispell to use hold_memory(): - affixes and affix tree - regis (REGex for ISpell, regis.c) A more general solution

Re: [HACKERS] patch: tsearch - some memory diet

2010-09-07 Thread Teodor Sigaev
on 32bit from 27MB (3399 blocks) to 13MB (1564 blocks) on 64bit from 55MB to cca 27MB. Good results. But, I think, there are more places in ispell to use hold_memory(): - affixes and affix tree - regis (REGex for ISpell, regis.c) -- Teodor Sigaev E-mail: teo.

[HACKERS] patch: tsearch - some memory diet

2010-09-07 Thread Pavel Stehule
Hello this simple patch reduce a persistent allocated memory for tsearch ispell dictionaries. on 32bit from 27MB (3399 blocks) to 13MB (1564 blocks) on 64bit from 55MB to cca 27MB. Regards Pavel Stehule tsearch_group_alloc.diff Description: Binary data -- Sent via pgsql-hackers mailing list