Re: [PERFORM] in-memory sorting

2010-08-19 Thread Tom Lane
Samuel Gendler writes: > Answered my own question. Cranking work_mem up to 350MB revealed that > the in-memory sort requires more memory than the disk sort. Yeah. The on-disk representation of sortable data is tighter than the in-memory representation for various reasons, mostly that we're will

Re: [PERFORM] in-memory sorting

2010-08-19 Thread Scott Marlowe
On Thu, Aug 19, 2010 at 1:06 AM, Samuel Gendler wrote: > Incidentally, if I set values on the connection before querying, is there an > easy way to get things back to default values or will my code need to know > the prior value and explicitly set it back?  Something like reset work_mem; -- Sen

Re: [PERFORM] in-memory sorting

2010-08-19 Thread Samuel Gendler
On Thu, Aug 19, 2010 at 12:06 AM, Samuel Gendler wrote: > Incidentally, if I set values on the connection before querying, is there > an easy way to get things back to default values or will my code need to > know the prior value and explicitly set it back? Something like > > > set work_mem = '5

Re: [PERFORM] in-memory sorting

2010-08-19 Thread Samuel Gendler
Incidentally, if I set values on the connection before querying, is there an easy way to get things back to default values or will my code need to know the prior value and explicitly set it back? Something like set work_mem = '512MB' query set value = 'default' or maybe BEGIN; set work_mem='

Re: [PERFORM] in-memory sorting

2010-08-18 Thread Scott Marlowe
Exactly, it's about the concurrency. I have a server with 128G ram but it runs dozens of queries at a time for hundreds of clients a second. The chance that something big for work_mem might jump up and bite me are pretty good there. Even so, at 16Meg it's not really big for that machine, and I m

Re: [PERFORM] in-memory sorting

2010-08-18 Thread Samuel Gendler
Yeah, although with 48GB of available memory and not that much concurrency, I'm not sure it matters that much. But point taken, I'll see about modifying the app such that work_mem gets set on a per-query basis. On Wed, Aug 18, 2010 at 11:24 PM, Scott Marlowe wrote: > On Wed, Aug 18, 2010 at 11:4

Re: [PERFORM] in-memory sorting

2010-08-18 Thread Scott Marlowe
On Wed, Aug 18, 2010 at 11:45 PM, Samuel Gendler wrote: > Answered my own question.  Cranking work_mem up to 350MB revealed that > the in-memory sort requires more memory than the disk sort. Note that unless you run VERY few client connections, it's usually better to leave work_mem somewhere in t

Re: [PERFORM] in-memory sorting

2010-08-18 Thread Pavel Stehule
Hello >> >> I'm not understanding why it is sorting on disk if it would fit within >> a work_mem segment - by a fairly wide margin.  Is there something else >> I can do to get that sort to happen in memory? >> Planner working with estimations. So there is some probability so planner expected a lar

Re: [PERFORM] in-memory sorting

2010-08-18 Thread Samuel Gendler
Answered my own question. Cranking work_mem up to 350MB revealed that the in-memory sort requires more memory than the disk sort. On Wed, Aug 18, 2010 at 10:23 PM, Samuel Gendler wrote: > I've got this explain: http://explain.depesz.com/s/Xh9 > > And these settings: > default_statistics_target =