Re: Out of Memory errors are frustrating as heck!

2019-04-23 Thread Tomas Vondra
On Tue, Apr 23, 2019 at 07:09:00PM -0400, Gunther wrote: On 4/23/2019 16:43, Justin Pryzby wrote: It wrote 40GB tempfiles - perhaps you can increase work_mem now to improve the query time. I now upped my shared_buffers back from 1 to 2GB and work_mem from 4 to 16MB. Need to set vm.overcom

Re: Out of Memory errors are frustrating as heck!

2019-04-23 Thread Gunther
On 4/23/2019 16:43, Justin Pryzby wrote: It wrote 40GB tempfiles - perhaps you can increase work_mem now to improve the query time. I now upped my shared_buffers back from 1 to 2GB and work_mem from 4 to 16MB. Need to set vm.overcommit_ratio from 50 to 75 (percent, with vm.overcommit_memory =

Re: Out of Memory errors are frustrating as heck!

2019-04-23 Thread Tomas Vondra
On Tue, Apr 23, 2019 at 04:37:50PM -0400, Gunther wrote: On 4/21/2019 23:09, Tomas Vondra wrote: What I think might work better is the attached v2 of the patch, with a single top-level condition, comparing the combined memory usage (spaceUsed + BufFile) against spaceAllowed. But it

Re: Out of Memory errors are frustrating as heck!

2019-04-23 Thread Tomas Vondra
On Tue, Apr 23, 2019 at 03:43:48PM -0500, Justin Pryzby wrote: On Tue, Apr 23, 2019 at 04:37:50PM -0400, Gunther wrote: On 4/21/2019 23:09, Tomas Vondra wrote: >What I think might work better is the attached v2 of the patch, with a Thanks for this, and I am trying this now. ... Aand, it's

Re: Out of Memory errors are frustrating as heck!

2019-04-23 Thread Justin Pryzby
On Tue, Apr 23, 2019 at 04:37:50PM -0400, Gunther wrote: > On 4/21/2019 23:09, Tomas Vondra wrote: > >What I think might work better is the attached v2 of the patch, with a > Thanks for this, and I am trying this now. ... > Aand, it's a winner! > > Unique (cost=5551524.36..5554207.33 rows=3461

Re: Out of Memory errors are frustrating as heck!

2019-04-23 Thread Gunther
On 4/21/2019 23:09, Tomas Vondra wrote: What I think might work better is the attached v2 of the patch, with a single top-level condition, comparing the combined memory usage (spaceUsed + BufFile) against spaceAllowed. But it also tweaks spaceAllowed once the size needed for BufFile gets over wor