Re: Out of Memory errors are frustrating as heck!

2019-04-14 Thread Justin Pryzby
On Sun, Apr 14, 2019 at 11:59:45PM -0400, Gunther wrote: > On 4/14/2019 23:24, Tom Lane wrote: > >Any chance of extracting a self-contained test case that reproduces this? > With 18 million rows involved in the base tables, hardly. Were you able to reproduce the problem with SELECT (without INSERT

Re: Out of Memory errors are frustrating as heck!

2019-04-14 Thread Gunther
On 4/14/2019 23:24, Tom Lane wrote: ExecutorState: 2234123384 total in 266261 blocks; 3782328 free (17244 chunks); 2230341056 used Oooh, that looks like a memory leak right enough. The ExecutorState should not get that big for any reasonable query. 2.2 GB is massive yes. Your error a

Re: Out of Memory errors are frustrating as heck!

2019-04-14 Thread Tom Lane
Gunther writes: >ExecutorState: 2234123384 total in 266261 blocks; 3782328 free (17244 > chunks); 2230341056 used Oooh, that looks like a memory leak right enough. The ExecutorState should not get that big for any reasonable query. Your error and stack trace show a failure in HashBatch

Re: Out of Memory errors are frustrating as heck!

2019-04-14 Thread Gunther
Thanks Justin Pryzby too, and Jeff Janes, responding to both of you for efficiency. Answers and more logs and the gdb backtrace below. The version is 10.2 latest. v10.7 is available; could you upgrade ? Sorry I meant 11.2 actually latest. What are these set to ? shared_buffers? work_mem? sh

Re: Out of Memory errors are frustrating as heck!

2019-04-14 Thread Jeff Janes
On Sun, Apr 14, 2019 at 9:06 PM Gunther wrote: > Thanks for looking at my problem Tom Lane and Jeff Janes. Sorry for not > having given enough detail. > > The version is 10.2 latest. The database was originally built with 10.1 > and then just started with 10.2. > Do you mean 11.2? The latest in

Re: Out of Memory errors are frustrating as heck!

2019-04-14 Thread Peter
On Sun, Apr 14, 2019 at 05:19:11PM -0400, Tom Lane wrote: ! Gunther writes: ! > For weeks now, I am banging my head at an "out of memory" situation. ! > There is only one query I am running on an 8 GB system, whatever I try, ! > I get knocked out on this out of memory. It is extremely impenetrab

Re: PostgreSQL upgrade.

2019-04-14 Thread Mark Kirkwood
On 10/04/19 8:20 PM, Andreas Kretschmer wrote: Am 10.04.19 um 07:40 schrieb Daulat Ram: We have two node postgresql database version 9.6 with streaming replication which is running on docker environment, os Linux (Ubuntu) and we have to migrate on PostgresQL11. I need your suggestions & st

Re: Out of Memory errors are frustrating as heck!

2019-04-14 Thread Justin Pryzby
On Sun, Apr 14, 2019 at 09:05:48PM -0400, Gunther wrote: > Thanks for looking at my problem Tom Lane and Jeff Janes. Sorry for not > having given enough detail. > > The version is 10.2 latest. v10.7 is available; could you upgrade ? What are these set to ? shared_buffers? work_mem? Was postgre

Re: Out of Memory errors are frustrating as heck!

2019-04-14 Thread Gunther
Thanks for looking at my problem Tom Lane and Jeff Janes. Sorry for not having given enough detail. The version is 10.2 latest. The database was originally built with 10.1 and then just started with 10.2. No dump and reload or pg_upgrade. Underlying system is 64bit Amazon Linux (CentOS like) r

Re: Out of Memory errors are frustrating as heck!

2019-04-14 Thread Jeff Janes
On Sun, Apr 14, 2019 at 4:51 PM Gunther wrote: > For weeks now, I am banging my head at an "out of memory" situation. There > is only one query I am running on an 8 GB system, whatever I try, I get > knocked out on this out of memory. > Is PostgreSQL throwing an error with OOM, or is getting kill

Re: Out of Memory errors are frustrating as heck!

2019-04-14 Thread Tom Lane
Gunther writes: > For weeks now, I am banging my head at an "out of memory" situation. > There is only one query I am running on an 8 GB system, whatever I try, > I get knocked out on this out of memory. It is extremely impenetrable to > understand and fix this error. I guess I could add a swap

Out of Memory errors are frustrating as heck!

2019-04-14 Thread Gunther
For weeks now, I am banging my head at an "out of memory" situation. There is only one query I am running on an 8 GB system, whatever I try, I get knocked out on this out of memory. It is extremely impenetrable to understand and fix this error. I guess I could add a swap file, and then I would