Re: Autovacuum degrades all other operations by keeping all buffers dirty?

2018-09-05 Thread David Pacheco
On Fri, Aug 31, 2018 at 3:50 PM, Andres Freund wrote: > On 2018-08-31 19:31:47 -0300, Alvaro Herrera wrote: > > On 2018-Aug-31, David Pacheco wrote: > > > > > From reading the 9.6.3 source, it looks like the autovacuum process > > > itself is single-threade

Autovacuum degrades all other operations by keeping all buffers dirty?

2018-08-31 Thread David Pacheco
Hello, We've been struggling with some major performance issues related to autovacuum. (I know this is a common problem.) For a while, we believed this was primarily related to I/O contention, but recent observations from our monitoring make me wonder if there's a deeper issue here, so I'm lookin

Re: Can PostgreSQL create new WAL files instead of reusing old ones?

2018-06-21 Thread David Pacheco
On Wed, Jun 20, 2018 at 10:35 AM, Jerry Jelinek wrote: > As Dave described in his original email on this topic, we'd like to avoid > recycling WAL files since that can cause performance issues when we have a > read-modify-write on a file that has dropped out of the cache. > > I have implemented a

Re: Can PostgreSQL create new WAL files instead of reusing old ones?

2018-04-17 Thread David Pacheco
On Tue, Apr 17, 2018 at 11:57 AM, Tom Lane wrote: > Alvaro Herrera writes: > > David Pacheco wrote: > >> tl;dr: We've found that under many conditions, PostgreSQL's re-use of > old > >> WAL > >> files appears to significantly degrade query latenc

Can PostgreSQL create new WAL files instead of reusing old ones?

2018-04-17 Thread David Pacheco
tl;dr: We've found that under many conditions, PostgreSQL's re-use of old WAL files appears to significantly degrade query latency on ZFS. The reason is complicated and I have details below. Has it been considered to make this behavior tunable, to cause PostgreSQL to always create new WAL files i

Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-12-04 Thread David Pacheco
Thanks again for helping out. On Mon, Dec 4, 2017 at 2:12 PM, Andres Freund wrote: > On 2017-12-04 13:57:52 -0800, David Pacheco wrote: > > On Mon, Dec 4, 2017 at 12:23 PM, Andres Freund > wrote: > > > FWIW, I'd like to see a report of this around the time the i

Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-12-04 Thread David Pacheco
On Mon, Dec 4, 2017 at 12:23 PM, Andres Freund wrote: > Hi, > > On 2017-11-20 11:12:08 -0800, David Pacheco wrote: > > $ ps -opid,rss,vsz,args -p 37627 > > PID RSS VSZ COMMAND > > 37627 2980 14968 /opt/postgresql/9.2.4/bin/postgres -D /manatee/pg/data > > &

Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-12-04 Thread David Pacheco
On Mon, Nov 20, 2017 at 11:12 AM, David Pacheco wrote: > I understand if the community isn't interested in fixing this case if > other users aren't seeing it much, but surely it's still a bug that this > unusual case can result in a deadlock? > I've filed bug

Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-11-20 Thread David Pacheco
Thanks again for looking at this issue. On Mon, Nov 20, 2017 at 10:34 AM, Andres Freund wrote: > Hi, > > On 2017-11-20 10:13:57 -0800, David Pacheco wrote: > > I expect what happened is that the syslogger process attempted to > allocate > > memory, failed because

Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-11-20 Thread David Pacheco
Responding to several points below. Tom Lane wrote: > David's report isn't too clear: did the syslogger process actually run > out of memory and exit of its own volition after an ENOMEM, or did it get > killed by the dreaded OOM killer? In either case, it's unclear whether > it was really using a