Re: fdatasync performance problem with large number of DB files

2021-02-23 Thread Michael Brown
On 2021-02-22 5:43 p.m., Tom Lane wrote: > Michael Brown writes: >> * is there a knob missing we can configure? > > No. The trouble with sync() is that per POSIX, it only schedules the > writes; there's no way to tell when the work has been done. I see > that Linux offers stronger promises in t

Re: fdatasync performance problem with large number of DB files

2021-02-22 Thread Tom Lane
Michael Brown writes: > I presume the reason postgres doesn't blindly run a sync() is that we > don't know what other I/O is on the system and it'd be rude to affect > other services. That makes sense, except for our environment the work > done by the recursive fsync is orders of magnitude more di

fdatasync performance problem with large number of DB files

2021-02-22 Thread Michael Brown
We've encountered a production performance problem with pg13 related to how it fsyncs the whole data directory in certain scenarios, related to what Paul (bcc'ed) described in a post to pgsql-hackers [1]. Background: We've observed the full recursive fsync is triggered when * pg_basebackup recei