Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-15 Thread Andres Freund
Hi, Thanks for pushing! Sorry to not get around to a review before... On 2021-01-15 11:42:50 +0900, Michael Paquier wrote: > On Fri, Jan 15, 2021 at 03:25:24PM +1300, Thomas Munro wrote: > > Thanks Michael! Another notch for the unnecessary system call > > hitlist: https://wiki.postgresql.org/wi

Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-14 Thread Michael Paquier
On Fri, Jan 15, 2021 at 03:25:24PM +1300, Thomas Munro wrote: > Thanks Michael! Another notch for the unnecessary system call > hitlist: https://wiki.postgresql.org/wiki/Syscall_Reduction A quick question. How much does it matter in terms of micro-performance for this code path depending on max/

Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-14 Thread Thomas Munro
On Fri, Jan 15, 2021 at 3:07 PM Michael Paquier wrote: > On Wed, Jan 13, 2021 at 04:27:25PM +0900, Michael Paquier wrote: > > I have been looking again at that, and the rebased version that Andres > > has provided would take care of that. Any thoughts? > > Hearing nothing, I have applied the thin

Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-14 Thread Michael Paquier
On Wed, Jan 13, 2021 at 04:27:25PM +0900, Michael Paquier wrote: > I have been looking again at that, and the rebased version that Andres > has provided would take care of that. Any thoughts? Hearing nothing, I have applied the thing on HEAD after more tests and more reads of the code. -- Michael

Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-12 Thread Michael Paquier
On Tue, Jan 12, 2021 at 12:10:24PM -0300, Alvaro Herrera wrote: > Apparently b2a5545bd63f changed; before that commit, that code > (including the quoted comment) was all in RemoveOldXlogFiles, and > endlogSegNo was calculated only once. But ISTM that even with that > formulation it had the problem

Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-12 Thread Michael Paquier
On Tue, Jan 12, 2021 at 11:30:13PM +1300, Thomas Munro wrote: > I haven't heard any user complaints, and I'd personally be happy with > a fix on master only. I have been looking again at that, and the rebased version that Andres has provided would take care of that. Any thoughts? -- Michael From

Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-12 Thread Alvaro Herrera
On 2021-Jan-11, Thomas Munro wrote: > I didn't check the migration history of this code but it seems that > endlogSegNo doesn't currently have the right scoping to achieve the > goal of that last comment, so checkpoints finish up repeatedly search > for the next free slot, starting at the low end

Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-12 Thread Thomas Munro
On Tue, Jan 12, 2021 at 8:27 PM Michael Paquier wrote: > Yeah, this rings a bell. I never went back to it even if the thing > looks rather clean at quick glance (not tested), but I may be able > to spend some cycles on that. I don't think that's critical enough > for a backpatch, so doing someth

Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-11 Thread Michael Paquier
On Tue, Jan 12, 2021 at 07:15:21PM +1300, Thomas Munro wrote: > Hah, I even knew that, apparently, but forgot. Adding Michael who > wrote a patch. It'd be nice to fix this, at least in 14. Yeah, this rings a bell. I never went back to it even if the thing looks rather clean at quick glance (not

Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-11 Thread Thomas Munro
On Tue, Jan 12, 2021 at 6:55 PM Andres Freund wrote: > I found this before as well: > https://postgr.es/m/CAB7nPqTB3VcKSSrW2Qj59tYYR2H4+n=5pzbdwou+x9iqvnm...@mail.gmail.com Hah, I even knew that, apparently, but forgot. Adding Michael who wrote a patch. It'd be nice to fix this, at least in 14

Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-11 Thread Andres Freund
Hi, On 2021-01-11 16:35:56 +1300, Thomas Munro wrote: > I noticed that RemoveXlogFile() has this code: > > /* > * Before deleting the file, see if it can be recycled as a future log > * segment. Only recycle normal files, pg_standby for example can > create > *

O(n^2) system calls in RemoveOldXlogFiles()

2021-01-10 Thread Thomas Munro
Hi, I noticed that RemoveXlogFile() has this code: /* * Before deleting the file, see if it can be recycled as a future log * segment. Only recycle normal files, pg_standby for example can create * symbolic links pointing to a separate archive directory.