On 9/24/21 12:28 PM, Robert Haas wrote:
On Thu, Sep 16, 2021 at 7:26 PM Bossart, Nathan <bossa...@amazon.com> wrote:
What do you think?
I think this is committable. I also went back and looked at your
previous proposal to do files in batches, and I think that's also
committable. After some reflection, I think I have a slight preference
for the batching approach.
It seems like it might lend itself to archiving multiple files in a
single invocation of the archive_command, and Alvaro just suggested it
again apparently not having realized that it had been previously
proposed by Andres, so I guess it has the further advantage of being
the thing that several committers intuitively feel like we ought to be
doing to solve this problem.
I also prefer this approach. Reducing directory scans is an excellent
optimization, but from experience I know that execution time for the
archive_command can also be a significant bottleneck. Begin able to
archive multiple segments per execution would be a big win in certain
scenarios.
So what I am inclined to do is commit
v1-0001-Improve-performance-of-pgarch_readyXlog-with-many.patch.
I read the patch and it looks good to me.
I do wish we had a way to test that history files get archived first,
but as I recall I was not able to figure out how to do reliably for [1]
without writing a custom archive_command just for testing. That is
something we might want to consider as we make this logic more complex.
Regards,
--
-David
da...@pgmasters.net
[1]
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=b981df4cc09aca978c5ce55e437a74913d09cccc
