On Fri, Jan 3, 2020 at 5:57 PM Thomas Munro <thomas.mu...@gmail.com> wrote: > On Fri, Jan 3, 2020 at 7:10 AM Tomas Vondra > <tomas.von...@2ndquadrant.com> wrote: > > Could we instead specify the number of blocks to prefetch? We'd probably > > need to track additional details needed to determine number of blocks to > > prefetch (essentially LSN for all prefetch requests).
Here is a new WIP version of the patch set that does that. Changes: 1. It now uses effective_io_concurrency to control how many concurrent prefetches to allow. It's possible that we should have a different GUC to control "maintenance" users of concurrency I/O as discussed elsewhere[1], but I'm staying out of that for now; if we agree to do that for VACUUM etc, we can change it easily here. Note that the value is percolated through the ComputeIoConcurrency() function which I think we should discuss, but again that's off topic, I just want to use the standard infrastructure here. 2. You can now change the relevant GUCs (wal_prefetch_distance, wal_prefetch_fpw, effective_io_concurrency) at runtime and reload for them to take immediate effect. For example, you can enable the feature on a running replica by setting wal_prefetch_distance=8kB (from the default of -1, which means off), and something like effective_io_concurrency=10, and telling the postmaster to reload. 3. The new code is moved out to a new file src/backend/access/transam/xlogprefetcher.c, to minimise new bloat in the mighty xlog.c file. Functions were renamed to make their purpose clearer, and a lot of comments were added. 4. The WAL receiver now exposes the current 'write' position via an atomic value in shared memory, so we don't need to hammer the WAL receiver's spinlock. 5. There is some rudimentary user documentation of the GUCs. [1] https://www.postgresql.org/message-id/13619.1557935593%40sss.pgh.pa.us
0001-Allow-PrefetchBuffer-to-be-called-with-a-SMgrRela-v2.patch
Description: Binary data
0002-Rename-GetWalRcvWriteRecPtr-to-GetWalRcvFlushRecP-v2.patch
Description: Binary data
0003-Add-WalRcvGetWriteRecPtr-new-definition-v2.patch
Description: Binary data
0004-Allow-PrefetchBuffer-to-report-missing-file-in-re-v2.patch
Description: Binary data
0005-Prefetch-referenced-blocks-during-recovery-v2.patch
Description: Binary data
