On Thu, Apr 8, 2021 at 3:27 AM Tomas Vondra <tomas.von...@enterprisedb.com> wrote: > On 4/7/21 1:24 PM, Thomas Munro wrote: > > I made one other simplifying change: previously, the prefetch module > > would read the WAL up to the "written" LSN (so, allowing itself to > > read data that had been written but not yet flushed to disk by the > > walreceiver), though it still waited until a record's LSN was > > "flushed" before replaying. That allowed prefetching to happen > > concurrently with the WAL flush, which was nice, but it felt a little > > too "special". I decided to remove that part for now, and I plan to > > look into making standbys work more like primary servers, using WAL > > buffers, the WAL writer and optionally the standard log-before-data > > rule. > > Not sure, but the removal seems unnecessary. I'm worried that this will > significantly reduce the amount of data that we'll be able to prefetch. > How likely it is that we have data that is written but not flushed? > Let's assume the replica is lagging and network bandwidth is not the > bottleneck - how likely is this "has to be flushed" a limit for the > prefetching?
Yeah, it would have been nice to include that but it'll have to be for v15 due to lack of time to convince myself that it was correct. I do intend to look into more concurrency of that kind for v15. I have pushed these patches, updated to be disabled by default. I will look into how I can run a BF animal that has it enabled during the recovery tests for coverage. Thanks very much to everyone on this thread for all the discussion and testing so far.
