At Wed, 25 Aug 2021 18:18:59 +0000, "Bossart, Nathan" <bossa...@amazon.com> wrote in > On 8/25/21, 5:33 AM, "alvhe...@alvh.no-ip.org" <alvhe...@alvh.no-ip.org> > wrote: > > On 2021-Aug-24, Bossart, Nathan wrote: > >> Another interesting thing I see is that the boundary stored in > >> earliestSegBoundary is not necessarily the earliest one. It's just > >> the first one that has been registered. I did this for simplicity for > >> the .ready file fix, but I can see it causing problems here. > > > > Hmm, is there really a problem here? Surely the flush point cannot go > > past whatever has been written. If somebody is writing an earlier > > section of WAL, then we cannot move the flush pointer to a later > > position. So it doesn't matter if the earliest point we have registered > > is the true earliest -- we only care for it to be the earliest that is > > past the flush point. > > Let's say we have the following situation (F = flush, E = earliest > registered boundary, and L = latest registered boundary), and let's > assume that each segment has a cross-segment record that ends in the > next segment. > > F E L > |-----|-----|-----|-----|-----|-----|-----|-----| > 1 2 3 4 5 6 7 8 > > Then, we write out WAL to disk and create .ready files as needed. If > we didn't flush beyond the latest registered boundary, the latest > registered boundary now becomes the earliest boundary. > > F E > |-----|-----|-----|-----|-----|-----|-----|-----| > 1 2 3 4 5 6 7 8 > > At this point, the earliest segment boundary past the flush point is > before the "earliest" boundary we are tracking.
We know we have some cross-segment records in the regin [E L] so we cannot add a .ready file if flush is in the region. I haven't looked the latest patch (or I may misunderstand the discussion here) but I think we shouldn't move E before F exceeds previous (or in the first picture above) L. Things are done that way in my ancient proposal in [1]. https://www.postgresql.org/message-id/attachment/117052/v4-0001-Avoid-archiving-a-WAL-segment-that-continues-to-t.patch + if (LogwrtResult.Write < firstSegContRecStart || + lastSegContRecEnd <= LogwrtResult.Write) + { <notify the last segment> By doing so, at the time of the second picutre, the pointers are set as: E F L |-----|-----|-----|-----|-----|-----|-----|-----| 1 2 3 4 5 6 7 8 Then the poiter are cleard at the time F reaches L, F |-----|-----|-----|-----|-----|-----|-----|-----| 1 2 3 4 5 6 7 8 Isn't this work correctly? As I think I mentioned in the thread, I don't think we don't have so many (more than several, specifically) segments in a region [E L]. [1] https://www.postgresql.org/message-id/20201216.110120.887433782054853494.horikyota.ntt%40gmail.com regards. -- Kyotaro Horiguchi NTT Open Source Software Center
