At Tue, 3 Aug 2021 20:46:57 +0000, "Bossart, Nathan" <bossa...@amazon.com> wrote in > + /* > + * Perform a full directory scan to identify the next log segment. There > + * may be one of the following scenarios which may require us to > perform a > + * full directory scan. > + * > + * 1. This is the first cycle since archiver has started and there is no > + * idea about the next anticipated log segment. > + * > + * 2. There is a timeline switch, i.e. the timeline ID tracked at > archiver > + * does not match with current timeline ID. Archive history file as > part of > + * this timeline switch. > + * > + * 3. The next anticipated log segment is not available. > > One benefit of the current implementation of pgarch_readyXlog() is > that .ready files created out of order will be prioritized before > segments with greater LSNs. IIUC, with this patch, as long as there > is a "next anticipated" segment available, the archiver won't go back > and archive segments it missed. I don't think the archive status > files are regularly created out of order, but XLogArchiveCheckDone() > has handling for that case, and the work to avoid creating .ready > files too early [0] seems to make it more likely. Perhaps we should > also force a directory scan when we detect that we are creating a > .ready file for a segment that is older than the "next anticipated" > segment. > > Nathan > > [0] https://postgr.es/m/DA71434B-7340-4984-9B91-F085BC47A778%40amazon.com
It works the current way always at the first iteration of pgarch_ArchiveCopyLoop() becuse in the last iteration of pgarch_ArchiveCopyLoop(), pgarch_readyXlog() erases the last anticipated segment. The shortcut works only when pgarch_ArchiveCopyLoop archives more than once successive segments at once. If the anticipated next segment found to be missing a .ready file while archiving multiple files, pgarch_readyXLog falls back to the regular way. So I don't see the danger to happen perhaps you are considering. In the first place, .ready are added while holding WALWriteLock in XLogWrite, and while removing old segments after a checkpoint (which happens while recovery). Assuming that no one manually remove .ready files on an active server, the former is the sole place doing that. So I don't see a chance that .ready files are created out-of-order way. regards. -- Kyotaro Horiguchi NTT Open Source Software Center