Hi Igniters! There are some updates for IEP-89 [1]. It proposes to use an incremental snapshot for PITR.
Incremental snapshots is a lightweight alternative for Ignite full snapshot. Incremental snapshots are created by a non-blocking Consistent Cut algorithm, and the incremental snapshot contains a collection of WAL segments that will be applied to the full snapshot. IEP contains some open questions: 1. How to handle rebalance data. 2. Whether to force creation of full snapshot after restoring an incremental snapshot. Any opinions, questions or comments are welcomed. Thanks! [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211884314 On Sun, May 29, 2022 at 7:38 PM Roman Puchkovskiy < roman.puchkovs...@gmail.com> wrote: > Maksim, > > Thank you for the added clarifications, the document has become a lot > clearer to me. > > пт, 27 мая 2022 г. в 18:55, Maksim Timonin <timoninma...@apache.org>: > > > > Hi Roman, > > > > Thanks for your comments! Yes, at the beginning I considered using a > single class for two types of records. But it is confusing, as you're > correctly noticed. I reworked IEP, now there are 2 different classes, and > both contain a list of transactions to include. Also I added info about > when those WAL records are actually written to 'WAL records' sections. > Please, have a look. > > > > Thanks, > > Maksim > > > > On Thu, May 26, 2022 at 2:54 PM Roman Puchkovskiy < > roman.puchkovs...@gmail.com> wrote: > >> > >> Hi Maksim. > >> > >> Do I understand correctly that 'consistent cut start' message always > >> contains IDs of transactions to include, while 'consistent cut finish' > >> message always contains IDs of transactions to exclude from the > >> consistent cut? (At least, this is the impression I got from the > >> example of parsing the WAL and the accompanying figure). If this is > >> the case, then it looks like the `include` and `check` fields are > >> mutually exclusive in ConsistentCutRecord. Would it make sense to > >> replace it with two classes, like ConsistentCutStartRecord(cutVer, > >> include) and ConsistentCutFinishRecord(cutVer, exclude)? > >> > >> Also, it seems that it could be beneficial to have a separate section > >> explaining when the corresponding records are written to WAL, to make > >> this information easier to find. Or, maybe, this could be added to the > >> current 'WAL records' section. > >> > >> пн, 16 мая 2022 г. в 12:52, Maksim Timonin <timoninma...@apache.org>: > >> > > >> > Dear Igniters, > >> > > >> > I just published IEP-89 [1] that proposes a new feature to Ignite - > (point > >> > in time recovery) PITR. I propose to implement the Consistent Cut > algorithm > >> > for this, actually I achieved a working PoC for Ignite. And based on > my > >> > research I wrote this IEP. > >> > > >> > Let's start a discussion here. Any questions or comments are welcomed. > >> > > >> > Thanks! > >> > > >> > > >> > [1] > >> > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211884314 >
