Hi Han, I have added a section on TTL of ReducingMergeState and AggregatingMergeState - HERE <https://docs.google.com/document/d/1HwEDRGoSZIUU1SYxTih4qp8FM6LjTdIrDs7CJHm4iB0/edit?tab=t.0#heading=h.mqp1qeixcg45> , please review.
Best, -Soumitra. On Mon, Jun 1, 2026 at 11:02 PM Soumitra Kumar <[email protected]> wrote: > Hi Han, > > Thanks for your review and encouragement! > > #1 - Users can migrate from ReducingState to ReducingMergeState, but it > has to be a conscious decision knowing the rocksdb implication. We should > plan to create a few howto docs monitoring and tuning rocksdb to get the > best out of the merge operators. Theoretically, it is possible to build an > automatic migration path, but I would not favor that because of the > different runtime characteristics of ReducingState and ReducingMergeState. > The checkpoints/savepoints for ReducingMergeState/AggregatingMergeState > state variables will serialize the Reduce/Aggregate function as well. > > #2 - "Will this introduce different semantics when State TTL is enabled" - > Can you elaborate on this? TBH, I have not planned the details of the TTL > of ReducingMergeState/AggregatingMergeState variables yet, but the TTL > should be applied on the variable, not on individual operands. I will add a > section on TTL of these variables in the FLIP. > > Best, > -Soumitra. > > On Mon, Jun 1, 2026 at 3:03 AM Han Yin <[email protected]> wrote: > >> Hi Sumatra, >> Thanks for the FLIP. The ability to leverage RocksDB merge operators in >> Reducing/Aggregating state is a really meaningful improvement. >> >> I share similar concerns about the user interface with the previous >> comments: >> • If new state types are introduced, can users migrate their existing >> jobs from ReducingState to ReducingMergeState? Since the core logic of the >> ReduceFunction remains the same, one would expect a straightforward >> migration path. If yes, will checkpoints/savepoints remain compatible >> across this switch (and back)? >> • Will this introduce different semantics when State TTL is enabled? >> >>
