Having thought about it more, HashMapStateBackend has won me over. I'll update the FLIP. If there aren't any more comments I'll open it up for voting on monday.
Seth On Wed, Sep 9, 2020 at 9:09 AM Seth Wiesman <sjwies...@gmail.com> wrote: > @Yun yes, this is really about making CheckpointStorage an orthogonal > concept. I think we can remain pragmatic and keep state-backend specific > configurations (async, incremental, etc) in the state backend themselves. I > view these as more advanced configurations and by the time someone is > changing the defaults they likely understand what is going on. My goal is > to help on-board users and so long as each state backend has a no-arg > default constructor that works for many users I think we've achieved that > goal. > > Regarding the checkpoint coordinator, that makes sense but I will consider > out of the scope of this FLIP. I want to focus on simplifying APIs. > > @Aljoscha Krettek <aljos...@apache.org> > > My feeling is that state backends and checkpointing are going to be > integral to Flink for many years, regardless or other enhancements so this > change is still valuable. > > Since this is a FLIP about improving the user api I'm happy to bikeshed > the names a little more than normal. HashMap makes sense, my other thought > was InMemory. > > Seth > > > > On Wed, Sep 9, 2020 at 8:04 AM Aljoscha Krettek <aljos...@apache.org> > wrote: > >> I like it a lot! >> >> I think it makes sense to clean this up despite the planned new >> fault-tolerance mechanisms. In the future, users will decide which >> mechanism to use and I can imagine that a lot of them will keep using >> the current mechanism for quite a while to come. But I'm happy to yield >> to Stephan's opinion here, he knows more about the progress of that work. >> >> The one nitpick I have is about naming: will users understand >> OnHeapStateBackend? I mean, do they know what on-heap/off-heap memory is >> and the tradeoffs? An alternative could be HashMapStateBackend, because >> that's essentially what it is. I wouldn't block anything on this, though. >> >> Aljoscha >> >> On 09.09.20 10:05, Konstantin Knauf wrote: >> > Thanks for the initiative. Big +1. Would be interested to hear if the >> > proposed interfaces still make sense in the face of the new >> fault-tolerance >> > work that is planned. Stephan/Piotr will know. >> > >> > On Tue, Sep 8, 2020 at 7:05 PM Seth Wiesman <sjwies...@gmail.com> >> wrote: >> > >> >> Hi Devs, >> >> >> >> I'd like to propose an update to how state backends and checkpoint >> storage >> >> are configured to help users better understand Flink. >> >> >> >> Apache Flink's durability story is a mystery to many users. One of the >> most >> >> common recurring questions from users comes from not understanding the >> >> relationship between state, state backends, and snapshots. Some of this >> >> confusion can be abated with learning material but the question is so >> >> pervasive that we believe Flink’s user APIs should be better >> communicate >> >> what different components are responsible for. >> >> >> >> >> >> >> >> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-142%3A+Disentangle+StateBackends+from+Checkpointing >> >> >> >> >> >> I look forward to a healthy discussion. >> >> >> >> >> >> Seth >> >> >> > >> > >> >>
