gt;> very
>> > > little difference. For us, as developers, the fewer things we
>> officially
>> > > support and claim are stable, the better.
>> > >
>> > > > 2. If self-contained and relocatable are the most important
>> > difference,
ost important
> > difference,
> > > why not include them in the proposal table?
> > >
> > > Good point. I will add this.
> > >
> > > > What does "Job full upgrade" means?
> > >
> > > I have clarified it to:
> > > > Arbi
; > category "Job upgrade w/o changing graph shape and record types"
> >
> > Best,
> > Piotrek
> >
> > pon., 17 sty 2022 o 11:25 Yun Tang napisał(a):
> >
> > > Hi everyone,
> > >
> > > Thanks for Piotr to drive this topic.
> > >
> > > I have several questions on this
are the most important
> difference,
> > why not include them in the proposal table?
> >
> > 1. What does "Job full upgrade" means?
> >
> > For the question of RocksDB upgrading, this depends on the backwards
> > compatibility [1], and it proves
aid.
>
>
> [1]
> https://github.com/facebook/rocksdb/wiki/RocksDB-Compatibility-Between-Different-Releases
>
> Best，
> Yun Tang
>
>
>
>
> From: Konstantin Knauf
> Sent: Friday, January 14, 2022 20:39
> To: dev ; Seth Wiesman
B-Compatibility-Between-Different-Releases
Best，
Yun Tang
From: Konstantin Knauf
Sent: Friday, January 14, 2022 20:39
To: dev ; Seth Wiesman ; Nico Kruber
; dander...@apache.org
Subject: Re: [DISCUSS] FLIP-203: Incremental savepoints
Hi everyone,
Thank you
Hi everyone,
Thank you, Piotr. Please find my thoughts on the topic below:
0) What exactly does the "State Processor API" row mean? Is it: Can it be
read by the State Processor API? Can it be written by the State Processor
API? Both? Something else?
1) If we take the assumption from FLIP-193 "th
> I have a very similar question to State Processor API. Is it the same
scenario in this case?
> Should it also be working with checkpoints but might be just untested?
I have used the State Processor API with aligned, full checkpoints. There
it has worked just fine.
David
On Thu, Jan 13, 2022 at
Thanks for the update, Piotr!
> Is `state.backend.incremental` the only configuration parameter that can
be
> used in this context?
According to FLIP-193 [1], all the existing checkpoint configurations are
actually for *Snapshot*, ownership (lifecycle) is the only difference
between Checkpoints an
Hi,
Thanks for the comments and questions. Starting from the top:
Seth: good point about schema evolution. Actually, I have a very similar
question to State Processor API. Is it the same scenario in this case?
Should it also be working with checkpoints but might be just untested?
And next questi
Hi Piotr,
would it be possible to provide a table that shows the
compatibility guarantees provided by the different snapshots going forward?
Like type of change (Topology. State Schema, Parallelism, ..) in one
dimension, and type of snapshot as the other dimension. Based on that, it
would be easie
Hi Piotr,
does this mean that we need to keep the checkpoints compatible across minor
versions? Or can we say, that the minor version upgrades are only
guaranteed with canonical savepoints?
My concern is especially if we'd want to change layout of the checkpoint.
D.
On Wed, Dec 29, 2021 at 5:
Thanks for the proposal Piotr! Overall I'm +1 for the idea, and below are
my two cents:
1. How about adding a "Term Definition" section and clarify what "native
format" (the "native" data persistence format of the current state backend)
and "canonical format" (the "uniform" format that supports sw
>> AFAIK state schema evolution should work both for native and canonical
>> savepoints.
Schema evolution does technically work for both formats, it happens after
the code paths have been unified, but the community has up until this point
considered that an unsupported feature. From my perspective
Hi Konstantin,
> In this context: will the native format support state schema evolution? If
> not, I am not sure, we can let the format default to native.
AFAIK state schema evolution should work both for native and canonical
savepoints.
Regarding what is/will be supported we will document as pa
Hi Piotr,
Thanks a lot for starting the discussion. Big +1.
In my understanding, this FLIP introduces the snapshot format as a *really*
user facing concept. IMO it is important that we document
a) that it is not longer the checkpoint/savepoint characteristics that
determines the kind of changes
Hi devs,
I would like to start a discussion about a previously announced follow up
of the FLIP-193 [1], namely allowing savepoints to be in native format and
incremental. The changes do not seem invasive. The full proposal is
written down as FLIP-203: Incremental savepoints [2]. Please take a look
17 matches
Mail list logo
