Hi, The idea sounds like a nice improvement to complete the feature. I don't have any comments on top of what has been written already above.
Bets, Piotrek czw., 19 sty 2023 o 09:57 Etienne Chauchot <echauc...@apache.org> napisał(a): > Hi, > > In the future we could add new compression algorithms by simply > extending /StreamCompressionDecorator/. For now there is only 2 > extensions: /UncompressedStreamCompressionDecorator/ and > /SnappyStreamCompressionDecorator/. > > But I agree, I'd stick to /SnappyStreamCompressionDecorator /which is in > use for keyed state compression. > > I'll add a comment on the FLIP saying that. > > Best > > Etienne > > Le 18/01/2023 à 16:36, Dawid Wysakowicz a écrit : > > For the compression formats I'd stick with what is supported for the > > keyed state, which is either enable or disable compression. The format > > used for keyed state backend compression is snappy. > > > > Best, > > > > Dawid > > > > On 18/01/2023 15:33, ConradJam wrote: > >> Thanks for driver this flip,I think have some question about this flip > >> > >> - Can we select the compression format, for example？ such as > >> snappy or > >> lz0 > >> - If compression format is supported, I think the metadata will > >> probably > >> have to include which compression format > >> > >> and I agree idea with Dawid > >> > >> Dawid Wysakowicz <dwysakow...@apache.org> 于2023年1月18日周三 20:55写道： > >> > >> > >>> It makes sense from my side. > >>> > >>> Could you, just for completeness, extend it with the info what will be > >>> the compression unit? If understand it correctly you envision each > >>> state > >>> to be compressed separately. > >>> > >>> Best, > >>> > >>> Dawid > >>> > >>> On 17/01/2023 15:47, Etienne Chauchot wrote: > >>>> Hi everyone, > >>>> > >>>> I just published a new FLIP to introduce operator state compression > >>>> (1). This feature is pretty straightforward to implement, so a PR is > >>>> already under review (2) but it appeared during the review that to be > >>>> able to insure backward compatibility of the operator state snapshots, > >>>> we needed to modify the snapshot format hence the FLIP. > >>>> > >>>> Please tell me if you have any thoughts about it. > >>>> > >>>> > >>>> [1] > >>>> > >>> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-290+Operator+state+compression > >>> > >>>> > >>>> [2] https://github.com/apache/flink/pull/21636 > >>>> > >>>> > >>>> Best, > >>>> > >>>> Etienne > >>>> > >>
