t
Yun Tang
*From:* Dawid Wysakowicz
*Sent:* Friday, January 20, 2023 16:41
*To:* dev@flink.apache.org; Yun Tang
*Subject:* Re: [DISCUSS] FLIP-290: Operator state compression
(FLINK-30113)
I would not unnecessarily extend the scope of the FLIP. From what I&#
bject:* Re: [DISCUSS] FLIP-290: Operator state compression
(FLINK-30113)
I would not unnecessarily extend the scope of the FLIP. From what I've
gathered having a compression for operator state is good enough.
If requests for more compression formats come I'd start it as a separate
effort
want to introduce more
compression formats in the future.
Best
Yun Tang
From: Dawid Wysakowicz
Sent: Friday, January 20, 2023 16:41
To: dev@flink.apache.org; Yun Tang
Subject: Re: [DISCUSS] FLIP-290: Operator state compression (FLINK-30113)
I would not unnecess
o introduce operator state compression.
[1] https://issues.apache.org/jira/browse/FLINK-11313
Best
Yun Tang
From: Yuan Mei
Sent: Friday, January 20, 2023 12:56
To: dev@flink.apache.org
Subject: Re: [DISCUSS] FLIP-290: Operator state compression (FLINK-30113)
: Operator state compression (FLINK-30113)
The proposal reads quite reasonable!
I do not have additional comments as long as the change can insure backward
compatibility. And many thanks to Dawid for catching this!
Best
Yuan
On Thu, Jan 19, 2023 at 6:03 PM Piotr Nowojski wrote:
> Hi,
>
>
The proposal reads quite reasonable!
I do not have additional comments as long as the change can insure backward
compatibility. And many thanks to Dawid for catching this!
Best
Yuan
On Thu, Jan 19, 2023 at 6:03 PM Piotr Nowojski wrote:
> Hi,
>
> The idea sounds like a nice improvement to comp
Hi,
The idea sounds like a nice improvement to complete the feature. I don't
have any comments on top of what has been written already above.
Bets,
Piotrek
czw., 19 sty 2023 o 09:57 Etienne Chauchot
napisał(a):
> Hi,
>
> In the future we could add new compression algorithms by simply
> extendi
Hi,
In the future we could add new compression algorithms by simply
extending /StreamCompressionDecorator/. For now there is only 2
extensions: /UncompressedStreamCompressionDecorator/ and
/SnappyStreamCompressionDecorator/.
But I agree, I'd stick to /SnappyStreamCompressionDecorator /which
Hi,
@Dawid yes each state is compressed separately as with keyed state. I'll
add this precision to the FLIP.
Best
Etienne
Le 18/01/2023 à 13:55, Dawid Wysakowicz a écrit :
It makes sense from my side.
Could you, just for completeness, extend it with the info what will be
the compression u
For the compression formats I'd stick with what is supported for the
keyed state, which is either enable or disable compression. The format
used for keyed state backend compression is snappy.
Best,
Dawid
On 18/01/2023 15:33, ConradJam wrote:
Thanks for driver this flip,I think have some ques
Thanks for driver this flip,I think have some question about this flip
- Can we select the compression format, for example？ such as snappy or
lz0
- If compression format is supported, I think the metadata will probably
have to include which compression format
and I agree idea with Daw
It makes sense from my side.
Could you, just for completeness, extend it with the info what will be
the compression unit? If understand it correctly you envision each state
to be compressed separately.
Best,
Dawid
On 17/01/2023 15:47, Etienne Chauchot wrote:
Hi everyone,
I just published
Hi everyone,
I just published a new FLIP to introduce operator state compression (1).
This feature is pretty straightforward to implement, so a PR is already
under review (2) but it appeared during the review that to be able to
insure backward compatibility of the operator state snapshots, we
13 matches
Mail list logo
