Thank you for the clarification.
On Thu, Jun 21, 2018 at 1:36 PM sihua zhou wrote:
> Yes, you can clear the state for a key(the currently active key), if you
> clear it, it means that you have also cleaned it from the state backend,
> and the future checpoints won't contains the key anymore unle
Yes, you can clear the state for a key(the currently active key), if you clear
it, it means that you have also cleaned it from the state backend, and the
future checpoints won't contains the key anymore unless you add it again.
Best, Sihua
On 06/21/2018 16:04，Garvit Sharma wrote：
Now, after
Now, after clearing state for a key, I don't want that redundant data in
the state backend. This is my concern.
Please let me know if there are any gaps.
Thanks,
On Thu, Jun 21, 2018 at 1:31 PM Garvit Sharma wrote:
> I am maintaining state data for a key in ValueState. As per [0] I can
> clear
I am maintaining state data for a key in ValueState. As per [0] I can
clear() state for that key.
[0]
https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/stream/state/state.html
Please let me know.
Thanks,
On Thu, Jun 21, 2018 at 1:19 PM sihua zhou wrote:
> Hi Garvit,
>
> Let's s
Hi Garvit,
Let's say you clearing the state at timestamp t1, then the checkpoints
completed before t1 will still contains the data you cleared. But the future
checkpoints won't contain the cleared data again. But I'm not sure what you
meaning by the cleared state, you can only clear a key-valu
So, would it delete all the files in HDFS associated with the cleared state?
On Thu, Jun 21, 2018 at 12:58 PM sihua zhou wrote:
> Hi Garvit,
>
> > Now, let's say, we clear the state. Would the state data be removed from
> HDFS too?
>
> The state data would not be removed from HDFS immediately, i
Hi,
Consider a managed keyed state backed by HDFS with checkpointing enabled.
Now, as the state grows the state data will be saved on HDFS.
Now, let's say, we clear the state. Would the state data be removed from
HDFS too?
How does Flink manage to clear the state data from state backend on
clear
