Re: Feature request: Removing state from operators

2020-11-02 Thread Peter Westermann
e: Feature request: Removing state from operators It seems like another option here would be to occasionally use the state processor API to purge a savepoint of all unnecessary state. On Fri, Oct 30, 2020 at 6:57 PM Steven Wu mailto:stevenz...@gmail.com>> wrote: not a solution, but a potential w

Re: Feature request: Removing state from operators

2020-10-30 Thread David Anderson
a state >> >> void dropState(StateDescriptor >> stateDescriptor); >> >> >> >> >> >> Thanks, >> >> Peter >> >> >> >> >> >> >> >> *From: *Congxian Qiu >> *Date: *Thursday, October

Re: Feature request: Removing state from operators

2020-10-30 Thread Steven Wu
state > > void dropState(StateDescriptor stateDescriptor); > > > > > > Thanks, > > Peter > > > > > > > > *From: *Congxian Qiu > *Date: *Thursday, October 29, 2020 at 10:38 AM > *To: *Robert Metzger > *Cc: *Peter Westermann , "user@flin

Re: Feature request: Removing state from operators

2020-10-29 Thread Peter Westermann
> listStates(); // Completely remove a state void dropState(StateDescriptor stateDescriptor); Thanks, Peter From: Congxian Qiu Date: Thursday, October 29, 2020 at 10:38 AM To: Robert Metzger Cc: Peter Westermann , "user@flink.apache.org" Subject: Re: Feature request: Removing state f

Re: Feature request: Removing state from operators

2020-10-29 Thread Congxian Qiu
Hi Peter Can applyToAllKeys[1] in KeyedStateBackend help you here? but currently, this is not exposed to users now. [1] https://github.com/apache/flink/blob/fada6fb6ac9fd7f6510f1f2d77b6baa06563e222/flink-runtime/src/main/java/org/apache/flink/runtime/state/KeyedStateBackend.java#L65 Best, Co

Re: Feature request: Removing state from operators

2020-10-27 Thread Robert Metzger
Hi Peter, I'm adding two committers to this thread who can help answering your question. On Mon, Oct 26, 2020 at 3:22 PM Peter Westermann wrote: > We use the feature for removing stateful operators via the > *allowNonRestoredState* relatively often and it works great. However, > there doesn’t s

Feature request: Removing state from operators

2020-10-26 Thread Peter Westermann
We use the feature for removing stateful operators via the allowNonRestoredState relatively often and it works great. However, there doesn’t seem to be anything like that for removing state from an existing operator (that we want to keep). Say my operator defines a MapState and a ValueState. Lat