Re: Serious memory leak in DefaultOperatorStateBackend

2018-02-22 Thread Gyula Fóra
hance the state was not registered > „fast > >>> enough“. As I see it, the proper way is the register the state under > the > >>> same name and clear it if you want to get rid of the data. There is > >>> currently no call that explicitly drops a state that

Re: Serious memory leak in DefaultOperatorStateBackend

2018-02-22 Thread Stefan Richter
urrently no call that explicitly drops a state that was once declared, and >> you might make a case that this is a feature to have for the future. Then >> again, we need a general decision about lazy and eager state IMO. >> >> Best, >> Stefan >> >>>

Re: Serious memory leak in DefaultOperatorStateBackend

2018-02-22 Thread Stefan Richter
e future. Then >>> again, we need a general decision about lazy and eager state IMO. >>> >>> Best, >>> Stefan >>> >>>> Am 22.02.2018 um 11:10 schrieb Gyula Fóra : >>>> >>>> Hi all, >>>> >>>&g

Re: Serious memory leak in DefaultOperatorStateBackend

2018-02-22 Thread Gyula Fóra
Best, > Stefan > > > Am 22.02.2018 um 11:10 schrieb Gyula Fóra : > > > > Hi all, > > > > We have discovered a fairly serious memory leak > > in DefaultOperatorStateBackend, with broadcast (union) list states. > > > > The problem seems to occur when

Re: Serious memory leak in DefaultOperatorStateBackend

2018-02-22 Thread Stefan Richter
Am 22.02.2018 um 11:10 schrieb Gyula Fóra : > > Hi all, > > We have discovered a fairly serious memory leak > in DefaultOperatorStateBackend, with broadcast (union) list states. > > The problem seems to occur when a broadcast state name is changed, in order > to drop some state

Serious memory leak in DefaultOperatorStateBackend

2018-02-22 Thread Gyula Fóra
Hi all, We have discovered a fairly serious memory leak in DefaultOperatorStateBackend, with broadcast (union) list states. The problem seems to occur when a broadcast state name is changed, in order to drop some state (intentionally). Flink does not drop the "garbage" broadcast