What's the advantage of using BroadcastState?

2018-08-18 Thread Paul Lam
Hi, AFAIK, the difference between a BroadcastStream and a normal DataStream is that the BroadcastStream is with a BroadcastState, but it seems that the functionality of BroadcastState can also be achieved by MapState in a CoMapFunction or something since the control stream is still broadcasted wit

Re: processWindowFunction

2018-08-18 Thread antonio saldivar
hi Vino it is possible to use global window, then set the trigger onElement comparing the element that has arrived with for example 10 mins, 20 mins and 60 mins of data? I have rules evaluating sum of amount for 10,20 or 60 mins for the same keyed element if the same id sum like $200 total within

Job Manager killed by Kubernetes during recovery

2018-08-18 Thread Bruno Aranda
Hi, I am experiencing an issue when a job manager is trying to recover using a HA setup. When the job manager starts again and tries to resume from the last checkpoints, it gets killed by Kubernetes (I guess), since I can see the following in the logs while the jobs are deployed: INFO org.apache

Re: Error in KyroSerializer

2018-08-18 Thread Rong Rong
This sounds very much related to FLINK-10160 [1]. Would you mind upgrading your Flink version to 1.4.3 and try again? Thanks, Rong [1] https://issues.apache.org/jira/browse/FLINK-10160 On Fri, Aug 17, 2018 at 4:20 PM Pankaj Chaudhary wrote: > Hi, > > I am on Flink 1.4.2 and as part of my opera

Re: Flink not rolling log files

2018-08-18 Thread Gary Yao
Hi, When using org.apache.log4j.rolling.RollingFileAppender, it is apparently not allowed to set: log4j.appender.file.file=${log.file} It works for me if I remove this property from the log4j.properties file. Moreover, you have configured: log4j.appender.file.RollingPolicy.FileNamePatt