Re: Checkpointing and save pointing

Hi, Boris TM will also need to write to the external volume. Best, Congxian On May 8, 2019, 03:56 +0800, Boris Lublinsky , wrote: > I am planning to use external volume for this. My understanding is that it > needs to be mounted only to the job manager, not the task managers. Is this > correct

Testing with ProcessingTime and FromElementsFunction with TestEnvironment

Hello! I am trying to write a test that runs in the TestEnviroment. I create a process that uses ProcessingTime, has a source constructed from a FromElementsFunction and runs data through a Keyed Stream into a ProcessingTimeSessionWindows.withGap(). The problem is that it appears that the env.exe

Checkpointing and save pointing

I am planning to use external volume for this. My understanding is that it needs to be mounted only to the job manager, not the task managers. Is this correct, or it needs to be mounted to both? Boris Lublinsky FDP Architect boris.lublin...@lightbend.com https://www.lightbend.com/

Getting async function call terminated with an exception

Hi, We are using flink 1.8.0 (but the flowing also happens in 1.7.2) I tried very simple unordered async call override def asyncInvoke(input: Foo, resultFuture: ResultFuture[ScoredFoo]) : Unit = { val r = ScoredFoo(Foo("a"), 80) Future.successful(r) } Running this stream seem to be stuck

Re: I want to use MapState on an unkeyed stream

But I only have one stream, nothing to connect it to. On 2019/05/07 00:15:59, Averell wrote: > From my understanding, having a fake keyBy (stream.keyBy(r => "dummyString")) > means there would be only one slot handling the data. > Would a broadcast function [1] work for your case? > > Regards,

Migration from flink 1.7.2 to 1.8.0

Hi We are migrating our app to Flink 1.8.0. We built a docker image like this as Hadoop is not anymore bundled : FROM myrepo:5/flink:1.8.0-scala_2.11-alpine ADD --chown=flink:flink https://my-artifactory-repo/artifactory/my-repo/org/apache/flink/flink-shaded-hadoop2-uber/2.8.3-1.8.0/flink-s

Re: Class loader premature closure - NoClassDefFoundError: org/apache/kafka/clients/NetworkClient

It is a blocker for exactly once support from flink kafka producer. This issue reported and closed. but still reproducible https://issues.apache.org/jira/browse/FLINK-10455 On Mon, May 6, 2019 at 10:20 AM Slotterback, Chris < chris_slotterb...@comcast.com> wrote: > Hey Flink users, > > > > Curre

Re: Apache Flink - How to destroy global window and release it's resources

Hi, There should be no window or trigger object kept per window. Aljoscha > On 15. Apr 2019, at 10:22, Fabian Hueske wrote: > > Hi, > > Aljoscha know the implementation best (since he implemented it). > > From my understanding (Aljoscha please correct me if I'm wrong), all Flink > managed

Flink on YARN: TaskManager heap auto-sizing?

In the Configuration section of the docs , the description for "taskmanager.heap.size" contains: "On YARN setups, this value is automatically configured to the size of the TaskManager's YARN container, minus a

Re: flink 1.7 HA production setup going down completely

im using 1.7.2. On Tue, May 7, 2019 at 5:50 PM miki haiat wrote: > Which flink version are you using? > I had similar issues with 1.5.x > > On Tue, May 7, 2019 at 2:49 PM Manjusha Vuyyuru > wrote: > >> Hello, >> >> I have a flink setup with two job managers coordinated by zookeeper. >> >> I s

Re: flink 1.7 HA production setup going down completely

Which flink version are you using? I had similar issues with 1.5.x On Tue, May 7, 2019 at 2:49 PM Manjusha Vuyyuru wrote: > Hello, > > I have a flink setup with two job managers coordinated by zookeeper. > > I see the below exception and both jobmanagers are going down: > > 2019-05-07 08:29:13,

flink 1.7 HA production setup going down completely

Hello, I have a flink setup with two job managers coordinated by zookeeper. I see the below exception and both jobmanagers are going down: 2019-05-07 08:29:13,346 INFO org.apache.flink.runtime.jobmanager.ZooKeeperSubmittedJobGraphStore - Released locks of job graph f8eb1b482d8ec8c1d3e94c4d0f79d

Re: TM occasionally hang in deploying state in Flink 1.5

Thanks Dawid, I’ve created an issue for this https://jira.apache.org/jira/browse/FLINK-12426 . Though we’re using 1.5 but this may affect later versions. I’m still investigating the root case but no result yet. This happens occasionally and isn'

How to export all not-null keyed ValueState

Hi, I have a keyed value state which is available for only about 1% the total number of keyed values that I have. Is there any way to get the values of all those state values? I looked at the queryable state option, but it looks like supporting querying by keyed value only. Thanks and best rega