Re: Updating external service and then processing response

Hi Michael, Yeah, I feel the updating of state in an external system is a job for a sink. However, I don't really like the idea of combining both a web service call and DB write in the one sink class because it's breaking single responsibility - I feel there should be a nicer way to compose these

Re: Setting the parallelism in a cluster of machines properly

Guys seriously I have done the process as described in the documentation of the standalone cluster 20 times. After I start the cluster with ./start-cluster.sh, I normally see with jps the JobManager process running in the master and the TaskManager processes running in slaves. Although every time I

Re: Updating external service and then processing response

If the external web service call does not modify the state of that external system all the approaches you list are probably ok. If there is external state modification then you want to ensure on restart the Flink job does not resend requests to that service or that it can handle duplicate reque

Re: Batch job stuck in Canceled state in Flink 1.5

Cluster is running on commit 2af481a On Sun, Apr 29, 2018 at 9:59 PM, Amit Jain wrote: > Hi, > > We are running numbers of batch jobs in Flink 1.5 cluster and few of those > are getting stuck at random. These jobs having the following failure after > which operator status changes to CANCELED and

Batch job stuck in Canceled state in Flink 1.5

Hi, We are running numbers of batch jobs in Flink 1.5 cluster and few of those are getting stuck at random. These jobs having the following failure after which operator status changes to CANCELED and stuck to same. Please find complete TM's log at https://gist.github.com/imamitjain/066d0e0ee2

JM/TM Disassociated

Hi, I have some simple elesticsearch sink on mesos with 6 TM , but for some reason after few hours JM/TM is Disassociated and killed all other TM as well . this is were everything collapse 2018-04-29 08:21:00,457 WARN akka.remote.ReliableDeliverySupervisor - Associati

Re: Raspberry Pi Memory Configuration

Thanks for the reply, but I tracked the problem down to a missing M in task m= anager.sh # export JVM_ARGS=3D"${JVM_ARGS} -Xms${TM_HEAP_SIZE}M -Xmx${TM_HEAP_SIZE}M -= XX:MaxDirectMemorySize=3D${TM_MAX_OFFHEAP_SIZE}" export JVM_ARGS=3D"${JVM_ARGS} -Xms${TM_HEAP_SIZE}M -Xmx${TM_HEAP_SIZE}M" I had t