Re: Cassandra POJO sink flink 1.4.0 in scala

Hi, shashank agarwal Not sure if I can answer fully your question, but after digging some code, I am not sure if C* connector totally supports Scala case class + CQL data mapping at the moment. I may be totally wrong, and you need to ask

Re: Cassandra POJO sink flink 1.4.0 in scala

Hi, shashank agarwal AFAIK, in java side, for a pojo data type, you don't need to set query since the CQL data mapping would take care of that whereas dealing with java tuples, you do need to provide a upsert query so that cassandra knows what to insert into the table. Scala tuple case is clear,

Re: org.apache.zookeeper.ClientCnxn, Client session timed out

Thanks! Great to know I do not have to worry duplicates inside Flink. One more question, why this happens? Because TM and JM both check leadership in different interval? > The TM noticed the loss of leadership before the JM did. On Wed, Dec 27, 2017, 13:52 Ufuk Celebi wrote: > On Wed, Dec 27, 2

Re: org.apache.zookeeper.ClientCnxn, Client session timed out

On Wed, Dec 27, 2017 at 4:41 PM, Hao Sun wrote: > Somehow TM detected JM leadership loss from ZK and self disconnected? > And couple of seconds later, JM failed to connect to ZK? > Yes, exactly as you describe. The TM noticed the loss of leadership before the JM did. > After all the cluster re

Re: A question about Triggers

This makes sense. Thanks. On Sat, Dec 23, 2017 at 10:58 AM, Fabian Hueske wrote: > Hi, > > all calls to onElement() or onTimer() are syncronized for any keys. Think > of a single thread calling these methods. > Event-time timers are called when a watermark passes the timer. Watermarks > are rec

org.apache.zookeeper.ClientCnxn, Client session timed out

Hi I need some help to figure out the root cause of this error. I am running flink 1.3.2 on K8S. My cluster has been up and running for almost two weeks and all of a sudden I see this familiar error again, my task manager is killed/lost. There are many ways cause this error, I need help to figure

Re: MergingWindow

Please check your email before sending it the next time as three emails for the same message is a little spammy ;-) This is internal code that is used to implement session windows as far as I can tell. The idea is to not merge the new window as it never had any state associated with it. The genera

Re: Apache Flink - broadcasting DataStream

Hey Mans! This refers to how sub tasks are connected to each other in your program. If you have a single sub task A1 and three sub tasks B1, B2, B3, broadcast will emit each incoming record at A1 to all B1, B2, B3: A1 --+-> B1 +-> B2 +-> B3 Does this help? On Mon, Dec 25, 2017 at 7:12

Re: flink yarn-cluster run job --files

The file URL needs to be accessible from all nodes, e.g. something like S3://... or hdfs://... >From the CLI: ``` Adds a URL to each user code classloader on all nodes in the cluster. The paths must specify a protocol (e.g. file://) and be accessible on all nodes (e.g. by means of a NFS share).

Re: Fetching TaskManager log failed

Thanks for reporting this issue. A few questions: - Which version of Flink are you using? - Does it work up to the point that the Exception is thrown? e.g. for smaller files it's OK? Let me pull in Chesnay (cc'd) who has worked on log fetching for the web runtime. – Ufuk On Tue, Dec 26, 2017 a

Re: How to stop FlinkKafkaConsumer and make job finished?

Hey Jaxon, I don't think it's possible to control this via the life-cycle methods of your functions. Note that Flink currently does not support graceful stop in a meaningful manner and you can only cancel running jobs. What comes to my mind to cancel on EOF: 1) Extend Kafka consumer to stop emit

Re: Flink network access control documentation

Hey Elias, thanks for opening a ticket (for reference: https://issues.apache.org/jira/browse/FLINK-8311). I fully agree with adding docs for this. I will try to write something down this week. Your point about JobManagers only coordinating via ZK is correct though. I had a look into the JobManage