Re: ResourceManager not using correct akka URI in standalone cluster (?)

2016-09-15 Thread AJ Heller
More information: >From the master node, I cannot `telnet localhost 6123` nor `telnet 6123` while the cluster is apparently running. Connection refused immediately. `netstat -n | grep 6123` is empty. There's no server listening. But the processes are running on all machines. Does it matter that

ResourceManager not using correct akka URI in standalone cluster (?)

2016-09-15 Thread AJ Heller
I'm running a standalone cluster on Amazon EC2. Leader election is happening according to the logs, and the Flink Dashboard is up and running, accessible remotely. The issue I'm having is that the SocketWordCount example is not working, the local connection is being refused! In the Flink Dashboard

Re: Streaming - memory management

2016-09-15 Thread Fabian Hueske
Thanks for the pointer and sorry for the late answer. I guess that depends on the semantics of "checkpointing". In Flink's terminology this means creating a copy of the state (and writing the copy to the external FS). It does not mean that the state is migrated or moved to the external FS. Best, F

Re: Why tuples are not ignored after watermark?

2016-09-15 Thread Fabian Hueske
No, this is not possible unless you use an external service such as a database. The assigners might run on different machines and Flink does not provide utilities for r/w shared state. Best, Fabian 2016-09-15 20:17 GMT+02:00 Saiph Kappa : > And is it possible to share state across parallel insta

Re: Why tuples are not ignored after watermark?

2016-09-15 Thread Saiph Kappa
And is it possible to share state across parallel instances with AssignerWithPunctuatedWatermarks? Thanks! On Wed, Sep 14, 2016 at 9:52 AM, Aljoscha Krettek wrote: > Hi, > the problem might be that your timestamp/watermark assigner is run in > parallel and that only one parallel instance of tho

Re: Streaming issue help needed

2016-09-15 Thread Gábor Gévay
Hello Vaidya, The error message is talking about the class "AbstractTime", which was removed long ago (before 1.0). Could you please double check that the Flink version is set appropriately everywhere, and that the binaries running in the cluster are really 1.1.2? Best, Gábor 2016-09-15 11:52

Re: Data Transfer between TM should be encrypted

2016-09-15 Thread Vijay Srinivasaraghavan
Hi Vinay, There are some delays and we expect the PR to be created next week.  RegardsVijay On Wednesday, September 14, 2016 5:41 PM, vinay patil wrote: Hi Vijay, Did you raise the PR for this task, I don't mind testing it out as well. Regards,Vinay Patil On Tue, Aug 30, 2016 at 6:28 PM

FlinkCEP for large key spaces and long timeouts between events

2016-09-15 Thread David Koch
Hello, Is FlinkCEP applicable to large key spaces with potentially long timeouts between events that define a pattern? Ideally, without ridiculous hardware. More concretely, we segment users (one key per user) based on sequences of events for that user. A segment "Abandoned Cart" may be defined

Re: Streaming issue help needed

2016-09-15 Thread Vaidyanathan Sivasubramanian
Dear Marton, Thanks very much for taking time to look into this issue. Actually I am on latest Flink version (1.1.2). Would the Hadoop version might be causing the problem (it is 2.6 with CDH 5.5.1)? Would appreciate any more pointers to resolve this. Thanks, Vaidya. > On 14-Sep-2016, at 4

Re: SQL for Flink

2016-09-15 Thread Timo Walther
Hi Radu, thanks for continuing the discussion we had at the conference here. Your proposals are all valid. If you have a look at the inital design document [1] for Table API/SQL we plan to add a SQL client at some point, but first we should focus on extending the set of supported operations.