Re: Would like to add a connector to DistributedLog.

2016-05-29 Thread Jia Zhai
Hi Gordon. Thanks a lot for the information and the Jira ticket. On Sun, May 29, 2016 at 1:37 PM, Tzu-Li (Gordon) Tai wrote: > Hi Jia, > > There's currently no dedicated documentation for composing streaming > connectors, but for examples, I would say that the Elasticsearch and Nifi > connectors

Re: Side-effects of DataSet::count

2016-05-29 Thread Márton Balassi
Hey Eron, Yes, DataSet#collect and count methods implicitly trigger a JobGraph execution, thus they also trigger writing to any previously defined sinks. The idea behind this behavior is to enable interactive querying (the one that you are used to get from a shell environment) and it is also a gre

Side-effects of DataSet::count

2016-05-29 Thread Eron Wright
I was curious as to how the `count` method on DataSet worked, and was surprised to see that it executes the entire program graph. Wouldn’t this cause undesirable side-effects like writing to sinks?Also strange that the graph is mutated with the addition of a sink (that isn’t subsequently r

[jira] [Created] (FLINK-3990) ZooKeeperLeaderElectionTest kills the JVM

2016-05-29 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-3990: --- Summary: ZooKeeperLeaderElectionTest kills the JVM Key: FLINK-3990 URL: https://issues.apache.org/jira/browse/FLINK-3990 Project: Flink Issue Type: Bug

Re: Would like to add a connector to DistributedLog.

2016-05-29 Thread Yijie Shen
Hi Jia, I'm currently working on a stream connector for DL, do you mind if we cooperate on FLINK-3987? If you don't mind, I would do some clean up locally in the next days and share it here. Hi Sijie, I think we should publish DL jars to public repository in order to be referenced by other project

Re: GSoC Project Proposal Draft: Code Generation in Serializers

2016-05-29 Thread Gábor Horváth
Hi! I would like to give you some status updates on the Google Summer of Code project. I started to implement the proposed features [1]. Status of code generation in general: * I can compile the generated code using Janino compiler * I can load the compiled classes and use them * For some mysteri