Re: Would like to add a connector to DistributedLog.

2016-05-28 Thread Sijie Guo
Ah, that's cool. Look forward to your flink connector. we are willing to offer any helps on DL. - Sijie On Sat, May 28, 2016 at 9:33 PM, Jia Zhai wrote: > +Distributedlog groups to get pertential help and instructions. > Thanks. > > On Sun, May 29, 2016 at 11:33 AM, Jia Zhai wrote: > >> Hi, >>

Re: Would like to add a connector to DistributedLog.

2016-05-28 Thread Tzu-Li (Gordon) Tai
Hi Jia, There's currently no dedicated documentation for composing streaming connectors, but for examples, I would say that the Elasticsearch and Nifi connectors are a good place to start learning. For a more full blown example I recommend taking a look at the Kafka connector. Basically, you'll ex

[jira] [Created] (FLINK-3989) Add guide to write a custom streaming source / sink in DataStream API documentation

2016-05-28 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-3989: -- Summary: Add guide to write a custom streaming source / sink in DataStream API documentation Key: FLINK-3989 URL: https://issues.apache.org/jira/browse/FLINK-3989

Re: Would like to add a connector to DistributedLog.

2016-05-28 Thread Jia Zhai
+Distributedlog groups to get pertential help and instructions. Thanks. On Sun, May 29, 2016 at 11:33 AM, Jia Zhai wrote: > Hi, > I would like to add a connector to DistributedLog, which recently > published by twitter. :) > Is there any special instructions or best practise to add a new > conne

Would like to add a connector to DistributedLog.

2016-05-28 Thread Jia Zhai
Hi, I would like to add a connector to DistributedLog, which recently published by twitter. :) Is there any special instructions or best practise to add a new connector? Any suggestion is very appreciated. The JIRA tickets(FLINK-3987 and FLINK-398

[jira] [Created] (FLINK-3988) Add data set connector to DistributedLog

2016-05-28 Thread Jia Zhai (JIRA)
Jia Zhai created FLINK-3988: --- Summary: Add data set connector to DistributedLog Key: FLINK-3988 URL: https://issues.apache.org/jira/browse/FLINK-3988 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-3987) Add data stream connector to DistributedLog

2016-05-28 Thread Jia Zhai (JIRA)
Jia Zhai created FLINK-3987: --- Summary: Add data stream connector to DistributedLog Key: FLINK-3987 URL: https://issues.apache.org/jira/browse/FLINK-3987 Project: Flink Issue Type: Improvement

Re: Iteration Intermediate Output

2016-05-28 Thread Vasiliki Kalavri
Hey, it would be great to add this feature indeed! Thanks for bringing it up Greg :) Would the best way be to extend the iteration operators to support intermediate outputs or revisit the idea of caching intermediate results and thus allow efficient for-loop iterations? -Vasia. On 26 May 2016 at

Re: Hotfixes on the master

2016-05-28 Thread Vasiliki Kalavri
Hi all, in principle I agree with Max. I personally avoid hotfixes and always open a PR, even for javadoc improvements. I believe the main problem is that we don't have a clear definition of what constitutes a "hotfix". Ideally, even cosmetic changes and documentation should be reviewed; I've see

[jira] [Created] (FLINK-3986) Rename the readFileStream(String filePath, long intervalMillis, WatchType watchType) from the StreamExecutionEnvironment

2016-05-28 Thread Kostas Kloudas (JIRA)
Kostas Kloudas created FLINK-3986: - Summary: Rename the readFileStream(String filePath, long intervalMillis, WatchType watchType) from the StreamExecutionEnvironment Key: FLINK-3986 URL: https://issues.apache.org/

[jira] [Created] (FLINK-3985) A metric with the name * was already registered

2016-05-28 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-3985: - Summary: A metric with the name * was already registered Key: FLINK-3985 URL: https://issues.apache.org/jira/browse/FLINK-3985 Project: Flink Issue Type: B