Re: Introduction

2016-07-31 Thread Márton Balassi
Welcome Neelesh, great to have you here. :) On Sun, Jul 31, 2016, 11:08 Neelesh Salian wrote: > Hello folks, > > I am Neelesh Salian; I recently joined the Flink community and I wanted to > take this opportunity to formally introduce myself. > > I have been working with the Hadoop and Spark ecos

Re: [DISCUSS] FLIP-6 - Flink Deployment and Process Model - Standalone, Yarn, Mesos, Kubernetes, etc.

2016-07-31 Thread Kurt Young
Thanks for the great proposal. There are still 2 issues i concerned with which i want to discuss with. #1 Who should decide the resources one operator uses, user or framework? Like how much cpu or memory will cost by my "map" operator, does it seem a little bit too low level for the users, should

Introduction

2016-07-31 Thread Neelesh Salian
Hello folks, I am Neelesh Salian; I recently joined the Flink community and I wanted to take this opportunity to formally introduce myself. I have been working with the Hadoop and Spark ecosystems over the past two years and found Flink really interesting in the Streaming use case. Excited to s

Re: [VOTE] Release Apache Flink 1.1.0 (RC1)

2016-07-31 Thread Ufuk Celebi
Probably related to shading :( What's strange is that Travis builds for Hadoop 2.6.3 with the release-1.1 branch do succeed (sometimes... Travis is super flakey at the moment, because of some corrupted cached dependencies): https://travis-ci.org/apache/flink/jobs/148348699 On Fri, Jul 29, 2016 at

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-07-31 Thread Aljoscha Krettek
@Stephan: For the Output, should we keep using a Collector (which exposes) the close() method which should never be called by users or create a new Output type that only has an "output" method. Collector can also be used but with a close() method that doesn't do anything. In the long run, I thought

Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

2016-07-31 Thread Aljoscha Krettek
Hi, regarding a), b) and c): The WindowOperator can be extended to have this signature: public class WindowOperator> that way the shape of state is generic and EvictingWindowOperator can use ListState there. regarding 2.: Yes, we can either take the current processing time/event time or the max t

Re: [DISCUSS] FLIP-6 - Flink Deployment and Process Model - Standalone, Yarn, Mesos, Kubernetes, etc.

2016-07-31 Thread Aljoscha Krettek
+1 I don't have much to say since this already seems very well worked out. Just some small remarks: - This sentence that describes TaskManager behavior will probably have to be adapted for FLIP-1, correct? "Loss of connection to the JobManager results in triggering master-failure recovery (curren