Re: Pulling Streaming out of staging and project restructure

2016-01-11 Thread Flavio Pompermaier
ectors > > > > > > > > > > > > If nobody disagrees, I'll open a PR for these changes. > > > > I created a JIRA for this discussion: > > > > https://issues.apache.org/jira/browse/FLINK-3205 > > > > > > > > > >

Re: Pulling Streaming out of staging and project restructure

2016-01-11 Thread Stephan Ewen
ges. > > > I created a JIRA for this discussion: > > > https://issues.apache.org/jira/browse/FLINK-3205 > > > > > > > > > On Fri, Oct 2, 2015 at 7:58 PM, fhueske wrote: > > > > > > > +1 > > > > > > > > > &g

Re: Pulling Streaming out of staging and project restructure

2016-01-11 Thread Flavio Pompermaier
ion: > > https://issues.apache.org/jira/browse/FLINK-3205 > > > > > > On Fri, Oct 2, 2015 at 7:58 PM, fhueske wrote: > > > > > +1 > > > > > > > > > From: Henry Saputra > > > Sent: Friday, October 2, 2015 19:34 > > > To

Re: Pulling Streaming out of staging and project restructure

2016-01-06 Thread Fabian Hueske
> > > > > > From: Henry Saputra > > Sent: Friday, October 2, 2015 19:34 > > To: dev@flink.apache.org > > Subject: Re: Pulling Streaming out of staging and project restructure > > > > > > +1 > > > > On Friday, October 2, 2015, Matthias J. Sax w

Re: Pulling Streaming out of staging and project restructure

2016-01-06 Thread Robert Metzger
8 PM, fhueske wrote: > +1 > > > From: Henry Saputra > Sent: Friday, October 2, 2015 19:34 > To: dev@flink.apache.org > Subject: Re: Pulling Streaming out of staging and project restructure > > > +1 > > On Friday, October 2, 2015, Matthias J. Sax wrote: > &g

RE: Pulling Streaming out of staging and project restructure

2015-10-02 Thread fhueske
+1 From: Henry Saputra Sent: Friday, October 2, 2015 19:34 To: dev@flink.apache.org Subject: Re: Pulling Streaming out of staging and project restructure +1 On Friday, October 2, 2015, Matthias J. Sax wrote: > It think, rename "flink-storm-compatibility-core" to just "fl

Re: Pulling Streaming out of staging and project restructure

2015-10-02 Thread Henry Saputra
+1 On Friday, October 2, 2015, Matthias J. Sax wrote: > It think, rename "flink-storm-compatibility-core" to just "flink-storm" > would be the cleanest solution. > > So in flink-contrib there would be two modules: > - flink-storm > - flink-storm-examples > > Please let me know if you have an

Re: Pulling Streaming out of staging and project restructure

2015-10-02 Thread Aljoscha Krettek
+1 On Fri, 2 Oct 2015 at 11:37 Márton Balassi wrote: > @Matthias: +1. > > On Fri, Oct 2, 2015 at 11:27 AM, Stephan Ewen wrote: > > > @matthias +1 for that approach > > > > On Fri, Oct 2, 2015 at 11:21 AM, Matthias J. Sax > wrote: > > > > > It think, rename "flink-storm-compatibility-core" to j

Re: Pulling Streaming out of staging and project restructure

2015-10-02 Thread Márton Balassi
@Matthias: +1. On Fri, Oct 2, 2015 at 11:27 AM, Stephan Ewen wrote: > @matthias +1 for that approach > > On Fri, Oct 2, 2015 at 11:21 AM, Matthias J. Sax wrote: > > > It think, rename "flink-storm-compatibility-core" to just "flink-storm" > > would be the cleanest solution. > > > > So in flink-

Re: Pulling Streaming out of staging and project restructure

2015-10-02 Thread Stephan Ewen
@matthias +1 for that approach On Fri, Oct 2, 2015 at 11:21 AM, Matthias J. Sax wrote: > It think, rename "flink-storm-compatibility-core" to just "flink-storm" > would be the cleanest solution. > > So in flink-contrib there would be two modules: > - flink-storm > - flink-storm-examples > >

Re: Pulling Streaming out of staging and project restructure

2015-10-02 Thread Matthias J. Sax
It think, rename "flink-storm-compatibility-core" to just "flink-storm" would be the cleanest solution. So in flink-contrib there would be two modules: - flink-storm - flink-storm-examples Please let me know if you have any objection about it. -Matthias On 10/02/2015 10:45 AM, Matthias J. S

Re: Pulling Streaming out of staging and project restructure

2015-10-02 Thread Matthias J. Sax
Sure. Will do that. -Matthias On 10/02/2015 10:35 AM, Stephan Ewen wrote: > @Matthias: How about getting rid of the storm-compatibility-parent and > making the core and examples projects directly projects in "contrib" > > On Fri, Oct 2, 2015 at 10:34 AM, Till Rohrmann wrote: > >> +1 for the ne

Re: Pulling Streaming out of staging and project restructure

2015-10-02 Thread Stephan Ewen
@Matthias: How about getting rid of the storm-compatibility-parent and making the core and examples projects directly projects in "contrib" On Fri, Oct 2, 2015 at 10:34 AM, Till Rohrmann wrote: > +1 for the new project structure. Getting rid of our code dump is a good > thing. > > On Fri, Oct 2,

Re: Pulling Streaming out of staging and project restructure

2015-10-02 Thread Till Rohrmann
+1 for the new project structure. Getting rid of our code dump is a good thing. On Fri, Oct 2, 2015 at 10:25 AM, Maximilian Michels wrote: > +1 Matthias, let's limit the overhead this has for the module maintainers. > > On Fri, Oct 2, 2015 at 12:17 AM, Matthias J. Sax wrote: > > I will commit s

Re: Pulling Streaming out of staging and project restructure

2015-10-02 Thread Maximilian Michels
+1 Matthias, let's limit the overhead this has for the module maintainers. On Fri, Oct 2, 2015 at 12:17 AM, Matthias J. Sax wrote: > I will commit something to flink-storm-compatibility tomorrow that > contains some internal package restructuring. I think, renaming the > three modules in this com

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Matthias J. Sax
I will commit something to flink-storm-compatibility tomorrow that contains some internal package restructuring. I think, renaming the three modules in this commit would be a smart move as both changes result in merge conflicts when rebasing open PRs. Thus we can limit this pain to a single time. I

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Henry Saputra
+1 I like the idea moving "staging" projects into appropriate modules. While we are at it, I would like to propose changing " flink-hadoop-compatibility" to "flink-hadoop". It is in my bucket list but would be nice if it is part of re-org. Supporting Hadoop in the connector implicitly means compa

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Maximilian Michels
+1 for the new Maven project structure +1 for removing the flink-testing-utils module +1 for moving flink-language-binding to flink-python On Thu, Oct 1, 2015 at 6:27 PM, Aljoscha Krettek wrote: > +1 For pulling out and the restructure. Enough good arguments have been > brought forward and I agre

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Aljoscha Krettek
+1 For pulling out and the restructure. Enough good arguments have been brought forward and I agree with all of them. On Thu, 1 Oct 2015 at 17:47 Ufuk Celebi wrote: > > > On 01 Oct 2015, at 16:48, Robert Metzger wrote: > > > > @Chesnay: Nothing prevents projects from getting stuck there. But at

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Ufuk Celebi
> On 01 Oct 2015, at 16:48, Robert Metzger wrote: > > @Chesnay: Nothing prevents projects from getting stuck there. But at least > we don't have two staging repositories (dist, staging). Also I would not > say that there has been no graduation out of staging. Yarn was also there > once, streamin

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Robert Metzger
@Chesnay: Nothing prevents projects from getting stuck there. But at least we don't have two staging repositories (dist, staging). Also I would not say that there has been no graduation out of staging. Yarn was also there once, streaming and gelly are currently leaving it. So I would actually say i

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Kostas Tzoumas
+1 to Robert and practicality :-) As I said before, I do not feel strongly about this, I was torn myself. On Thu, Oct 1, 2015 at 3:28 PM, Chesnay Schepler wrote: > If we remove flink-staging because projects tend to get stuck there, what > mechanism prevents the same happening with flink-contri

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Chesnay Schepler
If we remove flink-staging because projects tend to get stuck there, what mechanism prevents the same happening with flink-contrib? On 01.10.2015 15:19, Stephan Ewen wrote: +1 for Robert's comments. On Thu, Oct 1, 2015 at 3:16 PM, Robert Metzger wrote: Big +1 for graduating streaming out of

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Stephan Ewen
+1 for Robert's comments. On Thu, Oct 1, 2015 at 3:16 PM, Robert Metzger wrote: > Big +1 for graduating streaming out of staging. It is widely used, also in > production and we are spending a lot of effort into hardening it. > I also agree with the proposed new maven module structure. > > We hav

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Robert Metzger
Big +1 for graduating streaming out of staging. It is widely used, also in production and we are spending a lot of effort into hardening it. I also agree with the proposed new maven module structure. We have to carefully test the reworked structure for the scripts which are generating the hadoop1

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Kostas Tzoumas
+1 I wanted to suggest that we rename modules to fully accept streaming as first class, qualifying also "batch" as "batch" (e.g., flink-java --> flink-dataset-java, flink-streaming --> flink-datastream, etc). This would break maven dependencies (temporary hell :-) so it's not a decision to take l

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Chesnay Schepler
I like it in general. But while we're at it, what is the purpose of the flink-tests project, or rather which tests belong there instead of the individual projects? Where would new projects reside in, that previously would have been put into flink-staging? Lastly, I'd like to merge flink-lang

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Márton Balassi
Great to see streaming graduating. :) I like the outline, both getting rid of staging, having the examples together and generally flattening the structure are very reasonable to me. You have listed flink-streaming-examples under flink-streaming-connectors and left out some less prominent maven mo

Pulling Streaming out of staging and project restructure

2015-10-01 Thread Stephan Ewen
Hi all! We are making good headway with reworking the last parts of the Window API. After that, the streaming API should be good to be pulled out of staging. Since we are reorganizing the projects as part of that, I would shift a bit more to bring things a bit more up to date. In this restructur