I agree that it's a good idea to move the APIs into one module. But why should we merge client and compiler (optimizer) and the examples into one module? I think modules with clearly separated responsibilities can also help new contributors to navigate the code.
2015-03-17 16:16 GMT+01:00 Stephan Ewen <se...@apache.org>: > The good thing about the API projects is that there is no dependency from > Java code to Scala code. I think that caused most of the issues. > > We may still want to keep it separate. I am not fully decided on this > yet... > > Stephan > > On Tue, Mar 17, 2015 at 3:52 PM, Ufuk Celebi <u...@apache.org> wrote: > > > Thanks for bringing this up, Till. You are right, but I think the main > > issue was that tight interaction between Java and Scala was problematic. > I > > am not sure whether this is such a big problem for the the APIs. > > > > On Tue, Mar 17, 2015 at 3:21 PM, Till Rohrmann <trohrm...@apache.org> > > wrote: > > > > > Putting the Scala and Java API into the same module means that we'll > have > > > more mixed Java/Scala projects, right? I just want to check if everyone > > is > > > aware of it considering our latest experiences with these kind of > > modules. > > > > > > On Tue, Mar 17, 2015 at 2:21 PM, Ufuk Celebi <u...@apache.org> wrote: > > > > > > > +1 I like the proposed structure. > > > > > > > > The only thing I was wondering about is whether to name "core" => > > > "batch"? > > > > > > > > On Tue, Mar 17, 2015 at 11:37 AM, Márton Balassi < > > > balassi.mar...@gmail.com > > > > > > > > > wrote: > > > > > > > > > +1 for the proposed structure. > > > > > > > > > > I have no explicit preference for having batch and streaming scala > > > > together > > > > > or separated. That said streaming scala is considerably thin, it > does > > > not > > > > > really require an own maven submodule. > > > > > > > > > > Marked an older JIRA for the same issue as duplicate. [1] > > > > > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-1340 > > > > > > > > > > On Tue, Mar 17, 2015 at 10:46 AM, Stephan Ewen <se...@apache.org> > > > wrote: > > > > > > > > > > > To not let this discussion die, here is a concrete JIRA and a > > > proposed > > > > > > layout to restructure to. > > > > > > > > > > > > What remains to be discusses is whether we want to keep the > > > Scala/Java > > > > > APIs > > > > > > for batch/streaming in separate projects or in one project. > > > > > > > > > > > > Also, we need to find a good time to do this, when we are low on > > pull > > > > > > requests... > > > > > > > > > > > > https://issues.apache.org/jira/browse/FLINK-1712 > > > > > > > > > > > > Proposed Layout: > > > > > > > > > > > > - flink-hadoop (shaded fat jar) > > > > > > > > > > > > - Core (Core and Java and Scala) > > > > > > - Streaming (core + java + scala) > > > > > > - Runtime > > > > > > - Client (Client + Optimizer) > > > > > > > > > > > > - Examples (Java + Scala + Streaming Java + Streaming Scala) > > > > > > - Tests (test-utils (compile) and tests (test)) > > > > > > > > > > > > - Quickstarts > > > > > > - Quickstart Java > > > > > > - Quickstart Scala > > > > > > > > > > > > - connectors / Input/Output Formats > > > > > > - Avro > > > > > > - HBase > > > > > > - HadoopCompartibility > > > > > > - HCatalogue > > > > > > - JDBC > > > > > > - kafka > > > > > > - rabbit > > > > > > - ... > > > > > > > > > > > > - staging > > > > > > - Gelly > > > > > > - Gilbert (ML) > > > > > > - spargel (deprecated) > > > > > > - expression API > > > > > > > > > > > > - contrib > > > > > > > > > > > > - yarn > > > > > > > > > > > > - dist > > > > > > > > > > > > - yarn tests > > > > > > > > > > > > - java 8 > > > > > > > > > > > > On Mon, Jan 5, 2015 at 7:45 PM, Henry Saputra < > > > henry.sapu...@gmail.com > > > > > > > > > > > wrote: > > > > > > > > > > > > > Thanks Marton, having 2 threads discussing same thing can be > > > > confusing. > > > > > > > > > > > > > > - Henry > > > > > > > > > > > > > > On Mon, Jan 5, 2015 at 3:52 AM, Márton Balassi < > > > mbala...@apache.org> > > > > > > > wrote: > > > > > > > > Let us consider this thread the standard for the restructure, > > it > > > is > > > > > > > > perfectly in line with the wishes I have posted. > > > > > > > > > > > > > > > > +1 for keeping the 'flink-' prefix. > > > > > > > > > > > > > > > > On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra < > > > > > > henry.sapu...@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > >> @Robert, and @Stephan, sure I am ok with it, thanks for the > > > > > responses. > > > > > > > >> > > > > > > > >> - Henry > > > > > > > >> > > > > > > > >> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen < > > se...@apache.org> > > > > > > wrote: > > > > > > > >> > I think this works well together with Marton's > > restructuring. > > > > > > > >> > > > > > > > > >> > I would vote to keep the "flink-" prefix, because it > > > guarantees > > > > > that > > > > > > > the > > > > > > > >> > produced jars are prefixed with "flink-". Otherwise, we > will > > > > have > > > > > to > > > > > > > >> start > > > > > > > >> > configuring a lot... > > > > > > > >> > > > > > > > > >> > Greetings, > > > > > > > >> > Stephan > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra < > > > > > > > henry.sapu...@gmail.com> > > > > > > > >> > wrote: > > > > > > > >> > > > > > > > > >> >> Will this conflict with Marton's restructuring proposal > > which > > > > > > happens > > > > > > > >> >> in another thread (see "Project restructure" thread in > the > > > dev@ > > > > > > > list). > > > > > > > >> >> > > > > > > > >> >> Since we are doing refactoring, may I suggest that we > also > > > > remove > > > > > > > >> >> "flink-" prefix since maven group name will indicate it > is > > > part > > > > > of > > > > > > > >> >> Flink. > > > > > > > >> >> > > > > > > > >> >> - Henry > > > > > > > >> >> > > > > > > > >> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen < > > > se...@apache.org > > > > > > > > > > > > wrote: > > > > > > > >> >> > Hi everyone! > > > > > > > >> >> > > > > > > > > >> >> > I think that by now, quite a bit of the maven project > > > > structure > > > > > > > can be > > > > > > > >> >> > improved to get rid of some legacy artifacts. > Especially > > > the > > > > > > > >> >> "flink-addons" > > > > > > > >> >> > project seems to be a catch-all place for various > > projects. > > > > > > > >> >> > > > > > > > > >> >> > Here is a suggestion what we could do: > > > > > > > >> >> > > > > > > > > >> >> > 1) Move "flink-yarn" to the root. > > > > > > > >> >> > > > > > > > > >> >> > 2) Move "flink-streaming" to the root (this is planned > > > > anyways > > > > > > for > > > > > > > the > > > > > > > >> >> next > > > > > > > >> >> > release) > > > > > > > >> >> > > > > > > > > >> >> > 3) Create a project "flink-connectors", which will > > contain > > > > > > "avro", > > > > > > > >> >> "jdbc", > > > > > > > >> >> > and "hbase". Should we have them as separate > > sub-projects, > > > or > > > > > as > > > > > > > one > > > > > > > >> >> > project? > > > > > > > >> >> > > > > > > > > >> >> > 4) Consolidate the examples into a single project > > > > > > "flink-examples", > > > > > > > >> where > > > > > > > >> >> > Java, Scala, Streaming examples exist in different > > > packages. > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > > >> >> > Greetings, > > > > > > > >> >> > Stephan > > > > > > > >> >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > >
