Thanks for bringing this up, Till. You are right, but I think the main issue was that tight interaction between Java and Scala was problematic. I am not sure whether this is such a big problem for the the APIs.
On Tue, Mar 17, 2015 at 3:21 PM, Till Rohrmann <trohrm...@apache.org> wrote: > Putting the Scala and Java API into the same module means that we'll have > more mixed Java/Scala projects, right? I just want to check if everyone is > aware of it considering our latest experiences with these kind of modules. > > On Tue, Mar 17, 2015 at 2:21 PM, Ufuk Celebi <u...@apache.org> wrote: > > > +1 I like the proposed structure. > > > > The only thing I was wondering about is whether to name "core" => > "batch"? > > > > On Tue, Mar 17, 2015 at 11:37 AM, Márton Balassi < > balassi.mar...@gmail.com > > > > > wrote: > > > > > +1 for the proposed structure. > > > > > > I have no explicit preference for having batch and streaming scala > > together > > > or separated. That said streaming scala is considerably thin, it does > not > > > really require an own maven submodule. > > > > > > Marked an older JIRA for the same issue as duplicate. [1] > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-1340 > > > > > > On Tue, Mar 17, 2015 at 10:46 AM, Stephan Ewen <se...@apache.org> > wrote: > > > > > > > To not let this discussion die, here is a concrete JIRA and a > proposed > > > > layout to restructure to. > > > > > > > > What remains to be discusses is whether we want to keep the > Scala/Java > > > APIs > > > > for batch/streaming in separate projects or in one project. > > > > > > > > Also, we need to find a good time to do this, when we are low on pull > > > > requests... > > > > > > > > https://issues.apache.org/jira/browse/FLINK-1712 > > > > > > > > Proposed Layout: > > > > > > > > - flink-hadoop (shaded fat jar) > > > > > > > > - Core (Core and Java and Scala) > > > > - Streaming (core + java + scala) > > > > - Runtime > > > > - Client (Client + Optimizer) > > > > > > > > - Examples (Java + Scala + Streaming Java + Streaming Scala) > > > > - Tests (test-utils (compile) and tests (test)) > > > > > > > > - Quickstarts > > > > - Quickstart Java > > > > - Quickstart Scala > > > > > > > > - connectors / Input/Output Formats > > > > - Avro > > > > - HBase > > > > - HadoopCompartibility > > > > - HCatalogue > > > > - JDBC > > > > - kafka > > > > - rabbit > > > > - ... > > > > > > > > - staging > > > > - Gelly > > > > - Gilbert (ML) > > > > - spargel (deprecated) > > > > - expression API > > > > > > > > - contrib > > > > > > > > - yarn > > > > > > > > - dist > > > > > > > > - yarn tests > > > > > > > > - java 8 > > > > > > > > On Mon, Jan 5, 2015 at 7:45 PM, Henry Saputra < > henry.sapu...@gmail.com > > > > > > > wrote: > > > > > > > > > Thanks Marton, having 2 threads discussing same thing can be > > confusing. > > > > > > > > > > - Henry > > > > > > > > > > On Mon, Jan 5, 2015 at 3:52 AM, Márton Balassi < > mbala...@apache.org> > > > > > wrote: > > > > > > Let us consider this thread the standard for the restructure, it > is > > > > > > perfectly in line with the wishes I have posted. > > > > > > > > > > > > +1 for keeping the 'flink-' prefix. > > > > > > > > > > > > On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra < > > > > henry.sapu...@gmail.com> > > > > > > wrote: > > > > > > > > > > > >> @Robert, and @Stephan, sure I am ok with it, thanks for the > > > responses. > > > > > >> > > > > > >> - Henry > > > > > >> > > > > > >> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <se...@apache.org> > > > > wrote: > > > > > >> > I think this works well together with Marton's restructuring. > > > > > >> > > > > > > >> > I would vote to keep the "flink-" prefix, because it > guarantees > > > that > > > > > the > > > > > >> > produced jars are prefixed with "flink-". Otherwise, we will > > have > > > to > > > > > >> start > > > > > >> > configuring a lot... > > > > > >> > > > > > > >> > Greetings, > > > > > >> > Stephan > > > > > >> > > > > > > >> > > > > > > >> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra < > > > > > henry.sapu...@gmail.com> > > > > > >> > wrote: > > > > > >> > > > > > > >> >> Will this conflict with Marton's restructuring proposal which > > > > happens > > > > > >> >> in another thread (see "Project restructure" thread in the > dev@ > > > > > list). > > > > > >> >> > > > > > >> >> Since we are doing refactoring, may I suggest that we also > > remove > > > > > >> >> "flink-" prefix since maven group name will indicate it is > part > > > of > > > > > >> >> Flink. > > > > > >> >> > > > > > >> >> - Henry > > > > > >> >> > > > > > >> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen < > se...@apache.org > > > > > > > > wrote: > > > > > >> >> > Hi everyone! > > > > > >> >> > > > > > > >> >> > I think that by now, quite a bit of the maven project > > structure > > > > > can be > > > > > >> >> > improved to get rid of some legacy artifacts. Especially > the > > > > > >> >> "flink-addons" > > > > > >> >> > project seems to be a catch-all place for various projects. > > > > > >> >> > > > > > > >> >> > Here is a suggestion what we could do: > > > > > >> >> > > > > > > >> >> > 1) Move "flink-yarn" to the root. > > > > > >> >> > > > > > > >> >> > 2) Move "flink-streaming" to the root (this is planned > > anyways > > > > for > > > > > the > > > > > >> >> next > > > > > >> >> > release) > > > > > >> >> > > > > > > >> >> > 3) Create a project "flink-connectors", which will contain > > > > "avro", > > > > > >> >> "jdbc", > > > > > >> >> > and "hbase". Should we have them as separate sub-projects, > or > > > as > > > > > one > > > > > >> >> > project? > > > > > >> >> > > > > > > >> >> > 4) Consolidate the examples into a single project > > > > "flink-examples", > > > > > >> where > > > > > >> >> > Java, Scala, Streaming examples exist in different > packages. > > > > > >> >> > > > > > > >> >> > > > > > > >> >> > Greetings, > > > > > >> >> > Stephan > > > > > >> >> > > > > > >> > > > > > > > > > > > > > > >
