+1 for that. 2015-11-02 20:52 GMT+01:00 Stephan Ewen <se...@apache.org>:
> +1 for Max' suggestion to fix that for 1.0 (next release). > > Hot fixing of this thing so short before a release is a bit risky in my > opinion. It is easy to make errors (overlooking something, error not > visible because of cached older dependencies, ...), it happened more than > once during version upgrades, maven project re-organizations, etc. > > Doing it after 0.10 and having a few week to let it sink in and errors > surface would probably be much safer... > > On Mon, Nov 2, 2015 at 5:19 AM, Fabian Hueske <fhue...@gmail.com> wrote: > > > Ah OK. Sorry, I misunderstood your intention. > > > > 2015-11-02 14:07 GMT+01:00 Maximilian Michels <m...@apache.org>: > > > > > > That would mean to have "flink-java_2.10" and "flink-java_2.11" > > artifacts > > > > (and others that depend on flink-java and have no other Scala > > dependency) > > > > in the 0.10.0 release and only "flink-java" in the next 1.0 release. > > > > > > > > > My suggestion was to keep the Scala unsuffixed Scala 2.10 and add a > > > suffix for Scala 2.11. That's the way we are currently doing it (also > > > deployed on Maven like this). For the next release after 0.10, we can > > > do it properly. > > > > > > On Mon, Nov 2, 2015 at 1:30 PM, Chiwan Park <chiwanp...@apache.org> > > wrote: > > > > If we choose selective Scala version suffix for artifacts, we have to > > > tell which artifacts have the version suffix to newcomers. Some > artifacts > > > such as "flink-java”, "flink-streaming-java" are easily recognized. But > > > IMO, knowing whether artifacts such as "flink-ml", "flink-clients", > > > "flink-table" have the version suffix or not is difficult for > newcomers. > > > > > > > > This is why we are adding the version suffix to all Scala 2.11 > > artifacts > > > currently. For Scala 2.10 artifacts, we aren’t adding the version > suffix > > > for Flink with Java users. > > > > > > > > I’m for adding the version suffix to Scala 2.10 artifacts also. But > I’m > > > not sure that removing the version suffix from Java-only artifacts > would > > be > > > good. As I said above, It seems difficult for newcomers. > > > > > > > > Regards, > > > > Chiwan Park > > > > > > > > On November 2, 2015 at 8:19:15 PM, Fabian Hueske (fhue...@gmail.com) > > > wrote: > > > > > > > > That would mean to have "flink-java_2.10" and "flink-java_2.11" > > artifacts > > > > (and others that depend on flink-java and have no other Scala > > dependency) > > > > in the 0.10.0 release and only "flink-java" in the next 1.0 release. > > > > > > > > Do we want that? > > > > > > > > 2015-11-02 11:37 GMT+01:00 Maximilian Michels <m...@apache.org>: > > > > > > > >> I'm for leaving it as-is and renaming all artifacts which depend on > > > >> Scala for the release following 0.10. > > > >> > > > >> On Mon, Nov 2, 2015 at 11:32 AM, Fabian Hueske <fhue...@gmail.com> > > > wrote: > > > >> > OK, let me try to summarize the discussion (and please correct me > > if I > > > >> got > > > >> > something wrong). > > > >> > > > > >> > 1) Flink deploys Scala 2.11 snapshot artifacts. Therefore, we have > > to > > > >> > release 2.11 artifacts for the 0.10.0 release version as well. > > > >> > > > > >> > 2) Everybody agrees to appropriately tag all artifacts that have a > > > >> > (transitive) Scala dependency. ATM, that would also include > > flink-java > > > >> > which is a bit awkward. The Scala dependency in flink-java > > originates > > > >> from > > > >> > the Chill library which is used to obtain a Kryo serializer which > is > > > >> > initialized with serializers for Scala classes. We could resolve > > this > > > >> issue > > > >> > by providing Java and Scala specific implementations of the Kryo > > > >> > serializers and have KryoTypeInfos for Java and Scala. > > > >> > > > > >> > The question to answer right now is, do we want to have > "correctly" > > > >> labeled > > > >> > artifacts for the next 0.10.0 release or do we defer that for 1.0? > > > >> > If we want to solve it for 0.10.0 we need to cancel the current RC > > and > > > >> > provide a fix to remove the Scala dependency in flink-java, IMO. > > > >> > > > > >> > Opinions? > > > >> > > > > >> > Cheers, Fabian > > > >> > > > > >> > 2015-11-02 8:55 GMT+01:00 Stephan Ewen <se...@apache.org>: > > > >> > > > > >> >> +1 for the approach discusses here, and for removing Scala > > > dependencies > > > >> >> from modules that can be Scala independent. > > > >> >> > > > >> >> It would be nice if pure Java users would not see any Scala > > > versioning > > > >> (on > > > >> >> flink-core, flink-java, later also flink-sreaming-java). I guess > > for > > > any > > > >> >> runtime-related parts (including flink-client and currently all > > > >> streaming > > > >> >> projects), we need the Scala versions... > > > >> >> > > > >> >> On Sun, Nov 1, 2015 at 9:29 AM, Maximilian Michels < > m...@apache.org > > > > > > >> wrote: > > > >> >> > > > >> >> > Good point. Didn't know that. We can still add them for the > > > release. > > > >> >> > > > > >> >> > On Sat, Oct 31, 2015 at 1:51 PM, Alexander Alexandrov > > > >> >> > <alexander.s.alexand...@gmail.com> wrote: > > > >> >> > > My two cents - there are already Maven artifacts deployed for > > > 2.11 > > > >> in > > > >> >> the > > > >> >> > > SNAPSHOT repository. I think it might be confusing if they > > > suddenly > > > >> >> > > disappear for the stable release. > > > >> >> > > > > > >> >> > > > > > >> >> > > 2015-10-29 11:58 GMT+01:00 Maximilian Michels < > m...@apache.org > > >: > > > >> >> > > > > > >> >> > >> Seems like we agree that we need artifacts for different > > > versions > > > >> of > > > >> >> > Scala > > > >> >> > >> on Maven. There also seems to be a preference for including > > the > > > >> >> version > > > >> >> > in > > > >> >> > >> the artifact name. > > > >> >> > >> > > > >> >> > >> I've created an issue and marked it to be resolved for 1.0. > > For > > > the > > > >> >> 0.10 > > > >> >> > >> release, we will have binaries but no Maven artifacts. The > > > biggest > > > >> >> > >> challenge I see is to remove Scala from as many modules as > > > >> possible. > > > >> >> For > > > >> >> > >> example, flink-java depends on Scala at the moment.. > > > >> >> > >> > > > >> >> > >> https://issues.apache.org/jira/browse/FLINK-2940 > > > >> >> > >> > > > >> >> > >> On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV < > > > >> >> > fka...@redhat.com> > > > >> >> > >> wrote: > > > >> >> > >> > > > >> >> > >> > No idea if I get a vote ;) Nevertheless, +1 to have > binaries > > > for > > > >> >> both > > > >> >> > >> > versions in Maven and explicitly "scala versioned". > > > >> >> > >> > > > > >> >> > >> > Some background on this for those not as familiar with > scala > > > >> >> > versioning: > > > >> >> > >> > > > > >> >> > >> > It's considered best practice to label what version of > > scala a > > > >> >> library > > > >> >> > >> > uses in the artifact id. > > > >> >> > >> > > > > >> >> > >> > The reason is compiled scala code is only compatible with > > the > > > >> major > > > >> >> > >> > version of scala it was compiled for. For example, a > library > > > >> >> > compatible > > > >> >> > >> > with 2.10 is not compatible with 2.11. The same will be > true > > > with > > > >> >> 2.12 > > > >> >> > >> once > > > >> >> > >> > it is released. Mixing versions will result in undefined > > > behavior > > > >> >> > which > > > >> >> > >> > will likely manifest itself as runtime exceptions. > > > >> >> > >> > > > > >> >> > >> > The convention to fix this problem is for all published > > > >> libraries to > > > >> >> > >> > specify the version of scala they are compatible with. > > Leaving > > > >> out > > > >> >> the > > > >> >> > >> > scala version in a library is akin to saying "We don't > > depend > > > on > > > >> >> scala > > > >> >> > >> for > > > >> >> > >> > this library, so feel free to use whatever you want." Sbt > > > users > > > >> will > > > >> >> > >> > typically specify the version of scala they use and > tooling > > is > > > >> built > > > >> >> > >> around > > > >> >> > >> > ensuring consistency with the "%%" operator. > > > >> >> > >> > > > > >> >> > >> > E.g. > > > >> >> > >> > > > > >> >> > >> > scalaVersion := "2.11.4" > > > >> >> > >> > > > > >> >> > >> > // this resolves to to artifactID: "scalacheck_2.11" > > > >> >> > >> > libraryDependencies += "org.scalacheck" %% "scalacheck" % > > > >> "1.12.0" % > > > >> >> > >> "test" > > > >> >> > >> > > > > >> >> > >> > The most important part of this is that the scala version > is > > > >> >> explicit > > > >> >> > >> > which eliminates the problem for downstream users. > > > >> >> > >> > > > > >> >> > >> > Cheers, > > > >> >> > >> > Frederick > > > >> >> > >> > > > > >> >> > >> > > > > >> >> > >> > On 10/28/2015 10:55 AM, Fabian Hueske wrote: > > > >> >> > >> > > > > >> >> > >> >> +1 to have binaries for both versions in Maven and as > build > > > to > > > >> >> > download. > > > >> >> > >> >> > > > >> >> > >> >> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis < > > > >> >> > >> >> theodoros.vasilou...@gmail.com>: > > > >> >> > >> >> > > > >> >> > >> >> +1 for having binaries, I'm working on a Spark > application > > > >> >> currently > > > >> >> > >> with > > > >> >> > >> >>> Scala 2.11 and having to rebuild everything when > deploying > > > >> e.g. to > > > >> >> > EC2 > > > >> >> > >> >>> is a > > > >> >> > >> >>> pain. > > > >> >> > >> >>> > > > >> >> > >> >>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi < > > > u...@apache.org> > > > >> >> > wrote: > > > >> >> > >> >>> > > > >> >> > >> >>> I agree with Till, but is this something you want to > > > address in > > > >> >> this > > > >> >> > >> >>>> release already? > > > >> >> > >> >>>> > > > >> >> > >> >>>> I would postpone it to 1.0.0. > > > >> >> > >> >>>> > > > >> >> > >> >>>> – Ufuk > > > >> >> > >> >>>> > > > >> >> > >> >>>> On 26 Oct 2015, at 16:17, Till Rohrmann < > > > trohrm...@apache.org > > > >> > > > > >> >> > wrote: > > > >> >> > >> >>>>> > > > >> >> > >> >>>>> I would be in favor of deploying also Scala 2.11 > > > artifacts to > > > >> >> > Maven > > > >> >> > >> >>>>> > > > >> >> > >> >>>> since > > > >> >> > >> >>> > > > >> >> > >> >>>> more and more people will try out Flink with Scala > 2.11. > > > >> Having > > > >> >> the > > > >> >> > >> >>>>> dependencies in the Maven repository makes it > > considerably > > > >> >> easier > > > >> >> > for > > > >> >> > >> >>>>> people to get their Flink jobs running. > > > >> >> > >> >>>>> > > > >> >> > >> >>>>> Furthermore, I observed that people are not aware that > > our > > > >> >> > deployed > > > >> >> > >> >>>>> artifacts, e.g. flink-runtime, are built with Scala > > 2.10. > > > As > > > >> a > > > >> >> > >> >>>>> > > > >> >> > >> >>>> consequence, > > > >> >> > >> >>>> > > > >> >> > >> >>>>> they mix flink dependencies with other dependencies > > > pulling > > > >> in > > > >> >> > Scala > > > >> >> > >> >>>>> > > > >> >> > >> >>>> 2.11 > > > >> >> > >> >>> > > > >> >> > >> >>>> and then they wonder that the program crashes. It would > > be, > > > >> imho, > > > >> >> > >> >>>>> > > > >> >> > >> >>>> clearer > > > >> >> > >> >>> > > > >> >> > >> >>>> if all our dependencies which depend on a specific > Scala > > > >> version > > > >> >> > would > > > >> >> > >> >>>>> > > > >> >> > >> >>>> have > > > >> >> > >> >>>> > > > >> >> > >> >>>>> the corresponding Scala suffix appended. > > > >> >> > >> >>>>> > > > >> >> > >> >>>>> Adding the 2.10 suffix would also spare us the hassle > of > > > >> >> upgrading > > > >> >> > >> to a > > > >> >> > >> >>>>> newer Scala version in the future, because then the > > > artifacts > > > >> >> > >> wouldn't > > > >> >> > >> >>>>> share the same artifact name. > > > >> >> > >> >>>>> > > > >> >> > >> >>>>> Cheers, > > > >> >> > >> >>>>> Till > > > >> >> > >> >>>>> > > > >> >> > >> >>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels < > > > >> >> > m...@apache.org> > > > >> >> > >> >>>>> > > > >> >> > >> >>>> wrote: > > > >> >> > >> >>>> > > > >> >> > >> >>>>> Hi Flinksters, > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>>> We have recently committed an easy way to change > > Flink's > > > >> Scala > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>> version. > > > >> >> > >> >>> > > > >> >> > >> >>>> The > > > >> >> > >> >>>> > > > >> >> > >> >>>>> question arises now whether we should ship Scala 2.11 > as > > > >> >> binaries > > > >> >> > and > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>> via > > > >> >> > >> >>>> > > > >> >> > >> >>>>> Maven. For the rc0, I created all binaries twice, for > > > Scala > > > >> 2.10 > > > >> >> > and > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>> 2.11. > > > >> >> > >> >>>> > > > >> >> > >> >>>>> However, I didn't create Maven artifacts. This follows > > our > > > >> >> current > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>> shipping > > > >> >> > >> >>>> > > > >> >> > >> >>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0 > > Maven > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>> dependencies > > > >> >> > >> >>> > > > >> >> > >> >>>> but > > > >> >> > >> >>>> > > > >> >> > >> >>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries. > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>>> Should we also upload Maven dependencies for Scala > > 2.11? > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>>> If so, the next question arises: What version pattern > > > >> should we > > > >> >> > have > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>> for > > > >> >> > >> >>> > > > >> >> > >> >>>> the Flink Scala 2.11 dependencies? For Hadoop, we > append > > > >> -hadoop1 > > > >> >> > to > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>> the > > > >> >> > >> >>> > > > >> >> > >> >>>> VERSION, e.g. artifactID=flink-core, > > version=0.9.1-hadoop1. > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>>> However, it is common practice to append the suffix > to > > > the > > > >> >> > >> artifactID > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>> of > > > >> >> > >> >>> > > > >> >> > >> >>>> the Maven dependency, e.g. artifactID=flink-core_2.11, > > > >> >> > version=0.9.1. > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>> This > > > >> >> > >> >>>> > > > >> >> > >> >>>>> has mostly historic reasons but is widely used. > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>>> Whatever naming pattern we choose, it should be > > > consistent. > > > >> I > > > >> >> > would > > > >> >> > >> be > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>> in > > > >> >> > >> >>>> > > > >> >> > >> >>>>> favor of changing our artifact names to contain the > > Hadoop > > > >> and > > > >> >> > Scala > > > >> >> > >> >>>>>> version. This would also imply that all Scala > dependent > > > >> Maven > > > >> >> > >> modules > > > >> >> > >> >>>>>> receive a Scala suffix (also the default Scala 2.10 > > > >> modules). > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>>> Cheers, > > > >> >> > >> >>>>>> Max > > > >> >> > >> >>>>>> > > > >> >> > >> >>>>>> > > > >> >> > >> >>>> > > > >> >> > >> > > > > >> >> > >> > > > >> >> > > > > >> >> > > > >> > > > > > >
