If we choose selective Scala version suffix for artifacts, we have to tell which artifacts have the version suffix to newcomers. Some artifacts such as "flink-java”, "flink-streaming-java" are easily recognized. But IMO, knowing whether artifacts such as "flink-ml", "flink-clients", "flink-table" have the version suffix or not is difficult for newcomers.
This is why we are adding the version suffix to all Scala 2.11 artifacts currently. For Scala 2.10 artifacts, we aren’t adding the version suffix for Flink with Java users. I’m for adding the version suffix to Scala 2.10 artifacts also. But I’m not sure that removing the version suffix from Java-only artifacts would be good. As I said above, It seems difficult for newcomers. Regards, Chiwan Park On November 2, 2015 at 8:19:15 PM, Fabian Hueske (fhue...@gmail.com) wrote: That would mean to have "flink-java_2.10" and "flink-java_2.11" artifacts (and others that depend on flink-java and have no other Scala dependency) in the 0.10.0 release and only "flink-java" in the next 1.0 release. Do we want that? 2015-11-02 11:37 GMT+01:00 Maximilian Michels <m...@apache.org>: > I'm for leaving it as-is and renaming all artifacts which depend on > Scala for the release following 0.10. > > On Mon, Nov 2, 2015 at 11:32 AM, Fabian Hueske <fhue...@gmail.com> wrote: > > OK, let me try to summarize the discussion (and please correct me if I > got > > something wrong). > > > > 1) Flink deploys Scala 2.11 snapshot artifacts. Therefore, we have to > > release 2.11 artifacts for the 0.10.0 release version as well. > > > > 2) Everybody agrees to appropriately tag all artifacts that have a > > (transitive) Scala dependency. ATM, that would also include flink-java > > which is a bit awkward. The Scala dependency in flink-java originates > from > > the Chill library which is used to obtain a Kryo serializer which is > > initialized with serializers for Scala classes. We could resolve this > issue > > by providing Java and Scala specific implementations of the Kryo > > serializers and have KryoTypeInfos for Java and Scala. > > > > The question to answer right now is, do we want to have "correctly" > labeled > > artifacts for the next 0.10.0 release or do we defer that for 1.0? > > If we want to solve it for 0.10.0 we need to cancel the current RC and > > provide a fix to remove the Scala dependency in flink-java, IMO. > > > > Opinions? > > > > Cheers, Fabian > > > > 2015-11-02 8:55 GMT+01:00 Stephan Ewen <se...@apache.org>: > > > >> +1 for the approach discusses here, and for removing Scala dependencies > >> from modules that can be Scala independent. > >> > >> It would be nice if pure Java users would not see any Scala versioning > (on > >> flink-core, flink-java, later also flink-sreaming-java). I guess for any > >> runtime-related parts (including flink-client and currently all > streaming > >> projects), we need the Scala versions... > >> > >> On Sun, Nov 1, 2015 at 9:29 AM, Maximilian Michels <m...@apache.org> > wrote: > >> > >> > Good point. Didn't know that. We can still add them for the release. > >> > > >> > On Sat, Oct 31, 2015 at 1:51 PM, Alexander Alexandrov > >> > <alexander.s.alexand...@gmail.com> wrote: > >> > > My two cents - there are already Maven artifacts deployed for 2.11 > in > >> the > >> > > SNAPSHOT repository. I think it might be confusing if they suddenly > >> > > disappear for the stable release. > >> > > > >> > > > >> > > 2015-10-29 11:58 GMT+01:00 Maximilian Michels <m...@apache.org>: > >> > > > >> > >> Seems like we agree that we need artifacts for different versions > of > >> > Scala > >> > >> on Maven. There also seems to be a preference for including the > >> version > >> > in > >> > >> the artifact name. > >> > >> > >> > >> I've created an issue and marked it to be resolved for 1.0. For the > >> 0.10 > >> > >> release, we will have binaries but no Maven artifacts. The biggest > >> > >> challenge I see is to remove Scala from as many modules as > possible. > >> For > >> > >> example, flink-java depends on Scala at the moment.. > >> > >> > >> > >> https://issues.apache.org/jira/browse/FLINK-2940 > >> > >> > >> > >> On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV < > >> > fka...@redhat.com> > >> > >> wrote: > >> > >> > >> > >> > No idea if I get a vote ;) Nevertheless, +1 to have binaries for > >> both > >> > >> > versions in Maven and explicitly "scala versioned". > >> > >> > > >> > >> > Some background on this for those not as familiar with scala > >> > versioning: > >> > >> > > >> > >> > It's considered best practice to label what version of scala a > >> library > >> > >> > uses in the artifact id. > >> > >> > > >> > >> > The reason is compiled scala code is only compatible with the > major > >> > >> > version of scala it was compiled for. For example, a library > >> > compatible > >> > >> > with 2.10 is not compatible with 2.11. The same will be true with > >> 2.12 > >> > >> once > >> > >> > it is released. Mixing versions will result in undefined behavior > >> > which > >> > >> > will likely manifest itself as runtime exceptions. > >> > >> > > >> > >> > The convention to fix this problem is for all published > libraries to > >> > >> > specify the version of scala they are compatible with. Leaving > out > >> the > >> > >> > scala version in a library is akin to saying "We don't depend on > >> scala > >> > >> for > >> > >> > this library, so feel free to use whatever you want." Sbt users > will > >> > >> > typically specify the version of scala they use and tooling is > built > >> > >> around > >> > >> > ensuring consistency with the "%%" operator. > >> > >> > > >> > >> > E.g. > >> > >> > > >> > >> > scalaVersion := "2.11.4" > >> > >> > > >> > >> > // this resolves to to artifactID: "scalacheck_2.11" > >> > >> > libraryDependencies += "org.scalacheck" %% "scalacheck" % > "1.12.0" % > >> > >> "test" > >> > >> > > >> > >> > The most important part of this is that the scala version is > >> explicit > >> > >> > which eliminates the problem for downstream users. > >> > >> > > >> > >> > Cheers, > >> > >> > Frederick > >> > >> > > >> > >> > > >> > >> > On 10/28/2015 10:55 AM, Fabian Hueske wrote: > >> > >> > > >> > >> >> +1 to have binaries for both versions in Maven and as build to > >> > download. > >> > >> >> > >> > >> >> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis < > >> > >> >> theodoros.vasilou...@gmail.com>: > >> > >> >> > >> > >> >> +1 for having binaries, I'm working on a Spark application > >> currently > >> > >> with > >> > >> >>> Scala 2.11 and having to rebuild everything when deploying > e.g. to > >> > EC2 > >> > >> >>> is a > >> > >> >>> pain. > >> > >> >>> > >> > >> >>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <u...@apache.org> > >> > wrote: > >> > >> >>> > >> > >> >>> I agree with Till, but is this something you want to address in > >> this > >> > >> >>>> release already? > >> > >> >>>> > >> > >> >>>> I would postpone it to 1.0.0. > >> > >> >>>> > >> > >> >>>> – Ufuk > >> > >> >>>> > >> > >> >>>> On 26 Oct 2015, at 16:17, Till Rohrmann <trohrm...@apache.org > > > >> > wrote: > >> > >> >>>>> > >> > >> >>>>> I would be in favor of deploying also Scala 2.11 artifacts to > >> > Maven > >> > >> >>>>> > >> > >> >>>> since > >> > >> >>> > >> > >> >>>> more and more people will try out Flink with Scala 2.11. > Having > >> the > >> > >> >>>>> dependencies in the Maven repository makes it considerably > >> easier > >> > for > >> > >> >>>>> people to get their Flink jobs running. > >> > >> >>>>> > >> > >> >>>>> Furthermore, I observed that people are not aware that our > >> > deployed > >> > >> >>>>> artifacts, e.g. flink-runtime, are built with Scala 2.10. As > a > >> > >> >>>>> > >> > >> >>>> consequence, > >> > >> >>>> > >> > >> >>>>> they mix flink dependencies with other dependencies pulling > in > >> > Scala > >> > >> >>>>> > >> > >> >>>> 2.11 > >> > >> >>> > >> > >> >>>> and then they wonder that the program crashes. It would be, > imho, > >> > >> >>>>> > >> > >> >>>> clearer > >> > >> >>> > >> > >> >>>> if all our dependencies which depend on a specific Scala > version > >> > would > >> > >> >>>>> > >> > >> >>>> have > >> > >> >>>> > >> > >> >>>>> the corresponding Scala suffix appended. > >> > >> >>>>> > >> > >> >>>>> Adding the 2.10 suffix would also spare us the hassle of > >> upgrading > >> > >> to a > >> > >> >>>>> newer Scala version in the future, because then the artifacts > >> > >> wouldn't > >> > >> >>>>> share the same artifact name. > >> > >> >>>>> > >> > >> >>>>> Cheers, > >> > >> >>>>> Till > >> > >> >>>>> > >> > >> >>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels < > >> > m...@apache.org> > >> > >> >>>>> > >> > >> >>>> wrote: > >> > >> >>>> > >> > >> >>>>> Hi Flinksters, > >> > >> >>>>>> > >> > >> >>>>>> We have recently committed an easy way to change Flink's > Scala > >> > >> >>>>>> > >> > >> >>>>> version. > >> > >> >>> > >> > >> >>>> The > >> > >> >>>> > >> > >> >>>>> question arises now whether we should ship Scala 2.11 as > >> binaries > >> > and > >> > >> >>>>>> > >> > >> >>>>> via > >> > >> >>>> > >> > >> >>>>> Maven. For the rc0, I created all binaries twice, for Scala > 2.10 > >> > and > >> > >> >>>>>> > >> > >> >>>>> 2.11. > >> > >> >>>> > >> > >> >>>>> However, I didn't create Maven artifacts. This follows our > >> current > >> > >> >>>>>> > >> > >> >>>>> shipping > >> > >> >>>> > >> > >> >>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven > >> > >> >>>>>> > >> > >> >>>>> dependencies > >> > >> >>> > >> > >> >>>> but > >> > >> >>>> > >> > >> >>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries. > >> > >> >>>>>> > >> > >> >>>>>> Should we also upload Maven dependencies for Scala 2.11? > >> > >> >>>>>> > >> > >> >>>>>> If so, the next question arises: What version pattern > should we > >> > have > >> > >> >>>>>> > >> > >> >>>>> for > >> > >> >>> > >> > >> >>>> the Flink Scala 2.11 dependencies? For Hadoop, we append > -hadoop1 > >> > to > >> > >> >>>>>> > >> > >> >>>>> the > >> > >> >>> > >> > >> >>>> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1. > >> > >> >>>>>> > >> > >> >>>>>> However, it is common practice to append the suffix to the > >> > >> artifactID > >> > >> >>>>>> > >> > >> >>>>> of > >> > >> >>> > >> > >> >>>> the Maven dependency, e.g. artifactID=flink-core_2.11, > >> > version=0.9.1. > >> > >> >>>>>> > >> > >> >>>>> This > >> > >> >>>> > >> > >> >>>>> has mostly historic reasons but is widely used. > >> > >> >>>>>> > >> > >> >>>>>> Whatever naming pattern we choose, it should be consistent. > I > >> > would > >> > >> be > >> > >> >>>>>> > >> > >> >>>>> in > >> > >> >>>> > >> > >> >>>>> favor of changing our artifact names to contain the Hadoop > and > >> > Scala > >> > >> >>>>>> version. This would also imply that all Scala dependent > Maven > >> > >> modules > >> > >> >>>>>> receive a Scala suffix (also the default Scala 2.10 > modules). > >> > >> >>>>>> > >> > >> >>>>>> Cheers, > >> > >> >>>>>> Max > >> > >> >>>>>> > >> > >> >>>>>> > >> > >> >>>> > >> > >> > > >> > >> > >> > > >> >
