Hi Colin, We can take 2 different paths, both of them need a similar effort. — Offer Scala 3 along with Scala 2.12 and 2.13 — Offer Scala 3 only once Scala 2.12 is removed
On the draft PR I shared in the initial email ( https://github.com/apache/kafka/pull/11350), I used the first approach, and it didn't require any additional work. This means we can decide to take any of the 2 approaches freely. Offering Scala 3 only when 2.13 is removed means to wait for a bit, but it's perfectly doable. My personal.opinion is that, given Scala 3 is a major release (epoch according to Scala version schema), we might want to build in Scala 3 sooner rather than later. We could label this as experimental Scala 3 support. However, until Gradle 7.3 is released, we can't proceed any further with building with Scala 3. Currently, Gradle 7.3 is under RC3, so I expect it to be available soon. Regarding the changes I propose to add right now as I mentioned in the first email, you can see exactly what I meant in this PR: https://github.com/apache/kafka/pull/11432. Sometimes, Kafka uses some syntax or approaches that are deprecated and they are removed in Scala 3. This PR goes over those and uses the "accepted" syntax. Feel free to take a look and review it. Thanks for your feedback Colin! Best, ——— Josep Prat Aiven Deutschland GmbH Immanuelkirchstraße 26, 10405 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen m: +491715557497 w: aiven.io e: josep.p...@aiven.io On Mon, Nov 1, 2021, 23:45 Colin McCabe <cmcc...@apache.org> wrote: > Thanks for looking at this, Josep... I guess the way I always imagined > this happening was that a Scala 3 release would become one of our two > supported Scala versions. So instead of 2.12 and 2.13, we'd have 2.13 and > 3.x, for some X. Do you think that approach would be practical? > > best, > Colin > > > On Fri, Oct 8, 2021, at 09:05, Ismael Juma wrote: > > Yeah, changes that are good generally can be submitted any time. > > > > Ismael > > > > On Fri, Oct 8, 2021 at 7:35 AM Josep Prat <josep.p...@aiven.io.invalid> > > wrote: > > > >> Hi Ismael, > >> > >> Thanks for the reply. I don't think the demand is high at the moment, > but > >> it will increase overtime. > >> What do you think about me submitting a PR now with only the changes I > >> mention in group a)? They are small changes like dropping extra > >> parenthesis, or stopping shadowing names. The resulting code would be > >> cleaner Scala that can cross compile. > >> > >> Thanks again, > >> ——— > >> Josep Prat > >> > >> Aiven Deutschland GmbH > >> > >> Immanuelkirchstraße 26, 10405 Berlin > >> > >> Amtsgericht Charlottenburg, HRB 209739 B > >> > >> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >> > >> m: +491715557497 > >> > >> w: aiven.io > >> > >> e: josep.p...@aiven.io > >> > >> On Fri, Oct 8, 2021, 16:22 Ismael Juma <ism...@juma.me.uk> wrote: > >> > >> > Hi Josep, > >> > > >> > Thanks for looking into this. As you said, it seems like there are a > >> number > >> > of rough edges still. Do we have many users asking for this right > now? If > >> > not, it may make sense to wait for things to get a bit more mature > before > >> > adding the burden of a third Scala version. Ideally, we would drop > Scala > >> > 2.12 before we add Scala 3 support. > >> > > >> > Ismael > >> > > >> > On Fri, Oct 8, 2021 at 6:07 AM Josep Prat <josep.p...@aiven.io.invalid > > > >> > wrote: > >> > > >> > > Hi there, > >> > > For the last month I was actively working on migrating Apache Kafka > to > >> > > Scala 3. For what I could see, Apache Kafka is the biggest mixed > >> > Java-Scala > >> > > project to attempt a migration to Scala 3. This means, a lot of > >> > regressions > >> > > in regards to Java and Scala interoperability at bytecode were found > >> > within > >> > > this task. You can take a look at the PR here: > >> > > https://github.com/apache/kafka/pull/11350 > >> > > Currently, the status of the migration is as follows: > >> > > > >> > > - All tests pass in Scala 3 except one ( > >> > > > >> > > > >> > > >> > KafkaApis#getAllTopicMetadataShouldNotCreateTopicOrReturnUnknownTopicPartition). > >> > > Root cause for this is this bug with no easy workaround: Type > erased > >> > for > >> > > by-name parameters lampepfl/dotty#13638 > >> > > <https://github.com/lampepfl/dotty/issues/13638> > >> > > - All tests pass in Scala 2 > >> > > - It still uses a snapshot build of Gradle with support for > Scala 3 > >> > > - Spotbugs detected some new errors when the code was compiled > with > >> > > Scala 3. Some are simply false positives, but a small subset seem > >> like > >> > > they > >> > > might be legit bugs in Scala 3. I still need to come up with a > >> > minimized > >> > > reproducer for those cases. > >> > > > >> > > As one can see, most of the changes are arguably low-level and no > big > >> > > rewrite was needed. The biggest refactoring was splitting several > >> classes > >> > > into their own file. > >> > > > >> > > Changes made in this PR could be categorized in 3 different groups: > >> > > a) Needed changes because of changes in syntax or compiler being > >> > currently > >> > > more strict > >> > > b) Changes where a workaround is present but they are (or will be) > >> fixed > >> > in > >> > > upcoming Scala versions > >> > > c) Changes in the build file > >> > > > >> > > In order to have a smooth and manageable migration to Scala 3, I > >> propose > >> > > perform the migration in several steps: > >> > > > >> > > - Firstly apply all changes that would fall into category *a)* > >> > > - Once a new Scala version that incorporates the mentioned fixes > is > >> > > released, revisit point changes in point *b)* > >> > > - Once Type erased for by-name parameters lampepfl/dotty#13638 > >> > > <https://github.com/lampepfl/dotty/issues/13638> is resolved or > a > >> > > workaround is found and Gradle with Scala 3 support is released > we > >> > could > >> > > tackle the final step and incorporate changes mentioned in point > >> *c)* > >> > > - After this, we would run the Scala Migration Tool to > automatically > >> > > rewrite the code that uses removed syntax or language constructs. > >> (See > >> > > > >> > > > >> > > >> > https://docs.scala-lang.org/scala3/guides/migration/tooling-migration-mode.html > >> > > ) > >> > > > >> > > > >> > > What do you think about this approach? Please feel free to share > your > >> > > opinion on the proposed plan. > >> > > Also, I'm not sure if this should be written in a KIP form or not, > no > >> > > public interfaces needed to be modified. > >> > > > >> > > > >> > > For extra information, these are the bugs reported to Scala 3 so > far: > >> > > > >> > > - https://github.com/lampepfl/dotty/issues/13549 > >> > > - https://github.com/lampepfl/dotty/issues/13572 > >> > > - https://github.com/lampepfl/dotty/issues/13630 > >> > > - https://github.com/lampepfl/dotty/issues/13638 > >> > > - https://github.com/lampepfl/dotty/issues/13645 > >> > > > >> > > > >> > > Thank you in advance! > >> > > -- > >> > > > >> > > Josep Prat > >> > > > >> > > *Aiven Deutschland GmbH* > >> > > > >> > > Immanuelkirchstraße 26, 10405 Berlin > >> > > > >> > > Amtsgericht Charlottenburg, HRB 209739 B > >> > > > >> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >> > > > >> > > *m:* +491715557497 > >> > > > >> > > *w:* aiven.io > >> > > > >> > > *e:* josep.p...@aiven.io > >> > > > >> > > >> >
