We haven't changed anything significant in 1.14.
Whether the 2.13 job can run on Scala 2.12 depends a bit on the job (and
of course, used libraries!); it depends on the backwards-compatibility
from Scala, which APIs are used and what kind of Scala magic is being
employed.
We haven't really tested that scenario in 1.14 or below.
On 07/12/2021 09:28, guenterh.lists wrote:
Hi Chesnay,
thanks for the info - this is really good news for us.
I set up a playground using the snapshot from yesterday [1] and a
really quick and short Job using Scala 2.13 [2]
The job starts and returns correct results. Even the use of a case
class against the Java API is possible.
Then I made a second try with the same job (compiled with Scala
2.13.6) running on a Flink 1.14 cluster which was again successful.
My question:
Is this compilation with Scala versions >=2.13 already part of 1.14 or
is my example too small and simple that binary incompatibilities
between the versions doesn't matter?
Günter
[1]
https://gitlab.com/guenterh/flink_1_15_scala_2_13/-/tree/main/flink-1.15-SNAPSHOT
[2]
https://gitlab.com/guenterh/flink_1_15_scala_2_13/-/blob/main/flink_scala_213/build.sbt#L12
https://gitlab.com/guenterh/flink_1_15_scala_2_13/-/blob/main/flink_scala_213/src/main/scala/de/ub/unileipzig/Job.scala#L8
On 06.12.21 13:59, Chesnay Schepler wrote:
With regards to the Java APIs, you will definitely be able to use the
Java DataSet/DataStream APIs from Scala without any restrictions
imposed by Flink. This is already working with the current SNAPSHOT
version.
As we speak we are also working to achieve the same for the Table
API; we expect to achieve that but with some caveats (i.e., if you
use the Python API or the Hive connector then you still need to use
the Scala version provided by Flink).
As for the Scala APIs, we haven't really decided yet how this will
work in the future. However, one of the big benefits of the
Scala-free runtime is that it should now be easier for us to release
the APIs for more Scala versions.
On 06/12/2021 11:47, guenterh.lists wrote:
Dear list,
there have been some discussions and activities in the last months
about a Scala free runtime which should make it possible to use
newer Scala version (>= 2.13 / 3.x) on the application side.
Stephan Ewen announced the implementation is on the way [1] and
Martijn Vissr mentioned in the ask me anything session on version
1.14 that it is planned to make this possible in the upcoming 1.15
version (~ next February ) [2]
This would be very nice for our currently started project where we
are discussing the used tools and infrastructure. "Personally" I
would prefer that people with less experience on the JVM could make
their start and first experiences with a "pythonized" Scala using
the last versions of the language (2.13.x or maybe 3.x).
My question: Do you think your plans to provide the possibility of a
Scala free runtime with the upcoming version is still realistic?
Out of curiosity: If you can make this possible and applications
with current Scala versions are going to use the Java APIs of Flink
what's the future of the current Scala API of Flink where you have
to decide to use either Scala 2.11 or <2.12.8?
Is this then still possible as an alternative?
Thanks for some hints for our planning and decisions
Günter
[1] https://twitter.com/data_fly/status/1415012793347149830
[2] https://www.youtube.com/watch?v=wODmlow0ip0
