Hi Martijn
I do not maintain a large production application based on Flink, so it
would not be a problem for me to convert existing implementations to
whatever API.
I am working in the area of cultural heritage, which is mainly about the
processing of structured (meta)-data (scientific libraries, archives and
museums)
My impression: People without much background/experience with Java
implementations find it easier to get into the functional mindset as
supported in Scala. That's why I think it would be very unfortunate if
the use of Scala in Flink becomes more and more limited or neglected.
I think using the Java API in Scala is a possible way also in my
environment.
In the last weeks I tried to port the examples from the "Flink Course"
of Daniel Ciorcilan (https://rockthejvm.com/p/flink - he mainly offers
Scala courses), which are exclusively based on the native Scala API, to
the Java API. This has worked without any problems as far as I can see.
So far I haven't tried any examples based on the Table API or streaming
workflows in batch mode (which would be important for our environment).
My main trouble: So far I don't know enough about the limitations of
using the Java API in a Scala implementation and what that means. My
current understanding: the limitation is mainly in deriving the type
information in generic APIs with Scala types. For me it would be very
significant and helpful if there would be more information, descriptions
and examples about this topic.
So far unfortunately I had too little time to deal with a wrapper like
flink-scala-api (https://github.com/findify/flink-scala-api ) and the
current alternative is probably going to be deprecated in the future
(https://github.com/ariskk/flink4s/issues/17#issuecomment-1125806808 )
Günter
On 04.10.22 13:58, Martijn Visser wrote:
Hi Marton,
You're making a good point, I originally wanted to include already the
User mailing list to get their feedback but forgot to do so. I'll do
some more outreach via other channels as well.
@Users of Flink, I've made a proposal to deprecate and remove Scala
API support in a future version of Flink. Your feedback on this topic
is very much appreciated.
Regarding the large Scala codebase for Flink, a potential alternative
could be to have a wrapper for all Java APIs that makes them available
as Scala APIs. However, this still requires Scala maintainers and I
don't think that we currently have those in our community. The easiest
solution for them would be to use the Java APIs directly. Yes it would
involve work, but we won't actually be able to remove the Scala APIs
until Flink 2.0 so there's still time for that :)
Best regards,
Martijn
On Tue, Oct 4, 2022 at 1:26 AM Márton Balassi
<balassi.mar...@gmail.com> wrote:
Hi Martjin,
Thanks for compiling the FLIP. I agree with the sentiment that
Scala poses
considerable maintenance overhead and key improvements (like 2.13
or 2.12.8
supports) are hanging stale. With that said before we make this
move we
should attempt to understand the userbase affected.
A quick Slack and user mailing list search does return quite a bit of
results for scala (admittedly a cursory look at them suggest that
many of
them have to do with missing features in Scala that exist in Java
or Scala
versions). I would love to see some polls on this topic, we could
also use
the Flink twitter handle to ask the community about this.
I am aware of users having large existing Scala codebases for
Flink. This
move would pose a very large effort on them, as they would need to
rewrite
much of their existing code. What are the alternatives in your
opinion,
Martjin?
On Tue, Oct 4, 2022 at 6:22 AM Martijn Visser
<martijnvis...@apache.org>
wrote:
> Hi everyone,
>
> I would like to open a discussion thread on FLIP-265 Deprecate
and remove
> Scala API support. Please take a look at
>
>
https://cwiki.apache.org/confluence/display/FLINK/FLIP-265+Deprecate+and+remove+Scala+API+support
> and provide your feedback.
>
> Best regards,
>
> Martijn
> https://twitter.com/MartijnVisser82
> https://github.com/MartijnVisser
>
--
Günter Hipler
University library Leipzig
