Hi Patrick, Ideally it would be great if you could provide a small reproducer. It could be that something isn't working properly in hiding the internally used version of Scala from users, or something else. Without a reproducer, it's quite hard to debug.
Best regards, Martijn On Wed, Jan 10, 2024 at 2:49 PM Eifler, Patrick <patrick.eif...@sony.com> wrote: > > Hi Martijn, > > > > Many thanks for your reply. Yes I have seen the examples. I removed the > flink-scala dependency and only use the java libs for everything. So there > should no flink-scala API references in the stack. > > > > These are the flink dependencies we are using: > > > > "org.apache.flink" % "flink-core" % flinkVersion, > "org.apache.flink" % "flink-streaming-java" % flinkVersion, > "org.apache.flink" % "flink-table-api-java" % flinkVersion, > "org.apache.flink" % "flink-table-api-java-bridge" % flinkVersion, > "org.apache.flink" % "flink-table-runtime" % flinkVersion, > "org.apache.flink" % "flink-clients" % flinkVersion, > "org.apache.flink" % "flink-connector-base" % flinkVersion, > "org.apache.flink" % "flink-table-planner-loader" % flinkVersion, > "org.apache.flink" % "flink-connector-kafka" % flinkVersion, > "org.apache.flink" % "flink-statebackend-rocksdb" % flinkVersion % Provided, > "org.apache.flink" % "flink-avro" % flinkVersion, > "org.apache.flink" % "flink-avro-confluent-registry" % flinkVersion, > > > > And the code path where it fails is not directly related to Flink: (It does > not fail on the unit tests). > > > > Map( > PsNowTag -> new OutputTag[T](PsNowTag, typeInformation), > SCRNTag -> new OutputTag[T](SCRNTag, typeInformation), > GMSTag -> new OutputTag[T](GMSTag, typeInformation), > MuleTag -> new OutputTag[T](MuleTag, typeInformation), > FutureTag -> new OutputTag[T](FutureTag, typeInformation), > ) > > > > The failure looks like an incompatibility issue Scala runtime. Scala 2.12 vs. > Scala 2.13 collections are not compatible: > > > > java.lang.NoSuchMethodError: 'scala.collection.immutable.ArraySeq > scala.runtime.ScalaRunTime$.wrapRefArray(java.lang.Object[])' > > > > So this issue seems weird and does look like Flink is using the Scala 2.12 > runtime even if the flink-scala packages are not installed. The question is > why? > > Then I would wonder if it is possible to completely exclude Scala from the > Flink docker image. > > > > Best, > > > > Patrick > > -- > > Patrick Eifler > > Staff Software Engineer > > FTG Data Engineering > > Sony Interactive Entertainment > > Kemperplatz 1 > > 10785 Berlin > > > > From: Martijn Visser <martijnvis...@apache.org> > Date: Tuesday, 9. January 2024 at 16:41 > To: Eifler, Patrick <patrick.eif...@sony.com> > Cc: user@flink.apache.org <user@flink.apache.org> > Subject: Re: Flink 1.17 with Scala 2.13 or Scala 3 > > Hi Patrick, > > You're on the right track, because you can't use any of the Flink > Scala APIs in order to use any arbitrary Scala version. Have you seen > the examples with Scala 3? [2] Do you have an example of your > code/setup? > > Best regards, > > Martijn > > [1] https://flink.apache.org/2022/02/22/scala-free-in-one-fifteen/ > [2] https://github.com/sjwiesman/flink-scala-3 > > On Tue, Jan 9, 2024 at 4:16 PM Eifler, Patrick <patrick.eif...@sony.com> > wrote: > > > > Hi, > > > > > > > > The Flink image still has Scala 2.12 dependencies. I tried to run a flink > > job written in Scala 2.13 avoiding the usage of all flink-scala api’s but > > getting an incompatibility issue (regarding scala.collections) that I would > > normally expect in a project were Scala 2.13 and Scala 2.12 are running > > alongside. > > > > So I wonder what am I missing, as in the docs and jira tickets everyone > > says it works now with any Scala Version. > > Any pointers are appreciated, Thanks. > > > > > > > > Best, > > > > -- > > > > Patrick Eifler
