What do you mean by this? Spark is written in a combination of Scala and Java, and then compiled to Java Byte Code, as is typical for both Scala and Java. If there's additional byte code generation happening, it's java byte code, because the platform runs on the JVM.
On Sat, Nov 9, 2019 at 12:47 PM Bartosz Konieczny <bartkoniec...@gmail.com> wrote: > *This Message originated outside your organization.* > ------------------------------ > Hi there, > > Few days ago I got an intriguing but hard to answer question: > "Why Spark generates Java code and not Scala code?" > (https://github.com/bartosz25/spark-scala-playground/issues/18 > <https://github.com/bartosz25/spark-scala-playground/issues/18> > ) > > Since I'm not sure about the exact answer, I'd like to ask you to confirm > or not my thinking. I was looking for the reasons in the JIRA and the > research paper "Spark SQL: Relational Data Processing in Spark" ( > http://people.csail.mit.edu/matei/papers/2015/sigmod_spark_sql.pdf > <http://people.csail.mit.edu/matei/papers/2015/sigmod_spark_sql.pdf>) > but found nothing explaining why Java over Scala. The single task I found > was about why Scala and not Java but concerning data types ( > https://issues.apache.org/jira/browse/SPARK-5193 > <https://issues.apache.org/jira/browse/SPARK-5193>) > That's why I'm writing here. > > My guesses about choosing Java code are: > - Java runtime compiler libs are more mature and prod-ready than the > Scala's - or at least, they were at the implementation time > - Scala compiler tends to be slower than the Java's > https://stackoverflow.com/questions/3490383/java-compile-speed-vs-scala-compile-speed > <https://stackoverflow.com/questions/3490383/java-compile-speed-vs-scala-compile-speed> > - Scala compiler seems to be more complex, so debugging & maintaining it > would be harder > - it was easier to represent a pure Java OO design than mixed FP/OO in > Scala > ? > > Thank you for your help. > > -- > Bartosz Konieczny > data engineer > https://www.waitingforcode.com > <https://www.waitingforcode.com> > https://github.com/bartosz25/ > <https://github.com/bartosz25/> > https://twitter.com/waitingforcode > <https://twitter.com/waitingforcode> > >
