Switching this from user to dev
On Sat, Nov 9, 2019 at 9:47 AM Bartosz Konieczny
wrote:
> Hi there,
>
> Few days ago I got an intriguing but hard to answer question:
> "Why Spark generates Java code and not Scala code?"
> (https://github.com/bartosz25/spark-scala-playground/issues/18)
>
> Since
It’s mainly due to compilation speed. Scala compiler is known to be slow.
Even javac is quite slow. We use Janino which is a simpler compiler to get
faster compilation speed at runtime.
Also for low level code we can’t use (due to perf concerns) any of the
edges scala has over java, eg we can’t us
There are some cases, shuffle hash join performs even better than sort merge
join.
While, I noticed that ShuffledHashJoinExec does not implement CodegenSupport,
is there any concern? And if there is any chance to improve the performance of
ShuffledHashJoinExec?
