thanks for that detailed response!
On Mon, Feb 13, 2017 at 12:56 AM, Sumedh Wale wrote:
> The difference is closure invocation instead of a static java.lang.Math
> call. In many cases JIT may not be able to perform inlining and related
> code optimizations though in this specific case it should.
The difference is closure invocation
instead of a static java.lang.Math call. In many cases JIT may not
be able to perform inlining and related code optimizations though
in this specific case it should. This is highly dependent on the
specific case, but when inlin
yes agreed. however i believe nullSafeEval is not used for codegen?
On Fri, Feb 10, 2017 at 4:56 PM, Michael Armbrust
wrote:
> Function1 is specialized, but nullSafeEval is Any => Any, so that's still
> going to box in the non-codegened execution path.
>
> On Fri, Feb 10, 2017 at 1:32 PM, Koert
Function1 is specialized, but nullSafeEval is Any => Any, so that's still
going to box in the non-codegened execution path.
On Fri, Feb 10, 2017 at 1:32 PM, Koert Kuipers wrote:
> based on that i take it that math functions would be primary beneficiaries
> since they work on primitives.
>
> so i
based on that i take it that math functions would be primary beneficiaries
since they work on primitives.
so if i take UnaryMathExpression as an example, would i not get the same
benefit if i change it to this?
abstract class UnaryMathExpression(val f: Double => Double, name: String)
extends Un
With complex types it doesn't work as well, but for primitive types the
biggest benefit of whole stage codegen is that we don't even need to put
the intermediate data into rows or columns anymore. They are just variables
(stored in CPU registers).
On Fri, Feb 10, 2017 at 8:22 PM, Koert Kuipers wr
so i have been looking for a while now at all the catalyst expressions, and
all the relative complex codegen going on.
so first off i get the benefit of codegen to turn a bunch of chained
iterators transformations into a single codegen stage for spark. that makes
sense to me, because it avoids a b
