eshi Yamamuro [mailto:linguin.m.s@<mailto:linguin.m.s@>...]
Sent: Monday, September 12, 2016 7:12 PM
To: Mendelson, Assaf
Cc: dev@...
Subject: Re: UDF and native functions performance
Hi,
I think you'd better off comparing the gen'd code of `df.filter` and your gen'd
the same performance so I imagine this has something to do with some
> optimization that understands that range is ordered and therefore once the
> first condition fails, all would fail.
>
> The problem is I don’t see this in the plan, nor can I find it in the code.
>
>
>
>
>
> *Fr
once the
first condition fails, all would fail.
The problem is I don’t see this in the plan, nor can I find it in the code.
From: Takeshi Yamamuro [mailto:linguin@gmail.com]
Sent: Monday, September 12, 2016 7:12 PM
To: Mendelson, Assaf
Cc: dev@spark.apache.org
Subject: Re: UDF and native funct
Hi,
I think you'd better off comparing the gen'd code of `df.filter` and your
gen'd code
by using .debugCodegen().
// maropu
On Mon, Sep 12, 2016 at 7:43 PM, assaf.mendelson
wrote:
> I am trying to create UDFs with improved performance. So I decided to
> compare several ways of doing it.
>
> I
