eshi Yamamuro [mailto:linguin.m.s@<mailto:linguin.m.s@>...]
Sent: Monday, September 12, 2016 7:12 PM
To: Mendelson, Assaf
Cc: dev@...
Subject: Re: UDF and native functions performance
Hi,
I think you'd better off comparing the gen'd code of `df.filter` and your gen'd
the same performance so I imagine this has something to do with some
> optimization that understands that range is ordered and therefore once the
> first condition fails, all would fail.
>
> The problem is I don’t see this in the plan, nor can I find it in the code.
>
>
>
>
>
> *Fr
once the
first condition fails, all would fail.
The problem is I don’t see this in the plan, nor can I find it in the code.
From: Takeshi Yamamuro [mailto:linguin@gmail.com]
Sent: Monday, September 12, 2016 7:12 PM
To: Mendelson, Assaf
Cc: dev@spark.apache.org
Subject: Re: UDF and native funct
*override def *toString: String = *s"**$*child
>
> * < 10" **override def *eval(input: InternalRow): Any = {
> *val *value = child.eval(input)
> *if *(value == *null*)
> {
>
> *false *} *else *{
> child.dataType *match *{
>
case IntegerType => value.asInstanceOf[Int] < 10
}
}
}
override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
defineCodeGen(ctx, ev, c => s"($c) < 10")
}
}
private def withExpr(expr: Expression): Column = Column(expr)
