[
https://issues.apache.org/jira/browse/SPARK-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-16898:
------------------------------------
Assignee: Apache Spark
> Adds argument type information for typed logical plan like MapElements,
> TypedFilter, and AppendColumn
> -----------------------------------------------------------------------------------------------------
>
> Key: SPARK-16898
> URL: https://issues.apache.org/jira/browse/SPARK-16898
> Project: Spark
> Issue Type: Bug
> Reporter: Sean Zhong
> Assignee: Apache Spark
> Priority: Minor
>
> Typed logical plan like MapElements, TypedFilter, and AppendColumn contains a
> closure field: {{func: (T) => Boolean}}. For example class TypedFilter's
> signature is:
> {code}
> case class TypedFilter(
> func: AnyRef,
> deserializer: Expression,
> child: LogicalPlan) extends UnaryNode
> {code}
> From the above class signature, we cannot easily find:
> 1. What is the input argument's type of the closure {{func}}? How do we know
> which apply method to pick if there are multiple overloaded apply methods?
> 2. What is the input argument's schema?
> With this info, it is easier for us to define some custom optimizer rule to
> translate these typed logical plan to more efficient implementation, like the
> closure optimization idea in SPARK-14083.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
