ge-
From: Reynold Xin [mailto:r...@databricks.com]
Sent: Tuesday, July 29, 2014 11:44 AM
To: dev@spark.apache.org
Subject: Re: pre-filtered hadoop RDD use case
I am not sure if I agree that it lacks the mechanism to do pushdowns.
Hadoop InputFormat itself provides some basic mechanism to push
lementation seems to be in place, and more optimization is desired
> beyond just record-oriented execution pipelining.
>
>
>
> -Original Message-
> From: Reynold Xin [mailto:r...@databricks.com]
> Sent: Tuesday, July 29, 2014 12:55 AM
> To: dev@spark.apache.org
> Subject: Re: pre-f
[mailto:r...@databricks.com]
Sent: Tuesday, July 29, 2014 12:55 AM
To: dev@spark.apache.org
Subject: Re: pre-filtered hadoop RDD use case
Would something like this help?
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/rdd/PartitionPruningRDD.scala
On Th
e is PR in this I’d be happy to
> code
> it up and submit it.
>
>
> Thank you
> --
> Eugene Cheipesh
>
>
>
> --
> View this message in context:
> http://apache-spark-developers-list.1001551.n3.nabble.com/pre-filtered-hadoop-RDD-use-case-tp7484.html
> Sent from the Apache Spark Developers List mailing list archive at
> Nabble.com.
>
roblem and any
thoughts on the issue. If you think there is PR in this I’d be happy to code
it up and submit it.
Thank you
--
Eugene Cheipesh
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/pre-filtered-hadoop-RDD-use-case-tp7484.html
Sent from the
