0L = 1)
: +- Scan JSONRelation[id#0L] InputPaths: file:/persons.json,
PushedFilters: [EqualTo(id,1)]
+- Scan JSONRelation[id#1L] InputPaths: file:/cars.json
2016-05-12 16:34 GMT+01:00 Takeshi Yamamuro :
> Hi,
>
> What's the result of `df3.explain(true)`?
>
> // maropu
>
> On
I'm building spark from branch-1.6 source with mvn -DskipTests package and
I'm running the following code with spark shell.
*val* sqlContext *=* *new* org.apache.spark.sql.*SQLContext*(sc)
*import* *sqlContext.implicits._*
*val df = sqlContext.read.json("persons.json")*
*val df2 = sqlContext.r
In this case, isn't better to perform the filter earlier as possible even there
could be unhandled predicates?
Telmo Rodrigues
No dia 11/05/2016, às 09:49, Rishi Mishra escreveu:
> It does push the predicate. But as a relations are generic and might or might
> not handle
Hello,
I have a question about the Catalyst optimizer in Spark 1.6.
initial logical plan:
!'Project [unresolvedalias(*)]
!+- 'Filter ('t.id = 1)
! +- 'Join Inner, Some(('t.id = 'u.id))
! :- 'UnresolvedRelation `t`, None
! +- 'UnresolvedRelation `u`, None
logical plan after optimize
