In this case, isn't better to perform the filter earlier as possible even there could be unhandled predicates?
Telmo Rodrigues No dia 11/05/2016, às 09:49, Rishi Mishra <rmis...@snappydata.io> escreveu: > It does push the predicate. But as a relations are generic and might or might > not handle some of the predicates , it needs to apply filter of un-handled > predicates. > > Regards, > Rishitesh Mishra, > SnappyData . (http://www.snappydata.io/) > > https://in.linkedin.com/in/rishiteshmishra > >> On Wed, May 11, 2016 at 6:27 AM, Telmo Rodrigues >> <telmo.galante.rodrig...@gmail.com> wrote: >> Hello, >> >> I have a question about the Catalyst optimizer in Spark 1.6. >> >> initial logical plan: >> >> !'Project [unresolvedalias(*)] >> !+- 'Filter ('t.id = 1) >> ! +- 'Join Inner, Some(('t.id = 'u.id)) >> ! :- 'UnresolvedRelation `t`, None >> ! +- 'UnresolvedRelation `u`, None >> >> >> logical plan after optimizer execution: >> >> Project [id#0L,id#1L] >> !+- Filter (id#0L = cast(1 as bigint)) >> ! +- Join Inner, Some((id#0L = id#1L)) >> ! :- Subquery t >> ! : +- Relation[id#0L] JSONRelation >> ! +- Subquery u >> ! +- Relation[id#1L] JSONRelation >> >> >> Shouldn't the optimizer push down predicates to subquery t in order to the >> filter be executed before join? >> >> Thanks >
