Re: Problem while changing conventions with rules

[Karapost]

Hi Stamis,

I followed your suggestions and looked at the full log of the planner. 
From what I understood, the rule for converting a JdbcTableScan to 
LocationTableScan FIRES but the result IS NOT registered in the related 
subset.
I have only the JdbcTableScan in my tree.
This is what I see in the log (before the rule is fired):

Sets:
Set#0, type: RecordType(INTEGER l_orderkey,... , l_shipmode, VARCHAR(44) 
l_comment)
    rel#4:Subset#0.JDBC.tpch_africa, best=rel#2, importance=0.5
        rel#2:JdbcTableScan.JDBC.tpch_africa(table=[tpch_africa, 
lineitem]), rowcount=100.0, cumulative cost={100.0 rows, 101.0 cpu, 0.0 io}
rel#5:Subset#0.LocationConvention:JDBC.tpch_africa, best=null, 
importance=1.0

After the rule's been fired, I get the message that the 
LocationTableScanRel has been registered:

Pop match: rule 
[LocationTableScanRule:LocationConvention:JDBC.tpch_africa] rels 
[rel#2:JdbcTableScan.JDBC.tpch_africa(table=[tpch_africa, lineitem])]
Rule LocationTableScanRule:LocationConvention:JDBC.tpch_africa arguments 
[rel#2:JdbcTableScan.JDBC.tpch_africa(table=[tpch_africa, lineitem])] 
created rel#8:LocationTableScanRel
Register 
rel#8:LocationTableScanRel.LocationConvention:JDBC.tpch_africa(table=[tpch_africa, 
lineitem]) in rel#5:Subset#0.LocationConvention:JDBC.tpch_africa

But nothing has changed in the Sets! I would expect to have:

rel#5:Subset#0.LocationConvention:JDBC.tpch_africa, best=null, 
importance=1.0
    rel#8...

What am I doing wrong?

On 12/10/2018 09:17, Stamatis Zampetakis wrote:
Hi Alessandro,

 From a quick look, I cannot see what is going wrong with the planner. A few
suggestions:
i) when you send the planner dump it is better to keep
the indentation otherwise it is difficult to read;
ii) you can debug more easily the planner by increasing the logger level
(e.g., log4j.logger.org.apache.calcite.plan.RelOptPlanner=TRACE);
iii) the trace output is also more complete than the one you provided in
order to understand exactly what is happening.

Best,
Stamatis



Στις Πέμ, 11 Οκτ 2018 στις 5:33 μ.μ., ο/η Karapost <
karap...@win.tu-berlin.de> έγραψε:

Hi all,

I created a new convention called LocationConvention which encapsulates
a JdbcConvention. I implemented a LocationTableScanConverterRule which
matches a JdbcTableScaneRel and should (in theory) return a
LocationTableScanRel which is just a dummy implementation of a
TableScan. I also created a ProjectRel and ProjectRule which match a
LogicalProject with convention None.

I am sure that the rule is fired by the planner but from the error I get
it seems that it doesn't actually produce any result.
I call the planner with:

LocationConvention lc;
RelTraitSet rts = planner.getEmptyTraitSet().replace(lc);
optimized = planner.transform(0,rts,logicalPlan);

I get the following error:

Exception in thread "main"
org.apache.calcite.plan.RelOptPlanner$CannotPlanException: Node
[rel#7:Subset#1.LocationConvention:JDBC.tpch_africa] could not be
implemented; planner state:

Root: rel#7:Subset#1.LocationConvention:JDBC.tpch_africa
Original rel:
LogicalProject(subset=[rel#7:Subset#1.LocationConvention:JDBC.tpch_africa],

L_ORDERKEY=[$0]): rowcount = 100.0, cumulative cost = {100.0 rows, 100.0
cpu, 0.0 io}, id = 5
    JdbcTableScan(subset=[rel#4:Subset#0.JDBC.tpch_africa],
table=[[tpch_africa, lineitem]]): rowcount = 100.0, cumulative cost =
{100.0 rows, 101.0 cpu, 0.0 io}, id = 2

Sets:
Set#0, type: RecordType(INTEGER l_orderkey, INTEGER l_partkey, INTEGER
l_suppkey, INTEGER l_linenumber, DECIMAL(15, 2) l_quantity, DECIMAL(15,
2) l_extendedprice, DECIMAL(15, 2) l_discount, DECIMAL(15, 2) l_tax,
CHAR(1) l_returnflag, CHAR(1) l_linestatus, DATE l_shipdate, DATE
l_commitdate, DATE l_receiptdate, CHAR(25) l_shipinstruct, CHAR(10)
l_shipmode, VARCHAR(44) l_comment)
      rel#4:Subset#0.JDBC.tpch_africa, best=rel#2,
importance=0.49499999999999994
          rel#2:JdbcTableScan.JDBC.tpch_africa(table=[tpch_africa,
lineitem]), rowcount=100.0, cumulative cost={100.0 rows, 101.0 cpu, 0.0 io}
rel#12:AbstractConverter.JDBC.tpch_africa(input=rel#11:Subset#0.LocationConvention:JDBC.tpch_africa,convention=JDBC.tpch_africa),

rowcount=100.0, cumulative cost={inf}
      rel#11:Subset#0.LocationConvention:JDBC.tpch_africa, best=null,
importance=0.9899999999999999
      rel#31:Subset#0.ENUMERABLE, best=rel#30,
importance=0.49499999999999994
rel#32:AbstractConverter.ENUMERABLE(input=rel#11:Subset#0.LocationConvention:JDBC.tpch_africa,convention=ENUMERABLE),

rowcount=100.0, cumulative cost={inf}
rel#30:JdbcToEnumerableConverter.ENUMERABLE(input=rel#4:Subset#0.JDBC.tpch_africa),

rowcount=100.0, cumulative cost={110.0 rows, 111.0 cpu, 0.0 io}
Set#1, type: RecordType(INTEGER L_ORDERKEY)
      rel#6:Subset#1.NONE, best=null, importance=0.49999999999999994
rel#5:LogicalProject.NONE(input=rel#4:Subset#0.JDBC.tpch_africa,L_ORDERKEY=$0),

rowcount=100.0, cumulative cost={inf}
rel#8:AbstractConverter.NONE(input=rel#7:Subset#1.LocationConvention:JDBC.tpch_africa,convention=NONE),

rowcount=100.0, cumulative cost={inf}
      rel#7:Subset#1.LocationConvention:JDBC.tpch_africa, best=null,
importance=0.9999999999999999
      rel#15:Subset#1.JDBC.tpch_africa, best=rel#14,
importance=0.49999999999999994
rel#16:AbstractConverter.JDBC.tpch_africa(input=rel#7:Subset#1.LocationConvention:JDBC.tpch_africa,convention=JDBC.tpch_africa),

rowcount=100.0, cumulative cost={inf}
rel#14:JdbcProject.JDBC.tpch_africa(input=rel#4:Subset#0.JDBC.tpch_africa,L_ORDERKEY=$0),

rowcount=100.0, cumulative cost={180.0 rows, 181.0 cpu, 0.0 io}
      rel#24:Subset#1.ENUMERABLE, best=rel#23,
importance=0.49999999999999994
rel#25:AbstractConverter.ENUMERABLE(input=rel#7:Subset#1.LocationConvention:JDBC.tpch_africa,convention=ENUMERABLE),

rowcount=100.0, cumulative cost={inf}
rel#23:JdbcToEnumerableConverter.ENUMERABLE(input=rel#15:Subset#1.JDBC.tpch_africa),

rowcount=100.0, cumulative cost={190.0 rows, 191.0 cpu, 0.0 io}


My code is:

public class LocationTableScanRel extends TableScan implements LocationRel
{

    public LocationTableScanRel(final RelOptCluster cluster,
        final RelTraitSet traitSet,
        final RelOptTable table) {
      super(cluster, traitSet, table);
    }

    @Override public RelNode copy(final RelTraitSet traitSet, final
List<RelNode> inputs) {
      return new LocationTableScanRel(getCluster(),traitSet,table);
    }

}


public class LocationTableScanRule extends ConverterRule {

    private LocationConvention lc;
    public LocationTableScanRule(LocationConvention lc){
      super(JdbcTableScan.class,lc.getJdbcConvention(),
lc,"LocationTableScanRule:" + lc);
    }
    @Override public RelNode convert( RelNode rel) {
      final JdbcTableScan tscan = (JdbcTableScan) rel;
      System.out.println("Fire!");
      return new LocationTableScanRel(tscan.getCluster(),
tscan.getCluster().traitSetOf(getOutTrait()),tscan.getTable());
    }
}

Any help and advice are very welcome

Best

Alessandro