So I made some progress with updated code:
t = ibis.table([("a", "int64"), ("b", "int64")], name="table0")
test_table_0 = pa.Table.from_pydict({"a": [1, 2, 3], "b": [4, 5,
6]})
result = translate(t, self.compiler)
def table_provider(names):
if not names:
raise Exception("No names provided")
elif names[0] == 'table0':
return test_table_0
else:
raise Exception(f"Unknown table name {names}")
print(result)
result_buf =
pa._substrait._parse_json_plan(tobytes(MessageToJson(result)))
pa.substrait.run_query(result_buf, table_provider)
I think now the plan is passed properly and I got a "ArrowInvalid: Empty
substrait plan is passed"
Looking the plan reproduces by ibis-substrait, it looks like doesn't match
the expected format of Acero consumer. In particular, it looks like the
plan produced by ibis-substrait doesn't have a "relations" entry - any
thoughts on how this can be fixed? (I don't know if I am using the API
wrong or some format inconsistency between the two)
On Tue, Oct 4, 2022 at 1:54 PM Li Jin <[email protected]> wrote:
> Hi,
>
> I am testing integration between ibis-substrait and Acero but hit a
> segmentation fault. I think this might be cause the way I am
> integrating these two libraries are wrong, here is my code:
>
> Li Jin
> 1:51 PM (1 minute ago)
> to me
>
> class BasicTests(unittest.TestCase):
>
> """Test basic features"""
>
>
>
>
>
> @classmethod
>
> def setUpClass(cls):
>
> cls.compiler = SubstraitCompiler()
>
>
>
> def test_named_table(self):
>
> """Test basic"""
>
> t = ibis.table([("a", "int64"), ("b", "int64")], name="table0")
>
> result = translate(t, self.compiler)
>
>
>
> def table_provider(names):
>
> if not names:
>
> raise Exception("No names provided")
>
> elif names[0] == 'table0':
>
> return test_table_0
>
> else:
>
> raise Exception(f"Unknown table name {names}")
>
>
>
> test_table_0 = pa.Table.from_pydict({"a": [1, 2, 3], "b": [4, 5,
> 6]})
>
>
>
> print(type(result))
>
> print(result)
>
> result_bytes = result.SerializeToString()
>
>
>
> pa.substrait.run_query(result_bytes, table_provider)
>
>
> I wonder if someone has tried integration between these two before and can
> share some working code?
>
