Hello!
I am working on adding a custom data source node in Acero. I have a few
previous threads related to this topic.
Currently, I am able to register my custom factory method with Acero and
create a Custom source node, i.e., I can register and execute this with
Acero:
MySourceNodeOptions source_options = ...
Declaration source{"my_source", source_option}
The next step I want to do is to pass this through to the Acero substrait
consumer. From previous discussions, I am going to use "NamedTable '' as a
temporary way to define my custom data source in substrait. My question is
this:
What I need to do in substrait in order to register my own substrait
consumer rule/function for deserializing my custom named table protobuf
message into the declaration above. If this is not supported right now,
what is a reasonable/minimal change to make this work?
Thanks,
Li
