Hi everyone,
I’d like to discuss the possibility of sending filtering predicates to an
Arrow Flight server. Currently, it’s unclear what the "proper" approach is
for achieving this.
The GetFlightInfo method only accepts a FlightDescriptor without additional
parameters. If it allowed for extra parameters, this use case might be more
straightforward.
I’ve considered four possible approaches for passing predicates and
retrieving the appropriate endpoints from the server:
1.
*Include predicates in gRPC headers* when calling DoGet or DoExchange.
-
This approach is limited by gRPC header size constraints.
2.
*Embed predicates within the ticket returned by the Arrow Flight server.*
-
This requires the client to interpret the opaque binary ticket,
potentially blurring the client-server separation.
3.
*Use a DoAction request (e.g., "get_flight_endpoints")* to accept
filtering predicates and return the corresponding serialized endpoints.
4.
*Modify the FlightDescriptor to a command (CMD) type* and encode both
the original ticket and predicate information in the command field.
I’m leaning toward approach #3 due to its flexibility and the advantage of
avoiding unnecessary schema serialization in FlightInfo when the client
only needs endpoints.
Has anyone else encountered this challenge? If so, what solutions have
worked for you?
Looking forward to your thoughts.
Best,
Rusty
