Re: Flight: beginner questions for usage

2020-06-23 Thread Joris Peeters
OK, great, that clarifies a lot - I didn't appreciate how flexible the interpretation can be. Thanks for the doc pointer as well. I'll power onwards with the experiments. If I encounter areas of Flight that are still a bit rough around the edges - being in beta - then we might be able to contribut

Re: Flight: beginner questions for usage

2020-06-23 Thread David Li
Hey Joris, Your plan sounds right for Flight. As for semantics: The descriptor and ticket format are mostly application defined. For instance, I think some places (Dremio?) just put a raw SQL query as the "cmd" of a descriptor; putting serialized JSON or Protobuf is also certainly fine. I'd say

Flight: beginner questions for usage

2020-06-23 Thread Joris Peeters
Hello, I'm interested in using Flight for serving large amounts of data in a parallelised manner, and just building some Python prototypes, based on https://github.com/apache/arrow/blob/apache-arrow-0.17.1/python/examples/flight In my use-case, we'd have a bunch of worker servers, serving a numbe