hi Andy, AFAIK in the other implementations, the Flatbuffers serialization is an implementation detail of offering support for the columnar IPC protocol (which in turn depends on the columnar data structures, memory management, etc.). I'm not sure what a crate offering the Flatbuffers bindings alone would offer, outside of helping people create new Rust Arrow implementations, I guess. In any case, if you wanted to do that, we would need to make a release first.
- Wes On Sat, Nov 3, 2018 at 1:01 PM Andy Grove <andygrov...@gmail.com> wrote: > > Brief update on this. I have now been able to use flatbuffers to generate > Rust code from the Arrow schema, and it compiles. In theory, this can be > published as a standalone "arrow-format" crate but I'm wondering what the > plan should be here. > > I will start reviewing how this is handled in the other implementations but > if anyone has suggestions, please let me know. > > Thanks, > > Andy. > > > > On Mon, Sep 3, 2018 at 9:08 PM Andy Grove <andygrov...@gmail.com> wrote: > > > Flatbuffers now has support for Rust - the PR was just merged in the last > > couple days: > > https://github.com/google/flatbuffers/tree/master/rust/flatbuffers > > > > I hope to find some time in the next week or two to start experimenting > > with this for Arrow IPC. > > > > Andy. > > > > On Tue, Jun 19, 2018 at 6:09 PM Andy Grove <andygrov...@gmail.com> wrote: > > > >> The author of the Rust version of FlatBuffers is now working on a public > >> fork and seems to be making good progress. I just wanted to send a quick > >> update on this since I am now back onto working on Arrow IPC in Rust (I'm > >> sure this is going to take quite a while though so don't want to get > >> anyones hopes up too much). For anyone that is interested, here is the > >> FlatBuffers branch: https://github.com/rw/flatbuffers/tree/2018-02--rust > >> > >> Andy. > >> > >> > >> On Sat, May 19, 2018 at 4:05 PM Wes McKinney <wesmck...@gmail.com> wrote: > >> > >>> Sorry to hear, it's a bit of a rough situation with Flatbuffers and > >>> Rust. One possibility is to build an interface to the Flatbuffers data > >>> via flatcc (https://github.com/dvidelabs/flatcc) -- I wonder if these > >>> bindings are header-only like C++ and if that makes things any easier > >>> for you. > >>> > >>> If you defined an API in Rust for reading and writing the Arrow > >>> metadata that does not expose any Flatbuffers-specific details, then > >>> once there is a production-grade Rust implementation of Flatbuffers, > >>> the implementation details could be swapped out without disruption. > >>> > >>> > my recent attempts at contacting the author have been unsuccessful > >>> > >>> Have you e-mailed the author directly or only pings on GitHub? Pings > >>> may not be making it to their inbox. > >>> > >>> Appreciate your efforts on this; I think we'll see a lot more work in > >>> data processing in Rust in the coming years. > >>> > >>> - Wes > >>> > >>> On Fri, May 18, 2018 at 9:21 AM, Andy Grove <andygrov...@gmail.com> > >>> wrote: > >>> > Hi, > >>> > > >>> > Now that the refactor I've been working on has been merged, the next > >>> > priority for me personally with the Rust implementation is getting IPC > >>> and > >>> > integration testing working. > >>> > > >>> > Unfortunately the official Flatbuffers Rust version is not available > >>> yet > >>> > and my recent attempts at contacting the author have been unsuccessful > >>> so I > >>> > have started working with this fork of Flatbuffers which has Rust > >>> support: > >>> > https://github.com/josephDunne/flatbuffers. > >>> > > >>> > I was able to generate code from Schema.fbs but it doesn't compile and > >>> I've > >>> > started filing issues and debugging this. > >>> > > >>> > Working with this fork isn't ideal but I don't see what other choice I > >>> > have, other than just waiting for the official project to support Rust. > >>> > > >>> > I'm interested to hear if anyone has any alternate suggestions. I know > >>> it > >>> > would be possible to wrap C code but I'd like to keep the Rust > >>> > implementation as a pure Rust project if possible. > >>> > > >>> > Thanks, > >>> > > >>> > Andy. > >>> > >>
