Hi Renjie, I have not started on this but I would be interested in helping you with it.
At a high level I think there are two main parts to this work: 1. Translating DataFusion expressions to Gandiva protobuf 2. Implementing the code to make the native C call to Gandiva I could help with #1 pretty easily. I am concerned about the packaging implications of this. I feel quite strongly that there should be a "pure Rust" version of DataFusion/Arrow and that the Gandiva integration should be opt-in somehow, so maybe this is a separate project within the repository, or a feature that can be controlled by a feature flag in the Cargo.toml somehow. Thanks, Andy. On Sat, May 11, 2019 at 3:10 AM Renjie Liu <liurenjie2...@gmail.com> wrote: > > Hi: > @Andy Grove <andygrov...@gmail.com> Are you developing this? I'm > interested in this and want to join development. > > On Tue, Jan 8, 2019 at 3:18 PM Praveen Kumar <prav...@dremio.com> wrote: > >> Agree with Wes, the protobuf based interface should be the language >> neutral >> way to build expressions with Gandiva. >> >> On Mon, Jan 7, 2019 at 8:30 PM Andy Grove <andygrov...@gmail.com> wrote: >> >> > This makes sense to me know that I understand a little more about >> Gandiva. >> > This also fits well with my proposal to donate DataFusion in the other >> > thread. DataFusion can manage the overall logical query plan in Rust and >> > potentially delegate some subset of expression evaluation to Gandiva via >> > protobuf. >> > >> > Thanks, >> > >> > Andy. >> > >> > On Mon, Jan 7, 2019 at 7:51 AM Wes McKinney <wesmck...@gmail.com> >> wrote: >> > >> > > Gandiva supports a Protobuf-based interface -- this is how Java >> > > interacts with it via JNI. Rust could do the same -- that would >> > > probably be easier than wrapping the C++ class structure. It would >> > > also help drive new feature requirements in the serialized >> > > projection/filter expression trees >> > > >> > > - Wes >> > > >> > > On Mon, Jan 7, 2019 at 3:22 AM Krisztián Szűcs >> > > <szucs.kriszt...@gmail.com> wrote: >> > > > >> > > > I'm not sure, that a binding is a good idea. Both Arrow and Parquet >> > > > already have their own rust implementation, and a interfacing with >> > > > cpp isn't as easy and straightforward than it is with C. Otherwise >> > > > We could simply just maintain bindings for all of the cpp libraries, >> > > > rather than of having a hybrid solution. >> > > > >> > > > While We could spare the reimplementation of gandiva, it'd make >> > > > packaging more complicated and rust development way less >> > > > welcoming to new contributors. >> > > > >> > > > On Fri, Jan 4, 2019 at 3:39 PM Andy Grove <andygrov...@gmail.com> >> > wrote: >> > > > >> > > > > Now that the Rust implementation of Arrow is maturing, I'm >> interested >> > > in >> > > > > having bindings for Gandiva for query execution, rather than >> > > duplicating >> > > > > this in Rust. >> > > > > >> > > > > I will likely start looking at this soon but wanted to see if >> anyone >> > > else >> > > > > here is particularly interested in this area of functionality? >> > > > > >> > > > > Thanks, >> > > > > >> > > > > Andy. >> > > > > >> > > >> > >> > > > -- > Renjie Liu > Software Engineer, MVAD >
