Re: [Rust][DataFusion] DataFusion Overview / Architecture

2021-02-04 Thread Fernando Herrera
Hi Andy. I would like to take you offer and get a copy of your book. It would help me to understand better datafusion and help Andrew with the project documentation. Fernando On Thu, 4 Feb 2021, 18:01 Andy Grove, wrote: > That's correct, Remi. I built the Kotlin query engine from scratch as I w

Re: [Rust][DataFusion] DataFusion Overview / Architecture

2021-02-04 Thread Andy Grove
That's correct, Remi. I built the Kotlin query engine from scratch as I was writing the book, and it does follow the same basic design as DataFusion. I think it would be a useful reference for anyone writing up some DataFusion-specific documentation and I am happy to send a free copy to anyone who

Re: [Rust][DataFusion] DataFusion Overview / Architecture

2021-02-04 Thread Rémi Dettai
Hi Andrew! The book "How query engines work" ( https://leanpub.com/how-query-engines-work) that Andy wrote is pretty great! It documents query engine APIs in Kotlin and not Rust, as it was written during earlier Ballista experimentations, but almost all items still apply to DataFusion (feel free t

Re: [Rust] DataFusion TPCH benchmark overview

2021-02-04 Thread Daniël Heres
Thanks all for your input! I will create an umbrella ticket + linked failures / issues to track progress for TPCH support coming days and will share it here. Daniël Op do 4 feb. 2021 om 00:13 schreef Andrew Lamb : > This is awesome, thank you Daniel. I agree that focusing on enough SQL for > TP

Re: [Rust] DataFusion TPCH benchmark overview

2021-02-04 Thread Fernando Herrera
For me that would be great. I'm going to start reading the code and see what I can write to the arrow guide I'm working on. Thanks On Thu, 4 Feb 2021, 11:28 Andrew Lamb, wrote: > H Fernando, yes I would be delighted. > > I am planning on creating a high level overview w/ slides as a Tech Talk >

[Rust][DataFusion] DataFusion Overview / Architecture

2021-02-04 Thread Andrew Lamb
Does anyone have any high level architectural / overview material about DataFusion that they can share or point me at? I am planning on creating a high level / architectural overview of DataFusion (as it exists today) as a set of slides for a Tech Talk (will be open to the public) sometime in Marc

Re: [Rust] DataFusion TPCH benchmark overview

2021-02-04 Thread Andrew Lamb
H Fernando, yes I would be delighted. I am planning on creating a high level overview w/ slides as a Tech Talk (for work, but will be open to the public) sometime in March. How about I pull together some initial material, and then I can share that / go over it with anyone who is interested? What

[NIGHTLY] Arrow Build Report for Job nightly-2021-02-04-0

2021-02-04 Thread Crossbow
Arrow Build Report for Job nightly-2021-02-04-0 All tasks: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-04-0 Failed Tasks: - conda-linux-gcc-py36-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-04-0-drone-conda-linux

Re: [Rust] DataFusion TPCH benchmark overview

2021-02-04 Thread Fernando Herrera
Hi Andrew, I would like to work a little bit more on Datafusion, so I was wondering if you could give a small walkthrough of the code and how the queries are constructed. Do you think that could be possible? Fernando On Wed, Feb 3, 2021 at 11:13 PM Andrew Lamb wrote: > This is awesome, thank yo