[RESULT][VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 27.0.0 RC1

2023-06-30 Thread Andy Grove
On Fri, Jun 30, 2023 at 3:19 PM Andy Grove wrote: > The vote has passed with 3 binding +1 votes. Thank you to all who helped > with the release verification. > > On Fri, Jun 30, 2023 at 11:37 AM Andrew Lamb wrote: > >> +1 (binding) >> >> Sorry for the delay! I forgot to check my email >> >> I ve

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 27.0.0 RC1

2023-06-30 Thread Andy Grove
The vote has passed with 3 binding +1 votes. Thank you to all who helped with the release verification. On Fri, Jun 30, 2023 at 11:37 AM Andrew Lamb wrote: > +1 (binding) > > Sorry for the delay! I forgot to check my email > > I verified the release on mac x86_64 > > On Mon, Jun 26, 2023 at 9:33

Re: Apache Arrow | Graph Algorithms & Data Structures

2023-06-30 Thread Bechir Ben Daadouch
Hi, Adam, thank you :) Aldrin: First, I tried to implement the basic data structures needed for graph algorithms, namely a queue and a heapq (for breadth first search and dijkstra (weighted shortest path)). That's when I had some problems since Apache Arrow objects are immutable. This is how I

Re: Apache Arrow | Graph Algorithms & Data Structures

2023-06-30 Thread Bechir Ben Daadouch
Hi, @Adam, thank you :) @Aldrin: First, I tried to implement the basic data structures needed for graph algorithms, namely a queue and a heapq (for breadth first search and dijkstra (weighted shortest path)). That's when I had some problems since Apache Arrow objects are immutable. This is how

Re: Apache Arrow | Graph Algorithms & Data Structures

2023-06-30 Thread Gavin Ray
This isn't particularly efficient, but could you do something like this? https://replit.com/@GavinRay97/EnlightenedRichAdministration#main.py On Fri, Jun 30, 2023 at 1:10 PM Aldrin wrote: > > But I found out very quickly that I won't be able to... using only > Apache Arrow without resorting to

Re: [VOTE][RUST] Release Apache Arrow Rust 43.0.0 RC1

2023-06-30 Thread Andrew Lamb
+1 (binding) Thank you Raphael Verified on x86_64 mac On Fri, Jun 30, 2023 at 1:12 PM L. C. Hsieh wrote: > +1 (binding) > > Verified on M1 Mac. > > Thanks Raphael. > > > On Fri, Jun 30, 2023 at 8:27 AM Raphael Taylor-Davies > wrote: > > > > Hi, > > > > I would like to propose a release of Ap

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 27.0.0 RC1

2023-06-30 Thread Andrew Lamb
+1 (binding) Sorry for the delay! I forgot to check my email I verified the release on mac x86_64 On Mon, Jun 26, 2023 at 9:33 PM L. C. Hsieh wrote: > +1 (binding) > > Verified on Intel Mac. > > Thanks Andy. > > On Mon, Jun 26, 2023 at 3:05 PM Andy Grove wrote: > > > > Hi, > > > > I would lik

Re: [VOTE][RUST] Release Apache Arrow Rust 43.0.0 RC1

2023-06-30 Thread L. C. Hsieh
+1 (binding) Verified on M1 Mac. Thanks Raphael. On Fri, Jun 30, 2023 at 8:27 AM Raphael Taylor-Davies wrote: > > Hi, > > I would like to propose a release of Apache Arrow Rust Implementation, > version 43.0.0. > > This release candidate is based on commit: > 414235e7630d05cccf0b9f5032ebfc0858

Re: Apache Arrow | Graph Algorithms & Data Structures

2023-06-30 Thread Aldrin
> But I found out very quickly that I won't be able to... using only Apache > Arrow without resorting to other libraries. > I am aiming to assess the viability of Apache Arrow for graph algorithms and > data structures... > I also gave a shot at doing it similar to a certain SQL method... I'm

[VOTE][RUST] Release Apache Arrow Rust 43.0.0 RC1

2023-06-30 Thread Raphael Taylor-Davies
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 43.0.0. This release candidate is based on commit: 414235e7630d05cccf0b9f5032ebfc0858b8ae5b [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please downloa

Re: Apache Arrow | Graph Algorithms & Data Structures

2023-06-30 Thread Adam Lippai
Hi Bechir, GraphBlas is mainly an interface, a list of functions. This is similar to how BLAS is implemented by OpenBlas and MKL. Gabor Szárnyas has a great collection on how they are implanted and how they perform: https://github.com/GraphBLAS/GraphBLAS-Pointers Best regards, Adam Lippai On F

Re: Apache Arrow | Graph Algorithms & Data Structures

2023-06-30 Thread Bechir Ben Daadouch
Hey Adam, Thank you very much for taking the time to respond and for the suggestions :) I am aiming to assess the viability of Apache Arrow for graph algorithms and data structures, rather than addressing a particular issue. I am interested to determine the extent to which I can utilize Apache Ar

Re: Apache Arrow | Graph Algorithms & Data Structures

2023-06-30 Thread Adam Lippai
Hi, I’d recommend integrating with GraphBlas/suitesparse which are sparse matrix multiplication based algorithms for common graph problems. There might be an overlap in the data structures used, eg creating and a new arrow type like the recent tensor type addition, or simply creating zero copy or

Re: Apache Arrow | Graph Algorithms & Data Structures

2023-06-30 Thread Bechir Ben Daadouch
Thank you for taking the time to answer :) I don't have a fix Use-Case, but I am trying yo build a POC and evaluate whether Apache Arrow could be adequate in the context of graphs. But I found out very quickly that I won't be able to do all the necessary algorithm steps using only Apache Arrow wit

Re: detect memory leak between java and python

2023-06-30 Thread Antoine Pitrou
I think you should probably use an allocator that lives long enough, yes. Regards Antoine. Le 30/06/2023 à 10:23, Wenbo Hu a écrit : Hi, But gc is not predictable (though python's is more progressive than that of jvm), the following code fails randomly if `gc.collect` is not called. ``` #

Re: detect memory leak between java and python

2023-06-30 Thread Wenbo Hu
Hi, But gc is not predictable (though python's is more progressive than that of jvm), the following code fails randomly if `gc.collect` is not called. ``` # ENV # Python 3.8.16 (default, Jun 12 2023, 12:55:15) # [Clang 14.0.6 ] :: Anaconda, Inc. on darwin # Type "help", "copyright", "credits" or