Re: DISCUSS: Stabilize Arrow C Stream Interface?

2022-05-27 Thread Andrew Lamb
+1 to the idea on making it a stable interface On Thu, May 26, 2022 at 6:57 PM Jonathan Keane wrote: > I too am +1 (nonbinding) to marking it as stable > > -Jon > > > On Thu, May 26, 2022 at 1:05 PM Neal Richardson < > neal.p.richard...@gmail.com> > wrote: > > > +1 from me too to mark it as stab

[VOTE][RUST] Release Apache Arrow Rust 15.0.0 RC1

2022-05-27 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 15.0.0. This release candidate is based on commit: 6e6a9e148c0f50ed1e0eeebd667245e67d7826b6 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please download, ve

[RESULT] [VOTE][RUST] Release Apache Arrow Rust 15.0.0 RC1

2022-05-30 Thread Andrew Lamb
gt; > > > > > +1 (binding) > > > > > > > > > > Verified on Ubuntu 20.04.4 LTS > > > > > > > > > > Thanks, Andrew! > > > > > > > > > > On Fri, May 27, 2022 at 7:10 AM Remzi Yang < > 1371656737...@g

Re: [C++] Adding Run-Length Encoding to Arrow

2022-06-04 Thread Andrew Lamb
I think the biggest benefit of RLE is not on-the-wire compression, as that can be done via more general purpose compression schemes as Antoine mentions. The biggest benefit of RLE is that it allows operating directly and very efficiently on the "encoded" form -- for example, you can apply filters

Re: [VOTE] Mark C Stream Interface as Stable

2022-06-08 Thread Andrew Lamb
+1 (binding) On Wed, Jun 8, 2022 at 2:16 PM Will Jones wrote: > Hi, > > Given all feedback to discussion [1] has been positive, I would like to > propose marking the C Stream Interface as stable. > > I have prepared PRs in apache/arrow [2] and apache/arrow-rs [3] to remove > all "experimental" m

[VOTE][RUST] Release Apache Arrow Rust 16.0.0 RC1

2022-06-10 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 16.0.0. This release candidate is based on commit: c396dfb5035d22e57717b6dd365486b76eb611bc [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please download, ve

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 9.0.0 RC1

2022-06-10 Thread Andrew Lamb
+1 (binding) The verification script did not complete cleanly[1], but the errors appear to be bugs in the test normalization rather than anything in the code itself. Thanks for preparing the release Andrew [1] https://github.com/apache/arrow-datafusion/issues/2719 On Fri, Jun 10, 2022 at 11:3

Re: [VOTE][RUST] Release Apache Arrow Rust 16.0.0 RC1

2022-06-13 Thread Andrew Lamb
3: core::result::Result::unwrap > at > /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/core/src/ > result.rs:1078 > :23 >4: arrow::ipc::reader::tests::test_arrow_single_float_row > at ./src/ipc/reader.rs:1596:20 > > > Regards, > Mar

Re: [VOTE][RUST] Release Apache Arrow Rust 16.0.0 RC1

2022-06-13 Thread Andrew Lamb
Thanks! On Mon, Jun 13, 2022 at 3:36 PM Martin Grigorov wrote: > Hi Andrew, > > I used the `verify-release-candidate.sh` script on both machines. > > I will try to debug the test tomorrow! > > Regards, > Martin > > On Mon, Jun 13, 2022 at 10:31 PM Andrew La

[RESULT] [VOTE][RUST] Release Apache Arrow Rust 16.0.0 RC1

2022-06-13 Thread Andrew Lamb
t; :23 >4: arrow::ipc::reader::tests::test_arrow_single_float_row > at ./src/ipc/reader.rs:1596:20 > > > Regards, > Martin > > On Fri, Jun 10, 2022 at 8:54 PM Andrew Lamb wrote: > > > Hi, > > > > I would like to propose a release of

Re: Building Arrow-rs needs a lot of disk space

2022-06-13 Thread Andrew Lamb
Hi Martin, Rust in general has a known penchant for consuming large amounts of space -- any help to slim down resource usage would be most appreciated. Let us know if you need any help! Andrew On Mon, Jun 13, 2022 at 9:04 AM Martin Grigorov wrote: > Hi, > > While verifying 16.0.0 RC1 with `./d

Rust / Arrow sync Call - June 23 2022

2022-06-22 Thread Andrew Lamb
Andy Grove is hosting the next Rust / Arrow sync call tomorrow. More details such the meeting link can be found on the agenda document [1]. As always, all are invited and we'll send a summary of the meeting afterwards Thank you, Andrew [1] https://docs.google.com/document/d/1atCVnoff5SR4eM4Lwf2M1

Re: [ANNOUNCE] New Arrow committers: Dewey Dunnington, Alenka Frim, and Rok Mihevc

2022-06-22 Thread Andrew Lamb
Congratulations! On Wed, Jun 22, 2022 at 1:27 PM Dragoș Moldovan-Grünfeld < dragos.m...@gmail.com> wrote: > Congratulations! > > Sent from my iPhone > > > On 22 Jun 2022, at 18:13, Neal Richardson > wrote: > > > > On behalf of the Arrow PMC, I'm happy to announce that > > > > Dewey Dunnington >

[VOTE][RUST] Release Apache Arrow Rust 17.0.0 RC1

2022-06-24 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 17.0.0. This release candidate is based on commit: 9f7b6004d365b0c0bac8e30170b49bdd66cc7df0 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please download, ve

[RESULT] Re: [VOTE][RUST] Release Apache Arrow Rust 17.0.0 RC1

2022-06-27 Thread Andrew Lamb
pinning > to a specific commit, so this does not need to block the release for > everyone else. > > Regards, > Jörn > > From: Andrew Lamb > Sent: Friday, 24 June 2022 22:45 > To: dev > Subject: [VOTE][RUST] Release Apache Arrow Rust 17

[RUST][DISCUSS] Donate object_store_rs to Arrow

2022-06-28 Thread Andrew Lamb
Hello Rust / Arrow community, There is a proposal [1] in DataFusion to switch the abstraction used to read from remote object storage. Due to various reasons, this code is currently in its own crate/library [2] with unclear governance. We would like to contribute this code to the Apache Arrow pro

Re: [VOTE] Accept donation of Flight SQL JDBC driver

2022-07-01 Thread Andrew Lamb
+1 (binding) Thank you for driving this work On Thu, Jun 30, 2022 at 9:28 PM Yibo Cai wrote: > +1 > > > On 7/1/22 04:20, David Li wrote: > > Hello, > > > > This vote is to determine if the Arrow PMC is in favor of accepting the > donation of the Flight SQL JDBC driver. > > > > This process was

Re: [RUST][DISCUSS] Donate object_store_rs to Arrow

2022-07-08 Thread Andrew Lamb
like to see the crate adopted throughout the Rust Arrow ecosystem, and > if > > > going under Apache governance helps that end, I support that. > > > > > > My one question is: Do we have maintainers (not sure if it has to be a > > > committer or a PMC membe

[VOTE][RUST] Release Apache Arrow Rust 18.0.0 RC1

2022-07-08 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 18.0.0. This release candidate is based on commit: 330505c98c9db4d103b4965e8bd55065961dada7 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please download, ve

Re: [Rust] Enable GitHub discussions for Rust projects?

2022-07-11 Thread Andrew Lamb
> >> the experience didn't match the enthusiasm. > >> > >> (A couple of us Substrait folks have started thinking that we should > shift > >> traffic from the GH discussions to mailing lists, fyi. I suppose grass > is > >> always greener.) > >&g

[RESULT] [VOTE][RUST] Release Apache Arrow Rust 18.0.0 RC1

2022-07-11 Thread Andrew Lamb
Fri, Jul 8, 2022 at 9:55 PM Andrew Lamb wrote: > > > Hi, > > > > I would like to propose a release of Apache Arrow Rust Implementation, > > version 18.0.0. > > > > This release candidate is based on commit: > > 330505c98c9db4d103b4965e8bd55065961dada7 [1]

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 10.0.0 RC1

2022-07-13 Thread Andrew Lamb
+1 (binding) Thanks Andy! I know releases are a significant amount of work. Andrew On Tue, Jul 12, 2022 at 11:45 AM Andy Grove wrote: > Hi, > > I would like to propose a release of Apache Arrow DataFusion > Implementation, > version 10.0.0. > > This release candidate is based on commit: > d25e

[VOTE] Accept donation of Rust Object Store donation

2022-07-18 Thread Andrew Lamb
Hello, This vote is to determine if the Arrow PMC is in favor of accepting the donation of the Rust Object Store crate to arrow-rs. We have previously discussed this topic [1] and [2]. The proposed donation is at [3]. IP clearance work is underway in [4] and [5] The vote will be open for at lea

[Discuss][Rust][DataFusion] Errors with Context

2022-07-21 Thread Andrew Lamb
I wanted to mention that there is a proposed addition to DataFusionError, to add a `Context` variant in an open PR [1] . The addition to DataFusionError might not be obvious as the PR is large and is largely focused on subqueries, but given the variety of opinions on Error handling in Rust in gene

[Rust] Arrow Call Notes - 2022-07-21

2022-07-21 Thread Andrew Lamb
Hi, We had a call today. See more details in the agenda document [1] Attendees: Andrew Lamb, Dan Harris, Brent, Ian Joiner, Paddy Horan, Xudong Topics: * FlightSQL – described history and status of the progress in Rust * Discussed history and status of Row Format. Ian said he was interested in

[RESULT] Re: [VOTE] Accept donation of Rust Object Store donation

2022-07-22 Thread Andrew Lamb
22, 09:03 Martin Grigorov wrote: > > > +1 (non-binding) > > > > On Mon, Jul 18, 2022 at 6:55 PM Andrew Lamb > wrote: > > > > > Hello, > > > > > > This vote is to determine if the Arrow PMC is in favor of accepting the > > > donation

Re: [Rust] IPC Format / Feather support in Datafusion

2022-07-22 Thread Andrew Lamb
I am not familiar with feather -- there are some Draft PRs to add support for reading [1] and writing [2] the IPC file format which may be related to your question [1] https://github.com/apache/arrow-datafusion/pull/1858 [2] https://github.com/apache/arrow-datafusion/pull/1893 On Fri, Jul 22, 20

[VOTE][RUST] Release Apache Arrow Rust 19.0.0 RC1

2022-07-22 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 19.0.0. This release candidate is based on commit: c3e019f3011a902a344758969b1cfc3604f3c2d7 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please download, ve

Re: [Rust] IPC Format / Feather support in Datafusion

2022-07-22 Thread Andrew Lamb
to update it). I don't think I've > seen a formal discussion/decision on whether to keep using Feather or IPC > as the name (though I likely missed it). > > On Fri, Jul 22, 2022, at 12:21, Andrew Lamb wrote: > > I am not familiar with feather -- there are some Draft PRs to ad

[RESULT][VOTE][RUST] Release Apache Arrow Rust 19.0.0 RC1

2022-07-26 Thread Andrew Lamb
ntu 20.04.4 x86_64 and openEuler 20.03 aarch64. > > > > > > Regards, > > > Martin > > > > > > On Fri, Jul 22, 2022 at 7:56 PM Andrew Lamb > > wrote: > > > > > > > Hi, > > > > > > > > I would like to propose

Re: Proposal: renaming the 'master' branch to 'main'

2022-07-26 Thread Andrew Lamb
I am in favor of making the switch in arrow-rs as well -- can someone file an issue so to start that discussion? On Tue, Jul 26, 2022 at 9:43 AM Kevin Gurney wrote: > Hi All, > > Thank you for bringing up this topic again. > > @Neal - I am already the assignee for ARROW-15693, which involves upd

Re: [RUST][Go][proposal] Arrow Intermediate Representation to facilitate the transformation of row-oriented data sources into Arrow columnar representation

2022-07-29 Thread Andrew Lamb
I am +0 on a standard API -- in the Rust arrow-rs implementation we tend to borrow inspiration from the C++ / Java interfaces and then create appropriate Rust APIs. There is also a row based format in DataFusion [1] (Rust) and it is used to implement certain GroupBy and Sorts (similarly to what Sa

Re: [proposal] Arrow Intermediate Representation to facilitate the transformation of row-oriented data sources into Arrow columnar representation

2022-07-29 Thread Andrew Lamb
There has been a substantial amount of effort put into the arrow-rs Rust Parquet implementation to handle the corner cases of nested structs and list, and all the fun of various levels of nullability. Do let us know if you happen to try writing nested structures directly to parquet and have issues

[DISCUSS] [RUST] object_store release planning / schedule

2022-07-29 Thread Andrew Lamb
Hi, We have completed IP clearance, code merge, and CI for the Rust object store implementation. One final unresolved discussion is when to release new versions. I would like invite anyone with an opinion to discuss the proposal for a new release on [1] More details on the progress of the object

Re: [Flight][Java][JDBC] IP clearance of Flight JDBC Driver

2022-07-30 Thread Andrew Lamb
Thank you for working to make this happen David -- I don't know how much feedback you get from the community about JDBC/ODBC drivers for Arrow Flight SQL but I am excited, and I have encountered many other people who are similarly excited. I predict these technologies will be widely used and adopt

Re: [DISCUSS][Format] Dynamic data encodings in the IPC format and C ABI

2022-07-30 Thread Andrew Lamb
> but the decision must be declared up front in the schema. I am probably missing something obvious but given the description of the IPC format [1] perhaps we could support "on the fly changes" by sending additional `SCHEMA` messages in the stream, rather than just a single message at the beginnin

Re: [DISCUSS][Format] Dynamic data encodings in the IPC format and C ABI

2022-08-01 Thread Andrew Lamb
ty of implementations. For example, would RLE data be encoded as (run-length), or as (offset, run-length)? There are tradeoffs for both representations that depend a lot on the specific usecase. Andrew On Sat, Jul 30, 2022 at 3:36 PM Wes McKinney wrote: > On Sat, Jul 30, 2022 at 7:10 AM And

[VOTE][RUST] Release Apache Arrow Rust 20.0.0 RC2

2022-08-05 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 20.0.0. Note it includes the new object_store source as described in [5]. I don't think it is a release blocker but would welcome other opinions Also note this is RC2 (RC1 had an issue [6]) This release candidate

[RESULT] Re: [VOTE][RUST] Release Apache Arrow Rust 20.0.0 RC2

2022-08-08 Thread Andrew Lamb
n macOS 12.2.1 with Apple M1 Pro chip > > On Fri, Aug 5, 2022 at 5:07 PM Andrew Lamb wrote: > > > Hi, > > > > I would like to propose a release of Apache Arrow Rust Implementation, > > version 20.0.0. Note it includes the new object_store source as described > &

[VOTE][RUST] Release Apache Arrow Rust Object Store 0.4.0 RC2

2022-08-10 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.4.0. This is my first attempt to do a release so please let me know if you encounter any issues or have any suggestions. Note that the verification script is in a different (but similar) location Th

[RESULT] [VOTE][RUST] Release Apache Arrow Rust Object Store 0.4.0 RC2

2022-08-13 Thread Andrew Lamb
, code, review and comments. Andrew On Thu, Aug 11, 2022 at 9:41 AM Ian Joiner wrote: > +1 (Non-binding) > > Verified on macOS 12.2.1 with Apple M1 Pro chip > > Ian > > On Wed, Aug 10, 2022 at 10:28 AM Andrew Lamb wrote: > > > Hi, > > > > I would like

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 11.0.0 RC1

2022-08-17 Thread Andrew Lamb
This looks similar to [1] Do you by any chance have the ARROW_TEST_DATA environment set? If so I think it needs to end with a `/` or be unset to run the script. The difference is there is something wrong with the normalization: Expected: > files=[ARROW_TEST_DATA/csv/aggregate_test_100.csv], has_

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 11.0.0 RC1

2022-08-17 Thread Andrew Lamb
+1 (binding) Thanks Andy and everyone else who contributed to this release -- this looks like it will be great! On Tue, Aug 16, 2022 at 1:17 PM Ian Joiner wrote: > +1 (Non-binding) > > Verified on macOS 12.2.1 / Apple M1 Pro > > P.S. If verified with zsh instead of bash we got a command not fou

Re: [Rust] Proposal to use feature branches for changes that depend on unreleased versions of dependencies

2022-08-18 Thread Andrew Lamb
I think a feature branch sounds like a good idea to me. In the interim, I will work on making another sqlparser release On Thu, Aug 18, 2022 at 12:47 PM Andy Grove wrote: > With multiple contributors adding new features to sqlparser and arrow-rs > (often breaking changes), it is becoming more c

[VOTE][RUST] Release Apache Arrow Rust 21.0.0 RC2

2022-08-18 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 21.0.0. This release candidate is based on commit: 68934f0fa5da62ae9fcbb753d1dfd8f672a4eb1e [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please download, ve

Re: [VOTE][RUST] Release Apache Arrow Rust 21.0.0 RC2

2022-08-22 Thread Andrew Lamb
nd Array > > [Number(1), String(\"hello\")]"`, > > right: `"Json error: Expected JSON record to be an object, found > > Array([Number(1), String(\"hello\")])"`', src/json/reader.rs:2624:9 > > > > > > failures: > >

Re: [datafusion] datafusion cannot recognize chinese charactors?

2022-08-29 Thread Andrew Lamb
For completeness, I believe there is a corresponding issue [1] with resolution in the datafusion repo (double quotes are needed) Andrew [1] https://github.com/apache/arrow-datafusion/issues/3203 On Mon, Aug 22, 2022 at 2:16 PM Shao Grant wrote: > sorry, it's in rust not python, my bad.. > > >

[RESULT] [VOTE][RUST] Release Apache Arrow Rust 21.0.0 RC2

2022-08-29 Thread Andrew Lamb
ame json issue, already discussed. > > > > On Mon, Aug 22, 2022 at 6:45 AM Andy Grove > wrote: > > > > > That sounds fine to me. > > > > > > +1 (binding) > > > > > > Thanks, Andrew and LC. > > > > > > On Mon, Aug

Re: Usage of the name Feather?

2022-08-29 Thread Andrew Lamb
In the rust implementation we use the term "Arrow IPC" and I support your option 1: > The name Feather V2 is deprecated. Only the extension ".arrow" will be used for IPC files. Andrew On Mon, Aug 29, 2022 at 11:21 AM Matthew Topol wrote: > When I wrote "In-Memory Analytics with Apache Arrow" I

[VOTE][RUST] Release Apache Arrow Rust 22.0.0 RC1

2022-09-02 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 22.0.0. This release candidate is based on commit: e5b9d05ec50807666efe401729708d53216d79fc [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please download, ve

[Discuss][Rust][DataFusion] Supporting unsigned datatypes via SQL in DataFusion

2022-09-03 Thread Andrew Lamb
There is a discussion about supporting signed / unsigned types in DataFusion [1] that I think would benefit from some additional viewpoints. Thank you Andrew [1] https://github.com/apache/arrow-datafusion/issues/3325

Re: [ANNOUNCE] New Arrow PMC member: L. C. Hsieh

2022-09-04 Thread Andrew Lamb
Congratulations L. C. -- very well deserved Andrew On Sun, Sep 4, 2022 at 2:30 AM Gidon Gershinsky wrote: > Congrats Liang-Chi!! > > Cheers, Gidon > > > On Sun, Sep 4, 2022 at 7:37 AM Micah Kornfield > wrote: > > > Congrats! > > > > On Sat, Sep 3, 2022 at 8:19 PM QP Hou wrote: > > > > > Congr

Re: [ANNOUNCE] New Arrow PMC member: Weston Pace

2022-09-05 Thread Andrew Lamb
Congratulations Weston! On Mon, Sep 5, 2022 at 8:09 AM Rok Mihevc wrote: > Congrats Weston! > > Rok >

[RESULT][VOTE][RUST] Release Apache Arrow Rust 22.0.0 RC1

2022-09-05 Thread Andrew Lamb
>> > > >> Verified on Ubuntu 20.04.4 LTS > > >> > > >> Thanks, Andrew. > > >> > > >> On Fri, Sep 2, 2022 at 12:25 PM Ian Joiner > > wrote: > > >> > > >>> +1 (Non-binding) > > >>&

[ANNOUNCE] New Arrow committer: Yanghong Zhong

2022-09-08 Thread Andrew Lamb
On behalf of the Arrow PMC, I'm happy to announce that Yanghong Zhong has accepted an invitation to become a committer on Apache Arrow. Welcome, and thank you for your contributions! Andrew

Re: [Discussion] Arrow-rs minor and patch releases

2022-09-08 Thread Andrew Lamb
This is a great discussion to have. Thank you for raising it. We actually used to do fewer major releases and more minor releases (which was done by backporting API compatible changes to an active_release branch). However this took significant maintainer time (mostly my own, as I recall), though i

[VOTE][RUST] Release Apache Arrow Rust Object Store 0.5.0 RC1

2022-09-08 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.5.0. This release candidate is based on commit: dd58805b1c46691fcbe5b46412b2581ae3bd2a58 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please

[ANNOUNCE] New Arrow committer: Remzi Yang

2022-09-10 Thread Andrew Lamb
On behalf of the Arrow PMC, I'm happy to announce that Remzi Yang has accepted an invitation to become a committer on Apache Arrow. Welcome, and thank you for your contributions! Andrew

Re: [VOTE][RUST] Release Apache Arrow Rust Object Store 0.5.0 RC1

2022-09-11 Thread Andrew Lamb
> > > > > > +1 (binding) > > > > > > > > Verified on M1 Mac. > > > > > > > > On Thu, Sep 8, 2022 at 12:14 PM QP Hou wrote: > > > > > > > > > > +1 (binding) > > > > > > > > &

Re: [VOTE][RUST] Release Apache Arrow Rust Object Store 0.5.0 RC1

2022-09-11 Thread Andrew Lamb
h both release script and manually > > Jeremy Dyer > > On Sun, Sep 11, 2022 at 7:13 AM Andrew Lamb wrote: > > > Hi Jeremy, > > > > Thank you for the report. It appears that while quick-xml 0.24 was > yanked, > > they have subsequently released 0.24.1 (see d

Re: Subject: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 12.0.0 RC1

2022-09-12 Thread Andrew Lamb
+1 (binding) Thanks Andy. This is a pretty epic release. Thanks to everyone who contributed! We should write another blog post soon about all this great work -- I'll start that conversation as a github issue shortly Andrew On Mon, Sep 12, 2022 at 2:56 PM Andy Grove wrote: > Hi, > > I would li

Re: [datafusion] I want to learn datafusion but don't know where to get help,

2022-09-15 Thread Andrew Lamb
Hi Shao, Some other sources of information about DataFusion are the API docs [1] as well as the github repository [2]. There are also several examples of use in [3] Hope that helps, Andrew [1] https://docs.rs/datafusion/11.0.0/datafusion/ [2] https://github.com/apache/arrow-datafusion [3] http

[VOTE][RUST] Release Apache Arrow Rust 23.0.0 RC2

2022-09-16 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 23.0.0. This release candidate is based on commit: 5a55406cf24171600a143a83a95046c7513fd92c [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Note there is a kno

Re: [VOTE][RUST] Release Apache Arrow Rust 23.0.0 RC2

2022-09-16 Thread Andrew Lamb
ow-23.0.0.X.yNgAGJ0B/apache-arrow-rs-23.0.0/arrow)` > > > Ian > > On Fri, Sep 16, 2022 at 2:05 PM Andrew Lamb wrote: > > > Hi, > > > > I would like to propose a release of Apache Arrow Rust Implementation, > > version 23.0.0. > > > > This r

Re: [VOTE][RUST][Ballista] Release Apache Arrow Ballista 0.8.0 RC2

2022-09-16 Thread Andrew Lamb
+1 (binding) I ran the verify script and reviewed the changelog -- very cool On Fri, Sep 16, 2022 at 4:03 PM Andy Grove wrote: > Hi, > > I would like to propose a release of Apache Arrow Ballista Implementation, > version 0.8.0. > > This release candidate is based on commit: > 04f236b407cfeff9e

Re: [VOTE][RUST] Release Apache Arrow Rust 23.0.0 RC2

2022-09-17 Thread Andrew Lamb
We have resolved the issue (it was a verification scrip issue). I hope the voting on this release candidate can continue On Fri, Sep 16, 2022, 16:53 Andrew Lamb wrote: > Thank you for the report Ian. I am sorry -- I think there is a bug in the > verification script; I have filed an is

[RESULT] [VOTE][RUST] Release Apache Arrow Rust 23.0.0 RC2

2022-09-19 Thread Andrew Lamb
; > > > Verified on my macOS 12.2.1 / Apple M1 Pro > > > > > > On Fri, Sep 16, 2022 at 2:05 PM Andrew Lamb > > wrote: > > > > > > > Hi, > > > > > > > > I would like to propose a release of Apache Arrow Ru

[ANNOUNCE] New Arrow committer: Dan Harris

2022-09-20 Thread Andrew Lamb
On behalf of the Arrow PMC, I'm happy to announce that Dan Harris has accepted an invitation to become a committer on Apache Arrow. Welcome, and thank you for your contributions! Andrew

[VOTE][RUST] Release Apache Arrow Rust 24.0.0 RC1

2022-09-30 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 24.0.0. This release candidate is based on commit: 6477db16ca051b656bc0fe1f11bca85a23dbcd14 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please download, ve

[WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow

2022-10-02 Thread Andrew Lamb
We are working on a series of blog posts[1][2][3] that we plan to publish to the arrow blog about how structured data is represented using the two formats. Among other things, I hope this may help other Arrow / Parquet implementers as well as draw attention to the fact that parquet and arrow both

[RESULT][VOTE][RUST] Release Apache Arrow Rust 24.0.0 RC1

2022-10-03 Thread Andrew Lamb
> >> Thanks, Andrew! > > > >> > > > >>> On Fri, Sep 30, 2022 at 2:26 PM Jeremy Dyer > > wrote: > > > >>> > > > >>> +1 (non-binding) > > > >>> > > > >>> Confirmed on Ubuntu 22.04.1 L

Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow

2022-10-05 Thread Andrew Lamb
We have published the first post: http://localhost:4000/blog/2022/10/05/arrow-parquet-encoding-part-1/ On Sun, Oct 2, 2022 at 7:00 AM Andrew Lamb wrote: > We are working on a series of blog posts[1][2][3] that we plan to publish > to the arrow blog about how structured data is represented

Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow

2022-10-05 Thread Andrew Lamb
t; > krassovskysa...@gmail.com> > wrote: > > > Hi, we aren’t able to connect to your localhost 😀 > > > > > On Oct 5, 2022, at 12:44 PM, Andrew Lamb wrote: > > > > > > We have published the first post: > > > http://localhost:4000/blog/2022

[VOTE][RUST] Release Apache Arrow Rust Object Store 0.5.1 RC1

2022-10-07 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.5.1. This release candidate is based on commit: 8a54e95850fe27ac5865a02ef4be2de0937de5b3 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 13.0.0 RC1

2022-10-07 Thread Andrew Lamb
+1 (binding) I ran the release verification script (macos x86_64) as well as downloaded the tarball and poked around. Looks nice :thumbsup: -- thanks Andy On Fri, Oct 7, 2022 at 7:26 AM Andy Grove wrote: > Hi, > > I would like to propose a release of Apache Arrow DataFusion > Implementation, >

Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow

2022-10-08 Thread Andrew Lamb
Second post is now live (this time with non localhost url :thumbsup:): https://arrow.apache.org/blog/2022/10/08/arrow-parquet-encoding-part-2/ Andrew On Wed, Oct 5, 2022 at 4:57 PM Andrew Lamb wrote: > Let's just say the posting process has been a comedy of errors by me > 🤦‍♂️.

Re: [VOTE][RUST] Release Apache Arrow Rust Object Store 0.5.1 RC1

2022-10-10 Thread Andrew Lamb
Andrew On Sat, Oct 8, 2022 at 12:06 PM Ashish wrote: > +1 (non-binding) > > verified on M1 Mac > > On Fri, Oct 7, 2022 at 8:32 AM Andrew Lamb wrote: > > > Hi, > > > > I would like to propose a release of Apache Arrow Rust Object > > Store Implementation, ver

[VOTE][RUST] Release Apache Arrow Rust 25.0.0 RC1

2022-10-14 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 25.0.0. This release candidate is based on commit: 1eb19b5394b84eaa0dbb24f65e74018defb3332b [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please download, ve

Re: [VOTE][RUST] Release Apache Arrow Rust 25.0.0 RC1

2022-10-17 Thread Andrew Lamb
").unwrap(); > > Let's see if anyone else hits this or whether it is just me / my > environment. > > > On Fri, Oct 14, 2022 at 3:31 PM Andrew Lamb wrote: > > > Hi, > > > > I would like to propose a release of Apache Arrow Rust Implementation, >

[RESULT][VOTE][RUST] Release Apache Arrow Rust 25.0.0 RC1

2022-10-17 Thread Andrew Lamb
I tried this again, but this time ran the verify script from the repo > > > > rather than the one in the tarball, and that worked fine (although I > do > > > not > > > > understand why). > > > > > > > > +1 (binding) > > >

Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow

2022-10-17 Thread Andrew Lamb
And the final installment: https://arrow.apache.org/blog/2022/10/17/arrow-parquet-encoding-part-3/ On Sat, Oct 8, 2022 at 9:47 AM Andrew Lamb wrote: > Second post is now live (this time with non localhost url :thumbsup:): > > https://arrow.apache.org/blog/2022/10/08/arrow-parquet-enco

Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow

2022-10-20 Thread Andrew Lamb
I think it is a great idea to tweet it out -- can someone help me do so? On Wed, Oct 19, 2022 at 8:26 PM David Li wrote: > Andrew, what do you think about tweeting this series out through > @ApacheArrow? > > -David > > On Mon, Oct 17, 2022, at 16:50, Andrew Lamb wrote:

Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow

2022-10-20 Thread Andrew Lamb
et as @ApacheArrow > from https://tweetdeck.twitter.com/ after you accept the > invitation. > > Thanks, > -- > kou > > In > "Re: [WEBSITE] Blog posts on representing Structured Data with Parquet > and Arrow" on Thu, 20 Oct 2022 07:17:11 -0400, > Andrew

[RUST][DataFusion] Project Update Blog

2022-10-22 Thread Andrew Lamb
We are working on a blog post [1] that highlights the advances made in DataFusion over the last 5 months (a lot!). Any contributions and/or review from the community would be greatly appreciated. Thanks, Andrew [1] https://github.com/apache/arrow-site/pull/254

Re: [DISCUSS] Move issue tracking to

2022-10-23 Thread Andrew Lamb
It is my opinion that github issues have served us very well in the Arrow Rust implementation. I haven't heard anyone complain about github or suggest we should look for alternatives. One of the major benefits is that everyone who contributes to the project already has to have a github account, s

Re: [VOTE][RUST][Ballista] Release Apache Arrow Ballista 0.9.0 RC2

2022-10-24 Thread Andrew Lamb
+1 The verification process was very heavy weight (running TPCH, etc) and I didn't let it complete. -- I wonder if we really need to run the full test suites as part of doing so Finished release [optimized] target(s) in 27m 55s The changelog shows a very impressive list of features. Well don

[Rust] Sync up call

2022-10-25 Thread Andrew Lamb
It has been a while and I haven't been on the Rust syncup call for a while due to conflicting commitments. Thus I plan to host such a call (and will record it) this week. More details and suggest topics can be found in the agenda document [1]. All items of substance will be written and discussed

Re: [RUST][DataFusion] Project Update Blog

2022-10-25 Thread Andrew Lamb
The blog post is now available publicly at https://arrow.apache.org/blog/2022/10/25/datafusion-13.0.0/ Andrew On Sat, Oct 22, 2022 at 6:33 AM Andrew Lamb wrote: > We are working on a blog post [1] that highlights the advances made in > DataFusion over the last 5 months (a lot!

Re: [ANNOUNCE] New Arrow committer: Ben Baumgold

2022-10-27 Thread Andrew Lamb
Congratulations! On Thu, Oct 27, 2022 at 11:21 AM Raúl Cumplido wrote: > Congratulations Ben! > > El jue, 27 oct 2022 a las 5:11, Weston Pace () > escribió: > > > Congratulations Ben! > > > > On Wed, Oct 26, 2022 at 2:05 PM David Li wrote: > > > > > > Welcome Ben! > > > > > > On Wed, Oct 26, 20

[Rust] Sync meetingThursday October 27, 2022

2022-10-27 Thread Andrew Lamb
15:00 UTC Thursday October 27, 2022 Attendees: Batuhan, Wolfram Shulte, Michael Angerman, vin jake, Andy Grove, Andrew Lamb, Mustafa, Ian Joiner, Xudong Discussion topics: [Andrew Lamb] Ticket backlog, help reviewing PRs, Governance: #3957 [Batuhan Taskaya] Short & long term plans on impro

[Rust][DataFusion] Proposed PR guidelines

2022-10-27 Thread Andrew Lamb
There are some proposed guidelines[1] for PRs trying to formalize best practice as the community grows. We would welcome feedback. Andrew [1] https://github.com/apache/arrow-datafusion/pull/3985

[VOTE][RUST] Release Apache Arrow Rust 26.0.0 RC1

2022-10-28 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 26.0.0. This release candidate is based on commit: 779804317d9c9d80e72a955deb8594eb45a8308a [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please download, ve

Re: [ANNOUNCE] New Arrow committer: Eric Patrick Hanson

2022-10-28 Thread Andrew Lamb
Congratulations! On Thu, Oct 27, 2022 at 4:20 PM Joris Van den Bossche < jorisvandenboss...@gmail.com> wrote: > Congratulations, and welcome Eric! > > On Thu, 27 Oct 2022 at 13:53, Eric Hanson wrote: > > > > thanks, I'm excited to join! > > > > On 2022/10/26 21:38:53 Sutou Kouhei wrote: > > > On

Re: [ANNOUNCE] New Arrow committer: Will Jones

2022-10-28 Thread Andrew Lamb
Congratulations Will On Fri, Oct 28, 2022 at 10:21 AM Nic wrote: > Fantastic, congratulations Will! > > On Fri, 28 Oct 2022 at 10:52, Raúl Cumplido > wrote: > > > Congratulations Will! > > > > El vie, 28 oct 2022 a las 11:46, Antoine Pitrou () > > escribió: > > > > > > > > Welcome Will, and tha

Re: [ANNOUNCE] New Arrow PMC member: Jacob Quinn

2022-10-30 Thread Andrew Lamb
Congratulations Jacob and thank you for all the work on the Julia implementation On Fri, Oct 28, 2022 at 6:14 AM Patrick Horan wrote: > Congrats Jacob! > > On Tue, Oct 25, 2022, at 5:06 PM, Sutou Kouhei wrote: > > The Project Management Committee (PMC) for Apache Arrow has invited > > Jacob Quin

Re: [VOTE][RUST] Release Apache Arrow Rust 26.0.0 RC1

2022-10-31 Thread Andrew Lamb
rified on Ubuntu 22.04/AMD/System76 > > On Fri, Oct 28, 2022 at 4:14 PM Andrew Lamb wrote: > > > Hi, > > > > I would like to propose a release of Apache Arrow Rust Implementation, > > version 26.0.0. > > > > This release candidate is based on c

[VOTE][RUST] Release Apache Arrow Rust 26.0.0 RC2

2022-10-31 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 26.0.0. This release candidate is based on commit: 36b8dfcf28382cb3344dd1a1979372e7c8ee582a [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Note this is the SE

Re: [VOTE][RUST] Release Apache Arrow Rust 26.0.0 RC1

2022-10-31 Thread Andrew Lamb
Thank you all again for your help. We have resolved the issue and created an second release candidate [1] -- please help review that RC if you have a moment [1]: https://lists.apache.org/thread/pj9o553v6vdmnqxjv0ogo63pz2hrng8k On Mon, Oct 31, 2022 at 6:55 AM Andrew Lamb wrote: > Tha

Re: [ANNOUNCE] New Arrow PMC member: Nicola Crane

2022-10-31 Thread Andrew Lamb
Congratulations Nic! On Thu, Oct 27, 2022 at 5:10 PM Wes McKinney wrote: > Congratulations! > > On Wed, Oct 26, 2022 at 4:56 PM Ian Joiner wrote: > > > > Congrats Nic! > > > > Ian > > > > On Tuesday, October 25, 2022, Sutou Kouhei wrote: > > > > > The Project Management Committee (PMC) for Apa

Re: [ANNOUNCE] New Arrow committer: Bogumił Kamiński

2022-10-31 Thread Andrew Lamb
Congratulations On Thu, Oct 27, 2022 at 4:20 PM Joris Van den Bossche < jorisvandenboss...@gmail.com> wrote: > Congrats! > > On Wed, 26 Oct 2022 at 23:56, Ian Joiner wrote: > > > > Congrats Bogumił! > > > > Ian > > > > On Tuesday, October 25, 2022, Sutou Kouhei wrote: > > > > > Hi, > > > > > >

  1   2   3   4   5   6   7   8   9   10   >