Re: [ANNOUNCE] New Arrow PMC member: Curt Hagenlocher

Congrats! On Thu, Oct 31, 2024 at 10:15 AM Dewey Dunnington wrote: > Congrats! > > On Wed, Oct 30, 2024 at 8:38 PM Bryce Mecum wrote: > > > > Congrats Curt! > > > > On Wed, Oct 30, 2024 at 2:56 PM Sutou Kouhei wrote: > > > > > > The Project Management Committee (PMC) for Apache Arrow has > > >

Re: [RUST][LAZY CONSENSUS] Change default branch in the arrow-rs repo to `main`

+1 non binding. On Thu, Oct 10, 2024 at 5:17 AM Andrew Lamb wrote: > Hi, > > As we have discussed[1] we propose to change the default branch from > `master` to `main` in the https://github.com/apache/arrow-rs repository, > for consistency with other projects. > > There is also a related INFRA ti

[DataFusion] Question about async/await?

cond question is did you test against thread pool execution mode? It would be highly appreciated if you can answer my question. -- Renjie Liu Software Engineer, MVAD

[jira] [Created] (ARROW-7348) [Rust] Add api to return references of buffer of null bitmap.

Renjie Liu created ARROW-7348: - Summary: [Rust] Add api to return references of buffer of null bitmap. Key: ARROW-7348 URL: https://issues.apache.org/jira/browse/ARROW-7348 Project: Apache Arrow

[jira] [Created] (ARROW-7312) [Rust] ArrowError should implement std::error:Error

Renjie Liu created ARROW-7312: - Summary: [Rust] ArrowError should implement std::error:Error Key: ARROW-7312 URL: https://issues.apache.org/jira/browse/ARROW-7312 Project: Apache Arrow Issue

[jira] [Created] (ARROW-7113) [Rust] Buffer should accept memory owned by others

Renjie Liu created ARROW-7113: - Summary: [Rust] Buffer should accept memory owned by others Key: ARROW-7113 URL: https://issues.apache.org/jira/browse/ARROW-7113 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-6948) [Rust] [Parquet] Fix bool array support in arrow reader.

Renjie Liu created ARROW-6948: - Summary: [Rust] [Parquet] Fix bool array support in arrow reader. Key: ARROW-6948 URL: https://issues.apache.org/jira/browse/ARROW-6948 Project: Apache Arrow

Re: [DISCUSS] Understanding Arrow's CI problems and needs

Do we have ticket to track this? ?? Outlook for Android From: Andy Grove Sent: Saturday, October 12, 2019 11:46:18 PM To: dev Subject: Re: [DISCUSS] Understanding Arrow's CI problems and needs I've started a new section to discuss proposa

Re: [DISCUSS] Proposal about integration test of arrow parquet reader

catch regressions even when readers/writers can > read > > > their own data but they have either incorrect or undefined behavior in > > > regards to the specification (for example I would imagine checking in a > > > file as part of the fix for ARROW-6844 > > &

Re: [DISCUSS] Proposal about integration test of arrow parquet reader

Thanks wes. Sure I'll fix it. Wes McKinney 于 2019年10月11日周五 上午6:10写道： > I just merged the PR https://github.com/apache/arrow-testing/pull/11 > > Various aspects of this make me uncomfortable so I hope they can be > addressed in follow up work > > On Thu, Oct 10, 201

Re: [DISCUSS] Proposal about integration test of arrow parquet reader

I've create ticket to track here: https://issues.apache.org/jira/browse/ARROW-6845 For this moment, can we check in those pregenerated data to unblock rust version's arrow reader? On Thu, Oct 10, 2019 at 1:20 PM Renjie Liu wrote: > It would be fine in that case. > > Wes McKi

[jira] [Created] (ARROW-6845) Setup process to generate random data for integration tests

Renjie Liu created ARROW-6845: - Summary: Setup process to generate random data for integration tests Key: ARROW-6845 URL: https://issues.apache.org/jira/browse/ARROW-6845 Project: Apache Arrow

Re: [DISCUSS] Proposal about integration test of arrow parquet reader

It would be fine in that case. Wes McKinney 于 2019年10月10日周四 下午12:58写道： > On Wed, Oct 9, 2019 at 10:16 PM Renjie Liu > wrote: > > > > 1. There already exists a low level parquet writer which can produce > > parquet file, so unit test should be fine. But writer

Re: [DISCUSS] Proposal about integration test of arrow parquet reader

can also participate (since we ought to be > doing integration tests with Java, and we can also read JSON files to > Arrow). > > On Tue, Oct 8, 2019 at 11:54 PM Renjie Liu > wrote: > > > > On Wed, Oct 9, 2019 at 12:11 PM Andy Grove > wrote: > > > > > I

Re: [DISCUSS] Proposal about integration test of arrow parquet reader

outlining where we call Java to generate some files and then call Rust to > parse them? > > Thanks, > > Andy. > > > > > > > > On Tue, Oct 8, 2019 at 9:48 PM Renjie Liu wrote: > > > Hi: > > > > I'm developing rust version of reader wh

[DISCUSS] Proposal about integration test of arrow parquet reader

Hi: I'm developing rust version of reader which reads parquet into arrow array. To verify the correct of this reader, I use the following approach: 1. Define schema with protobuf. 2. Generate json data of this schema using other language with more sophisticated implementation (e.g. java

Re: Parquet to Arrow in Java

Hi: I'm working on the rust part and expecting to finish this recently. I'm also interested in the java version because we are trying to embed arrow in spark to implement vectorized processing. Maybe we can work together. Micah Kornfield 于 2019年8月5日周一 下午1:50写道： > Hi Anoop, > I think a contribut

[jira] [Created] (ARROW-6069) [Rust] [Parquet] Implement Converter to convert record reader to arrow primitive array.

Renjie Liu created ARROW-6069: - Summary: [Rust] [Parquet] Implement Converter to convert record reader to arrow primitive array. Key: ARROW-6069 URL: https://issues.apache.org/jira/browse/ARROW-6069

[jira] [Created] (ARROW-5901) [Rust] Implement PartialEq to compare array and json values

Renjie Liu created ARROW-5901: - Summary: [Rust] Implement PartialEq to compare array and json values Key: ARROW-5901 URL: https://issues.apache.org/jira/browse/ARROW-5901 Project: Apache Arrow

[jira] [Created] (ARROW-5823) [Rust] Fix build break.

Renjie Liu created ARROW-5823: - Summary: [Rust] Fix build break. Key: ARROW-5823 URL: https://issues.apache.org/jira/browse/ARROW-5823 Project: Apache Arrow Issue Type: Bug Reporter

[jira] [Created] (ARROW-5792) [Rust] [Parquet] A visitor trait for parquet types.

Renjie Liu created ARROW-5792: - Summary: [Rust] [Parquet] A visitor trait for parquet types. Key: ARROW-5792 URL: https://issues.apache.org/jira/browse/ARROW-5792 Project: Apache Arrow Issue

[jira] [Created] (ARROW-5755) [Rust] [Parquet] Add derived clone for Type

Renjie Liu created ARROW-5755: - Summary: [Rust] [Parquet] Add derived clone for Type Key: ARROW-5755 URL: https://issues.apache.org/jira/browse/ARROW-5755 Project: Apache Arrow Issue Type: New

[jira] [Created] (ARROW-5463) [Rust] Implement AsRef for Buffer

Renjie Liu created ARROW-5463: - Summary: [Rust] Implement AsRef for Buffer Key: ARROW-5463 URL: https://issues.apache.org/jira/browse/ARROW-5463 Project: Apache Arrow Issue Type: New Feature

[jira] [Created] (ARROW-5316) [Rust] Interfaces for gandiva bindings.

Renjie Liu created ARROW-5316: - Summary: [Rust] Interfaces for gandiva bindings. Key: ARROW-5316 URL: https://issues.apache.org/jira/browse/ARROW-5316 Project: Apache Arrow Issue Type: Sub-task

[jira] [Created] (ARROW-5315) [Rust] Gandiva binding.

Renjie Liu created ARROW-5315: - Summary: [Rust] Gandiva binding. Key: ARROW-5315 URL: https://issues.apache.org/jira/browse/ARROW-5315 Project: Apache Arrow Issue Type: New Feature

Re: Rust bindings for Gandiva

is a > separate project within the repository, or a feature that can be controlled > by a feature flag in the Cargo.toml somehow. > > Thanks, > > Andy. > > > > > On Sat, May 11, 2019 at 3:10 AM Renjie Liu > wrote: > >> >> Hi: >> @Andy Grove

Re: [ANNOUNCE] New Arrow committer: Neville Dipale

Congrats! Chao Sun 于 2019年5月12日周日 上午12:38写道： > Congrats Neville! > > On Sat, May 11, 2019 at 9:36 AM Micah Kornfield > wrote: > > > Congrats!! > > > > On Saturday, May 11, 2019, paddy horan wrote: > > > > > Congrats Neville! Thank you for your contributions! > > > > > > Get Outlook for iOS

Re: Rust bindings for Gandiva

> > wrote: > > > > > > > > > Now that the Rust implementation of Arrow is maturing, I'm > interested > > > in > > > > > having bindings for Gandiva for query execution, rather than > > > duplicating > > > > > this in Rust. > > > > > > > > > > I will likely start looking at this soon but wanted to see if > anyone > > > else > > > > > here is particularly interested in this area of functionality? > > > > > > > > > > Thanks, > > > > > > > > > > Andy. > > > > > > > > > > > -- Renjie Liu Software Engineer, MVAD

[jira] [Created] (ARROW-5298) [Rust] Add debug implementation for Buffer

Renjie Liu created ARROW-5298: - Summary: [Rust] Add debug implementation for Buffer Key: ARROW-5298 URL: https://issues.apache.org/jira/browse/ARROW-5298 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-5281) [Rust] [Parquet] Move DataPageBuilder to test_common

Renjie Liu created ARROW-5281: - Summary: [Rust] [Parquet] Move DataPageBuilder to test_common Key: ARROW-5281 URL: https://issues.apache.org/jira/browse/ARROW-5281 Project: Apache Arrow Issue

[jira] [Created] (ARROW-5162) [Rust] [Parquet] Rename mod reader to arrow.

Renjie Liu created ARROW-5162: - Summary: [Rust] [Parquet] Rename mod reader to arrow. Key: ARROW-5162 URL: https://issues.apache.org/jira/browse/ARROW-5162 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-5127) [Rust] [Parquet] Add page iterator

Renjie Liu created ARROW-5127: - Summary: [Rust] [Parquet] Add page iterator Key: ARROW-5127 URL: https://issues.apache.org/jira/browse/ARROW-5127 Project: Apache Arrow Issue Type: Sub-task

[jira] [Created] (ARROW-5126) [Rust] [Parquet] Convert parquet column desc to arrow data type

Renjie Liu created ARROW-5126: - Summary: [Rust] [Parquet] Convert parquet column desc to arrow data type Key: ARROW-5126 URL: https://issues.apache.org/jira/browse/ARROW-5126 Project: Apache Arrow

Re: [ANNOUNCE] New Arrow committer: Paddy Horan

Congrats! Micah Kornfield 于 2019年3月1日周五 上午7:26写道： > Congrats! > > On Thu, Feb 28, 2019 at 3:14 PM Bryan Cutler wrote: > > > Congratulations Paddy! > > > > On Thu, Feb 28, 2019 at 7:14 AM Wes McKinney > wrote: > > > > > Welcome Paddy and thank you! > > > > > > > > > On Thu, Feb 28, 2019 at 4:29

Re: [ANNOUNCE] New Arrow committer: Chao Sun

Congrats! Micah Kornfield 于 2019年3月1日周五 上午7:26写道： > Congrats! > > On Thu, Feb 28, 2019 at 3:02 PM Bryan Cutler wrote: > > > Congratulations Chao! > > > > On Thu, Feb 28, 2019 at 9:27 AM Neville Dipale > > wrote: > > > > > Congratulations Chao and Paddy! I'm loving the increase in velocity on >

Re: [jira] [Created] (ARROW-4678) [Rust] Minimize unstable feature usage

being worked on - maybe ~1 year timeframe? > > The first set of features are easy enough to get rid of - I'll make a PR > to do that (https://github.com/sfackler/arrow/tree/more-stable). I'm a > bit less sure of what to do with specialization and packed_simd, though. > > > > -- > This message was sent by Atlassian JIRA > (v7.6.3#76005) > -- Renjie Liu Software Engineer, MVAD

[jira] [Created] (ARROW-4634) [Rust] [Parquet] Reorganize test_common mod to allow more test util codes.

Renjie Liu created ARROW-4634: - Summary: [Rust] [Parquet] Reorganize test_common mod to allow more test util codes. Key: ARROW-4634 URL: https://issues.apache.org/jira/browse/ARROW-4634 Project: Apache

Re: [Rust] Rust 0.13.0 release

0.12 was ~January 20 I think it would be good to release again > by the end of March > > On Wed, Feb 13, 2019 at 7:29 AM Renjie Liu > wrote: > > > > Hi, Andy: > > Thanks for bringing this thread. I'm working on the arrow reader for > > parquet and expect

Re: [Rust] Rust 0.13.0 release

Hi, Andy: Thanks for bringing this thread. I'm working on the arrow reader for parquet and expecting to make progress recently. BTW, what's the time line of 0.13.0？ Chao Sun 于 2019年2月13日周三 上午10:34写道： > I’m also interested in the Parquet/Arrow integration and may help there. > This is however a

[jira] [Created] (ARROW-4525) [Rust] [Parquet] Convert ArrowError to ParquetError

Renjie Liu created ARROW-4525: - Summary: [Rust] [Parquet] Convert ArrowError to ParquetError Key: ARROW-4525 URL: https://issues.apache.org/jira/browse/ARROW-4525 Project: Apache Arrow Issue

Re: [ANNOUNCE] New Arrow PMC member: Andy Grove

Congratilations! Bryan Cutler 于 2019年2月5日周二 下午12:08写道： > Congratulations Andy! > > On Mon, Feb 4, 2019, 3:29 PM Philipp Moritz > > Congratulations! > > > > On Mon, Feb 4, 2019 at 3:16 PM Krisztián Szűcs < > szucs.kriszt...@gmail.com> > > wrote: > > > > > Congrats Andy! :) > > > > > > On Mon, Fe

Re: [Rust] code style: restrict line width to 90 characters?

+1 for this suggestio. Chao Sun 于 2019年1月26日周六 上午2:39写道： > Hi Neville, there's no limit today: you'll need to add > > max_width = 90 > comment_width = 90 > > to rustfmt.toml to limit both source code and comment to 90 characters. > > Chao > > On Fri, Jan 25, 2019 at 10:34 AM Neville Dipale > wr

[jira] [Created] (ARROW-4365) [Rust] [Parquet] Implement RecordReader

Renjie Liu created ARROW-4365: - Summary: [Rust] [Parquet] Implement RecordReader Key: ARROW-4365 URL: https://issues.apache.org/jira/browse/ARROW-4365 Project: Apache Arrow Issue Type: Bug

Re: [VOTE] Accept donation of Rust DataFusion library for Apache Arrow

+1 (non-binding) I also tried to write a similar engine, and glad to merge with datadusion paddy horan 于 2019年1月24日周四 上午5:29写道： > +1 (non-binding) > > Thanks Andy > > Get Outlook for iOS > > > From: Chao Sun > Sent: Wednesday, January 23,

[jira] [Created] (ARROW-4219) [Rust] [Parquet] Implement ArrowReader

Renjie Liu created ARROW-4219: - Summary: [Rust] [Parquet] Implement ArrowReader Key: ARROW-4219 URL: https://issues.apache.org/jira/browse/ARROW-4219 Project: Apache Arrow Issue Type: Sub-task

[jira] [Created] (ARROW-4218) [Rust][Parquet]Implement ColumnReader

Renjie Liu created ARROW-4218: - Summary: [Rust][Parquet]Implement ColumnReader Key: ARROW-4218 URL: https://issues.apache.org/jira/browse/ARROW-4218 Project: Apache Arrow Issue Type: Sub-task

Re: [Rust] move parquet into a separate sub-crate

erence on parquet in cpp/src/arrow. > 2. The path dependency used in workspace has to be changed to a version > number when we do "cargo publish". This should be added to the release > instructions and committer who performs the job should do the extra step. > > Thoughts? > > Chao > -- Renjie Liu Software Engineer, MVAD

Re: [DISCUSS] Rust add adapter for parquet

oing > >> with C++. It's possible you might run into the same kinds of issues > >> that led us to consider the monorepo arrangement. > >> > >> Thanks > >> Wes > >> On Sun, Aug 19, 2018 at 11:11 PM Renjie Liu > >> wrote: > >&g

[jira] [Created] (ARROW-3706) [Rust] Add record batch reader trait.

Renjie Liu created ARROW-3706: - Summary: [Rust] Add record batch reader trait. Key: ARROW-3706 URL: https://issues.apache.org/jira/browse/ARROW-3706 Project: Apache Arrow Issue Type: New Feature

Re: [DISCUSS] Rust add adapter for parquet

Hi, Chao: I've opened an jira issue for that and planning to work on that. On Mon, Aug 20, 2018 at 11:03 AM Renjie Liu wrote: > Yes, it's a mistake, sorry for that > > > On Mon, Aug 20, 2018 at 10:57 AM Chao Sun wrote: > >> (s/flink/arrow - it is a mistake

Re: [DISCUSS] Rust add adapter for parquet

<https://github.com/sunchao/parquet-rs/issues/140> about this recently. > Feel free to share your comments on the github. > > Best, > Chao > > On Sun, Aug 19, 2018 at 7:39 PM, Renjie Liu > wrote: > >> cc:Sunchao and Any >> >> >> -- Forward

[jira] [Created] (ARROW-3085) [Rust] Add an adapter for parquet.

Renjie Liu created ARROW-3085: - Summary: [Rust] Add an adapter for parquet. Key: ARROW-3085 URL: https://issues.apache.org/jira/browse/ARROW-3085 Project: Apache Arrow Issue Type: New Feature

[DISCUSS] Rust add adapter for parquet

Hi, all: Now the rust component is approaching a stable state and rust reader for parquet is ready. I think it maybe a good time to start an adapter for parquet, just like adapter for orc in cpp. How you guys think about it? -- Liu, Renjie Software Engineer, MVAD

Re: [ANNOUNCE] New Arrow committers: Andy Grove and Krisztián Szűcs

Congrats Andy, Krisztian! Andy Grove 于 2018年8月16日周四 上午7:47写道： > Congrats to you too, Krisztian! > > I'm also honored to be part of this project and look forward to > contributing more in the near future. > > Andy. > > On Wed, Aug 15, 2018 at 5:29 PM Krisztián Szűcs > > wrote: > > > I feel honor

Re: Rust tasks for 0.10.0?

+1 for pushing to crates.io. On Wed, Jul 25, 2018 at 10:52 AM Andy Grove wrote: > Hi, > > I'm wondering what we should do with the Rust implementation for the 0.10.0 > release. I would like to have an official release pushed to crates.io for > sure. > > Since the Rust implementation is so new th

[jira] [Created] (ARROW-2852) [Rust] Mark Array as Sync and Send

Renjie Liu created ARROW-2852: - Summary: [Rust] Mark Array as Sync and Send Key: ARROW-2852 URL: https://issues.apache.org/jira/browse/ARROW-2852 Project: Apache Arrow Issue Type: Bug

Re: rust using nightly channel

y so we > might not have to wait too long. > > Thanks, > > Andy. > > > > On Tue, Apr 10, 2018 at 4:38 AM, Renjie Liu > wrote: > > > Hi: > > Can we use experimental features in nightly channel? There are many > useful > > features that can only be u

rust using nightly channel

Hi: Can we use experimental features in nightly channel? There are many useful features that can only be use in nightly channel, e.g. the Alloc api, since arrow requires control over low level primitives such as memory allocation, simd execution, etc. -- Liu, Renjie Software Engineer, MVAD

Re: Rust Arrow status and plans for this week

Hello Uwe: My JIRA id is liurenjie1024 and it seems that I have been given contibutor permission. On Tue, Apr 10, 2018 at 3:00 PM Uwe L. Korn wrote: > Hello Andy, > > this is very exciting. Once we have basic documentation, we should have a > look at streamlining the release process in the ASF i

Re: Rust Arrow status and plans for this week

Cool! I'm also trying to use arrow-rs in my project and would like to contribute to arrow-rs, can anybody give me contributor permission? On Tue, Apr 10, 2018 at 10:31 AM Jacques Nadeau wrote: > Super cool, congrats on the progress! > > The IPC/interop is top priority for me as well. > > On Mon,

[jira] [Created] (ARROW-2435) [Rust] Add memory pool abstraction.

Renjie Liu created ARROW-2435: - Summary: [Rust] Add memory pool abstraction. Key: ARROW-2435 URL: https://issues.apache.org/jira/browse/ARROW-2435 Project: Apache Arrow Issue Type: Improvement