Re: Existence/name/scope for minimal C/C++ Arrow C Data interface helpers

2022-06-03 Thread Dewey Dunnington
verlap with these other efforts? Are > > > those headers something we could|should "just" PR into apache/arrow > > > and write up how to use them? If not what is the work to make them so > > > that they could be (the answer of course could be design something

Re: [VOTE] Mark C Stream Interface as Stable

2022-06-08 Thread Dewey Dunnington
+1 (non-binding) On Wed, Jun 8, 2022 at 2:29 PM Antoine Pitrou wrote: > > +1 (binding) > > > Le 08/06/2022 à 20:15, Will Jones a écrit : > > Hi, > > > > Given all feedback to discussion [1] has been positive, I would like to > > propose marking the C Stream Interface as stable. > > > > I have pr

Re: Existence/name/scope for minimal C/C++ Arrow C Data interface helpers

2022-06-10 Thread Dewey Dunnington
/paleolimbot/arrow-c/pull/1/files On Fri, Jun 3, 2022 at 12:41 PM Dewey Dunnington wrote: > Hi all, > > Based on the points raised above and a few adventures implementing some of > this in related projects, I put together a brief design document proposing > a scope and structure to per

Re: Existence/name/scope for minimal C/C++ Arrow C Data interface helpers

2022-06-14 Thread Dewey Dunnington
/paleolimbot/arrow-c/pull/5 On Fri, Jun 10, 2022 at 12:27 PM Dewey Dunnington wrote: > Hi all, > > As promised, I converted the design document [1] into an initial PR [2]. > Rather than draft the whole header, I started with README + implementations > + testing for error hand

Re: Existence/name/scope for minimal C/C++ Arrow C Data interface helpers

2022-06-16 Thread Dewey Dunnington
onvey the > message that there is a parallel C API for Arrow. > > > Le 15/06/2022 à 05:18, Dewey Dunnington a écrit : > > Hi all, > > > > I drafted a second PR [1] drafting a design for storing parsed > information > > obtained from a struct ArrowSchema (i.

Re: [ANNOUNCE] New Arrow committers: Dewey Dunnington, Alenka Frim, and Rok Mihevc

2022-06-25 Thread Dewey Dunnington
o our new committers! > > > > > >> > > > > > >> > > > > > >> Le 22/06/2022 à 20:02, Andrew Lamb a écrit : > > > > > >> > Congratulations! > > > > > >> > > > > > > >> > On Wed, Jun 22, 2022 at 1:27 P

Re: Existence/name/scope for minimal C/C++ Arrow C Data interface helpers

2022-06-25 Thread Dewey Dunnington
b.com/paleolimbot/nanoarrow/pull/9 [2] https://github.com/paleolimbot/nanoarrow/pull/10 On Wed, Jun 15, 2022 at 12:18 AM Dewey Dunnington wrote: > Hi all, > > I drafted a second PR [1] drafting a design for storing parsed information > obtained from a struct ArrowSchema (i.e., parsing the f

Re: Existence/name/scope for minimal C/C++ Arrow C Data interface helpers

2022-07-06 Thread Dewey Dunnington
row-c (or some other > arrow-$SOMETHING) > > Otherwise we could be looking at having to do an IP clearance / > software grant at a later time. > > Thanks, > Wes > > On Sat, Jun 25, 2022 at 8:52 PM Dewey Dunnington > wrote: > > > > Hi all, > > > > Than

Re: Usage of the name Feather?

2022-08-30 Thread Dewey Dunnington
Ian has a very good point...I would be in favour of calling them "Arrow files" wherever possible since there's no need to know or care what interprocess communication is to use them! On Mon, Aug 29, 2022 at 6:50 PM Ian Cook wrote: > +1 We should explicitly discourage further use of “Feather” to

Re: [C++] Read Flight data source into Acero

2022-09-07 Thread Dewey Dunnington
We do this [1] in the R package to use RecordBatchReaders as input (it powers the DuckDB integration and a few other things we need to evaluate R things that don't have a corresponding Acero node). [1] https://github.com/apache/arrow/blob/master/r/src/compute-exec.cpp#L396-L406 to On Wed, Sep 7,

Re: PRs for RLE support

2022-09-14 Thread Dewey Dunnington
> * Should we encode "run lengths" or "run ends"? In addition to the points mentioned above, this seems the most consistent with the variable-length binary/list layouts > encoding the run ends as a buffer (similar to list array for example) makes it difficult to calculate offsets I don't have a

Re: PRs for RLE support

2022-09-14 Thread Dewey Dunnington
tion doing excessive slicing could cache the offsets > themselves. > * Add an optional buffer offset to the spec that can be populated in > cases where random array access is not possible. > > On Wed, Sep 14, 2022 at 10:53 AM Dewey Dunnington > wrote: > > > > > * Shoul

Re: [VOTE] Adopt ADBC database client connectivity specification

2022-09-21 Thread Dewey Dunnington
+1! (non-binding) On Wed, Sep 21, 2022 at 12:47 PM Gavin Ray wrote: > +1 (non-binding/I'm not important) > > On Wed, Sep 21, 2022 at 11:40 AM David Li wrote: > > > Hello, > > > > We have been discussing [1] standard interfaces for Arrow-based database > > access and have been working on impleme

Re: [ANNOUNCE] New Arrow PMC member: Nicola Crane

2022-10-26 Thread Dewey Dunnington
Congrats, Nic! On Wed, Oct 26, 2022 at 10:10 AM Jacob Wujciak wrote: > Congratulations! 🎉 > > On Wed, Oct 26, 2022 at 3:04 PM Larry White wrote: > > > Congratulations, Nic! > > > > On Wed, Oct 26, 2022 at 2:31 AM Alenka Frim > .invalid> > > wrote: > > > > > 🎉 Congratulations Nic! Well deserved

Re: [VOTE] Move issue tracking to GitHub Issues

2022-10-27 Thread Dewey Dunnington
+1 (non-binding)! On Thu, Oct 27, 2022 at 8:54 AM Eric Hanson wrote: > +1 > > On 2022/10/26 23:02:33 Neal Richardson wrote: > > I propose that we move issue tracking from the ASF's Jira to GitHub > Issues. > > This has been discussed on [1] and [2] and there seems to be consensus. A > > number o

Re: [VOTE] Disable ASF Jira issue reporting

2022-11-24 Thread Dewey Dunnington
I would be a strong +1 for using GitHub for all new top-level issues. We already have a title formatting requirement on JIRAs (e.g., [R] Some issue description) and so I don't see any reason why labels could not be added in retrospect even if they don't exist yet. On Thu, Nov 24, 2022 at 7:30 AM J

Re: [ANNOUNCE] New Arrow committer: Raúl Cumplido

2022-12-06 Thread Dewey Dunnington
Congrats! Welcome! On Tue, Dec 6, 2022 at 10:35 AM Larry White wrote: > Congrats, Raúl! > > On Tue, Dec 6, 2022 at 9:20 AM David Li wrote: > > > Welcome Raúl! > > > > On Tue, Dec 6, 2022, at 08:41, Neal Richardson wrote: > > > Congratulations! > > > > > > > > > > > >> On Dec 6, 2022, at 6:11 AM

Re: [ANNOUNCE] New Arrow committer: Jacob Wujciak

2022-12-15 Thread Dewey Dunnington
Congrats, Jacob! On Thu, Dec 15, 2022 at 9:26 PM Matt Topol wrote: > Congrats Jacob!! > > On Thu, Dec 15, 2022, 7:53 PM Neal Richardson > > wrote: > > > Congrats! > > > > On Thu, Dec 15, 2022 at 7:00 PM Ian Cook wrote: > > > > > Herzlichen Glückwunsch, Jacob! > > > > > > On Thu, Dec 15, 2022 a

Pre-release feedback for 'nanoarrow'

2023-01-02 Thread Dewey Dunnington
Hi all, Following a discussion on the arrow-dev list a few months ago [1], I spent some time over the past few months building the 'nanoarrow' library [2] with considerable contributions from David Li and others in the Arrow developer community. The focus of the library is producing and consuming

Re: [DISCUSS] State of the Arrow Project 2022

2023-01-03 Thread Dewey Dunnington
First, a +1000 on Will's blog post! [1] Continue: Building tools that benefit users of all languages, with particular kudos to ADBC for providing an ABI-stable way to write database drivers that can be used by practitioners in C++, Ruby, Python, Java, Go, and (soon!) R. Start: I wonder if this

Re: [VOTE] Release Apache Arrow 11.0.0 - RC0

2023-01-22 Thread Dewey Dunnington
Just a note that I wasn't able to produce an error building Arrow C++ using clang-dev [1]. That isn't to say one doesn't still exist (I will test more thoroughly in the coming days), but it does suggest that it's something we can/should handle at the packaging stage if it does pop up (rather than b

Re: R arrow package question

2023-02-01 Thread Dewey Dunnington
In case it hasn't already been mentioned here, I wonder if manually setting `schema()` would help. You're correct that the invalid value isn't scientific notation (i.e., it's a blank string) so maybe that column should be a string column instead. You could get the guessed schema from the original o

Re: [VOTE] Release Apache Arrow ADBC 0.2.0 - RC1

2023-02-14 Thread Dewey Dunnington
I ran: ``` export DOCKER_DEFAULT_ARCHITECTURE=linux/amd64 export PKG_CONFIG_PATH=/opt/homebrew/Cellar/libpq/15.2/lib/pkgconfig/:/opt/homebrew/Cellar/openssl@3 /3.0.8/lib/pkgconfig/ export CGO_CXXFLAGS="-std=c++11" TEST_GLIB=0 ./dev/release/verify-release-candidate.sh 0.2.0 1 ``` I had to manually

Re: [VOTE] Format: Fixed shape tensor Canonical Extension Type

2023-02-21 Thread Dewey Dunnington
+1! I put together a quick R implementation as well to see how the permutation field fits with our native column-major storage [1]. It worked great! Thank you for all of your work assembling all of our collective opinions on this :-) [1] https://gist.github.com/paleolimbot/c42f068c2b8b98255dbfbe37

Re: [VOTE] Format: Fixed shape tensor Canonical Extension Type

2023-02-22 Thread Dewey Dunnington
I don't think having both dimension names and permutation is redundant...dimension names can also serve as human-readable tags that help a human interpret the values. If reading a NetCDF, for example, one might store the dimension variable names. When determining type equality it may be useful that

[VOTE] Release Apache Arrow nanoarrow 0.1.0 - RC1

2023-03-01 Thread Dewey Dunnington
Hello, I would like to propose the following release candidate (RC1) of Apache Arrow nanoarrow [0] version 0.1.0. This is an initial release consisting of 31 resolved GitHub issues [1]. Special thanks to David Li for his reviews and support during the preparation of this initial release candidate

Re: [VOTE] Release Apache Arrow nanoarrow 0.1.0 - RC1

2023-03-01 Thread Dewey Dunnington
hat-rout-fail > > I'm not sure whether they are caused by my wrong setup or > not. Could someone check the failures? > > Thanks, > -- > kou > > In > "[VOTE] Release Apache Arrow nanoarrow 0.1.0 - RC1" on Wed, 1 Mar 2023 > 13:03:44 -0400, >

Re: [VOTE][Format] Fixed shape tensor Canonical Extension Type

2023-03-06 Thread Dewey Dunnington
+1 (non-binding)! On Mon, Mar 6, 2023 at 9:59 AM Nic Crane wrote: > +1 > > On Mon, 6 Mar 2023 at 12:41, Alenka Frim > wrote: > > > Hi all, > > > > I am starting a new voting thread with this email as the first voting > > thread [1] opened up new > > comments and suggestions and we wanted to tak

Re: [VOTE] Release Apache Arrow nanoarrow 0.1.0 - RC1

2023-03-07 Thread Dewey Dunnington
ow' R > > package (8.0.0). I upgraded it to 11.0.0 and it > > worked. Thanks to Dewey for helping me! > > > > Thanks, > > -- > > kou > > > > In > > "Re: [VOTE] Release Apache Arrow nanoarrow 0.1.0 - RC1" on Wed, 1 Mar > >

Re: [RESULT][VOTE] Release Apache Arrow nanoarrow 0.1.0 - RC1

2023-03-09 Thread Dewey Dunnington
ent example: > [RESULT][VOTE][RUST] Release Apache Arrow Rust Object Store 0.5.5 RC1 > https://lists.apache.org/thread/ybqvx5k2lnyotnh6yq5xzzp80x09fl1c > > > Thanks, > -- > kou > > In > "Re: [VOTE] Release Apache Arrow nanoarrow 0.1.0 - RC1" on Tue, 7 Mar

Re: [ANNOUNCE] New Arrow PMC member: Will Jones

2023-03-13 Thread Dewey Dunnington
Congrats, Will! On Mon, Mar 13, 2023 at 3:07 PM Matt Topol wrote: > > Congrats Will! > > On Mon, Mar 13, 2023, 2:02 PM Jacob Wujciak > wrote: > > > Congratulations Will, well deserved! > > > > On Mon, Mar 13, 2023 at 6:58 PM Andrew Lamb wrote: > > > > > The Project Management Committee (PMC) fo

Re: [VOTE] Release Apache Arrow ADBC 0.3.0 - RC1

2023-03-20 Thread Dewey Dunnington
+1 (non-binding)! I ran `USE_CONDA=1 TEST_APT=0 TEST_YUM=0 dev/release/verify-release-candidate.sh 0.3.0 1` successfully on MacOS Monterey (M1) and MacOS Ventura (M1). Additionally, I verified R with and without conda (`TEST_DEFAULT=0 TEST_R=1 dev/release/verify-release-candidate.sh 0.3.0 1` and

Re: [DISCUSSION] C-Data API for Non-CPU Use Cases

2023-04-10 Thread Dewey Dunnington
I left some comments on the PR as well...I think this is an important addition and I'm excited to see this discussion! If there is further information that needs to be passed along in the future, schema metadata could be used. Even with schema metadata, the device type and ID will always need to b

Re: [DISCUSS] Migrate s390x from Travis to ASF Jenkins

2023-04-20 Thread Dewey Dunnington
I wonder if the main use of the s390x job is to ensure that Arrow works on big endian? If there are any claims about working with/on big endian in the code base (e.g., the existence of a ARROW_IS_LITTLE_ENDIAN macro) I think it's essential that it is tested somewhere. Deciding to abandon big endian

Re: [ANNOUNCE] New Arrow PMC member: Matt Topol

2023-05-04 Thread Dewey Dunnington
Congrats! On Thu, May 4, 2023 at 6:31 AM Alenka Frim wrote: > > Congratulations Matt!! > > On Thu, May 4, 2023 at 9:22 AM Joris Van den Bossche < > jorisvandenboss...@gmail.com> wrote: > > > Congrats Matt! > > > > On Thu, 4 May 2023 at 06:31, Nic Crane wrote: > > > > > > Congratulations! > > > >

Re: [VOTE] Release Apache Arrow ADBC 0.4.0 - RC0

2023-05-12 Thread Dewey Dunnington
+1! On Windows/Git Bash I ran: PATH="$PATH:/c/PROGRA~1/R/R-43~1.0/bin" TEST_DEFAULT=0 TEST_R=1 dev/release/verify-release-candidate.sh 0.4.0 0 On Ubuntu 22.04 I ran: TEST_APT=0 TEST_YUM=0 USE_CONDA=1 dev/release/verify-release-candidate.sh 0.4.0 0 I wasn't able to verify on MacOS/aarch64 because

Re: [DISCUSS][Format] Draft implementation of string view array format

2023-05-16 Thread Dewey Dunnington
Very cool! In addition to performance mentioned above, I could see this being useful for the R bindings - we already have a global string pool and a mechanism for keeping a vector of them alive. I don't see the C Data interface in the PR although I may have missed it - is that a part of the propo

Re: [VOTE][Format] Add experimental ArrowDeviceArray to C-Data API

2023-05-29 Thread Dewey Dunnington
+1 (non-binding)! Reading the discussion on that PR is illuminating as to how difficult this can be...thank you! On Fri, May 26, 2023 at 3:54 PM Benjamin Kietzman wrote: > > +1, thanks for all your work on this! > > On Fri, May 26, 2023 at 11:09 AM Matt Topol wrote: > > > That makes 1 binding an

Re: [VOTE][Format] Add experimental ArrowDeviceArray to C-Data API

2023-06-02 Thread Dewey Dunnington
I've already given my vote here, but wanted to share a proof-of-concept C implementation (== copy an arbitrary valid ArrowArray to given a suitable device implementation) of the proposed spec that includes Apple Metal [1] and could include CUDA as well (I did Metal first since Matt already worked u

Re: [VOTE] Release Apache Arrow 12.0.1 - RC1

2023-06-12 Thread Dewey Dunnington
+1! I ran TEST_DEFAULT=0 TEST_CPP=1 ARROW_CMAKE_OPTIONS="-DProtobuf_SOURCE=BUNDLED -DARROW_FLIGHT=OFF -DARROW_FLIGHT_SQL=OFF" ./verify-release-candidate.sh ...on MacOS Ventura aarch64. (Flight disabled because of protobuf issues). On Mon, Jun 12, 2023 at 10:28 AM Joris Van den Bossche wrote: >

[VOTE] Release Apache Arrow nanoarrow 0.2.0 - RC0

2023-06-16 Thread Dewey Dunnington
Hello, I would like to propose the following release candidate (RC0) of Apache Arrow nanoarrow version 0.2.0. This release consists of 17 resolved GitHub issues [1]. This release candidate is based on commit: a7b824de6cb99ce458e1a5cd311d69588ceb0570 [2] The source release rc0 is hosted at [3]. T

[VOTE] Release Apache Arrow nanoarrow 0.2.0 - RC1

2023-06-19 Thread Dewey Dunnington
Hello, I would like to propose the following release candidate (RC1) of Apache Arrow nanoarrow version 0.2.0. This release consists of 17 resolved GitHub issues [1]. This release candidate is based on commit: f71063605e288d9a8dd73cfdd9578773519b6743 [2] The source release rc1 is hosted at [3]. T

Re: [VOTE] Release Apache Arrow nanoarrow 0.2.0 - RC0

2023-06-19 Thread Dewey Dunnington
main > * gcc (Debian 12.2.0-14) 12.2.0 > * R version 4.3.0 (2023-04-21) -- "Already Tomorrow" > > > Thanks, > -- > kou > > In > "[VOTE] Release Apache Arrow nanoarrow 0.2.0 - RC0" on Fri, 16 Jun 2023 > 17:15:41 -0300, > Dewey Dunnin

Re: [VOTE] Release Apache Arrow nanoarrow 0.2.0 - RC1

2023-06-19 Thread Dewey Dunnington
My vote is +1 (non-binding). Verified on MacOS M1 (both Homebrew and Conda). On Mon, Jun 19, 2023 at 3:58 PM Dewey Dunnington wrote: > > Hello, > > I would like to propose the following release candidate (RC1) of > Apache Arrow nanoarrow version 0.2.0. This release consists o

Re: [VOTE] Release Apache Arrow ADBC 0.5.0 - RC0

2023-06-19 Thread Dewey Dunnington
+1 (non-binding) I ran the following on MacOS M1: USE_CONDA=1 TEST_APT=0 TEST_YUM=0 ./verify-release-candidate.sh 0.5.0 0 On Mon, Jun 19, 2023 at 12:12 PM Jean-Baptiste Onofré wrote: > > +1 (non binding) > > Regards > JB > > On Fri, Jun 16, 2023 at 2:06 AM David Li wrote: > > > > Hello, > > >

Re: [VOTE] Release Apache Arrow nanoarrow 0.2.0 - RC1

2023-06-20 Thread Dewey Dunnington
t; >> dev/release/verify-release-candidate.sh 0.2.0 1 > >> > >> with: > >> > >>* Apache Arrow C++ main > >>* gcc (Debian 12.2.0-14) 12.2.0 > >>* R version 4.3.0 (2023-04-21) -- "Already Tomorrow" > >> >

[RESULT][VOTE] Release Apache Arrow nanoarrow 0.2.0 - RC1

2023-06-22 Thread Dewey Dunnington
Thank you everybody for verifying and voting! With 3 binding +1s and 3 non-binding +1s, the vote passes! I have opened a PR to improve the verification instructions (particularly on conda where most problems occurred) [1]. Apache Arrow nanoarrow 0.2.0 has the following post-release tasks. I believ

Re: [RESULT][VOTE] Release Apache Arrow nanoarrow 0.2.0 - RC1

2023-06-23 Thread Dewey Dunnington
> https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/post-01-upload.sh > ? > > > Thanks, > -- > kou > > In > "[RESULT][VOTE] Release Apache Arrow nanoarrow 0.2.0 - RC1" on Thu, 22 Jun > 2023 16:05:50 -0300, > Dewey Dunnington wrote:

[ANNOUNCE] Apache Arrow nanoarrow 0.2.0 Released

2023-06-23 Thread Dewey Dunnington
The Apache Arrow community is pleased to announce the 0.2.0 release of Apache Arrow nanoarrow. This initial release covers 19 resolved issues from 6 contributors[1]. The release is available now from [2]. Release notes are available at: https://github.com/apache/arrow-nanoarrow/blob/apache-arrow-

Re: [ANNOUNCE] New Arrow PMC member: Dewey Dunnington

2023-06-23 Thread Dewey Dunnington
>>>>> Well-deserved Dewey, congratulations! > >> >>>>>> > >> >>>>>> On Fri, 23 Jun 2023 at 11:53, Vibhatha Abeykoon >> > > >> >>>>>> wrote: > >> >>>>>> > >> >

Re: [RESULT][VOTE] Release Apache Arrow nanoarrow 0.2.0 - RC1

2023-06-23 Thread Dewey Dunnington
] Removed old artifacts from SVN [x] Bumped versions on main [1] https://arrow.apache.org/blog/2023/06/22/nanoarrow-0.120-release/ On Fri, Jun 23, 2023 at 9:28 AM Dewey Dunnington wrote: > > Thanks for offering! Sorry for being slow to update the thread...David > Li ran the upload script

Re: [VOTE] Release Apache Arrow ADBC 0.5.1 - RC1

2023-06-23 Thread Dewey Dunnington
+1! I ran USE_CONDA=1 TEST_APT=0 TEST_YUM=0 ./verify-release-candidate.sh 0.5.1 1 on MacOS M1. On Fri, Jun 23, 2023 at 8:50 PM David Li wrote: > > My vote: +1 (Ubuntu Linux 20.04/x86_64; macOS 13.4/AArch64) > > On Fri, Jun 23, 2023, at 17:51, Matt Topol wrote: > > +1 tested on Pop!_Os 22.04 with

Re: Do we need CODEOWNERS ?

2023-07-04 Thread Dewey Dunnington
Just a note that for me, the main problem is that I get automatic review requests for PRs that have nothing to do with R (I think this happens when a rebase occurs that contained an R commit). Because that happens a lot, it means I miss actual review requests and sometimes mentions because they ble

Re: [ANNOUNCE] New Arrow committer: Kevin Gurney

2023-07-04 Thread Dewey Dunnington
Congrats! On Tue, Jul 4, 2023 at 2:08 PM Matt Topol wrote: > > Welcome! > > On Tue, Jul 4, 2023, 11:06 AM Joris Van den Bossche < > jorisvandenboss...@gmail.com> wrote: > > > Congrats Kevin! > > > > On Tue, 4 Jul 2023 at 13:47, David Li wrote: > > > > > > Welcome Kevin! > > > > > > On Tue, Jul 4

Re: [VOTE] Apache Arrow ADBC (API) 1.1.0

2023-08-15 Thread Dewey Dunnington
ill be 0.7.0 of the packages. (So I will not merge the > > linked PR until after we release ADBC 0.6.0.) > > > > This vote will be open for at least 72 hours. > > > > [ ] +1 Adopt the ADBC 1.1.0 specification > > [ ] 0 > > [ ] -1 Do not adopt the specificatio

Re: [Vote][Format] C Data Interface Format string for REE

2023-08-16 Thread Dewey Dunnington
+1! Looking forward to implementing this in nanoarrow! On Wed, Aug 16, 2023 at 11:18 AM Ian Cook wrote: > > +1 (non-binding) > > On Wed, Aug 16, 2023 at 10:16 AM Matt Topol > wrote: > > > > Hey All, > > > > As proposed by Felipe [1] I'm starting a vote on the proposed update to the > > Format Sp

Re: [Discuss] Do we need a release verification script?

2023-08-23 Thread Dewey Dunnington
I can certainly empathize with the difficulty of maintaining a release verification script and also the difficulty of remembering which combination of environment variables are needed on my system to make it work. The general sentiment of "anybody should be able to check that this release actually

Re: [VOTE] Release Apache Arrow ADBC 0.6.0 - RC0

2023-08-26 Thread Dewey Dunnington
My vote: +1 I ran USE_CONDA=1 dev/release/verify-release-candidate.sh 0.6.0 0 ...on Ubuntu 22.04. On Thu, Aug 24, 2023 at 12:13 PM David Li wrote: > > My vote: +1 > > Tested sources, binaries on macOS 13.5/AArch64 > > On Thu, Aug 24, 2023, at 07:02, Raúl Cumplido wrote: > > +1 non-binding > > >

Re: [DISCUSS] Proposal to add VariableShapeTensor Canonical Extension Type

2023-09-01 Thread Dewey Dunnington
Thank you for proposing this! I left a comment on the PR as well, but I'm excited for this to standardize a few concepts that I have run into whilst working on ADBC and GeoArrow: - Properly returning an array with >1 dimension from the PostgreSQL ADBC driver - As the basis for encoding raster tile

Re: [LAST CALL][DISCUSS] Unsigned integers in Utf8View

2023-09-19 Thread Dewey Dunnington
Hi all, Sorry for the late reply! I would lean towards signed integers because we don't use unsigned integers anywhere in the existing specification (other than as a data type). While they are allowed as dictionary index values, the spec specifically discourages their use [1]. If the times have c

Re: [VOTE] Release Apache Arrow ADBC 0.7.0 - RC0

2023-09-22 Thread Dewey Dunnington
+1! I ran: export DOCKER_DEFAULT_PLATFORM=linux/amd64 && USE_CONDA=1 dev/release/verify-release-can didate.sh 0.7.0 0 On Thu, Sep 21, 2023 at 12:52 AM Sutou Kouhei wrote: > > +1 > > I ran the following on Debian GNU/Linux sid: > > JAVA_HOME=/usr/lib/jvm/default-java \ > TEST_PYTHON=0 \ >

[VOTE] Release Apace Arrow nanoarrow 0.3.0 - RC0

2023-09-26 Thread Dewey Dunnington
Hello, I would like to propose the following release candidate (rc0) of Apache Arrow nanoarrow [0] version 0.3.0. This is an initial release consisting of 42 resolved GitHub issues from 4 contributors [1]. This release candidate is based on commit: c00cd7707bcddb4dab9a7d19bf63e87c06d36c63 [2] Th

Re: [Format] C Data Interface integration testing

2023-09-26 Thread Dewey Dunnington
Thank you for setting this up! I look forward to adding nanoarrow as soon as time allows. Cheers, -dewey On Tue, Sep 26, 2023 at 9:48 AM Antoine Pitrou wrote: > > > Hello, > > We have added some infrastructure for integration testing of the C Data > Interface between Arrow implementations. We a

Re: [VOTE][Format] Variable shape tensor canonical extension type

2023-09-29 Thread Dewey Dunnington
+1! Thank you for iterating on this with all of us! On Fri, Sep 29, 2023 at 11:28 AM Alenka Frim wrote: > > +1 > Thanks for pushing this through! > > On Wed, Sep 27, 2023 at 2:44 PM Rok Mihevc wrote: > > > Hi all, > > > > Following the discussion [1][2] I would like to propose a vote to add > >

Re: [VOTE] Release Apace Arrow nanoarrow 0.3.0 - RC0

2023-09-29 Thread Dewey Dunnington
My vote is +1 (verified on MacOS 13.6 aarch64) On Fri, Sep 29, 2023 at 10:04 AM Jean-Baptiste Onofré wrote: > > +1 (non binding) > > Tested on MacOS 13.5 (aarch64). > > Regards > JB > > On Tue, Sep 26, 2023 at 5:23 PM Dewey Dunnington > wrote: > > > >

[RESULT] Release Apace Arrow nanoarrow 0.3.0 - RC0

2023-09-29 Thread Dewey Dunnington
blog post [ ] Sent announcement to annou...@apache.org [ ] Removed old artifacts from SVN [ ] Bumped versions on main On Fri, Sep 29, 2023 at 1:14 PM Dewey Dunnington wrote: > > My vote is +1 (verified on MacOS 13.6 aarch64) > > On Fri, Sep 29, 2023 at 10:04 AM Jean-Baptiste Onof

[ANNOUNCE] Apache Arrow nanoarrow 0.3.0 Released

2023-09-29 Thread Dewey Dunnington
The Apache Arrow community is pleased to announce the 0.3.0 release of Apache Arrow nanoarrow. This release covers 42 resolved issues from 4 contributors[1]. The release is available now from [2]. Release notes are available at: https://github.com/apache/arrow-nanoarrow/blob/apache-arrow-nanoarro

Re: [RESULT] Release Apace Arrow nanoarrow 0.3.0 - RC0

2023-10-04 Thread Dewey Dunnington
] Removed old artifacts from SVN [x] Bumped versions on main On Fri, Sep 29, 2023 at 1:16 PM Dewey Dunnington wrote: > > The vote passes with 4 +1 binding and 4 +1 non-binding votes! > > I will take care of the following post-release tasks: > > [ ] Closed GitHub milestone >

Re: [Vote][Format] C data interface format string for ListView and LargeListView arrays

2023-10-05 Thread Dewey Dunnington
I'm sorry for missing earlier discussion on this or a PR into the format where this discussion may have occurred...is there a reason that +lv and +Lv were chosen over a single-character version (i.e., maybe +v and +V)? A single-character version is (slightly) easier to parse in C. On Thu, Oct 5, 2

Re: [Vote][Format] C data interface format string for ListView and LargeListView arrays

2023-10-05 Thread Dewey Dunnington
discussion. > > The union types also use two characters, so I didn’t think it would be a > problem. > > — > Felipe > > On Thu, 5 Oct 2023 at 15:26 Dewey Dunnington > wrote: > > > I'm sorry for missing earlier discussion on this or a PR into the > > f

Re: [Vote][Format] C data interface format string for ListView and LargeListView arrays

2023-10-05 Thread Dewey Dunnington
+vl and +vL sound good to me! On Thu, Oct 5, 2023 at 5:06 PM Ben Harkins wrote: > > Not sure how consequential it'd be in practice, but my first thought is > that "+vl" and "+vL" (or "+v"/"+V") would require fewer logic changes and > extra checks for parsers. Plus, establishing a v-prefixed conve

Re: [Vote][Format] C data interface format string for ListView and LargeListView arrays

2023-10-05 Thread Dewey Dunnington
t's not like you > > have to backtrack anyway. > > > > +1 from me on Felipe's proposal. > > > > Regards > > > > Antoine. > > > > > > Le 05/10/2023 à 20:33, Felipe Oliveira Carvalho a écrit : > > > This mailing list thread

Re: [Vote][Format] (new proposal) C data interface format string for ListView and LargeListView arrays

2023-10-06 Thread Dewey Dunnington
+1! On Fri, Oct 6, 2023, 8:03 PM Matt Topol wrote: > +1 > > On Fri, Oct 6, 2023, 6:55 PM Benjamin Kietzman > wrote: > > > +1 > > > > On Fri, Oct 6, 2023, 17:27 Felipe Oliveira Carvalho > > > wrote: > > > > > Hello, > > > > > > I'm writing to propose "+vl" and "+vL" as format strings for list-v

Re: [DISCUSS][Swift] repo for swift similar to arrow-rs

2023-10-10 Thread Dewey Dunnington
Hi Alva, I would encourage you to do whatever will make life more pleasant for you and other contributors to the Swift Arrow implementation. I have found development of an Arrow subproject (nanoarrow) in a separate repository very pleasant. While I don't run integration tests there, it's not becau

Re: [ANNOUNCE] New Arrow PMC member: Jonathan Keane

2023-10-15 Thread Dewey Dunnington
Congrats, Jon! On Sun, Oct 15, 2023 at 7:53 AM Nic Crane wrote: > > Congrats Jon! > > On Sun, 15 Oct 2023, 05:52 Jacob Wujciak-Jens, > wrote: > > > Congratulations 🎉! > > > > Raúl Cumplido schrieb am So., 15. Okt. 2023, > > 00:58: > > > > > Congratulations Jon! > > > > > > El dom, 15 oct 2023,

Re: Apache Arrow file format

2023-10-18 Thread Dewey Dunnington
Plenty of opinions here already, but I happen to think that IPC streams and/or Arrow File/Feather are wildly underutilized. For the use-case where you're mostly just going to read an entire file into R or Python it's a bit faster (and far superior to a CSV or pickling or .rds files in R). > you're

Re: [VOTE][Format] C data interface format strings for Utf8View and BinaryView

2023-10-18 Thread Dewey Dunnington
+1! On Wed, Oct 18, 2023 at 2:14 PM Matt Topol wrote: > > +1 > > On Wed, Oct 18, 2023 at 1:05 PM Antoine Pitrou wrote: > > > +1 > > > > Le 18/10/2023 à 19:02, Benjamin Kietzman a écrit : > > > Hello all, > > > > > > I propose "vu" and "vz" as format strings for the Utf8View and > > > BinaryView

Re: [DISCUSS][Format] C data interface for Utf8View

2023-10-26 Thread Dewey Dunnington
Ben kindly explained to me offline that the need for the buffer sizes is because when Arrow C++ imports an Array it creates Buffer class wrappers around the imported pointers. Arrow C++ does not have a notion of a buffer of unknown size to my knowledge, which leaves two undesirable alternatives: (1

Re: [DISCUSS][Format] C data interface for Utf8View

2023-10-26 Thread Dewey Dunnington
to another device?) I don't think there's any barrier to accessing the content of all the array elements but I could be mistaken. On Thu, Oct 26, 2023 at 1:04 PM Antoine Pitrou wrote: > > > Le 26/10/2023 à 17:45, Dewey Dunnington a écrit : > > The lack of buffer sizes is some

Re: [DISCUSS][Format] C data interface for Utf8View

2023-10-26 Thread Dewey Dunnington
I'm afraid I've derailed the discussion into solving a bigger problem than strictly necessary. I don't think this is the time to solve the general problem of the C data interface having no way to communicate buffer sizes, particularly since there's no immediate agreement on its utility or implement

Re: [DISCUSS][Format] C data interface for Utf8View

2023-10-26 Thread Dewey Dunnington
s constantly omitting buffer sizes and consumers constantly recalculating them. On Thu, Oct 26, 2023 at 4:35 PM Dewey Dunnington wrote: > > I'm afraid I've derailed the discussion into solving a bigger problem > than strictly necessary. I don't think this is the time

Re: [DISCUSS][Format] C data interface for Utf8View

2023-10-29 Thread Dewey Dunnington
In the absence of a general solution to the C data interface omitting buffer sizes, I think the original proposal is the best way forward...this is the first type to be added whose buffer sizes cannot be calculated without looping over every element of the array; the buffer sizes are needed to effi

Re: [VOTE] Release Apache Arrow ADBC 0.8.0 - RC0

2023-11-07 Thread Dewey Dunnington
+1! I ran: TEST_APT=0 TEST_YUM=0 USE_CONDA=1 dev/release/verify-release-candidate.sh 0.8.0 0 On Fri, Nov 3, 2023 at 12:18 PM David Li wrote: > > Hello, > > I would like to propose the following release candidate (RC0) of Apache Arrow > ADBC version 0.8.0. This is a release consisting of 42 reso

Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB interface

2023-11-07 Thread Dewey Dunnington
For argument's sake, I might suggest that the process you described in your initial note would probably work best in another repo: you would be able to iterate faster and release/version at your own pace. The flexibility you get from moving to a separate repo comes at the cost of extra responsibili

Re: [ANNOUNCE] New Arrow PMC member: Raúl Cumplido

2023-11-13 Thread Dewey Dunnington
Congrats, Raùl! On Mon, Nov 13, 2023 at 3:54 PM Dane Pitkin wrote: > > Congrats, Raul! > > On Mon, Nov 13, 2023 at 2:45 PM Kevin Gurney > wrote: > > > Congratulations, Raúl! > > > > > > From: Nic Crane > > Sent: Monday, November 13, 2023 2:31 PM > > To: dev@arro

Re: [DISCUSS] Protocol for exchanging Arrow data over REST APIs

2023-11-21 Thread Dewey Dunnington
I also think a set of best practices for Arrow over HTTP would be a valuable resource for the community...even if it never becomes a specification of its own, it will be beneficial for API developers and consumers of those APIs to have a place to look to understand how Arrow can help improve throug

Re: [ANNOUNCE] New Arrow committer: Felipe Oliveira Carvalho

2023-12-07 Thread Dewey Dunnington
Congrats! On Thu, Dec 7, 2023 at 4:28 PM Andrew Lamb wrote: > > Congratulations! > > On Thu, Dec 7, 2023 at 3:09 PM Kevin Gurney > wrote: > > > Congratulations, Felipe! > > > > From: Daniël Heres > > Sent: Thursday, December 7, 2023 2:59 PM > > To: dev@arrow.apa

Re: [VOTE][Julia] Release Apache Arrow Julia 2.7.0 RC1

2023-12-08 Thread Dewey Dunnington
+1 I ran export PATH="/Applications/Julia-1.9.app/Contents/Resources/julia/bin:$PATH" dev/release/verify_rc.sh 2.7.0 1 ...on MacOS M1 Ventura On Tue, Dec 5, 2023 at 4:38 PM Sutou Kouhei wrote: > > Hi, > > I would like to propose the following release candidate (RC1) of > Apache Arrow Julia ver

Re: [DISCUSS] Semantics of extension types

2023-12-13 Thread Dewey Dunnington
Thank you for opening the discussion here and opening it up! I agree that attaching semantics as metadata and/or documenting them in a central repository is an unreasonable burden to put on extension type authors and Arrow implementations in general. I also agree that operations other than filter

Re: [DISCUSS] Semantics of extension types

2023-12-15 Thread Dewey Dunnington
I also like these equivalence traits...in addition to being easy for extension type authors to specify when registering an extension type in Arrow C++, implementations that allow registration like pyarrow and arrow/R would be able to specify them easily, whereas implementing methods, compute functi

Re: [VOTE] Release Apache Arrow 14.0.2 - RC3

2023-12-15 Thread Dewey Dunnington
+1 I ran TEST_DEFAULT=0 TEST_CPP=1 dev/release/verify-release-candidate.sh 14.0.2 3 on MacOS M1. I do get one failing test (gandiva-internals-test) but this has failed for me for the last three versions. Note that the R bindings will have to patch the static libraries we host for convenience inst

Re: [VOTE] Release Apache Arrow ADBC 0.9.0 - RC0

2024-01-04 Thread Dewey Dunnington
+1 I ran: export DOCKER_DEFAULT_PLATFORM=linux/amd64 && USE_CONDA=1 dev/release/verify-release-candidate.sh 0.9.0 0 ...on MacOS M1 Ventura On Thu, Jan 4, 2024 at 9:47 AM Jean-Baptiste Onofré wrote: > > +1 (non binding) > > I checked: > - LICENSE is OK but maybe worth to keep only LICENSE.txt at

[VOTE] Release Apache Arrow nanoarrow 0.4.0 - RC0

2024-01-29 Thread Dewey Dunnington
Hello, I would like to propose the following release candidate (rc0) of Apache Arrow nanoarrow [0] version 0.4.0. This release consists of 46 resolved GitHub issues from 5 contributors [1]. This release candidate is based on commit: 3f83f4c48959f7a51053074672b7a330888385b1 [2] The source release

Re: [VOTE] Release Apache Arrow nanoarrow 0.4.0 - RC0

2024-01-30 Thread Dewey Dunnington
l mar, 30 ene 2024 a las 0:30, David Li () escribió: > > > > +1 (binding) > > > > Tested on Debian Linux 'bookworm' > > > > On Mon, Jan 29, 2024, at 10:45, Dane Pitkin wrote: > > > +1 (non-binding) > > > > > > Verified on MacOS

Re: [DISCUSS] Status and future of @ApacheArrow Twitter account

2024-01-30 Thread Dewey Dunnington
I also find it a useful tool to follow other projects...there may be a good replacement for it at some point but in the meantime I would love to see releases + blog posts tweeted (or retweeted by) the official account. -dewey On Tue, Jan 30, 2024 at 6:01 AM Raúl Cumplido wrote: > > El lun, 29 en

Re: [VOTE][Julia] Release Apache Arrow Julia 2.7.1 RC1

2024-01-31 Thread Dewey Dunnington
+1 Tested on MacOS Sonoma (aarch64). I ran export PATH="/Applications/Julia-1.9.app/Contents/Resources/julia/bin:${PATH}" && dev/release/verify_rc.sh 2.7.1 1 On Wed, Jan 31, 2024 at 2:01 PM Jacob Quinn wrote: > > +1, tested on macos. > > -Jacob > > On Wed, Jan 31, 2024 at 10:11 AM Ben Baumgold

[RESULT] Release Apache Arrow nanoarrow 0.4.0 - RC0

2024-02-01 Thread Dewey Dunnington
> > * Apache Arrow C++ main > * gcc (Debian 13.2.0-9) 13.2.0 > * R version 4.3.2 (2023-10-31) -- "Eye Holes" > * Python 3.11.7 > > Thanks, > -- > kou > > In > "[VOTE] Release Apache Arrow nanoarrow 0.4.0 - RC0" on Mon, 29 Jan 2024 > 1

[ANNOUNCE] Apache Arrow nanoarrow 0.4.0 Released

2024-02-12 Thread Dewey Dunnington
The Apache Arrow community is pleased to announce the 0.4.0 release of Apache Arrow nanoarrow. This initial release covers 44 resolved issues from 5 contributors[1]. The release is available now from [2], release notes are available at [3], and a blot post documenting new contributions is availabl

Re: [RESULT] Release Apache Arrow nanoarrow 0.4.0 - RC0

2024-02-12 Thread Dewey Dunnington
] Release blog post [x] Sent announcement to annou...@apache.org [x] Removed old artifacts from SVN [x] Bumped versions on main On Thu, Feb 1, 2024 at 3:21 PM Dewey Dunnington wrote: > > With 4 binding +1 and 1 non-binding +1, the vote carries! > > If somebody is up for reviewing the

Re: [ANNOUNCE] Apache Arrow nanoarrow 0.4.0 Released

2024-02-16 Thread Dewey Dunnington
Thanks for the suggestion! I opened up a PR to update that language [1]. Cheers! -dewey [1] https://github.com/apache/arrow-nanoarrow/pull/389 On Mon, Feb 12, 2024 at 2:57 PM Antoine Pitrou wrote: > > > Hi Dewey, > > Le 12/02/2024 à 15:01, Dewey Dunnington a écrit : > > A

  1   2   >