Re: ADBC -> Arrow references for C#

Hi, How about just checking out apache/arrow in GitHub Actions because this is a temporary problem until the next Apache Arrow release? We can checkout apache/arrow-adbc and apache/arrow by the following steps: - uses: actions/checkout@v3 - uses: actions/checkout@v3 with: repositor

Re: New datatype: Huge integers & decimals

A further advantage of third-party extension types is that they give you a way to experiment without as much concern for compatibility. I think writing an extension type if possible, and promoting it to an official type (extension or otherwise) only if necessary, is a good general approach. On Tu

Re: New datatype: Huge integers & decimals

Hello Arrow devs, I actually have a use case where we'd like to support a new number type in Arrow, but instead of larger numbers, smaller ones. :) For machine learning use cases, we at Lance would like to support bfloat16 [1]. These are 16-bit floating point numbers that trade significant digits

Re: [VOTE][Format] Add experimental ArrowDeviceArray to C-Data API

To clarify: > Depends on what we're voting on? Voting on adopting the spec and adding it (while still leaving it labeled as "experimental" in the docs) to the format. --Matt On Tue, May 23, 2023 at 3:29 PM Matthew Topol wrote: > @Antoine: I've updated the PR with a prose description of the C

[VOTE][RUST][DataFusion] Release DataFusion Python Bindings 25.0.0 RC1

Hi, I would like to propose a release of Apache Arrow DataFusion Python Bindings, version 25.0.0. This release candidate is based on commit: 82b4a9553ed56cae664a7c637c7ad4d88fb21d35 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. The Python whee

Re: [VOTE][Format] Add experimental ArrowDeviceArray to C-Data API

@Antoine: I've updated the PR with a prose description of the C Device Data interface. Sorry for the lack of that in the first place. --Matt On Tue, May 23, 2023 at 10:34 AM Antoine Pitrou wrote: > > Also, I forgot to say, but thanks a lot for doing this! We can hope this > will drastically imp

Re: New datatype: Huge integers & decimals

Your question seems unspecific, but we now have the possibility of standardizing canonical extension types (which are, of course, optional to implement and support): https://arrow.apache.org/docs/format/CanonicalExtensions.html Le 23/05/2023 à 19:45, Ian Joiner a écrit : That’s a possibil

Re: New datatype: Huge integers & decimals

That’s a possibility. Do we consider officially support them? On Tuesday, May 23, 2023, Antoine Pitrou wrote: > > I'm not sure what you're actually proposing here. A new extension type > perhaps? > > > Le 23/05/2023 à 19:13, Ian Joiner a écrit : > >> Hi, >> >> We need to have really large integ

Re: New datatype: Huge integers & decimals

I'm not sure what you're actually proposing here. A new extension type perhaps? Le 23/05/2023 à 19:13, Ian Joiner a écrit : Hi, We need to have really large integers (with 128, 256 and 512 bits) as well as decimals (up to at least decimal1024) because they do actually exist in crypto / web

New datatype: Huge integers & decimals

Hi, We need to have really large integers (with 128, 256 and 512 bits) as well as decimals (up to at least decimal1024) because they do actually exist in crypto / web3 space. See https://docs.rs/primitive-types/latest/primitive_types/ for an example of what needs to be supported. If accepted we

Re: ADBC -> Arrow references for C#

So far we've waited for official releases (and that reminds me I need to bump Java). Except Golang can easily reference different versions as necessary (and we've had to do that before to pick up fixes, we should bump to the released 12.0.0 if we haven't already though). That said for C# I thin

RE: ADBC -> Arrow references for C#

How do the other languages in apache/arrow-adbc consume bits from apache/arrow before they are officially released in a major version? For example, I see Java uses the 11.0.0 version (https://github.com/apache/arrow-adbc/blob/main/java/pom.xml#L31). What would happen if it is dependent on a bug

ADBC -> Arrow references for C#

We recently put up feat(csharp): adding C# functionality by davidhcoe * Pull Request #697 * apache/arrow-adbc (github.com). This PR introduces C# functionality for ADBC and is dependent on capabilities introduced in GH-33856: [C#] Implement C Data

Re: [VOTE][Format] Add experimental ArrowDeviceArray to C-Data API

Also, I forgot to say, but thanks a lot for doing this! We can hope this will drastically improve interoperability between non-CPU data frameworks and libraries. Regards Antoine. Le 23/05/2023 à 16:32, Antoine Pitrou a écrit : Depends on what we're voting on? The C declarations seem fi

Arrow community meeting May 24 at 16:00 UTC

Hi all, Our biweekly Arrow community meeting is tomorrow at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 The notes for this and future instances of this meeting will be captured in this Goog

Re: [VOTE][Format] Add experimental ArrowDeviceArray to C-Data API

Depends on what we're voting on? The C declarations seem fine to me (I'm a bit lukewarm on the reserved bits, but I understand the motivation), however I've posted comments as to how to document the interface. The current PR entirely lacks a prose description of the C Device Data Interface.

[RESULT][VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 25.0.0 RC3

Resending with RESULT subject line. On Tue, May 23, 2023 at 8:04 AM Andy Grove wrote: > The vote has passed with 5 +1 votes (3 binding). Thanks, everyone. I have > published the source tarball and also published the crates. > > On Tue, May 23, 2023 at 7:02 AM vin jake wrote: > >> +1 (non-bindin

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 25.0.0 RC3

The vote has passed with 5 +1 votes (3 binding). Thanks, everyone. I have published the source tarball and also published the crates. On Tue, May 23, 2023 at 7:02 AM vin jake wrote: > +1 (non-binding) > > Verified on M1 macbook. > > Thanks Andy > > Andy Grove 于 2023年5月20日周六 21:41写道： > > > Hi, >

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 25.0.0 RC3

+1 (non-binding) Verified on M1 macbook. Thanks Andy Andy Grove 于 2023年5月20日周六 21:41写道： > Hi, > > I would like to propose a release of Apache Arrow DataFusion > Implementation, > version 25.0.0. > > *Note that the vote is for RC3 because I ran into some issues with RC1 and > RC2 (see [4]).* >

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 25.0.0 RC3

+1 (non-binding) Verified on M1 Mac. Thanks Andy Andrew Lamb 于2023年5月23日周二 01:51写道： > +1 (binding) > > Thanks Andy > > On Sat, May 20, 2023 at 1:05 PM L. C. Hsieh wrote: > > > +1 (binding) > > > > Verified on M1 Mac. > > > > Thanks Andy. > > > > On Sat, May 20, 2023 at 6:41 AM Andy Grove > w