Re: [VOTE] Release Apache Arrow 13.0.0 - RC0

Apparently, we never managed to test the C API support in the C# library on downlevel versions of .NET until now, and they don't actually work :(. This gap is in part because the tests don't get run on anything but .NET 7.0. I've submitted a PR to address these issues, though the testing for .NET 4

Are interval components unsigned?

The Parquet specification clearly states that the components of a Parquet interval are unsigned integers. I couldn't find an equivalent statement for Arrow, and the C++ implementation has these as signed. Is it correct to assume that the intent for Arrow intervals is that they should be non-negativ

Re: Are interval components unsigned?

> Micah > > [1] https://lists.apache.org/thread/pqs6qnjvw1gxfkxz02bntvvyqxvw34mm > > On Fri, Oct 13, 2023 at 4:40 PM Curt Hagenlocher > wrote: > > > The Parquet specification clearly states that the components of a Parquet > > interval are unsigned integers. I couldn't find an equiv

Re: [ANNOUNCE] New Arrow committer: Curt Hagenlocher

nday, October 15, 2023 5:32 PM > > To: dev@arrow.apache.org > > Subject: Re: [ANNOUNCE] New Arrow committer: Curt Hagenlocher > > > > Congratulations! > > > > On Sun, Oct 15, 2023, 8:51 AM Gang Wu wrote: > > > > > Congrats! > > > > > >

Language-specific discussion (with C# example)

I'm curious what other (sub-) communities do about implementation-specific considerations that aren't directly tied to the Arrow standard. I don't see much of that kind of discussion on the dev list; does that mean these happen largely in the context of specific pull requests -- or perhaps not at a

decimal64

Recently, someone opened an issue on GitHub ([C++] Decimal64/32 support? · Issue #38622 · apache/arrow (github.com) ) asking for support for narrower decimal types. They were advised to start a thread on the mailing list, and as they haven't done so yet

Re: decimal64

decimal64 type. On Thu, Nov 9, 2023 at 8:17 AM Raphael Taylor-Davies wrote: > Perhaps my maths is incorrect, but a decimal64 would have a maximum > precision of 18, not 19? log(9223372036854775807) = 18.9? > > On 09/11/2023 16:01, Curt Hagenlocher wrote: > > Recently, someo

Re: decimal64

ov 9, 2023 at 8:23 AM Curt Hagenlocher wrote: > Obviously the limits don't match up exactly, so it depends on whether you > want to express the decimal bounds in a way that captures all the possible > underlying values or in a way that allows all the decimal values to be > represe

Re: decimal64

If Arrow had a decimal64 type, someone could choose to use that for a PostgreSQL money column knowing that there are edge cases where they may get an undesired result. On Thu, Nov 9, 2023 at 8:42 AM Antoine Pitrou wrote: > > Le 09/11/2023 à 17:23, Curt Hagenlocher a écrit : >

Re: decimal64

scale is > > fixed anyway, and you're probably not going to multiply money values > > together. > > > > > > Le 09/11/2023 à 17:54, Curt Hagenlocher a écrit : > >> If Arrow had a decimal64 type, someone could choose to use that for a > >> Pos

CIDR 2024

Who's going to CIDR in January? (And who else is shocked that it's already going to be 2024...?) -Curt

Re: CIDR 2024

6/12/2023 à 01:15, Wes McKinney a écrit : > > I will also be there. > > > > On Mon, Dec 4, 2023 at 12:58 PM Tony Wang wrote: > > > >> I am > >> > >> Get Outlook for Android<https://aka.ms/AAb9ysg> > >> ___

Java, dictionary ids and schema equality

I've (mostly) fixed the C# implementation of dictionary IPC but I'm getting a failing integration test. The Java checks are explicitly validating that the dictionary IDs in the schema match the values it expects. None of the other implementations seem to do that, though they're obviously passing an

Re: ADBC: xdbc_data_type and xdbc_sql_data_type

Interestingly, the description of sql_data_type in FlightSql.proto includes "The value of the SQL DATA TYPE which has the same values as data_type value." On Thu, Jan 11, 2024 at 10:06 AM David Li wrote: > Those values are inherited from Flight SQL [1] which effectively borrowed > types from J

Re: [ANNOUNCE] New Committer Joel Lubinitsky

Congrats Joel! On Mon, Apr 1, 2024 at 10:36 AM Wes McKinney wrote: > Congrats! > > On Mon, Apr 1, 2024 at 11:01 AM Andrew Lamb wrote: > > > Congratulations Joel. > > > > On Mon, Apr 1, 2024 at 11:53 AM Raúl Cumplido > > wrote: > > > > > Congratulations and welcome Joel! > > > > > > > > > El lu

Re: ADBC - OS-level driver manager

The advantage to system-wide registration of drivers (however that's accomplished) is of course that it allows driver authors to provide a single installer or set of instructions for the driver to be installed without regard for different usage scenarios. So if Tableau and Excel can both use ODBC d

Re: [DISCUSS][C#][GLib] Formalize use of the GLib libraries for native library bindings

As a ParquetSharp user, I think this is a great idea. I agree with Kou that the best user experience includes distributing the native parts, but I believe these can be separated out into individual "runtime" NuGet packages and referenced in a way that only the required ones are downloaded -- see N

Re: [DISCUSS] Statistics through the C data interface

> would it be easier to request statistics at a higher level of abstraction? What if there were a "single table provider" level of abstraction between ADBC and ArrowArrayStream as a C API; something that can report statistics and apply simple predicates? On Thu, May 23, 2024 at 5:57 AM Dewey Dun

Re: [DISCUSS] Variant Spec Location

This seems to straddle that line, in that you can also view this as a way to represent semi-structured data in a manner that allows for more efficient querying and computation by breaking out some of its components into a more structured form. (I also happen to want a canonical Arrow representatio

Re: [VOTE] Allow Decimal32 and Decimal64 bitwidths in Arrow Format

+1 (non-binding) I can upload a PR for the corresponding C# change within the next day. On Wed, Sep 4, 2024 at 2:21 PM Matt Topol wrote: > Based on various discussions among the ecosystem and to continue expanding > the zero-copy interoperability for Arrow to be used with different > libraries

Re: [Format][Important] Needed clarification of timezone-less timestamps

At some point, you just have to trust that a user is doing semantically-meaningful operations. After all, they could also choose to subtract a temperature from an elevation, or add feet to meters. It's important to define the precise semantics of an operation, including the assumptions it makes abo

Re: [ANNOUNCE] New Arrow PMC chair: Neil Richardson

Congratulations Neil! -Curt On Wed, Oct 30, 2024 at 4:28 AM Andrew Lamb wrote: > I am pleased to announce that the Arrow Project has a new PMC chair and VP > as per our tradition of rotating the chair once a year. Andy Grove has > resigned and > Neil Richardson was duly elected by the PMC and a

Re: [ANNOUNCE] New Arrow PMC member: Curt Hagenlocher

t; > bryceme...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > Congrats Curt! > > > > > > > > > > > > > > On Wed, Oct 30, 2024 at 2:56 PM Sutou Kouhei < > k...@clear-code.com > > > > > > > > wrote: > > > > > > > > > > > > > > > > The Project Management Committee (PMC) for Apache Arrow has > > > > > > > > invited Curt Hagenlocher to become a PMC member and we are > > > > > > > > pleased to announce that Curt Hagenlocher has accepted. > > > > > > > > > > > > > > > > Congratulations and welcome! > > > > > > > > > > > > > > > > >

Re: [ANNOUNCE] New Arrow committer: Adam Reeve

Congratulations Adam! On Mon, Nov 18, 2024 at 4:47 PM Ian Cook wrote: > Congratulations Adam! > > On Mon, Nov 18, 2024 at 19:31 Sutou Kouhei wrote: > > > On behalf of the Arrow PMC, I'm happy to announce that Adam Reeve > > has accepted an invitation to become a committer on Apache > > Arrow. W

Re: [RESULT][VOTE] Split C# release process

Yes, but can we weight each repository by the number of stars it has? :P I think arrow-dotnet is a better name, but it's a mild preference at best. On Thu, Mar 20, 2025 at 6:01 PM Sutou Kouhei wrote: > Hi, > > The vote carries with 5 +1 binding votes. > > Let's discuss the new repository name a

Re: [VOTE] Split C# release process

+1 On Sun, Mar 16, 2025 at 6:22 PM Sutou Kouhei wrote: > +1 > > In <20250317.101329.2226182884160584389@clear-code.com> > "[VOTE] Split C# release process" on Mon, 17 Mar 2025 10:13:29 +0900 > (JST), > Sutou Kouhei wrote: > > > Hi, > > > > I would like to propose splitting C# release pr

Re: [VOTE] Enable GitHub Discussions for apache/arrow-*

+1 On Fri, Mar 21, 2025 at 8:17 AM Gang Wu wrote: > +1 > > On Fri, Mar 21, 2025 at 10:58 PM Dewey Dunnington < > dewey.dunning...@gmail.com> wrote: > > > +1! > > > > On Fri, Mar 21, 2025 at 7:58 AM Neal Richardson < > > neal.p.richard...@gmail.com> > > wrote: > > > > > +1 > > > > > > On Fri, Mar

Re: [DISCUSS] Turtle canonical extension type

You bet your sweet ass I am. On Tue, Apr 1, 2025 at 11:58 AM Simon Perkins wrote: > I concur. +1 > > On Tue, Apr 1, 2025 at 8:52 PM Weston Pace wrote: > > > I've written a draft at [1] but for simplicity's sake I will include the > > text of the proposal inline below. > > > > [1] https://github

Re: [RESULT][VOTE] Split C# release process

gt; > -- > > > kou > > > > > > In > > > > "Re: [RESULT][VOTE] Split C# release process" on Fri, 21 Mar 2025 > 15:36:03 +1300, > > > Adam Reeve wrote: > > > > > >> I'd also prefer arrow-dotnet, as I thin

Copyright and dependencies

I'm sure this is the wrong audience for legal-adjacent advice :P but I thought I'd start here anyway. I'm interested in incorporating cldr/common/supplemental/windowsZones.xml at main · unicode-org/cldr into the .

Re: Copyright and dependencies

gt; Maybe you can reply here with a link to the PR when you vendor it, and > others here can review and make sure it follows the ASF guidelines? > > Neal > > > [1]: https://apache.org/legal/resolved.html#category-a > > On Tue, Apr 29, 2025 at 5:07 PM Curt Hagenlocher > wr

Re: Copyright and dependencies

SF. Could you check this? > > ("UNICODE, INC. LICENSE AGREEMENT - DATA FILES AND SOFTWARE" > in category A case? But > https://www.unicode.org/copyright.html doesn't have the > "Exhibit1" anchor now...) > > Thanks, > -- > kou > > In >

Re: [DISCUSS] Split C# release process

I'm definitely game for trying this. On Wed, Mar 12, 2025 at 7:50 PM Sutou Kouhei wrote: > Hi, > > This is a similar discussion to the "[DISCUSS] Split Go > release process" thread[1], the "[DISCUSS] Split Java > release process" thread[2] and the "[DISCUSS] Split R release > process" thread[3]:

Re: Arrow Flight Endpoint Location URLs

This feels like it's trying to use Flight as a catalog service. On Wed, Mar 26, 2025 at 6:04 PM Rusty Conover wrote: > Hi Jacob and Matt, > > I appreciate the opportunity to review the document. It helped clarify some > things for me, but I’d like to propose an approach that is slightly > differ