Re: [DISCUSS] Fix Arrow Flight Core tests

2023-12-13 Thread Sutou Kouhei
Hi, I think (2) is better because https://arrow.apache.org/docs/developers/java/building.html is for building checked out repository instead of a release source archive. Thanks, -- kou In "Re: [DISCUSS] Fix Arrow Flight Core tests" on Wed, 13 Dec 2023 20:42:29 +0100, Jean-Baptiste Onofré

Re: [Discussion][C++][FlightRPC] What stage to submit a PR for Flight SQL ODBC driver

2023-12-13 Thread David Li
Yes, I can handle this process on the PMC side. Thanks for the help. On Wed, Dec 13, 2023, at 16:55, Laurent Goujon wrote: > David, would you be the PMC contact for the flightsql-odbc driver? If yes, > and if there's no prior objection for bringing flightsql-odbc code to Arrow > project, I can sta

Re: [DISC][Java]: Migrate Arrow Java to JPMS Java Platform Module System

2023-12-13 Thread David Li
That seems fine. Is there somewhere more canonical we should be putting the libraries? For instance I poked netty-tcnative and it puts the libraries under META-INF/native. On Wed, Dec 13, 2023, at 17:52, James Duong wrote: > Another issue popped up while doing modules with native libraries. > >

Re: [DISC][Java]: Migrate Arrow Java to JPMS Java Platform Module System

2023-12-13 Thread James Duong
Another issue popped up while doing modules with native libraries. We currently put native libraries in the arch folder in the root of each built JAR. JPMS is interpreting this as a package name (eg x86_64). This is breaking when a user tries to use multiple modules with native libs (or in the

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 34.0.0 RC1

2023-12-13 Thread Andrew Lamb
There is now a PR with a proposed fix ready as well[1] [1]: https://github.com/apache/arrow-datafusion/pull/8533 On Wed, Dec 13, 2023 at 3:35 PM L. C. Hsieh wrote: > Thanks Andrew for finding the regression before releasing. > > Agreed to hold on the release for now. > > On Wed, Dec 13, 2023 at

Re: [Discussion][C++][FlightRPC] What stage to submit a PR for Flight SQL ODBC driver

2023-12-13 Thread Laurent Goujon
David, would you be the PMC contact for the flightsql-odbc driver? If yes, and if there's no prior objection for bringing flightsql-odbc code to Arrow project, I can start the internal discussion to get the legal approval from Dremio and work on the ip clearance with you (JB can be my backup on tha

Re: [VOTE] Release Apache Arrow 14.0.2 - RC3

2023-12-13 Thread Raúl Cumplido
Hi, A couple of minor nits for the release verification. The PR with the verification tasks [1] shows a couple of binary verification failures. "verify-rc-binaries-binary-linux-amd64" --> fails due to an issue downloading artifacts from artifactory, this is being worked on [2] and we have opened

[VOTE] Release Apache Arrow 14.0.2 - RC3

2023-12-13 Thread Raúl Cumplido
Hi, I would like to propose the following release candidate (RC3) of Apache Arrow version 14.0.2. This is a release consisting of 30 resolved GitHub issues[1]. This release candidate is based on commit: 740889f413af9b1ae1d81eb1e5a4a9fb4ce9cf97 [2] The source release rc3 is hosted at [3]. The bin

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 34.0.0 RC1

2023-12-13 Thread L. C. Hsieh
Thanks Andrew for finding the regression before releasing. Agreed to hold on the release for now. On Wed, Dec 13, 2023 at 12:26 PM Andy Grove wrote: > > Thanks, Andrew. > > That does seem quite serious. I agree it would be better to hold off on the > release until this is resolved. > > On Wed, D

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 34.0.0 RC1

2023-12-13 Thread Andy Grove
Thanks, Andrew. That does seem quite serious. I agree it would be better to hold off on the release until this is resolved. On Wed, Dec 13, 2023 at 1:09 PM Andrew Lamb wrote: > I have found a regression [1] that I think is fairly serious and should be > fixed prior to a release. > > If others t

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 34.0.0 RC1

2023-12-13 Thread Andrew Lamb
I have found a regression [1] that I think is fairly serious and should be fixed prior to a release. If others think we should release anyways I can plan to make a patch release shortly with the fix [1] https://github.com/apache/arrow-datafusion/issues/8532 On Wed, Dec 13, 2023 at 2:36 AM Kun L

Re: [DISC][Java]: Migrate Arrow Java to JPMS Java Platform Module System

2023-12-13 Thread Jean-Baptiste Onofré
Agree with David on that. Regards JB On Wed, Dec 13, 2023 at 8:17 PM David Li wrote: > > I think we will just have to live with requiring different flags for > different modules. If we can detect the lack of the flag at runtime somehow > and present an appropriate error message (the way we do

Re: [DISCUSS] Fix Arrow Flight Core tests

2023-12-13 Thread Jean-Baptiste Onofré
That's my point: 1. we should document that the build has to be launched with -DskipTests by default 2. or include submodule in the source distribution we distributed I think (1) is an acceptable and easy path. If there are no objections, I will create a PR about that (also updating https://arrow.

Re: [DISCUSS] Fix Arrow Flight Core tests

2023-12-13 Thread David Li
Could we include submodules in the source distribution instead? Also, you can still build from the source distribution, you just have to -DskipTests. On Wed, Dec 13, 2023, at 09:54, Jean-Baptiste Onofré wrote: > Hi David and foxes, > > I checked the release source distribution and I think we sh

Re: [DISC][Java]: Migrate Arrow Java to JPMS Java Platform Module System

2023-12-13 Thread David Li
I think we will just have to live with requiring different flags for different modules. If we can detect the lack of the flag at runtime somehow and present an appropriate error message (the way we do for the memory module) that would be best. On Wed, Dec 13, 2023, at 12:33, James Duong wrote:

Re: [DISCUSS] Semantics of extension types

2023-12-13 Thread Dewey Dunnington
Thank you for opening the discussion here and opening it up! I agree that attaching semantics as metadata and/or documenting them in a central repository is an unreasonable burden to put on extension type authors and Arrow implementations in general. I also agree that operations other than filter

Re: [DISCUSS] Semantics of extension types

2023-12-13 Thread Antoine Pitrou
Hi, For now, I would suggest that each implementation decides on their own strategy, because we don't have a clear idea of which is better (and extension types are probably not getting a lot of use yet). Regards Antoine. Le 13/12/2023 à 17:39, Benjamin Kietzman a écrit : The main proble

Re: [DISC][Java]: Migrate Arrow Java to JPMS Java Platform Module System

2023-12-13 Thread James Duong
For the work on modularization, I’ve merged the flight-grpc and flight-core modules. I haven’t moved any other packages. I also encountered this problem with flight-core. Flight-core derives from io.netty.buffer (specifically CompositeByteBuf), but we cannot use Netty buffer with JPMS without r

Re: [DISCUSS] Semantics of extension types

2023-12-13 Thread Benjamin Kietzman
The main problem I see with adding properties to ExtensionType is I'm not sure where that information would reside. Allowing type authors to declare in which ways the type is equivalent (or not) to its storage is appealing, but it seems to need an official extension field like ARROW:extension:seman

[DISCUSS] Semantics of extension types

2023-12-13 Thread Benjamin Kietzman
Hello all, Recently, a PR to arrow c++ [1] was opened to allow implicit casting from any extension type to its storage type in acero. This raises questions about the validity of applying operations to an extension array's storage. For example, some extension type authors may intend different order

Re: [DISC][Java]: Migrate Arrow Java to JPMS Java Platform Module System

2023-12-13 Thread Jean-Baptiste Onofré
Hi, We have a "split packages" between flight-core and flight-grpc. I don't think anyone is using only flight-core to create a new "transport". I think it's acceptable to combine flight-core and flight-grpc, and maybe reshape a bit (imho some classes from flight-grpc should be actually in flight-c

Re: [DISCUSS] Fix Arrow Flight Core tests

2023-12-13 Thread Jean-Baptiste Onofré
Hi David and foxes, I checked the release source distribution and I think we should maybe include a profile to include TestTls related tests. As an Apache project, we are supposed to be able to build from the source distribution without any external requirements (in the case the project goes to t

Arrow R Package Development Sync Call - Thurs 14th Dec

2023-12-13 Thread Nic Crane
The fortnightly Arrow R package dev community call is on Thursday 14th December at 17:30 UTC (12:30 ET). Video call link: https://meet.google.com/ghw-qfvv-cjb The meeting notes can be found here; please feel free to add items to the agenda: https://docs.google.com/document/d/1nSIfJw8mfqtvScqvSVqm