Re: NullVector getField() confusion

2021-01-22 Thread Micah Kornfield
This sounds like a bug from the an interop perspective. Record batches including those with null columns, I think should be round trippable (I'm surprised this isn't covered in our integration testing). On Mon, Jan 11, 2021 at 8:44 AM Matt Boughen wrote: > Hi > > I'm confused by NullVector::get

Re: [C++] read Parquet columns into 64-bit offset types

2021-01-22 Thread Micah Kornfield
This seems like a bug or a miss. I opened: https://issues.apache.org/jira/browse/ARROW-11353 to track a fix. On Sun, Jan 17, 2021 at 9:18 PM Steve Kim wrote: > > This should be possible already, at least on git master but perhaps also > > in 2.0.0. Which problem are you encountering? > > With

Re: [MATLAB] Developing a MATLAB Interface for Apache Arrow

2021-01-22 Thread Kevin Gurney
Hi Antoine, Thanks for your input! As you pointed out, I am in fact familiar with the matlab/ directory! :-) Several MathWorkers, including myself, helped contribute to this code a while back. We are hoping to use it as a starting point as we build out a more fully fledged MATLAB interface to

Re: Please Review: Application for a Media Type

2021-01-22 Thread Wes McKinney
Thank you for taking the lead on this. I gave a brief read through and I think it makes sense using Thrift or Protocol Buffers as a guideline. Would be good for some others to review who might be familiar with IANA media formats On Wed, Jan 20, 2021 at 6:17 PM Weston Pace wrote: > > Per a previou

Re: [MATLAB] Developing a MATLAB Interface for Apache Arrow

2021-01-22 Thread Antoine Pitrou
Hello Kevin, You could use a Google Docs or similar to share the design document and allow people to comment. Inside a Google Doc, you can use "File -> Share" to create a sharable URL with specific permissions (such as commenting but not editing). I was about to mention the matlab/ directory in

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-22 Thread Neal Richardson
Crossbow does (at least some) Windows verification, and it appears that it passed: https://github.com/apache/arrow/pull/9245 On Fri, Jan 22, 2021 at 5:46 AM Neville Dipale wrote: > Hi Krisztian, > > The full output is at > https://gist.github.com/nevi-me/88a6279dd90aea30aa4caaa15fb0cc53 > > I al

Re: [MATLAB] Developing a MATLAB Interface for Apache Arrow

2021-01-22 Thread Kevin Gurney
It seems like the mailing list stripped out the design doc I attached for some reason. Here is a link to the same document hosted online instead: https://mathworks-my.sharepoint.com/:b:/p/kgurney/EU3Kdz0cubRJrkEyI1bNR88BKnH4S2siU2EHHNQwxTgHUg?e=wzLDx4 Note: This link is only a temporary solutio

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-22 Thread Neville Dipale
Hi Krisztian, The full output is at https://gist.github.com/nevi-me/88a6279dd90aea30aa4caaa15fb0cc53 I also ran dev/release/verify-release-candidate-wheels.bat 3.0.0 2 Getting the below error, it seems to be a Python 3.7 bug, but I'm not yet finding a solution for it online. (C:\tmp\arrow-verif

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-22 Thread Krisztián Szűcs
Thanks Neville for testing it! There should be more context about the failures above the summary. Could you please post the errors? On Fri, Jan 22, 2021 at 2:05 PM Neville Dipale wrote: > > (+0 non-binding) > > Getting test failures (see end of my mail). > > This is my first time verifying (Win

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-22 Thread Neville Dipale
(+0 non-binding) Getting test failures (see end of my mail). This is my first time verifying (Windows 10; Insider Preview if relevant), so I'm likely missing something in my config. I'll read the verification script and try again. I ran the below using PowerShell: $env:ARROW_GANDIVA=0; $env:ARR

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-22 Thread Neville Dipale
This is my first time verifying, do I also need to set the env vars below? ARROW_GANDIVA=0 ARROW_PLASMA=0 TEST_DEFAULT=0 TEST_SOURCE=1 TEST_CPP=1 TEST_PYTHON=1 TEST_JAVA=1 TEST_INTEGRATION_CPP=1 TEST_INTEGRATION_JAVA=1 Otherwise, I'm currently running: ./dev/release/verify-release-candidate.bat 3

[NIGHTLY] Arrow Build Report for Job nightly-2021-01-22-0

2021-01-22 Thread Crossbow
Arrow Build Report for Job nightly-2021-01-22-0 All tasks: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-22-0 Failed Tasks: - centos-8-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-22-0-travis-centos-8-aarch64 - con

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-22 Thread Krisztián Szűcs
Could anyone verify the release on a windows machine? On Thu, Jan 21, 2021 at 4:43 AM Bryan Cutler wrote: > > +1 (non-binding) > > I verified binaries and source with the following: > ARROW_TMPDIR=/tmp/arrow-test ARROW_GANDIVA=0 ARROW_PLASMA=0 TEST_DEFAULT=0 > TEST_SOURCE=1 TEST_CPP=1 TEST_PYTHON