Re: [DISCUSS][Java] Builders for java classes

2019-10-24 Thread Fan Liya
Hi Micah, IMO, we need an adapter from on-heap array to off-heap array. This is useful because many third-party Java libraries populate data to an on-heap array. And I see this API in your design: IntVectorBuilder addAll(int[] values); So I am +1 for this. Best, Liya Fan On Thu, Oct 24, 2019

Re: [DISCUSS][Java] Builders for java classes

2019-10-24 Thread Ravindra Pindikura
On Thu, Oct 24, 2019 at 10:01 AM Micah Kornfield wrote: > As part a PR Ji Liu has made to help populate data for test cases [1], the > question came up on whether we should provide a more builder classes in > java for ValueVectors. The proposed implementation would wrap the existing > Writer cl

[NIGHTLY] Arrow Build Report for Job nightly-2019-10-24-0

2019-10-24 Thread Crossbow
Arrow Build Report for Job nightly-2019-10-24-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-24-0 Failed Tasks: - docker-clang-format: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-24-0-circle-docker-clang-format - docke

Re: [VOTE] Clarifications and forward compatibility changes for Dictionary Encoding

2019-10-24 Thread Antoine Pitrou
Le 24/10/2019 à 04:39, Micah Kornfield a écrit : > > 3. Clarifies that the file format, can only contain 1 "NON-delta" > dictionary batch and multiple "delta" dictionary batches. This is a bit weird. If the file format can carry delta dictionaries, it means order is significant, so it may as

[jira] [Created] (ARROW-6984) Update LZ4 to 1.9.2 for CVE-2019-17543

2019-10-24 Thread Sangeeth Keeriyadath (Jira)
Sangeeth Keeriyadath created ARROW-6984: --- Summary: Update LZ4 to 1.9.2 for CVE-2019-17543 Key: ARROW-6984 URL: https://issues.apache.org/jira/browse/ARROW-6984 Project: Apache Arrow Iss

[jira] [Created] (ARROW-6985) Steadily increasing time to load file using read_parquet

2019-10-24 Thread Casey (Jira)
Casey created ARROW-6985: Summary: Steadily increasing time to load file using read_parquet Key: ARROW-6985 URL: https://issues.apache.org/jira/browse/ARROW-6985 Project: Apache Arrow Issue Type: Bug

Re: Possible Arrow 0.15.1 release

2019-10-24 Thread Krisztián Szűcs
I had to fix the OSX wheel build issues, once [1] is green I can start the release procedure, although we have three new issues in the release [2]. ARROW-6983: We have a fix for this. ARROW-6984: I'll bump LZ4's version. ARROW-6977: should be resolved by ARROW-6983? [1]: https://github.com/apache

Re: [VOTE] Clarifications and forward compatibility changes for Dictionary Encoding

2019-10-24 Thread Micah Kornfield
Hi Antoine, There is a defined order for dictionaries in metadata. What isn't well defined is relative ordering between record batches and Delta dictionaries. However, this point seems confusing. I can't think of a real-world use case we're it would be valuable enough to include, so I will remov

Re: Possible Arrow 0.15.1 release

2019-10-24 Thread Antoine Pitrou
No, ARROW-6977 is something else. I'll whip up a PR. Le 24/10/2019 à 17:25, Krisztián Szűcs a écrit : > I had to fix the OSX wheel build issues, once [1] is green I can start the > release procedure, although we have three new issues in the release [2]. > > ARROW-6983: We have a fix for this.

Re: [DISCUSS] Result vs Status

2019-10-24 Thread Omer F. Ozarslan
Hi, I don't have much experience on customized clang-tidy plugins, but this might be a good use case for such a plugin from what I read here and there (frankly this was a good excuse for me to have a look at clang tooling as well). I wanted to ensure it isn't obviously overkill before this suggest

Re: [DISCUSS] Result vs Status

2019-10-24 Thread Omer F. Ozarslan
Forgot to mention most of those lines are longer than line width while out is usually (always?) last parameter, so probably that's why grep possibly underestimates their number. On Thu, Oct 24, 2019 at 4:33 PM Omer F. Ozarslan wrote: > > Hi, > > I don't have much experience on customized clang-ti

Re: [DISCUSS] Result vs Status

2019-10-24 Thread Micah Kornfield
Hi Omer, I think this is really cool. It is quite possible it was underestimated (I agree about line lengths), but I think the clang query is double counting somehow. For instance: "grep -r Status *" only returns ~9000 results in total for me. Similarly using grep for "FinishTyped" returns 18 r

[jira] [Created] (ARROW-6986) [R] Add basic Expression class

2019-10-24 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-6986: -- Summary: [R] Add basic Expression class Key: ARROW-6986 URL: https://issues.apache.org/jira/browse/ARROW-6986 Project: Apache Arrow Issue Type: New Featu

Re: [DISCUSS] Result vs Status

2019-10-24 Thread Omer F. Ozarslan
Hi Micah, You're right. Quite possible that clang-query counted same function separately for each include in each file. (I was iterating each file separately, but providing all of them at once didn't change the result either.) It's cool and wrong, so not very useful apparently. :-) Best, Omer O

[jira] [Created] (ARROW-6987) [CI] Travis OSX failing to install sdk headers

2019-10-24 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-6987: - Summary: [CI] Travis OSX failing to install sdk headers Key: ARROW-6987 URL: https://issues.apache.org/jira/browse/ARROW-6987 Project: Apache Arrow

[jira] [Created] (ARROW-6988) [CI][R] Buildbot's R Conda is failing

2019-10-24 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-6988: - Summary: [CI][R] Buildbot's R Conda is failing Key: ARROW-6988 URL: https://issues.apache.org/jira/browse/ARROW-6988 Project: Apache Arrow

[jira] [Created] (ARROW-6989) [Python][C++] Assert is triggered when decimal type inference occurs on a value with out of range precision

2019-10-24 Thread Micah Kornfield (Jira)
Micah Kornfield created ARROW-6989: -- Summary: [Python][C++] Assert is triggered when decimal type inference occurs on a value with out of range precision Key: ARROW-6989 URL: https://issues.apache.org/jira/browse

[jira] [Created] (ARROW-6990) [C++] Support casting between decimal types with compatible precision/scales

2019-10-24 Thread Micah Kornfield (Jira)
Micah Kornfield created ARROW-6990: -- Summary: [C++] Support casting between decimal types with compatible precision/scales Key: ARROW-6990 URL: https://issues.apache.org/jira/browse/ARROW-6990 Projec

[jira] [Created] (ARROW-6991) [Packaging][deb] Add support for Ubuntu 19.10

2019-10-24 Thread Kouhei Sutou (Jira)
Kouhei Sutou created ARROW-6991: --- Summary: [Packaging][deb] Add support for Ubuntu 19.10 Key: ARROW-6991 URL: https://issues.apache.org/jira/browse/ARROW-6991 Project: Apache Arrow Issue Type: