[jira] [Created] (ARROW-5506) [C++] Generic columnar format functionality

2019-06-03 Thread Andrei Gudkov (JIRA)
Andrei Gudkov created ARROW-5506: Summary: [C++] Generic columnar format functionality Key: ARROW-5506 URL: https://issues.apache.org/jira/browse/ARROW-5506 Project: Apache Arrow Issue Type:

Re: [DISCUSS][Java] How to solve the problem of OutOfMemoryException when there is sufficient memory?

2019-06-03 Thread Fan Liya
I can think of an example in Flink SQL: A hash table (for hash join) consists of two parts: 1. The bucket area, which is used to store the hash code & data address. 2. The record area, which stores the actual data. The size of the record area can be estimated by: estimated_record_count * average_

Re: [DISCUSS][Java] How to solve the problem of OutOfMemoryException when there is sufficient memory?

2019-06-03 Thread Fan Liya
Hi @Ravindra Pindikura and @Wes McKinney, Thanks for your feedback. According to [1], the major benefit of buddy allocator is that, it avoids memory fragmentation, and thus reduces the maintenance cost and compaction overhead. However, for applications that know exactly how much memory they need (

Re: doput without ticket?

2019-06-03 Thread ming zhang
Hi Wes A client usually needs to contact a control plane to find out an "execution plan" before it can write to data plane end points. The control plane might authorize the put, decide where to write, and encode all that into a ticket as part of the flightinfo. the ticket could be used not just as

[jira] [Created] (ARROW-5505) [R] Stop masking base R functions

2019-06-03 Thread Neal Richardson (JIRA)
Neal Richardson created ARROW-5505: -- Summary: [R] Stop masking base R functions Key: ARROW-5505 URL: https://issues.apache.org/jira/browse/ARROW-5505 Project: Apache Arrow Issue Type: Improv

[jira] [Created] (ARROW-5504) [R] move use_threads argument to global option

2019-06-03 Thread Neal Richardson (JIRA)
Neal Richardson created ARROW-5504: -- Summary: [R] move use_threads argument to global option Key: ARROW-5504 URL: https://issues.apache.org/jira/browse/ARROW-5504 Project: Apache Arrow Issue

[jira] [Created] (ARROW-5503) [R] add read_json()

2019-06-03 Thread Neal Richardson (JIRA)
Neal Richardson created ARROW-5503: -- Summary: [R] add read_json() Key: ARROW-5503 URL: https://issues.apache.org/jira/browse/ARROW-5503 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-5502) [R] file readers should mmap

2019-06-03 Thread Neal Richardson (JIRA)
Neal Richardson created ARROW-5502: -- Summary: [R] file readers should mmap Key: ARROW-5502 URL: https://issues.apache.org/jira/browse/ARROW-5502 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-5501) [R] read/write_feather/arrow?

2019-06-03 Thread Neal Richardson (JIRA)
Neal Richardson created ARROW-5501: -- Summary: [R] read/write_feather/arrow? Key: ARROW-5501 URL: https://issues.apache.org/jira/browse/ARROW-5501 Project: Apache Arrow Issue Type: Improvemen

[jira] [Created] (ARROW-5500) [R] read_csv_arrow() signature should match readr::read_csv()

2019-06-03 Thread Neal Richardson (JIRA)
Neal Richardson created ARROW-5500: -- Summary: [R] read_csv_arrow() signature should match readr::read_csv() Key: ARROW-5500 URL: https://issues.apache.org/jira/browse/ARROW-5500 Project: Apache Arrow

[jira] [Created] (ARROW-5499) [R] Alternate bindings for when libarrow is not found

2019-06-03 Thread Neal Richardson (JIRA)
Neal Richardson created ARROW-5499: -- Summary: [R] Alternate bindings for when libarrow is not found Key: ARROW-5499 URL: https://issues.apache.org/jira/browse/ARROW-5499 Project: Apache Arrow

[jira] [Created] (ARROW-5498) [C++] Add support for Flatbuffers 1.11.0 with MinGW

2019-06-03 Thread Sutou Kouhei (JIRA)
Sutou Kouhei created ARROW-5498: --- Summary: [C++] Add support for Flatbuffers 1.11.0 with MinGW Key: ARROW-5498 URL: https://issues.apache.org/jira/browse/ARROW-5498 Project: Apache Arrow Issue

Re: [DISCUSS][Java] How to solve the problem of OutOfMemoryException when there is sufficient memory?

2019-06-03 Thread Wes McKinney
Note that we prefer using jemalloc in C++, which also employs the buddy allocator strategy On Mon, Jun 3, 2019 at 4:33 PM Jacques Nadeau wrote: > > The Netty allocator is a buddy allocator. It always allocates in power of two > sizes. As far as I know, this hasn't changed recently. > -- > Jacque

[jira] [Created] (ARROW-5497) [R][Release] Build and publish R package docs

2019-06-03 Thread Neal Richardson (JIRA)
Neal Richardson created ARROW-5497: -- Summary: [R][Release] Build and publish R package docs Key: ARROW-5497 URL: https://issues.apache.org/jira/browse/ARROW-5497 Project: Apache Arrow Issue

[jira] [Created] (ARROW-5496) [R][CI] Fix relative paths in R codecov.io reporting

2019-06-03 Thread Neal Richardson (JIRA)
Neal Richardson created ARROW-5496: -- Summary: [R][CI] Fix relative paths in R codecov.io reporting Key: ARROW-5496 URL: https://issues.apache.org/jira/browse/ARROW-5496 Project: Apache Arrow

Re: [DISCUSS] Formalizing "extension type" metadata in the Arrow binary protocol

2019-06-03 Thread Wes McKinney
hi Micah, I have just updated my PR per your comments with more examples of extension types. https://github.com/apache/arrow/pull/4332 Are there more comments about this? I can start a vote in a couple of days absent further opinions. Can someone volunteer to review David's Java PR? I would lik

[jira] [Created] (ARROW-5495) [C++] Use HTTPS consistently for downloading dependencies

2019-06-03 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-5495: --- Summary: [C++] Use HTTPS consistently for downloading dependencies Key: ARROW-5495 URL: https://issues.apache.org/jira/browse/ARROW-5495 Project: Apache Arrow

[jira] [Created] (ARROW-5494) [Python] Create FileSystem bindings

2019-06-03 Thread Antoine Pitrou (JIRA)
Antoine Pitrou created ARROW-5494: - Summary: [Python] Create FileSystem bindings Key: ARROW-5494 URL: https://issues.apache.org/jira/browse/ARROW-5494 Project: Apache Arrow Issue Type: Improv

[jira] [Created] (ARROW-5493) [Integration/Go] add Go support for IPC integration tests

2019-06-03 Thread Sebastien Binet (JIRA)
Sebastien Binet created ARROW-5493: -- Summary: [Integration/Go] add Go support for IPC integration tests Key: ARROW-5493 URL: https://issues.apache.org/jira/browse/ARROW-5493 Project: Apache Arrow

Re: doput without ticket?

2019-06-03 Thread Wes McKinney
hi Ming, See comment on the FlightDescriptor member of FlightData: https://github.com/apache/arrow/blob/master/format/Flight.proto#L281 The descriptor can contain a serialized CMD if you need to pass additional information to the server when you are starting a DoPut operation. The schema is foun

[jira] [Created] (ARROW-5492) [R] Add "columns" option to read_parquet to read subset of columns

2019-06-03 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-5492: --- Summary: [R] Add "columns" option to read_parquet to read subset of columns Key: ARROW-5492 URL: https://issues.apache.org/jira/browse/ARROW-5492 Project: Apache Arrow

Re: ARROW-4714: Providing JNI interface to Read ORC file via Arrow C++

2019-06-03 Thread Wes McKinney
hi Yurui -- we'll hopefully get this merged this week, I think it's just a matter of maintainer bandwidth at the moment. Thanks On Wed, May 29, 2019 at 11:09 PM Yurui Zhou wrote: > > Hey guys: > > Currently all the comments has been resolved and all the builds and tests are > passed. > > Is the

[jira] [Created] (ARROW-5491) Remove unecessary semicolons following MACRO definitions

2019-06-03 Thread Brian Hulette (JIRA)
Brian Hulette created ARROW-5491: Summary: Remove unecessary semicolons following MACRO definitions Key: ARROW-5491 URL: https://issues.apache.org/jira/browse/ARROW-5491 Project: Apache Arrow

[jira] [Created] (ARROW-5490) [C++] Remove ARROW_BOOST_HEADER_ONLY

2019-06-03 Thread Antoine Pitrou (JIRA)
Antoine Pitrou created ARROW-5490: - Summary: [C++] Remove ARROW_BOOST_HEADER_ONLY Key: ARROW-5490 URL: https://issues.apache.org/jira/browse/ARROW-5490 Project: Apache Arrow Issue Type: Task

[jira] [Created] (ARROW-5489) [C++

2019-06-03 Thread Francois Saint-Jacques (JIRA)
Francois Saint-Jacques created ARROW-5489: - Summary: [C++ Key: ARROW-5489 URL: https://issues.apache.org/jira/browse/ARROW-5489 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-5488) [R] Workaround when C++ lib not available

2019-06-03 Thread JIRA
Romain François created ARROW-5488: -- Summary: [R] Workaround when C++ lib not available Key: ARROW-5488 URL: https://issues.apache.org/jira/browse/ARROW-5488 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-5487) [CI] [Python] Failure in docs build

2019-06-03 Thread Antoine Pitrou (JIRA)
Antoine Pitrou created ARROW-5487: - Summary: [CI] [Python] Failure in docs build Key: ARROW-5487 URL: https://issues.apache.org/jira/browse/ARROW-5487 Project: Apache Arrow Issue Type: Bug

[jira] [Created] (ARROW-5486) [GLib] Add binding of gandiva::FunctionRegistry and related things

2019-06-03 Thread Kenta Murata (JIRA)
Kenta Murata created ARROW-5486: --- Summary: [GLib] Add binding of gandiva::FunctionRegistry and related things Key: ARROW-5486 URL: https://issues.apache.org/jira/browse/ARROW-5486 Project: Apache Arrow

[jira] [Created] (ARROW-5485) [Gandiva][Crossbow] OSx builds failing

2019-06-03 Thread Praveen Kumar Desabandu (JIRA)
Praveen Kumar Desabandu created ARROW-5485: -- Summary: [Gandiva][Crossbow] OSx builds failing Key: ARROW-5485 URL: https://issues.apache.org/jira/browse/ARROW-5485 Project: Apache Arrow