Re: How to include arrow and parquet in another project's CMakeLists.txt

2020-05-01 Thread Sutou Kouhei
Hi, Adding find_package(Parquet REQUIRED) to CMakeLists.txt and parquet_shared to target_link_libraries() will work: --- project(TestLib) cmake_minimum_required(VERSION 2.8.12) set(CMAKE_MACOSX_RPATH 1) set(CMAKE_LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib") find_package(JlCxx) get_target

[jira] [Created] (ARROW-8668) [Packaging][APT][Yum][ARM] Use Travis CI's ARM machine to build packages

2020-05-01 Thread Kouhei Sutou (Jira)
Kouhei Sutou created ARROW-8668: --- Summary: [Packaging][APT][Yum][ARM] Use Travis CI's ARM machine to build packages Key: ARROW-8668 URL: https://issues.apache.org/jira/browse/ARROW-8668 Project: Apache

Re: [Discuss] Proposal for optimizing Datasets over S3/object storage

2020-05-01 Thread David Li
(Apologies for the double-email) In the original coalescing PR, an "AsyncContext" abstraction was discussed. I could imagine being able to hold arbitrary attributes/metrics for tasks that a scheduler could then take into account, while making it easier for applications to thread through all the di

Re: [Discuss] Proposal for optimizing Datasets over S3/object storage

2020-05-01 Thread David Li
Wes, >From the outline there it seems like a path forward. I think the mode that would be helpful here is some sort of simple ordering/dependency so that the scheduler knows not to schedule subtasks of B until all subtasks of A have started (but not necessarily finished). I think the other part w

Re: [Discuss] Proposal for optimizing Datasets over S3/object storage

2020-05-01 Thread Wes McKinney
I just wrote up a ticket about a general purpose multi-consumer scheduler API, do you think this could be the beginning of a resolution? https://issues.apache.org/jira/browse/ARROW-8667 We may also want to design in some affordances so that no consumer is ever 100% blocked, even if that causes te

[jira] [Created] (ARROW-8667) [C++] Add multi-consumer Scheduler API to sit one layer above ThreadPool

2020-05-01 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8667: --- Summary: [C++] Add multi-consumer Scheduler API to sit one layer above ThreadPool Key: ARROW-8667 URL: https://issues.apache.org/jira/browse/ARROW-8667 Project: Apache

[jira] [Created] (ARROW-8666) [Java] DenseUnionVector has no way to set offset/validity directly

2020-05-01 Thread David Li (Jira)
David Li created ARROW-8666: --- Summary: [Java] DenseUnionVector has no way to set offset/validity directly Key: ARROW-8666 URL: https://issues.apache.org/jira/browse/ARROW-8666 Project: Apache Arrow

[jira] [Created] (ARROW-8665) [Java] DenseUnionWriter#setPosition fails with NullPointerException

2020-05-01 Thread David Li (Jira)
David Li created ARROW-8665: --- Summary: [Java] DenseUnionWriter#setPosition fails with NullPointerException Key: ARROW-8665 URL: https://issues.apache.org/jira/browse/ARROW-8665 Project: Apache Arrow

[jira] [Created] (ARROW-8664) Add isSet skip check to all Vector types

2020-05-01 Thread Ryan Murray (Jira)
Ryan Murray created ARROW-8664: -- Summary: Add isSet skip check to all Vector types Key: ARROW-8664 URL: https://issues.apache.org/jira/browse/ARROW-8664 Project: Apache Arrow Issue Type: Task

[jira] [Created] (ARROW-8663) Small correction to building.rst

2020-05-01 Thread Chad Dougherty (Jira)
Chad Dougherty created ARROW-8663: - Summary: Small correction to building.rst Key: ARROW-8663 URL: https://issues.apache.org/jira/browse/ARROW-8663 Project: Apache Arrow Issue Type: Bug

Re: Pyarrow building from source along with CPP Libraries to link to another Cython API

2020-05-01 Thread Vibhatha Abeykoon
Sorry for the confusion, this must go to the user mailing list. With Regards, Vibhatha Abeykoon, Research Assistant, Intelligent Systems Engineering, Indiana University Bloomington, Cell : +1-812-955-1394 Web: https://www.vibhatha.org On Thu, Apr 3

[jira] [Created] (ARROW-8662) [CI] Consolidate appveyor scripts

2020-05-01 Thread Krisztian Szucs (Jira)
Krisztian Szucs created ARROW-8662: -- Summary: [CI] Consolidate appveyor scripts Key: ARROW-8662 URL: https://issues.apache.org/jira/browse/ARROW-8662 Project: Apache Arrow Issue Type: Improv

[NIGHTLY] Arrow Build Report for Job nightly-2020-05-01-0

2020-05-01 Thread Crossbow
Arrow Build Report for Job nightly-2020-05-01-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-05-01-0 Failed Tasks: - test-conda-python-3.6: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-05-01-0-azure-test-conda-python-3.6 - te