Re: [Python] manylinux2014 and _GLIBCXX_USE_CXX11_ABI setting

2021-09-10 Thread Philipp Moritz
t the discussion there. On Fri, Sep 10, 2021 at 1:45 AM Antoine Pitrou wrote: > > Le 10/09/2021 à 10:05, Philipp Moritz a écrit : > > Thanks for your answer Antoine! > > > > Considering your first comment, there is a section in > > https://www.python.org/dev/peps/pep-

Re: [Python] manylinux2014 and _GLIBCXX_USE_CXX11_ABI setting

2021-09-10 Thread Philipp Moritz
will need a bit more coordination across the Python community. I'm happy to get a discussion started over there. On Fri, Sep 10, 2021 at 12:40 AM Antoine Pitrou wrote: > > Le 10/09/2021 à 09:12, Philipp Moritz a écrit : > > Dear all, > > > > how do you think _GLIB

[Python] manylinux2014 and _GLIBCXX_USE_CXX11_ABI setting

2021-09-10 Thread Philipp Moritz
Dear all, how do you think _GLIBCXX_USE_CXX11_ABI should be set for manylinux2014 (and manylinux2010) wheels? Should it be 0 or 1? Unfortunately I don't see https://www.python.org/dev/peps/pep-0599/ or https://github.com/pypa/manylinux specifying it. I think for manylinux1 the common wisdom was to

[jira] [Created] (ARROW-7004) [Plasma] Make it possible to bump up object in LRU cache

2019-10-28 Thread Philipp Moritz (Jira)
Philipp Moritz created ARROW-7004: - Summary: [Plasma] Make it possible to bump up object in LRU cache Key: ARROW-7004 URL: https://issues.apache.org/jira/browse/ARROW-7004 Project: Apache Arrow

[jira] [Created] (ARROW-6824) [Plasma] Support batched create and seal requests for small objects

2019-10-09 Thread Philipp Moritz (Jira)
Philipp Moritz created ARROW-6824: - Summary: [Plasma] Support batched create and seal requests for small objects Key: ARROW-6824 URL: https://issues.apache.org/jira/browse/ARROW-6824 Project: Apache

Re: [Discuss] C++ filenames: hyphens or underscores?

2019-08-06 Thread Philipp Moritz
I also have a small preference for underscores but would also be fine with dashes. It seems to be more common (therefore blends better with vendored code) and agrees with the styleguide and is closest to the exiting code. Also as an aside, having file_names names like variable_names is nice. Compa

[jira] [Created] (ARROW-5904) [Java] [Plasma] Fix compilation of Plasma Java client

2019-07-10 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5904: - Summary: [Java] [Plasma] Fix compilation of Plasma Java client Key: ARROW-5904 URL: https://issues.apache.org/jira/browse/ARROW-5904 Project: Apache Arrow

[jira] [Created] (ARROW-5751) [Packaging][Python] Python 2.7 wheels broken on macOS: libcares.2.dylib not found

2019-06-26 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5751: - Summary: [Packaging][Python] Python 2.7 wheels broken on macOS: libcares.2.dylib not found Key: ARROW-5751 URL: https://issues.apache.org/jira/browse/ARROW-5751

[jira] [Created] (ARROW-5690) [Packaging] macOS wheels broken: libprotobuf.18.dylib missing

2019-06-22 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5690: - Summary: [Packaging] macOS wheels broken: libprotobuf.18.dylib missing Key: ARROW-5690 URL: https://issues.apache.org/jira/browse/ARROW-5690 Project: Apache Arrow

Re: How should a Python/C++ project depend on Arrow (issues with ABI and wheel)?

2019-06-21 Thread Philipp Moritz
Dear all, I agree with Wes and Antoine, the way things are currently handled is not sustainable. If we are using wheels, it can only work if everybody is using the same toolchain. In the past the Arrow contributors have tried to "fix" TensorFlows non-compliance with the manylinux1 standards from t

[jira] [Created] (ARROW-5671) [crossbow] mac os python wheels failing

2019-06-20 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5671: - Summary: [crossbow] mac os python wheels failing Key: ARROW-5671 URL: https://issues.apache.org/jira/browse/ARROW-5671 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-5670) [crossbow] mac os python 3.5 wheel failing

2019-06-20 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5670: - Summary: [crossbow] mac os python 3.5 wheel failing Key: ARROW-5670 URL: https://issues.apache.org/jira/browse/ARROW-5670 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-5669) [crossbow] manylinux1 wheel building failing

2019-06-20 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5669: - Summary: [crossbow] manylinux1 wheel building failing Key: ARROW-5669 URL: https://issues.apache.org/jira/browse/ARROW-5669 Project: Apache Arrow Issue

Re: [ANNOUNCE] New Arrow committer: Francois Saint-Jacques

2019-06-12 Thread Philipp Moritz
Congrats François :) On Wed, Jun 12, 2019 at 3:37 PM Antoine Pitrou wrote: > > Welcome on the team François :-) > > > Le 12/06/2019 à 17:45, Wes McKinney a écrit : > > On behalf of the Arrow PMC I'm happy to announce that Francois has > > accepted an invitation to become an Arrow committer! > >

Re: [ANNOUNCE] New Arrow committer: Neville Dipale

2019-05-11 Thread Philipp Moritz
Congrats Neville! On Sat, May 11, 2019 at 6:09 PM Renjie Liu wrote: > Congrats! > > Chao Sun 于 2019年5月12日周日 上午12:38写道: > > > Congrats Neville! > > > > On Sat, May 11, 2019 at 9:36 AM Micah Kornfield > > wrote: > > > > > Congrats!! > > > > > > On Saturday, May 11, 2019, paddy horan wrote: > >

Re: Use arrow as a general data serialization framework in distributed stream data processing

2019-04-25 Thread Philipp Moritz
Hey Shawn, Thanks for these benchmarks! This is indeed a workload we would like to support well in Arrow/Plasma/Ray (if you are using Ray, using Plasma as a shared memory transport but some of the issues this raises will apply more widely to Arrow and other possible IPC/RPC transports like Flight

Re: [VOTE] Add 64-bit offset list, binary, string (utf8) types to the Arrow columnar format

2019-04-25 Thread Philipp Moritz
+1 (binding) On Thu, Apr 25, 2019 at 1:34 PM Wes McKinney wrote: > +1 (binding) > > On Thu, Apr 25, 2019 at 3:33 PM Wes McKinney wrote: > > > > In a recent mailing list discussion [1] Micah Kornfield has proposed > > to add new list and variable-size binary and unicode types to the > > Arrow co

Re: [DISCUSS] 64-bit offset variable width types (i.e.Large List, Last String, Large bytes)

2019-04-25 Thread Philipp Moritz
iously PRs would need both C++ and Java implementation > before > > > being merged. > > > > > > Could a PMC member facilitate this? > > > > > > Philipp if approved, do you have bandwidth to finish up the PR for > > > LargeList? > > > &

Re: [DISCUSS] 64-bit offset variable width types (i.e.Large List, Last String, Large bytes)

2019-04-15 Thread Philipp Moritz
@Micah: I wanted to make it possible to support serializing large objects (existing large pandas dataframes with an "object" column and also large python types with the pyarrow serialization). On Mon, Apr 15, 2019 at 8:22 PM Micah Kornfield wrote: > To summarize my understanding of the thread so

Re: [DISCUSS] 64-bit offset variable width types (i.e.Large List, Last String, Large bytes)

2019-04-11 Thread Philipp Moritz
Thanks for getting the discussion started, Micah! I'm +1 on this change and also slightly prefer 1. As Antoine mentions, there doesn't seem to be a clear benefit from 2, unless we want to also support 8 or 16 bit indices in the future, which seems unlikely. So going with 1 is ok I think. Best, Ph

Re: Support for numpy matrix

2019-03-30 Thread Philipp Moritz
Hey Mitar, It might be as simple as adding a handler here: https://github.com/apache/arrow/blob/master/python/pyarrow/serialization.py#L300 Do you want to try that? -- Philipp. On Sat, Mar 30, 2019 at 3:22 PM Mitar wrote: > Hi! > > I do not know where to start looking into this? Not sure if I

[jira] [Created] (ARROW-5027) [Python] Add JSON Reader

2019-03-27 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5027: - Summary: [Python] Add JSON Reader Key: ARROW-5027 URL: https://issues.apache.org/jira/browse/ARROW-5027 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-5022) [C++] Implement more "Datum" types for AggregateKernel

2019-03-26 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5022: - Summary: [C++] Implement more "Datum" types for AggregateKernel Key: ARROW-5022 URL: https://issues.apache.org/jira/browse/ARROW-5022 Project: Ap

[jira] [Created] (ARROW-5002) [C++] Implement GroupBy

2019-03-24 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5002: - Summary: [C++] Implement GroupBy Key: ARROW-5002 URL: https://issues.apache.org/jira/browse/ARROW-5002 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-4983) [Plasma] Unmap memory when the client is destroyed

2019-03-21 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4983: - Summary: [Plasma] Unmap memory when the client is destroyed Key: ARROW-4983 URL: https://issues.apache.org/jira/browse/ARROW-4983 Project: Apache Arrow

[jira] [Created] (ARROW-4958) [C++] Purely static linking broken

2019-03-18 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4958: - Summary: [C++] Purely static linking broken Key: ARROW-4958 URL: https://issues.apache.org/jira/browse/ARROW-4958 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-4939) [Python] Add wrapper for "sum" kernel

2019-03-17 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4939: - Summary: [Python] Add wrapper for "sum" kernel Key: ARROW-4939 URL: https://issues.apache.org/jira/browse/ARROW-4939 Project: Apache Arrow

[jira] [Created] (ARROW-4912) [C++, Python] Allow specifying column names to CSV reader

2019-03-15 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4912: - Summary: [C++, Python] Allow specifying column names to CSV reader Key: ARROW-4912 URL: https://issues.apache.org/jira/browse/ARROW-4912 Project: Apache Arrow

[jira] [Created] (ARROW-4905) [C++][Plasma] Remove dlmalloc from client library

2019-03-15 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4905: - Summary: [C++][Plasma] Remove dlmalloc from client library Key: ARROW-4905 URL: https://issues.apache.org/jira/browse/ARROW-4905 Project: Apache Arrow

Re: CMake refactor Heads-up

2019-03-15 Thread Philipp Moritz
Thanks Uwe for doing this! It's amazing how well the arrow build system is working given it supports so many different configurations (which makes Uwe's refactor all the more impressive). Big thanks for everybody helping to maintain it. On Fri, Mar 15, 2019 at 8:08 AM Wes McKinney wrote: > Than

Re: [ANNOUNCE] New Arrow committer: Micah Kornfield

2019-03-08 Thread Philipp Moritz
Congrats Micah! On Fri, Mar 8, 2019 at 11:28 AM Wes McKinney wrote: > On behalf of the Arrow PMC, I'm happy to announce that Micah has > accepted an invitation to become a committer on Apache Arrow. > > Welcome, and thank you for your contributions! >

[jira] [Created] (ARROW-4797) [Plasma] Avoid store crash if not enough memory is available

2019-03-07 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4797: - Summary: [Plasma] Avoid store crash if not enough memory is available Key: ARROW-4797 URL: https://issues.apache.org/jira/browse/ARROW-4797 Project: Apache Arrow

Re: [Discuss][Format] Checksum/Hash signature for data

2019-03-05 Thread Philipp Moritz
Hey Micah, in plasma, we are using xxhash to compute a hash/checksum [1] (it is computed in parallel using multiple threads) and have good experience with it -- all data in Ray is checksummed this way. Initially there were problems with uninitialized bits in the arrow representation, but that has

Re: [Discuss][Format] Checksum/Hash signature for data

2019-03-05 Thread Philipp Moritz
(I meant to say SHA256 instead of SHA1) On Tue, Mar 5, 2019 at 9:45 PM Philipp Moritz wrote: > Hey Micah, > > in plasma, we are using xxhash to compute a hash/checksum [1] (it is > computed in parallel using multiple threads) and have good experience with > it -- all data in Ray

[jira] [Created] (ARROW-4757) Nested chunked array support

2019-03-04 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4757: - Summary: Nested chunked array support Key: ARROW-4757 URL: https://issues.apache.org/jira/browse/ARROW-4757 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-4690) Building TensorFlow compatible wheels for Arrow

2019-02-26 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4690: - Summary: Building TensorFlow compatible wheels for Arrow Key: ARROW-4690 URL: https://issues.apache.org/jira/browse/ARROW-4690 Project: Apache Arrow Issue

Re: TensorFlow, PyTorch, and manylinux1

2019-02-06 Thread Philipp Moritz
onths till manylinux2014 or whatever will support c++14. On Wed, Feb 6, 2019 at 9:14 AM Philipp Moritz wrote: > The problems arose if some functionality of C++11 were used. It > led to certain symbols being statically linked into the shared library > which clashed with other shared lib

Re: TensorFlow, PyTorch, and manylinux1

2019-02-06 Thread Philipp Moritz
6, 2019 at 12:38 PM Antoine Pitrou > <mailto:anto...@python.org>> wrote: > > > > > > Le 06/02/2019 à 01:06, Philipp Moritz a écrit : > > > Thanks for the meeting! One question concerning a point that is > still > > > not super clear to

[jira] [Created] (ARROW-4491) [Python] Remove usage of std::to_string and std::stoi

2019-02-05 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4491: - Summary: [Python] Remove usage of std::to_string and std::stoi Key: ARROW-4491 URL: https://issues.apache.org/jira/browse/ARROW-4491 Project: Apache Arrow

Re: TensorFlow, PyTorch, and manylinux1

2019-02-05 Thread Philipp Moritz
Thanks for the meeting! One question concerning a point that is still not super clear to me: Say we define a new manylinux standard based on gcc >=5 (with stable c++11 support). There will still be a lot of wheels form the manylinux1 days that are built against gcc 4.8 that might use the c++11 fea

Re: [ANNOUNCE] New Arrow committer: Ravindra Pindikura

2019-02-04 Thread Philipp Moritz
Congratulations! On Mon, Feb 4, 2019 at 9:44 PM Ravindra Pindikura wrote: > Thanks everyone. It’s been a pleasure to work with the arrow community, > and I feel honored. > > > On Feb 5, 2019, at 9:28 AM, Bryan Cutler wrote: > > > > Congratulations Ravindra! > > > > On Mon, Feb 4, 2019, 7:40 PM

Re: [ANNOUNCE] New Arrow PMC member: Andy Grove

2019-02-04 Thread Philipp Moritz
Congratulations! On Mon, Feb 4, 2019 at 3:16 PM Krisztián Szűcs wrote: > Congrats Andy! :) > > On Mon, Feb 4, 2019 at 4:39 PM Wes McKinney wrote: > > > The Project Management Committee (PMC) for Apache Arrow has invited > > Andy Grove to become a PMC member and we are pleased to announce that >

[jira] [Created] (ARROW-4475) [Python] Serializing objects that contain themselves

2019-02-04 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4475: - Summary: [Python] Serializing objects that contain themselves Key: ARROW-4475 URL: https://issues.apache.org/jira/browse/ARROW-4475 Project: Apache Arrow

[jira] [Created] (ARROW-4453) [Python] Create Cython wrappers for sparse array

2019-02-01 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4453: - Summary: [Python] Create Cython wrappers for sparse array Key: ARROW-4453 URL: https://issues.apache.org/jira/browse/ARROW-4453 Project: Apache Arrow

[jira] [Created] (ARROW-4452) [Python] Serializing sparse torch tensors

2019-02-01 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4452: - Summary: [Python] Serializing sparse torch tensors Key: ARROW-4452 URL: https://issues.apache.org/jira/browse/ARROW-4452 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-4378) [Plasma] Release objects upon Create

2019-01-25 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4378: - Summary: [Plasma] Release objects upon Create Key: ARROW-4378 URL: https://issues.apache.org/jira/browse/ARROW-4378 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-4285) [Python] Use proper builder interface for serialization

2019-01-17 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4285: - Summary: [Python] Use proper builder interface for serialization Key: ARROW-4285 URL: https://issues.apache.org/jira/browse/ARROW-4285 Project: Apache Arrow

[jira] [Created] (ARROW-4269) [Python] AttributeError: module 'pandas.core' has no attribute 'arrays'

2019-01-15 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4269: - Summary: [Python] AttributeError: module 'pandas.core' has no attribute 'arrays' Key: ARROW-4269 URL: https://issues.apache.org/jira/browse/ARROW-4269

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-13 Thread Philipp Moritz
+1 (non-binding) I didn't do very exhaustive tests, but the full Ray test suite passes based on that commit. On Sun, Jan 13, 2019 at 8:59 PM Wes McKinney wrote: > +1 (binding) > > Ran release verification scripts (for source release) on Ubuntu 14.04 > (with ARROW_HAVE_CUDA=yes) and Windows / Vi

[jira] [Created] (ARROW-4249) [Plasma] Remove reference to logging.h from plasma/common.h

2019-01-13 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4249: - Summary: [Plasma] Remove reference to logging.h from plasma/common.h Key: ARROW-4249 URL: https://issues.apache.org/jira/browse/ARROW-4249 Project: Apache Arrow

[jira] [Created] (ARROW-4217) [Plasma] Remove custom object metadata

2019-01-09 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4217: - Summary: [Plasma] Remove custom object metadata Key: ARROW-4217 URL: https://issues.apache.org/jira/browse/ARROW-4217 Project: Apache Arrow Issue Type

Re: plan for Go implementation of Plasma

2018-12-19 Thread Philipp Moritz
Hey Dustin, Thanks for getting in touch! Here are two additional ways to do it: 5. Native go client library: If Go has support to ship file descriptors over unix domain sockets (which I think it has, see https://github.com/opencontainers/runc/blob/master/libcontainer/utils/cmsg.go) and interact w

TensorFlow, PyTorch, and manylinux1

2018-12-15 Thread Philipp Moritz
Dear all, As some of you know, there is a standard in Python called manylinux ( https://www.python.org/dev/peps/pep-0513/) to package binary executables and libraries into a “wheel” in a way that allows the code to be run on a wide variety of Linux distributions. This is very convenient for Python

[jira] [Created] (ARROW-4025) [Python] TensorFlow/PyTorch arrow ThreadPool workarounds not working in some settings

2018-12-13 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4025: - Summary: [Python] TensorFlow/PyTorch arrow ThreadPool workarounds not working in some settings Key: ARROW-4025 URL: https://issues.apache.org/jira/browse/ARROW-4025

[jira] [Created] (ARROW-4024) [Python] Cython compilation error on cython==0.27.3

2018-12-13 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4024: - Summary: [Python] Cython compilation error on cython==0.27.3 Key: ARROW-4024 URL: https://issues.apache.org/jira/browse/ARROW-4024 Project: Apache Arrow

[jira] [Created] (ARROW-3958) [Plasma] Reduce number of IPCs

2018-12-07 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3958: - Summary: [Plasma] Reduce number of IPCs Key: ARROW-3958 URL: https://issues.apache.org/jira/browse/ARROW-3958 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-3950) [Plasma] Don't force loading the TensorFlow op on import

2018-12-06 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3950: - Summary: [Plasma] Don't force loading the TensorFlow op on import Key: ARROW-3950 URL: https://issues.apache.org/jira/browse/ARROW-3950 Project: Apache

[jira] [Created] (ARROW-3934) [Gandiva] Don't compile precompiled tests if ARROW_GANDIVA_BUILD_TESTS=off

2018-12-03 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3934: - Summary: [Gandiva] Don't compile precompiled tests if ARROW_GANDIVA_BUILD_TESTS=off Key: ARROW-3934 URL: https://issues.apache.org/jira/browse/ARROW-3934 Pr

[jira] [Created] (ARROW-3919) [Python] Support 64 bit indices for pyarrow.serialize and pyarrow.deserialize

2018-11-30 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3919: - Summary: [Python] Support 64 bit indices for pyarrow.serialize and pyarrow.deserialize Key: ARROW-3919 URL: https://issues.apache.org/jira/browse/ARROW-3919

[jira] [Created] (ARROW-3746) [Gandiva] [Python] Make it possible to list all functions registered with Gandiva

2018-11-09 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3746: - Summary: [Gandiva] [Python] Make it possible to list all functions registered with Gandiva Key: ARROW-3746 URL: https://issues.apache.org/jira/browse/ARROW-3746

Re: [ANNOUNCE] New Arrow committers: Romain François, Sebastien Binet, Yosuke Shiro

2018-11-08 Thread Philipp Moritz
Welcome everybody! On Thu, Nov 8, 2018 at 12:57 PM Antoine Pitrou wrote: > > It's nice to have new people onboard. Welcome everyone :-) > > Le 08/11/2018 à 20:56, Wes McKinney a écrit : > > On behalf of the Arrow PMC, I'm happy to announce that Romain > > François, Sebastien Binet, and Yosuke S

Re: [ANNOUNCE] New Arrow PMC member: Krisztián Szűcs

2018-11-08 Thread Philipp Moritz
Congrats and welcome Krisztián! On Thu, Nov 8, 2018 at 11:48 AM Wes McKinney wrote: > The Project Management Committee (PMC) for Apache Arrow has invited > Krisztián Szűcs to become a PMC member and we are pleased to announce > that he has accepted. > > Congratulations and welcome, Krisztián! >

[jira] [Created] (ARROW-3718) [Gandiva] Remove spurious gtest include

2018-11-08 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3718: - Summary: [Gandiva] Remove spurious gtest include Key: ARROW-3718 URL: https://issues.apache.org/jira/browse/ARROW-3718 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-3721) [Gandiva] [Python] Support all Gandiva literals

2018-11-08 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3721: - Summary: [Gandiva] [Python] Support all Gandiva literals Key: ARROW-3721 URL: https://issues.apache.org/jira/browse/ARROW-3721 Project: Apache Arrow Issue

[jira] [Created] (ARROW-3659) Clang Travis build (matrix entry 2) might not actually be using clang

2018-10-30 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3659: - Summary: Clang Travis build (matrix entry 2) might not actually be using clang Key: ARROW-3659 URL: https://issues.apache.org/jira/browse/ARROW-3659 Project

[jira] [Created] (ARROW-3602) [Gandiva] [Python] Add preliminary Cython bindings for Gandiva

2018-10-23 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3602: - Summary: [Gandiva] [Python] Add preliminary Cython bindings for Gandiva Key: ARROW-3602 URL: https://issues.apache.org/jira/browse/ARROW-3602 Project: Apache Arrow

[jira] [Created] (ARROW-3589) [Gandiva] Make it possible to compile gandiva without JNI

2018-10-22 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3589: - Summary: [Gandiva] Make it possible to compile gandiva without JNI Key: ARROW-3589 URL: https://issues.apache.org/jira/browse/ARROW-3589 Project: Apache Arrow

[jira] [Created] (ARROW-3243) [C++] Upgrade jemalloc to version 5

2018-09-16 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3243: - Summary: [C++] Upgrade jemalloc to version 5 Key: ARROW-3243 URL: https://issues.apache.org/jira/browse/ARROW-3243 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-3199) [Plasma] Check for EAGAIN in recvmsg and sendmsg

2018-09-08 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3199: - Summary: [Plasma] Check for EAGAIN in recvmsg and sendmsg Key: ARROW-3199 URL: https://issues.apache.org/jira/browse/ARROW-3199 Project: Apache Arrow

[jira] [Created] (ARROW-3159) [Plasma] Plasma C++ and Python integration test for tensors

2018-09-01 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3159: - Summary: [Plasma] Plasma C++ and Python integration test for tensors Key: ARROW-3159 URL: https://issues.apache.org/jira/browse/ARROW-3159 Project: Apache Arrow

[jira] [Created] (ARROW-3157) [C++] Improve buffer creation for typed data

2018-09-01 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3157: - Summary: [C++] Improve buffer creation for typed data Key: ARROW-3157 URL: https://issues.apache.org/jira/browse/ARROW-3157 Project: Apache Arrow Issue

[jira] [Created] (ARROW-3116) [Plasma] Add "ls" to object store

2018-08-24 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3116: - Summary: [Plasma] Add "ls" to object store Key: ARROW-3116 URL: https://issues.apache.org/jira/browse/ARROW-3116 Project: Apache Arrow

[jira] [Created] (ARROW-3105) [Plasma] Improve flushing error message

2018-08-21 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3105: - Summary: [Plasma] Improve flushing error message Key: ARROW-3105 URL: https://issues.apache.org/jira/browse/ARROW-3105 Project: Apache Arrow Issue Type

Re: [VOTE] Combining Arrow C++ development process with Apache Parquet C++

2018-08-21 Thread Philipp Moritz
+1 for the monorepo plan and push access to Parquet C++ committers -- Philipp. On Tue, Aug 21, 2018 at 3:12 PM, Antoine Pitrou wrote: > > +1 from me. > > Regards > > Antoine. > > > Le 21/08/2018 à 18:49, Wes McKinney a écrit : > > hi folks, > > > > As discussed on the Arrow and Parquet mailing

Re: [ANNOUNCE] New Arrow PMC member: Antoine Pitrou

2018-08-21 Thread Philipp Moritz
Awesome, congratulations and welcome! On Tue, Aug 21, 2018 at 1:28 PM, Phillip Cloud wrote: > Welcome! Great to have you on the team! > > On Tue, Aug 21, 2018 at 4:27 PM Wes McKinney wrote: > > > The Project Management Committee (PMC) for Apache Arrow has invited > > Antoine Pitrou to become a

Re: [VOTE] Accept donation of Gandiva to Apache Arrow

2018-08-16 Thread Philipp Moritz
+1 On Thu, Aug 16, 2018, 10:02 AM Siddharth Teotia wrote: > +1 > > On Thu, Aug 16, 2018 at 9:57 AM, Julian Hyde wrote: > > > +1 > > On Thu, Aug 16, 2018 at 8:56 AM Wes McKinney > wrote: > > > > > > Dear all, > > > > > > The developers of Gandiva, an LLVM-based vectorized expression > > > evalu

[jira] [Created] (ARROW-3062) [Python] Extend fast libtensorflow_framework.so compatibility workaround to Python 2.7

2018-08-15 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3062: - Summary: [Python] Extend fast libtensorflow_framework.so compatibility workaround to Python 2.7 Key: ARROW-3062 URL: https://issues.apache.org/jira/browse/ARROW-3062

[jira] [Created] (ARROW-3018) [Plasma] Improve random ObjectID generation

2018-08-07 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3018: - Summary: [Plasma] Improve random ObjectID generation Key: ARROW-3018 URL: https://issues.apache.org/jira/browse/ARROW-3018 Project: Apache Arrow Issue

[jira] [Created] (ARROW-2976) [Python] Directory in pyarrow.get_library_dirs() on Travis doesn't contain libarrow.so

2018-08-03 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2976: - Summary: [Python] Directory in pyarrow.get_library_dirs() on Travis doesn't contain libarrow.so Key: ARROW-2976 URL: https://issues.apache.org/jira/browse/ARROW

[jira] [Created] (ARROW-2975) [Plasma] TensorFlow op: Compilation only working if arrow found by pkg-config

2018-08-03 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2975: - Summary: [Plasma] TensorFlow op: Compilation only working if arrow found by pkg-config Key: ARROW-2975 URL: https://issues.apache.org/jira/browse/ARROW-2975

[jira] [Created] (ARROW-2954) [Plasma] Store object_id only once in object table

2018-07-31 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2954: - Summary: [Plasma] Store object_id only once in object table Key: ARROW-2954 URL: https://issues.apache.org/jira/browse/ARROW-2954 Project: Apache Arrow

[jira] [Created] (ARROW-2953) [Plasma] Store memory usage

2018-07-31 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2953: - Summary: [Plasma] Store memory usage Key: ARROW-2953 URL: https://issues.apache.org/jira/browse/ARROW-2953 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-2940) [Python] Import error with pytorch 0.3

2018-07-30 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2940: - Summary: [Python] Import error with pytorch 0.3 Key: ARROW-2940 URL: https://issues.apache.org/jira/browse/ARROW-2940 Project: Apache Arrow Issue Type

Re: [DISCUSS] Solutions for improving the Arrow-Parquet C++ development morass

2018-07-29 Thread Philipp Moritz
I do not claim to have insight into parquet-cpp development. However, from our experience developing Ray, I can say that the monorepo approach (for Ray) has improved things a lot. Before we tried various schemes to split the project into multiple repos, but the build system and test infrastructure

[jira] [Created] (ARROW-2920) [Python] Segfault with pytorch 0.4

2018-07-26 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2920: - Summary: [Python] Segfault with pytorch 0.4 Key: ARROW-2920 URL: https://issues.apache.org/jira/browse/ARROW-2920 Project: Apache Arrow Issue Type: Bug

Re: [DISCUSS] Contribution of Gandiva to Apache Arrow

2018-07-25 Thread Philipp Moritz
+1 on merging it and also agreed with Uwe that we will need to deal with LLVM version conflicts. In addition it would be good to come up with a plan on how it can be useful for other DataFrame open source projects. Having end-to-end applications that let people profit from this code will help adopt

Re: Plasma Java API to read RecordBatch from Python process

2018-07-20 Thread Philipp Moritz
Hey Jieun, Currently we can only transfer RecordBatch objects using Plasma between C++ and Python unfortunately. I just opened a JIRA for doing it with Java too in https://issues.apache.org/jira/browse/ARROW-2892. The necessary pieces are there (in particular there is a low level API to access Pl

[jira] [Created] (ARROW-2892) [Plasma] Implement interface to get Java arrow objects from Plasma

2018-07-20 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2892: - Summary: [Plasma] Implement interface to get Java arrow objects from Plasma Key: ARROW-2892 URL: https://issues.apache.org/jira/browse/ARROW-2892 Project: Apache

[jira] [Created] (ARROW-2890) [Plasma] Make Python PlasmaClient.release private

2018-07-20 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2890: - Summary: [Plasma] Make Python PlasmaClient.release private Key: ARROW-2890 URL: https://issues.apache.org/jira/browse/ARROW-2890 Project: Apache Arrow

Re: Pyarrow Plasma client.release() fault

2018-07-20 Thread Philipp Moritz
Also you should avoid calling release directly, because it will also be called automatically here: https://github.com/apache/arrow/blob/master/python/pyarrow/_plasma.pyx#L222 Instead, you should call "del buffer" on the PlasmaBuffer. I'll submit a PR to make the release method private. The only

[jira] [Created] (ARROW-2866) [Plasma] TensorFlow op: Investiate outputting multiple output Tensors for the reading op

2018-07-16 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2866: - Summary: [Plasma] TensorFlow op: Investiate outputting multiple output Tensors for the reading op Key: ARROW-2866 URL: https://issues.apache.org/jira/browse/ARROW-2866

Re: Passing Arrow object across language

2018-07-16 Thread Philipp Moritz
Hey Yuri, you can use the Arrow IPC mechanism to do this: - https://github.com/apache/arrow/blob/master/format/IPC.md - Python: https://arrow.apache.org/docs/python/ipc.html - C++: https://arrow.apache.org/docs/cpp/namespacearrow_1_1ipc.html - For Java, see the org.apache.arrow.vector.ipc namespa

[jira] [Created] (ARROW-2811) [Python] Test serialization for determinism

2018-07-07 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2811: - Summary: [Python] Test serialization for determinism Key: ARROW-2811 URL: https://issues.apache.org/jira/browse/ARROW-2811 Project: Apache Arrow Issue

[jira] [Created] (ARROW-2805) [Python] TensorFlow import workaround not working with tensorflow-gpu if CUDA is not installed

2018-07-06 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2805: - Summary: [Python] TensorFlow import workaround not working with tensorflow-gpu if CUDA is not installed Key: ARROW-2805 URL: https://issues.apache.org/jira/browse/ARROW-2805

[jira] [Created] (ARROW-2803) [C++] Put hashing function into src/arrow/util

2018-07-06 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2803: - Summary: [C++] Put hashing function into src/arrow/util Key: ARROW-2803 URL: https://issues.apache.org/jira/browse/ARROW-2803 Project: Apache Arrow Issue

[jira] [Created] (ARROW-2794) [Plasma] Add Delete method for multiple objects

2018-07-04 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2794: - Summary: [Plasma] Add Delete method for multiple objects Key: ARROW-2794 URL: https://issues.apache.org/jira/browse/ARROW-2794 Project: Apache Arrow Issue

[jira] [Created] (ARROW-2788) [Plasma] Defining Delete semantics

2018-07-03 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2788: - Summary: [Plasma] Defining Delete semantics Key: ARROW-2788 URL: https://issues.apache.org/jira/browse/ARROW-2788 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-2758) [Plasma] Use Scope enum in Plasma

2018-06-27 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2758: - Summary: [Plasma] Use Scope enum in Plasma Key: ARROW-2758 URL: https://issues.apache.org/jira/browse/ARROW-2758 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-2757) [Plasma] Huge pages test failing

2018-06-27 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2757: - Summary: [Plasma] Huge pages test failing Key: ARROW-2757 URL: https://issues.apache.org/jira/browse/ARROW-2757 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-2737) [Plasma] Integrate TensorFlow Op with arrow packaging scripts

2018-06-24 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2737: - Summary: [Plasma] Integrate TensorFlow Op with arrow packaging scripts Key: ARROW-2737 URL: https://issues.apache.org/jira/browse/ARROW-2737 Project: Apache Arrow

  1   2   3   >