Hi Anton,
Another possibility is to look at our C++ CSV reader and parser (in
src/arrow/csv). It's the only piece of Arrow that uses non-trivial
multi-threading right now (with tasks spawning new tasks dynamically,
see InferringColumnBuilder). It's based on the ThreadPool and TaskGroup
APIs (i
Hi Jed,
Le 03/05/2019 à 05:47, Jed Brown a écrit :
> I would caution to please not commit to the MKL/BLAS model in which the
> library creates threads internally. It's a disaster for managing
> oversubscription and affinity issues among groups of threads and/or
> multiple processes (e.g., MPI).
Antoine Pitrou created ARROW-5252:
-
Summary: [C++] Change variant implementation
Key: ARROW-5252
URL: https://issues.apache.org/jira/browse/ARROW-5252
Project: Apache Arrow
Issue Type: Improv
Francois Saint-Jacques created ARROW-5253:
-
Summary: [C++] external Snappy fails on Alpine
Key: ARROW-5253
URL: https://issues.apache.org/jira/browse/ARROW-5253
Project: Apache Arrow
Antoine Pitrou writes:
> Hi Jed,
>
> Le 03/05/2019 à 05:47, Jed Brown a écrit :
>> I would caution to please not commit to the MKL/BLAS model in which the
>> library creates threads internally. It's a disaster for managing
>> oversubscription and affinity issues among groups of threads and/or
>>
Thanks for your answers,
> -Original Message-
> From: Antoine Pitrou [mailto:anto...@python.org]
> Sent: Friday, May 3, 2019 03:54
> Le 03/05/2019 à 05:47, Jed Brown a écrit :
> > I would caution to please not commit to the MKL/BLAS model in which
I'm actually talking about threading laye
Le 03/05/2019 à 17:57, Jed Brown a écrit :
>
>>> The library is then free to use constructs like omp taskgroup/taskloop
>>> as granularity warrants; it will never utilize threads that the
>>> application didn't explicitly give it.
>>
>> I don't think we're planning to use OpenMP in Arrow, though
Hi Sidd,
Does setting the system property io.netty.tryReflectionSetAccessible to
true have any other adverse effect other than those warnings during build?
Bryan
On Thu, May 2, 2019 at 8:43 PM Jacques Nadeau wrote:
> I'm onboard with this change.
>
> On Fri, Apr 26, 2019 at 2:14 AM Siddharth T
"Malakhov, Anton" writes:
>> > the library creates threads internally. It's a disaster for managing
>> > oversubscription and affinity issues among groups of threads and/or
>> > multiple processes (e.g., MPI).
>
> This is exactly what I'm talking about referring as issues with threading
> compo
No need for apologies, Wes, I appreciate you keeping this on your radar.
I've made the changes and have pushed them to the PR branch. You can begin
your review when you get the chance.
--TPB
On Thu, May 2, 2019 at 3:32 PM Wes McKinney wrote:
> + Parquet dev list
>
> Thanks Tim for working on
hi David -- would you like to open a PR and corresponding JIRA issue
for discussion? We might want to hold a vote to formalize the
extension type mechanism (and to fix the metadata names -- I agree
that having an ARROW namespace would be better than what we are doing
now)
On Thu, May 2, 2019 at 7:
David Li created ARROW-5254:
---
Summary: [Flight][Java] DoAction does not support result streams
Key: ARROW-5254
URL: https://issues.apache.org/jira/browse/ARROW-5254
Project: Apache Arrow
Issue Type
David Li created ARROW-5255:
---
Summary: [Java] Implement user-defined data types API
Key: ARROW-5255
URL: https://issues.apache.org/jira/browse/ARROW-5255
Project: Apache Arrow
Issue Type: Bug
Sure, I've created https://issues.apache.org/jira/browse/ARROW-5255.
PR: https://github.com/apache/arrow/pull/4251
I'm not sure if what I'm doing with my Vector subclass is quite right,
but we'd especially like this in Java, so happy to work through any
feedback.
Also, as part of this discussion
Kouhei Sutou created ARROW-5256:
---
Summary: [Packaging][deb] Failed to build with LLVM 7.1.0
Key: ARROW-5256
URL: https://issues.apache.org/jira/browse/ARROW-5256
Project: Apache Arrow
Issue Typ
Wes McKinney created ARROW-5257:
---
Summary: [Website] Update site to use "official" Apache Arrow
logo, add clearly marked links to logo
Key: ARROW-5257
URL: https://issues.apache.org/jira/browse/ARROW-5257
I've just reviewed the format and C++ changes in
https://github.com/apache/arrow/pull/3644 which look good to me modulo
minor comments.
Can someone take a look at the Java changes soon so we move this
toward completion?
One question came up of whether "DurationInterval" is the name we
want. It mi
Hello!
I'm using the Java API for Arrow and am finding some ambiguity between the
length field in a RecordBatch and the "byte-width-adjusted" length field in
a Buffer.
As per https://arrow.apache.org/docs/format/Metadata.html under the "Record
data headers" section:
"A record batch is a collectio
18 matches
Mail list logo
