OK, great, that clarifies a lot - I didn't appreciate how flexible the
interpretation can be. Thanks for the doc pointer as well.
I'll power onwards with the experiments. If I encounter areas of Flight
that are still a bit rough around the edges - being in beta - then we
might be able to contribut
Hey Joris,
Your plan sounds right for Flight. As for semantics:
The descriptor and ticket format are mostly application defined. For
instance, I think some places (Dremio?) just put a raw SQL query as
the "cmd" of a descriptor; putting serialized JSON or Protobuf is also
certainly fine.
I'd say
Hi,
As discussed on the mailing list [1] I would like to add a "bit width"
field to our Decimal metadata to allow for supporting different
Decimal physical sizes other than 128-bit (where 32- and 64-bit
representations are relatively common) without requiring that we add a
new value to the Type en
Hello,
I'm interested in using Flight for serving large amounts of data in a
parallelised manner, and just building some Python prototypes, based on
https://github.com/apache/arrow/blob/apache-arrow-0.17.1/python/examples/flight
In my use-case, we'd have a bunch of worker servers, serving a numbe
If you configured CMake to build tests (-DARROW_BUILD_TESTS=ON) and
install locally, there should be a `libarrow_testing.so` that you need
to link against. What I meant is that this library is _not_ part of
pip/conda/dpkg/rpm.
François
Yes, I figured it out already by seeing "undefined reference".
Do you know if there is any way to build Arrow in a way that testing
is part of the library?
For some reason, in the installation directory I see random.h but when
I inspect libarrow.a with nm I don't see definitions of the functions.
It doesn't support it yet, but you can probably trivially add a method
for it. Time (timestamp, interval) types might require more changes
since the type factories are not always parameter-less. A small note,
this functionality (random generation) is part of the testing shared
library. AFAIK this i
Yeah, I wrote some code which does the thing (cannot use the latest
arrow version so far). The only missing feature is Date32/Date64.
Are they supported as part of ArrayOf or not implemented because it
might be possible (guess) to convert uint32 to Date32 for example?
On Mon, Jun 22, 2020 at 7:31
I'm in favor of the list of enums myself -- it seems like it will be
easier to work with and less error prone in general. Storage space
with this should not be an issue.
On Tue, Jun 23, 2020 at 7:31 AM Antoine Pitrou wrote:
>
>
>
> Le 23/06/2020 à 06:02, Micah Kornfield a écrit :
> >>
> > A bit-
Hey Talal,
Try adding the `kr.motd.maven` extension as per
https://github.com/rymurr/flight-spark-source/blob/master/pom.xml#L34
That should populate the ${os.detected.arch} variable required to pull in
the netty native dependencies.
Best,
Ryan Murray | Principal Consulting Engineer
+44754085
Le 23/06/2020 à 06:02, Micah Kornfield a écrit :
>>
> A bit-field could work, but it is a little easier to mess-up (e.g.
> endianness).
I don't think endianness can be an issue. A bitfield would have the
same representation as any "long" Flatbuffers field.
> I would lean towards a list of e
I think for Fllight, it'd be good to work out some negotiation that
can preserve compatibility with the existing protocol as well, e.g. by
taking advantage of headers. For some methods, like DoGet, the data
flow isn't bidirectional so negotiating via headers (or earlier in the
request flow, such as
Hello,
I have been trying to use apache arrow flight. But I am having trouble
using it.
It gives me following error:
Missing artifact io.netty:netty-transport-native-unix-common:jar:${
os.detected.name}-${os.detected.arch}:4.1.27.Final pom.xml
org.apache.arrow
arrow-flight
0.15.1
T
Arrow Build Report for Job nightly-2020-06-23-0
All tasks:
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-06-23-0
Failed Tasks:
- centos-7-aarch64:
URL:
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-06-23-0-travis-centos-7-aarch64
- debian-stre
FTR: We can use the latest(!) clang for all platform for conda and wheels. It
isn't probably even that much of a complicated setup.
On Mon, Jun 22, 2020, at 5:42 PM, Francois Saint-Jacques wrote:
> We should aim to improve the performance of the most widely used
> *default* packages, which are p
15 matches
Mail list logo
