For the index vs pointer question - DuckDB went with pointers as they are more
flexible, and DuckDB was designed to consume data (and strings) from a wide
variety of formats in a wide variety of languages. Pointers allows us to easily
zero-copy from e.g. Python strings, R strings, Arrow strings,
Hi -
I have a data set which is mostly a 2D table, however one column
(called Attributes) contains a List of Structs in each cell. Each
Struct has three fields: Attribute Tag, Attribute Type and Attribute
Value.
The definition of the Attributes Field is:
/**
* Attribute Tag - Two character tag.
Hi,
I recently started my first PR for Arrow (Java) but I need someone to
approve the check workflows before I can proceed.
PR is
https://github.com/apache/arrow/pull/15106
Thanks,
Mark
I would agree with this.
I’ve been working with the GO Arrow library last few weeks, and took a while to
get head around it all / how to use etc.
Even then not sure i’ve got it right.
Usage examples would be great.
Regards
Mark
> On Oct 14, 2020, at 4:08 PM, Fernando Herrera
>
Unclear if this is needed or not.
It would be ideal if even the streaming format coming in, was based on Arrow
concepts / datatypes / organization etc.
More thinking required.
Regards
Mark.
On 9/10/20, 5:25 AM, "Fan Liya" wrote:
+1 for introducing Arrow in streaming pro
method for appending inbound realtime sensor data into the in-memory model.
Still thinking about that one.
Regards
Mark.
[1] Large in obviously relative: In this case, a single plot may have 20-50
separate time series, each with between 20k to 10 million points each.
[2] The data
d, blocks could come out of a data base/source, through the data
service, across the wire (flight) and land in the consuming applications
memory without ever being decompressed or processed until final use.
Crazy thought ?
Regards
Mark.
[1]: https://www.vldb.org/pvldb/vol8/p1816-teller.pdf
taset
and visualization choices. So far arrow seems a good choice rather than any
'roll your own', and it will be nice to use same format on Client side as well
as in the Server system.
My use case is primarily 'Get', consuming large datasets for visualization. I
doubt I
Thanks Wes,
I'll likely work on that once I get my head around Arrow in general and confirm
will use for the project.
Considerations for how to account for the streaming append problem to an
otherwise immutable dataset is current concern. Still thinking through that.
Regards
reat if it can.
Regards
Mark.
-Original Message-
From: Sebastien Binet
Sent: Wednesday, August 12, 2020 1:53 PM
To: dev@arrow.apache.org
Subject: Re: Arrow Flight + Go, Arrow for Realtime
Mark,
AFAIK, nobody's actively working on Arrow-Flight for Go (I think somebody
started that w
ds to
'grow' as new data arrives, often at high speed).
Not language specific, just trying to understand the right pattern for using
Arrow for this, and couldn't' find much in the docs.
Regards
Mark.
Mark Waddle created ARROW-8967:
--
Summary: [Python] [Parquet] Table.to_pandas() fails to convert
valid TIMESTAMP_MILLIS fails to convert to pandas timestamp
Key: ARROW-8967
URL: https://issues.apache.org/jira/browse
Mark Hildreth created ARROW-8648:
Summary: [Rust] Optimize Rust CI Build Times
Key: ARROW-8648
URL: https://issues.apache.org/jira/browse/ARROW-8648
Project: Apache Arrow
Issue Type
Mark Hildreth created ARROW-8637:
Summary: Resolve Issues with `prettytable-rs` dependency
Key: ARROW-8637
URL: https://issues.apache.org/jira/browse/ARROW-8637
Project: Apache Arrow
Issue
Mark Harris created ARROW-8608:
--
Summary: Update vendored mpark/variant.h to latest to fix NVCC
compilation issues
Key: ARROW-8608
URL: https://issues.apache.org/jira/browse/ARROW-8608
Project: Apache
Mark Hildreth created ARROW-8590:
Summary: [Rust] Use Arrow pretty print utility in DataFusion
Key: ARROW-8590
URL: https://issues.apache.org/jira/browse/ARROW-8590
Project: Apache Arrow
Mark Keller created ARROW-8015:
--
Summary: Releasing pyarrow 0.16.0 for Windows Python 3.5
Key: ARROW-8015
URL: https://issues.apache.org/jira/browse/ARROW-8015
Project: Apache Arrow
Issue Type
-5679 but this seems to be open
for the time being.
Could you please let me know if you are planning on releasing this, or if
it’s gone for good?
--
Mark Keller
Software Engineer
mobile +1 650 484 6154 <+16504846154>
email mark.kel...@snowflake.com
Snowflake Inc.
450 Concar Drive
San
Mark Litwintschik created ARROW-6815:
Summary: Timestamps saved via Pandas and PyArrow unreadable in
Hive and Presto
Key: ARROW-6815
URL: https://issues.apache.org/jira/browse/ARROW-6815
Project
Mark Harris created ARROW-6205:
--
Summary: ARROW_DEPRECATED warning when including io/interfaces.h
from CUDA (.cu) source
Key: ARROW-6205
URL: https://issues.apache.org/jira/browse/ARROW-6205
Project
I already made sure that Matei is aware of this thread. He seemed
interested in talking with key Arrow developers.
On Thu, Dec 15, 2016 at 10:49 AM, Julian Hyde wrote:
> I think someone should reach out to Matei and Shoumik, and see if they
> would like to collaborate. Wes, would you like to do
21 matches
Mail list logo
