Domink's point is that the IPC reader currently first writes the whole
thing into a Vec, and then copies all of that to buffers using
IPC::Buffer offsets and lengths. Thus, it performs 2 memcopies of the whole
data and needs to hold 2x the required memory (the Vec and the
arrow::Buffers).
I notice
Thank you for filing the ticket.
I wonder if you mean this reader:
https://docs.rs/arrow/3.0.0/arrow/ipc/reader/struct.FileReader.html#method.try_new
If so, while it is called a `FileReader` I think that is somewhat
misleading. It requires something that implements `std::io::Read` -- which
`&[u8]
I just remembered a bigger issue I ran into. I wanted to read from IPC but
I don’t have a file. I do have the data as [u8] already. The current API
incurs more copies than necessary (I think) and therefore the performance
of reading IPC is worse than in JS. (
https://issues.apache.org/jira/project
I am looking forward to speaking with you then. I’ll talk about the
motivation.
My experience with the library has been good. I ran into a few limitations
that I filed Jiras for. I struggled a bit with some of the error handling
and Arc types but that’s probably because I am now very experienced
Hi Dominik,
That sounds really interesting -- thank you for the offer
I for one would enjoy seeing a demo and suggest that 10 minutes might be a
good length. The next call (details are also on the announcement [1]) is
scheduled for Wednesday March 10, 2021 at 09:00 PST / 12:00 EST / 17:00
UTC. T
Hi Dominic,
I would be interested in a demo. Im curious to see your implementation and
what advantages you have seen over javascript
thanks
Fernando
On Thu, Feb 25, 2021 at 10:39 PM Dominik Moritz wrote:
> Hello Rust Arrow Devs,
>
> I have been working on a wasm version of Arrow using the Rust
Hello Rust Arrow Devs,
I have been working on a wasm version of Arrow using the Rust library (
https://github.com/domoritz/arrow-wasm). I was wondering whether you would
be interested in having me demo it in the Arrow Rust sync call. If so, when
would be the next one and how much time would you wa
