Hello Rust Arrow Devs,
I have been working on a wasm version of Arrow using the Rust library (
https://github.com/domoritz/arrow-wasm). I was wondering whether you would
be interested in having me demo it in the Arrow Rust sync call. If so, when
would be the next one and how much time would you wa
Alright, there seems to be enough consensus around giving this a shot.
Since this sounds like a procedural change, I'll start a [VOTE] thread.
Neal
On Thu, Feb 25, 2021 at 9:36 AM Micah Kornfield
wrote:
> I think we should give this a try and see how it works out.
>
> On the other hand, indepen
Hi Fernando.
There are two independent use cases for input_file_name():
1. A lot of data still comes in from CSV and we process to Parquet after
careful data typing rules have been applied. We then materialize the
input_file_name to a column in the Parquet to be able to trace lineage of
the data.
Yeap... that did it. I had to update rust for it to work
On Thu, Feb 25, 2021 at 5:44 PM Daniël Heres wrote:
> I think you might have an older version of rust. 1.49 has stabilized the
> feature.
>
> I think we should test and specify a minimum rust version in our crates in
> order to catch those
I think you might have an older version of rust. 1.49 has stabilized the
feature.
I think we should test and specify a minimum rust version in our crates in
order to catch those errors.
On Thu, Feb 25, 2021, 18:36 Fernando Herrera
wrote:
> It was a simple "cargo test" in the rust folder
>
> err
I think we should give this a try and see how it works out.
On the other hand, independent version numbers could also be desirable in
> certain cases? Assume that following this proposal Rust makes a minor 3.1.0
> release, and then afterwards there is a Python regression we want to fix.
> The next
It was a simple "cargo test" in the rust folder
error[E0658]: binding by-move and by-ref in the same pattern is unstable
> --> arrow/src/json/reader.rs:71:58
>|
> 71 | (InferredType::Array(s), InferredType::Array(o)) => {
>| - by-ref pattern h
Could you possibly provide the exact error message / steps to reproduce the
problem you are seeing? I wonder if some dependent library pushed an
incompatible upgrade or something like that
On Thu, Feb 25, 2021 at 12:23 PM Fernando Herrera <
fernando.j.herr...@gmail.com> wrote:
> Today I was runni
I am also supportive of Neil's proposal -- thank you for writing it up.
On Thu, Feb 25, 2021 at 3:35 AM Joris Van den Bossche <
jorisvandenboss...@gmail.com> wrote:
> On Wed, 24 Feb 2021 at 23:03, Neal Richardson >
> wrote:
>
> > Hi all,
> > We've had some discussion about ways to reduce the cos
Today I was running Rust tests on my fork in master and got an error
message regarding flight. The message reads:
binding by-move and by-ref in the same pattern is unstable
Does anyone know what is wrong with flight?
Thanks
Fernando
Hi Antoine,
I struggled a bit to put all my thoughts in an email format, that will be
easily consumable.
So, I wrote up a github markdown to add some more detail to the issue, we are
facing.
Could you take a look, and let us know your thoughts?
https://github.com/mathworks/matlab-arrow-support-
Hi Ying,
If you manage to get the debugger to work nicely with VScode, could you
share the instructions on how to do set this up? I usually just use gdb,
but that can be a bit crude, would love to use a visual debugger sometimes.
Regards,
Maarten Breddels
Software engineer / consultant / data sc
Hi Ying,
You could also try the --gtest_break_on_failure flag (or equivalently the
GTEST_BREAK_ON_FAILURE=1 environment).
Ben
On Thu, Feb 25, 2021, 05:00 Antoine Pitrou wrote:
>
> Hi Ying,
>
> Have you tried using the given test executable as a debug target?
> (something like build/debug/arro
I see. You are storing the file name when reading a json, csv, and parquet
file.
Just out of curiosity, how would you use the file name in spark?
Are you using it for file statistics?
On Thu, Feb 25, 2021 at 9:36 AM Mike Seddon wrote:
> Hi Fernando,
>
> After Andrew's reply I have moved the fil
Arrow Build Report for Job nightly-2021-02-25-0
All tasks:
https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-25-0
Failed Tasks:
- conda-linux-gcc-py36-aarch64:
URL:
https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-25-0-drone-conda-linux
Hi Ying,
Have you tried using the given test executable as a debug target?
(something like build/debug/arrow-orc-writer.exe)
Also, it has various command-line options to change behaviour and narrow
down the tests (I suggest trying --gtest_filter=...).
Regards
Antoine.
Le 25/02/2021 à 09:
Hi Fernando,
After Andrew's reply I have moved the filename metadata into the Schema and
actually changed the ScalarFunctionImplementation signature to: Arc Result + Send + Sync>;
I have a functional (WIP) repo already:
https://github.com/seddonm1/arrow/compare/master...seddonm1:input-file
I ne
Hi Mike,
I've been thinking how you are considering adding metadata to the
RecordBatch.
The struct it is now defined as
pub struct RecordBatch {
> schema: SchemaRef,
> columns: Vec>,
> }
Are you suggesting something like this?
pub struct RecordBatch {
> schema: SchemaRef,
> co
On Wed, 24 Feb 2021 at 23:03, Neal Richardson
wrote:
> Hi all,
> We've had some discussion about ways to reduce the cost of releasing and
> ways to allow maintainers of subprojects to make more frequent maintenance
> releases. Specifically, see these two recent mailing list threads:
>
> *
>
> htt
Hi,
To facilitate faster debugging I’d like to integrate make unittest debugging
into VSCode (on Mac) so that when I run a test that might show some bugs
breakpoints can stop the execution so that I can dig around a bit. Does anyone
know how that can be done? I know it is a stupid question but
20 matches
Mail list logo
