Just following up here - what's the status? It looks like there's some
unaddressed comments on the PR?
On Tue, Nov 23, 2021, at 13:54, Micah Kornfield wrote:
> Sorry I just took a closer look and left some comments. I think the one
> substantive issue, is the document linked talks about differen
Sorry I just took a closer look and left some comments. I think the one
substantive issue, is the document linked talks about different
length columns in the Bag, and this isn't mentioned in the flatbuffers?
Could you comment/update the documentations in flatbuffers accordingly?
Thanks,
Micah
On
Thanks for putting that up.
It doesn't look like there's been too much discussion here. If people agree
it's useful, maybe the next step is to draft an implementation in Java or C++
for feedback? There was some discussion on the use cases in the document, do we
feel like we need to clarify that
I put the draft up here: https://github.com/apache/arrow/pull/11646
Thanks.
On Mon, Nov 8, 2021 at 1:57 PM David Li wrote:
> Hey Nate,
>
> Thanks for doing this! Would you be interested in putting that commit up
> as a draft PR for discussion? I think we can discuss there.
>
> I'm not sure anyo
Hey Nate,
Thanks for doing this! Would you be interested in putting that commit up as a
draft PR for discussion? I think we can discuss there.
I'm not sure anyone is actively working on RLE or other encoding schemes at the
moment.
-David
On Mon, Nov 8, 2021, at 13:19, Nate Bauernfeind wrote:
I've written up the ColumnBag proposal addressing items 1 and 2 on the
list. I'm open to any and all feedback/suggestions.
I'd be happy to add item 3 (binary metadata) to the proposed change set.
Let me know if you want me to whip up the initial suggestion for that
version (and whether or not to k
I'm still interested in RLE related effort, but not sure about my available
bandwidth (which is why I haven't made more of an effort there).
On Tue, Aug 3, 2021 at 6:00 PM Wes McKinney wrote:
> Another Flatbuffers/Message.fbs project we should rekindle soon, in
> addition to the schema evolution
Another Flatbuffers/Message.fbs project we should rekindle soon, in
addition to the schema evolution/replacement question which has been
raised with Flight, is that of sparse/compressed data (e.g. RLE). I
have a vacation plus some travel coming up so won't be able to devote
meaningful attention to
Hey Nate,
For the first two points, semantically I'm tempted to think of it more like the
ability to send a "bag of columns" according to some schema (and hence columns
could have differing lengths or even be absent). This could be a new structure
alongside a record batch, which is semantically
Wes suggested that maybe there are enough new ideas that it may make sense
to evolve-past the existing structures rather than to bolt-on new
functionality. I would like to learn what requirements exist should new
structures be adopted, and if applicable, would like to turn this into a
full POC prop
10 matches
Mail list logo
