comphead closed issue #4658: ExternalSorter Fails to Spill Dictionaries
URL: https://github.com/apache/datafusion/issues/4658
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
davidhewitt commented on issue #4658:
URL: https://github.com/apache/datafusion/issues/4658#issuecomment-2680773931
Ok great, I'll work on that today 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to g
tustvold commented on issue #4658:
URL: https://github.com/apache/datafusion/issues/4658#issuecomment-2679171994
> I assume by row format you mean
[arrow-row](https://arrow.apache.org/rust/arrow_row/index.html), however it's
not clear to me if there's a standard way to serialize these to a
davidhewitt commented on issue #4658:
URL: https://github.com/apache/datafusion/issues/4658#issuecomment-2678889088
It looks like an alternative to using the row format in datafusion might be
to support delta-encoded dictionaries in `arrow-rs`.
https://github.com/apache/arrow-rs/issues/6783
davidhewitt commented on issue #4658:
URL: https://github.com/apache/datafusion/issues/4658#issuecomment-2678487900
@tustvold we're hitting this issue in present-day at Pydantic. It's possibly
been made more relevant by the [recent fixes to spill
heuristics](https://github.com/apache/datafu
