barbarj commented on issue #24277:
URL: https://github.com/apache/datafusion/issues/24277#issuecomment-5259698244
I'll implement this soon, but at high-level, the design I'm thinking is:
### Serialization
For evaluation purposes, we only need the hashes in the `HashMap` case. The
protobuf then looks like, with the contents of `ArrayMapNode` to be determined:
```protobuf
message PhysicalHashTableLookupExprNode {
repeated PhysicalExprNode on_columns = 1;
uint64 seed = 2;
string description = 3;
oneof map {
HashSetMapNode hash_map = 4;
ArrayMapNode array_map = 5;
}
}
message HashSetMapNode {
repeated fixed64 build_hashes = 1;
}
```
We add a `hashes` access method:
```rust
pub trait JoinHashMapType {
pub fn hashes(&self) -> Vec<u64>;
}
```
### Deserialization
For the `ArrayMapNode` case, we just deserialize to the `ArrayMap` variant
of `HashTableLookupExpr`. For the `HashSetMapNode` case, we add a new impl of
`JoinHashMapType` with `contain_hashes`/`len`/`is_empty` implemented. It will
return internal errors from the build-side mutation and get_matched_indices
methods. This will allow it be evaluated after deserialization but not updated.
(Will document that this should be constructed during deserialization only and
is an immutable snapshot of the expression).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
