alamb commented on issue #11268:
URL: https://github.com/apache/datafusion/issues/11268#issuecomment-2211125762
> It's a good idea. I also considered how to implement SQL syntax to support
this scalar function, but I had no idea how to do it. It seems
`UserDefinedSQLPlanner` is what I need. It's helpful for me.
>
> I will try to implement the syntax for `MAP { x : 1 }`. I guess I can
follow how the `RawDictionaryExpr` works.
👍 ❤️
> Besides the SQL syntax, I'm considering what the scalar function would
look like. Should we consider the data layout of `arrow::MapArray`? If needed,
I think its parameters could be different from DuckDB's `map_from_entries`
because the data layout of `arrow::MapArray` allows many slots for a map value,
like:
>
> ```
> [{a:1,b:2},{c:3},{d:4},{}] `
> ```
>
> The function would be used like this:
>
> ```
> map_from_entries_array([a, 1, b, 2], [c, 3], [d,4], [])
> ```
>
> If we don't need to consider it, we could only allow one element for
`MapArray`. It would be similar to how `named_struct()` is used:
I think following the model for `named_struct` makes the most sense here
From my reading of
https://docs.rs/arrow/latest/arrow/array/struct.MapArray.html the idea is that
each element of a `MapArray` is itself a map with some arbitrary number of
key/value pairs
I think we could use
https://docs.rs/arrow/latest/arrow/array/builder/struct.MapBuilder.html to
create the actual array/value
## Syntax Proposal
Maybe we could use a function like `make_map` to follow the naming
convention of `make_array`:
https://datafusion.apache.org/user-guide/sql/scalar_functions.html#make-array
```sql
-- create {'foo': 1, 'bar': 2}
select make_map('foo', 1, 'bar', 2);
```
## Implementation suggestion
I recommend doing this in 2 PRs:
1. Implement test the function (e.g. `make_map`)
2. Implement the rewrite from the map literal to `make_map`
## Notes / caveats
I am not sure how complete the MapArray support in arrow-rs is, so it
wouldn't surprise me if we hit various unimplemented errors when we tried to do
basic operations like compare maps or sort them. I don't think that is a reason
not to proceed here, but I wanted to point it out
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
