[
https://issues.apache.org/jira/browse/HUDI-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17943388#comment-17943388
]
Lin Liu commented on HUDI-7833:
-------------------------------
The reason is that the new data does not have correct keys to join with the
existing records on the storage. I have fixed the data generation process to
ensure the keys are matched.
> Validate that fg reader works with nested column as record key
> --------------------------------------------------------------
>
> Key: HUDI-7833
> URL: https://issues.apache.org/jira/browse/HUDI-7833
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: Jonathan Vexler
> Assignee: Lin Liu
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.0.2
>
> Original Estimate: 2h
> Time Spent: 1h
> Remaining Estimate: 1h
>
> Ensure that fg reader works if the record key is a nested column
>
> Progress:
> Created a PR to reproduce the problem:
> [https://github.com/apache/hudi/pull/12253]
> From the PR, we turn on and off the fg reader, and run write operations
> (insert, update and delete), and read. We want to test if the update and
> delete could succeed on the map typed key column.
> From the test result, we can see that with or without fg reader enabled, the
> test failed for map typed key. We can conclude that nested keys are not
> supported in Hudi so far.
> I did some investigation on the root cause for this specific test:
> in `BuiltinKeyGenerator.combineRecordKeyInternal`, the `UnsafeMapData`
> object's hash is returned, without caring the content of the map object.
> to fix it, we need to create a until function that deserialize these map
> objects, and generate the hash based on their content.
> We should also do this for all other nested data types.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
