[
https://issues.apache.org/jira/browse/HUDI-2719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan reassigned HUDI-2719:
-----------------------------------------
Assignee: Nishith Agarwal
> Add back support for copying over extra metadata from previous commit
> ----------------------------------------------------------------------
>
> Key: HUDI-2719
> URL: https://issues.apache.org/jira/browse/HUDI-2719
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: sivabalan narayanan
> Assignee: Nishith Agarwal
> Priority: Major
>
> Add back support for copying over extra metadata from previous commit .
> We had a config named 'hoodie.write.meta.key.prefixes'. when this config is
> added, hudi copies over any extra metadata from previous commit with matching
> key prefixes as per `hoodie.write.meta.key.prefixes`. for eg: if
> `hoodie.write.meta.key.prefixes` set to `abc` and if there are other config
> options as "abc.key1" and "abc.key2", these two will be copied over from
> previous commit metadata and get overridden with current writer's commit
> metadata.
>
> Our initial understanding is that this was added to support copying over
> delta streamer checkpoint for multi -writer scenairo. But since the config is
> not very straight forward to use, we simplified and introduced a new config
> as per [https://github.com/apache/hudi/pull/3820.]
> Added the old one "hoodie.write.meta.key.prefixes" as an alternative. Since
> multi-writer in itself is still evolving, we felt simplifying the config for
> deltastreamer checkpoint makes sense.
> But if someone out there was using the older config to copy some other extra
> metadata for debugging or some other purposes, we need to add the support
> back.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
