sivabalan narayanan created HUDI-8474:
-----------------------------------------
Summary: Design and Impl MDT repartitioner to assist with writing
to MDT
Key: HUDI-8474
URL: https://issues.apache.org/jira/browse/HUDI-8474
Project: Apache Hudi
Issue Type: Sub-task
Components: metadata, writer-core
Reporter: sivabalan narayanan
Assignee: sivabalan narayanan
Fix For: 1.0.0
We need a repartitoner for MDT where we take in HoodieData<HoodieRecords> and
return 1 spark task pertaining to 1 file slice in MDT.
For eg, for FILES, its typically 1 file slice.
for col stats, RLI, etc its based on how user has configured it.
We should be doing sort within partitioner as well since w/ hfile we might have
to sort the keys.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
