sivabalan narayanan created HUDI-8475:
-----------------------------------------
Summary: Generate all required stats required for MDT within DT
write handles
Key: HUDI-8475
URL: https://issues.apache.org/jira/browse/HUDI-8475
Project: Apache Hudi
Issue Type: Sub-task
Components: metadata, writer-core
Reporter: sivabalan narayanan
Assignee: sivabalan narayanan
Fix For: 1.0.0
As of now, some stats are sent to HoodieBackedTableMetadataWriter from the
write handles, while some are generated by reading the base files or log
on-demand within HoodieBackedTableMetadataWriter.update() call.
With the new dag design, we wanted to populate all stats within the DT write
handles only and send them back via WriteStatus.
We do not plan to collect entire WriteStatus in the driver, but just the
HoodieWRiteStat, and so we should be able to manage with WriteStatus holding
all the required stats for all indexes in MDT.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
