yihua opened a new issue, #20074: URL: https://github.com/apache/hudi/issues/20074
The file group reader and the log and buffer code it reaches take a `HoodieTableMetaClient` but use only the table config, the base path, a storage factory and, in two places, the timeline. Those two places (the committed-instant check on log blocks of tables before version 8, and the schema-history lookup for schema-on-read) load the timeline on the executor, so every task repeats that `.hoodie` I/O and tasks of one query can see different timelines. Proposal: pass the reader a small table state with the table config, the base path, and the committed instants and schema history captured on the driver where the timeline is already loaded. part of #20064 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
