danny0405 commented on code in PR #13699:
URL: https://github.com/apache/hudi/pull/13699#discussion_r2273598475
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java:
##########
@@ -150,6 +151,7 @@ public abstract class HoodieTable<T, I, K, O> implements
Serializable {
private transient FileSystemViewManager viewManager;
protected final transient HoodieEngineContext context;
+ private final ReaderContextFactory<T> readerContextFactoryForWrite;
Review Comment:
The hoodie table holds all the info required there: the engine context, the
meta client, and the write config. It looks not right we pass around a "reader"
context factory for all the write path, even for write path that is not COW
table merging scenarios.
There are also discrepancies in write path, only spark needs this factory,
Flink and Java can get the factory directly from the engine context in the
hoodie table.
The broadcast is already there for all the write executors before my change,
if we have some manner to resolve the serialization issue of the engine context
or reader context itself, that would be the best. Or if we can limit the
factory only been initialized for COW table updates in write handles, that
would be great.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
