924060929 commented on code in PR #69806:
URL: https://github.com/apache/doris/pull/69806#discussion_r4451588581
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/plugin/PluginDrivenExternalTable.java:
##########
@@ -148,17 +166,63 @@ public Optional<ConnectorTableHandle>
resolveConnectorTableHandle(
}
}
});
- String dbName = db != null ? db.getRemoteName() : "";
- // Some catalogs reject table requests for views with non-404 errors.
Classify views before
- // probing the table endpoint so snapshot loading can continue to view
expansion.
- Optional<ConnectorTableHandle> handle = supportsView() && isView()
- ? Optional.empty() : metadata.getTableHandle(session, dbName,
getRemoteName());
- // A view has no table handle. Check live view existence before
treating an empty handle as a
- // dropped table; the cached isView flag can outlive a remote view
deletion.
- if (!handle.isPresent() && !metadata.viewExists(session, dbName,
getRemoteName())) {
- observation.ifPresent(Runnable::run);
+ }
+
+ /** A synthetic system relation repairs the shared source identity rather
than its transient name. */
+ protected PluginDrivenExternalTable lookupRepairSource() {
+ return this;
+ }
+
+ /** Capture before resolving a handle, then retain through every provider
read made with that handle. */
+ public MetadataRead observeConnectorMetadata(ConnectorSession session,
ConnectorMetadata metadata) {
+ PluginDrivenExternalTable source = lookupRepairSource();
+ Connector connector = ((PluginDrivenExternalCatalog)
source.catalog).getConnector();
+ return new MetadataRead(source,
source.observeConnectorTableLookup(connector), session, metadata);
Review Comment:
Fixed in 99fb90a156f01180fe40acdea228eab96d47c9f6. Eager and scan partition
views now capture MetadataRead before resolving the handle and retain it
through snapshot binding, filter application and partition enumeration. This
reuses the existing identity/generation guard without adding freshness RPCs or
changing pruning degradation. Native ThriftHmsClient -> CachingHmsClient ->
HiveConnectorMetadata tests now exercise a real deferred scan pin and eager
partition enumeration, both for confirmed absence and an old error arriving
after same-name recreation. Filter application/listing and pinned partition
reads have additional cleanup/recreation cases. All 120 missing-table tests
pass; affected validation covers 1448 unique cases across 97 classes. FE build
and Checkstyle pass.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/plugin/PluginDrivenExternalTable.java:
##########
@@ -1823,7 +1899,7 @@ private long fetchRowCount(ConnectorSession session,
ConnectorMetadata metadata)
// feature flag because the listing can be a costly remote round-trip;
the connector self-samples,
// pins its classloader, and degrades to -1 rather than throwing.
if (GlobalVariable.enable_get_row_count_from_file_list) {
- long dataSize = metadata.estimateDataSizeByListingFiles(session,
handleOpt.get());
+ long dataSize = read.execute(() ->
metadata.estimateDataSizeByListingFiles(session, handleOpt.get()));
Review Comment:
Fixed in 99fb90a156f01180fe40acdea228eab96d47c9f6. Hive file estimation now
preserves only confirmed typed table absence through its ordinary-error
fallback; the FE executes guarded cleanup before returning UNKNOWN. Ordinary
listing/permission failures still degrade to -1. Sampled ANALYZE file-size
reads retain a pre-handle MetadataRead while keeping their original failure
propagation and finally-based scope closure. Native Thrift/HMS cache tests
cover file estimates and sampled file-size reads, each for actual absence and
delayed same-name recreation, and assert the cross-statement scopes close.
Connector tests verify ordinary-error degradation and TCCL restoration. All 120
missing-table tests and the affected 1448-case/97-class validation pass; FE
build and Checkstyle pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
