szehon-ho commented on code in PR #54411:
URL: https://github.com/apache/spark/pull/54411#discussion_r2844643407
##########
sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2DataFrameSessionCatalogSuite.scala:
##########
@@ -111,7 +113,25 @@ class InMemoryTableSessionCatalog extends
TestV2SessionCatalogBase[InMemoryTable
val identToUse =
Option(InMemoryTableSessionCatalog.customIdentifierResolution)
.map(_(ident))
.getOrElse(ident)
- super.loadTable(identToUse)
+
+ // For single-part namespaces, follow Iceberg's pattern: first try to load
the table
+ // normally, fall back to metadata table resolution only on
NoSuchTableException
+ try {
+ super.loadTable(identToUse)
+ } catch {
+ case _: AnalysisException if identToUse.name().toLowerCase(Locale.ROOT)
== "snapshots" =>
Review Comment:
So in this code path, we get AnlaysisException like:
```
[REQUIRES_SINGLE_PART_NAMESPACE] spark_catalog requires a single-part
namespace, but got identifier `default`.`metadata_test_tbl`.`snapshots`.
SQLSTATE: 42K05
org.apache.spark.sql.AnalysisException: [REQUIRES_SINGLE_PART_NAMESPACE]
spark_catalog requires a single-part namespace, but got identifier
`default`.`metadata_test_tbl`.`snapshots`. SQLSTATE: 42K05
at
org.apache.spark.sql.errors.QueryCompilationErrors$.requiresSinglePartNamespaceError(QueryCompilationErrors.scala:1550)
at
org.apache.spark.sql.connector.catalog.CatalogV2Implicits$IdentifierHelper.asTableIdentifier(CatalogV2Implicits.scala:171)
at
org.apache.spark.sql.execution.datasources.v2.V2SessionCatalog.loadTable(V2SessionCatalog.scala:91)
at
org.apache.spark.sql.connector.catalog.DelegatingCatalogExtension.loadTable(DelegatingCatalogExtension.java:73)
at
org.apache.spark.sql.connector.InMemoryTableSessionCatalog.org$apache$spark$sql$connector$TestV2SessionCatalogBase$$super$loadTable(DataSourceV2DataFrameSessionCatalogSuite.scala:103)
at
org.apache.spark.sql.connector.TestV2SessionCatalogBase.loadTable(TestV2SessionCatalogBase.scala:69)
at
org.apache.spark.sql.connector.TestV2SessionCatalogBase.loadTable$(TestV2SessionCatalogBase.scala:62)
at
org.apache.spark.sql.connector.InMemoryTableSessionCatalog.loadTable(DataSourceV2DataFrameSessionCatalogSuite.scala:120)
at
org.apache.spark.sql.connector.catalog.CatalogV2Util$.getTable(CatalogV2Util.scala:483)
at
org.apache.spark.sql.connector.catalog.CatalogV2Util$.loadTable(CatalogV2Util.scala:458)
at
org.apache.spark.sql.catalyst.analysis.RelationResolution.$anonfun$resolveRelation$4(RelationResolution.scala:131)
at scala.Option.orElse(Option.scala:477)
```
its testing the V2SessionCatalog so it doesnt hit that code path. Probably
we can change the TestV2SessionCatalog (or V2SessionCatalog itself) to catch
it and return the right error, but it may be too much.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
