[
https://issues.apache.org/jira/browse/SPARK-60206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-60206:
-----------------------------------
Labels: pull-request-available (was: )
> Metric views resolve unqualified names in the reader's current namespace
> instead of the creation namespace
> ----------------------------------------------------------------------------------------------------------
>
> Key: SPARK-60206
> URL: https://issues.apache.org/jira/browse/SPARK-60206
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.2.0
> Reporter: Lisa Cao
> Priority: Major
> Labels: pull-request-available
>
> A metric view resolves unqualified names against the current catalog and
> namespace of whoever queries it, not the ones it was created in. Regular
> views use the creation namespace.
> {code:sql}
> CREATE TABLE default.t USING parquet AS SELECT * FROM VALUES ('a', 1), ('b',
> 2) AS v(k, x);
> CREATE DATABASE other;
> CREATE TABLE other.t USING parquet AS SELECT * FROM VALUES ('z', 1000) AS
> v(k, x);
> USE default;
> CREATE VIEW default.mv WITH METRICS LANGUAGE YAML AS $$
> version: "0.1"
> source: t
> dimensions:
> - name: k
> expr: k
> measures:
> - name: total
> expr: sum(x)
> $$;
> CREATE VIEW default.v AS SELECT sum(x) AS total FROM t;
> USE other;
> SELECT measure(total) FROM default.mv; -- 1000, reads other.t
> SELECT total FROM default.v; -- 3
> {code}
> The same happens for:
> * SQL sources ({{source: SELECT * FROM t}})
> * functions in dimension expressions ({{expr: f(k)}} calls {{other.f}})
> * subqueries in {{filter}}
> * temp views: with {{CREATE TEMPORARY VIEW t}} (or
> {{df.createOrReplaceTempView("t")}}) in the reader's session, the metric view
> reads the temp view, even from {{default}}
> * v2 catalog metric views, including after {{USE}} of another catalog
> From a namespace without {{t}} it fails with {{TABLE_OR_VIEW_NOT_FOUND}}.
> Reproduced on 4.2.0 and 4.3.0-rc1, with in-memory and Hive catalogs, and over
> Spark Connect.
> The creation namespace is stored ({{view.catalogAndNamespace.*}}), but
> {{SessionCatalog.parseMetricViewDefinition}} only applies the view's SQL
> configs. Regular views resolve under
> {{AnalysisContext.withAnalysisContext(view.desc)}} ({{ViewResolution}});
> metric views skip it.
> Workaround: fully qualify names in the definition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
