[
https://issues.apache.org/jira/browse/SPARK-53348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18016358#comment-18016358
]
Dongjoon Hyun commented on SPARK-53348:
---------------------------------------
This is reverted from `branch-4.0` first due to the non-ANSI CI failures.
Previously, both `master` and `branch-4.0` non-ANSI CIs are broken.
-
https://github.com/apache/spark/commit/f3c2d39f32c9122bf0e496082b34572185ce8f38
- https://github.com/apache/spark/actions/runs/17197538374/job/48928049245
{code}
[info] - Default ANSI value is stored for views *** FAILED *** (18 milliseconds)
[info] "[fals]e" did not equal "[tru]e" (DefaultANSIValueSuite.scala:104)
[info] Analysis:
[info] "[fals]e" -> "[tru]e"
{code}
> Always persist ANSI value when creating a view or assume it when querying if
> not stored
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-53348
> URL: https://issues.apache.org/jira/browse/SPARK-53348
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Mihailo Aleksic
> Assignee: Mihailo Aleksic
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> I propose that we always store ANSI value when creating a view because
> otherwise users can be affected by unwanted behavior. For example if user
> creates a view on version that has ANSI = false by default he expects this
> not to fail.
> CREATE VIEW view AS SELECT CAST('abc' AS INT) AS a;
> SELECT * FROM view;
> But if user queries the view on the version which has ANSI = true by default,
> above query is going to fail (because when we don't store the value, and we
> store it only if explicitly set, we use the default one). Number of this and
> similar use cases is huge, because ANSI impact area is huge and thus I
> propose that we always store the value.
> If the value is not stored, I propose that we use createVersion field to
> determine whether the ANSI value should be true (Spark 4.0.0 and above) or
> false (lower than Spark 4.0.0). If the createVersion field wasn't stored
> during view creation, I propose that we assume that the ANSI = false because
> number of those views is incomparable larger than the ones expecting ANSI =
> true
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
