[
https://issues.apache.org/jira/browse/SPARK-51747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amanda Liu updated SPARK-51747:
-------------------------------
Description:
Before this, DataSourceStrategy caches the first query plan and does not
respect options.
For example,
{code:java}
spark.sql("CREATE TABLE t(a string, b string) USING CSV".stripMargin)
spark.sql("INSERT INTO TABLE t VALUES ('a;b', 'c')")
spark.sql("SELECT * FROM t").show()
spark.sql("SELECT * FROM t WITH ('delimiter' = ';')"){code}
Expected output:
{code:java}
+----+----+
|col1|col2|
+----+----+
| a;b| c|
+----+----+
+----+----+
|col1|col2|
+----+----+
| a| b,c|
+----+----+ {code}
Output before this JIRA:
{code:java}
+----+----+
|col1|col2|
+----+----+
| a;b| c|
+----+----+
+----+----+
|col1|col2|
+----+----+
| a;b| c|
+----+----+ {code}
was:
Before this, DataSourceStrategy caches the first query plan and does not
respect options.
For example,
```
spark.sql("CREATE TABLE t(a string, b string) USING CSV".stripMargin)
spark.sql("INSERT INTO TABLE t VALUES ('a;b', 'c')")
spark.sql("SELECT * FROM t").show()
spark.sql("SELECT * FROM t WITH ('delimiter' = ';')")
```
> Data source cached plan should account for options
> --------------------------------------------------
>
> Key: SPARK-51747
> URL: https://issues.apache.org/jira/browse/SPARK-51747
> Project: Spark
> Issue Type: Task
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Amanda Liu
> Priority: Major
>
> Before this, DataSourceStrategy caches the first query plan and does not
> respect options.
> For example,
>
> {code:java}
> spark.sql("CREATE TABLE t(a string, b string) USING CSV".stripMargin)
> spark.sql("INSERT INTO TABLE t VALUES ('a;b', 'c')")
>
> spark.sql("SELECT * FROM t").show()
> spark.sql("SELECT * FROM t WITH ('delimiter' = ';')"){code}
>
>
> Expected output:
>
> {code:java}
> +----+----+
> |col1|col2|
> +----+----+
> | a;b| c|
> +----+----+
> +----+----+
> |col1|col2|
> +----+----+
> | a| b,c|
> +----+----+ {code}
>
>
> Output before this JIRA:
>
> {code:java}
> +----+----+
> |col1|col2|
> +----+----+
> | a;b| c|
> +----+----+
> +----+----+
> |col1|col2|
> +----+----+
> | a;b| c|
> +----+----+ {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
