[
https://issues.apache.org/jira/browse/SPARK-51744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarno Rajala updated SPARK-51744:
---------------------------------
Description:
When {{{}spark.sql.parser.quotedRegexColumnNames=true{}}}, the following query
fails:
{{select `X Y` as xy from (select 1 as `X Y`)}}
giving the error message
{{[INVALID_USAGE_OF_STAR_OR_REGEX] Invalid usage of regular expression 'X Y' in
expression `alias`. SQLSTATE: 42000}}
Similar errors occur with other expressions like {{{}`X Y`/2{}}}. This severely
restricts use of columns with spaces when {{{}quotedRegexColumnNames=true{}}}.
There are many ways that this could be fixed.
I would suggest at least that backticked columns not be treated as regexes if
they do not contain any regex tokens.
This issue is related to SPARK-36698. However, that issue is a feature request
and not specifically about column names with spaces. However, the proposed PR
would presumably fix this issue as well.
was:
When {{{}spark.sql.parser.quotedRegexColumnNames=true{}}}, the following query
fails:
{{select `X Y` as xy from (select 1 as `X Y`)}}
giving the error message
{{[INVALID_USAGE_OF_STAR_OR_REGEX] Invalid usage of regular expression 'X Y' in
expression `alias`. SQLSTATE: 42000}}
Similar errors occur with other expressions like {{{}`X Y`/2{}}}. This severely
restricts use of columns with spaces when {{{}quotedRegexColumnNames=true{}}}.
There are many ways that this could be fixed.
I would suggest at least that backticked columns not be treated as regexes if
they do not contain any regex tokens.
This issue is related to SPARK-36698. However, that issue is a feature request
and not specifically about column names with spaces.
> Columns with spaces cannot be aliased or used in expressions when
> quotedRegexColumnNames=true
> ---------------------------------------------------------------------------------------------
>
> Key: SPARK-51744
> URL: https://issues.apache.org/jira/browse/SPARK-51744
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.3.2, 3.5.0
> Reporter: Jarno Rajala
> Priority: Minor
>
> When {{{}spark.sql.parser.quotedRegexColumnNames=true{}}}, the following
> query fails:
> {{select `X Y` as xy from (select 1 as `X Y`)}}
> giving the error message
> {{[INVALID_USAGE_OF_STAR_OR_REGEX] Invalid usage of regular expression 'X Y'
> in expression `alias`. SQLSTATE: 42000}}
> Similar errors occur with other expressions like {{{}`X Y`/2{}}}. This
> severely restricts use of columns with spaces when
> {{{}quotedRegexColumnNames=true{}}}.
> There are many ways that this could be fixed.
> I would suggest at least that backticked columns not be treated as regexes if
> they do not contain any regex tokens.
> This issue is related to SPARK-36698. However, that issue is a feature
> request and not specifically about column names with spaces. However, the
> proposed PR would presumably fix this issue as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
