[
https://issues.apache.org/jira/browse/HIVE-22963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ganesha Shreedhara updated HIVE-22963:
--------------------------------------
Description:
Parsing of query fails when we use single or double quotes in from/to string of
translate function in 2.3*/3.1.1 version of hive. Parsing of the same query is
successful in 2.1.1 version of hive.
*Steps to reproduce:*
{code:java}
CREATE TABLE test_table (data string);
INSERT INTO test_table VALUES("d\"a\"t\"a");
select translate(data, '"', '') from test_table;
{code}
Parsing fails with the following exception:
{code:java}
NoViableAltException(355@[157:5: ( ( Identifier LPAREN )=>
partitionedTableFunction | tableSource | subQuerySource | virtualTableSource
)])NoViableAltException(355@[157:5: ( ( Identifier LPAREN )=>
partitionedTableFunction | tableSource | subQuerySource | virtualTableSource
)]) at org.antlr.runtime.DFA.noViableAlt(DFA.java:158) at
org.antlr.runtime.DFA.predict(DFA.java:116) at
org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource0(HiveParser_FromClauseParser.java:2942)
at
org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:2880)
at
org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:1451)
at
org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1341)
at
org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45811) at
org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:39699)
at
org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:39951)
at
org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:39597)
at
org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:38786)
at
org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:38674)
at
org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2340)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1369)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:208) at
org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77) at
org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70) at
org.apache.hadoop.hive.ql.Driver.compile(Driver.java:507) at
org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1388) at
org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1528) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:1308) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:1298) at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:276) at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221) at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:465) at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:992) at
org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:916) at
org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:795) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.util.RunJar.run(RunJar.java:223) at
org.apache.hadoop.util.RunJar.main(RunJar.java:136)FAILED: ParseException line
1:40 cannot recognize input near 'tt' ';' '<EOF>' in from source
0org.apache.hadoop.hive.ql.parse.ParseException: line 1:40 cannot recognize
input near 'tt' ';' '<EOF>' in from source 0 at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:211) at
org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77) at
org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70) at
org.apache.hadoop.hive.ql.Driver.compile(Driver.java:507) at
org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1388) at
org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1528) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:1308) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:1298) at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:276) at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221) at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:465) at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:992) at
org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:916) at
org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:795) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.util.RunJar.run(RunJar.java:223) at
org.apache.hadoop.util.RunJar.main(RunJar.java:136){code}
Parsing is successful when double quote is escaped.
The following query works:
{code:java}
SELECT translate(data, '\"', ' ') from test_table;
{code}
But the behaviour which looks buggy is when the double quote is not escaped in
from/to string of translate function, it seems to be interpreting the double
quote in from/to string of next translate function as ending quote.
Parsing is successful for the following queries.
{code:java}
select translate(data, '"', ''), translate(data, '"', '') from test_table;
select translate(data, '"', ''), translate(data, ' ', '"') from test_table;
select translate(data, "'", ""), translate(data, "'", "") from test_table;
select translate(data, "'", ""), translate(data, " ", "'") from test_table;
select translate(data, '"', '"') from test_table;
select translate(data, "'", "'") from test_table;
{code}
The same behaviour exists when we use any other built-in functions or UDFs.
was:
Parsing of query fails when we use single or double quotes in from/to string of
translate function in 2.3*/3.1.1 version of hive. Parsing of the same query is
successful in 2.1.1 version of hive.
*Steps to reproduce:*
{code:java}
CREATE TABLE test_table (data string);
INSERT INTO test_table VALUES("d\"a\"t\"a");
select translate(data, '"', '') from test_table;
{code}
Parsing fails with the following exception:
{code:java}
NoViableAltException(355@[157:5: ( ( Identifier LPAREN )=>
partitionedTableFunction | tableSource | subQuerySource | virtualTableSource
)])NoViableAltException(355@[157:5: ( ( Identifier LPAREN )=>
partitionedTableFunction | tableSource | subQuerySource | virtualTableSource
)]) at org.antlr.runtime.DFA.noViableAlt(DFA.java:158) at
org.antlr.runtime.DFA.predict(DFA.java:116) at
org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource0(HiveParser_FromClauseParser.java:2942)
at
org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:2880)
at
org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:1451)
at
org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1341)
at
org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45811) at
org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:39699)
at
org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:39951)
at
org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:39597)
at
org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:38786)
at
org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:38674)
at
org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2340)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1369)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:208) at
org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77) at
org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70) at
org.apache.hadoop.hive.ql.Driver.compile(Driver.java:507) at
org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1388) at
org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1528) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:1308) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:1298) at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:276) at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221) at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:465) at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:992) at
org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:916) at
org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:795) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.util.RunJar.run(RunJar.java:223) at
org.apache.hadoop.util.RunJar.main(RunJar.java:136)FAILED: ParseException line
1:40 cannot recognize input near 'tt' ';' '<EOF>' in from source
0org.apache.hadoop.hive.ql.parse.ParseException: line 1:40 cannot recognize
input near 'tt' ';' '<EOF>' in from source 0 at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:211) at
org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77) at
org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70) at
org.apache.hadoop.hive.ql.Driver.compile(Driver.java:507) at
org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1388) at
org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1528) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:1308) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:1298) at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:276) at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221) at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:465) at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:992) at
org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:916) at
org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:795) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.util.RunJar.run(RunJar.java:223) at
org.apache.hadoop.util.RunJar.main(RunJar.java:136){code}
Parsing is successful when double quote is escaped.
The following query works:
{code:java}
SELECT translate(data, '\"', ' ') from test_table;
{code}
But the behaviour which looks buggy is when the double quote is not escaped in
from/to string of translate function, it seems to be interpreting the double
quote in from/to string of next translate function as ending quote.
Parsing is successful for the following queries.
{code:java}
select translate(data, '"', ''), translate(data, '"', '') from test_table;
select translate(data, '"', ''), translate(data, ' ', '"') from test_table;
select translate(data, "'", ""), translate(data, "'", "") from test_table;
select translate(data, "'", ""), translate(data, " ", "'") from test_table;
select translate(data, '"', '"') from test_table;
select translate(data, "'", "'") from test_table;
{code}
> HiveParser is misinterpreting quotes in parameters of built-in functions or
> UDF
> -------------------------------------------------------------------------------
>
> Key: HIVE-22963
> URL: https://issues.apache.org/jira/browse/HIVE-22963
> Project: Hive
> Issue Type: Bug
> Components: Parser
> Affects Versions: 3.1.1, 2.3.6
> Reporter: Ganesha Shreedhara
> Priority: Major
>
> Parsing of query fails when we use single or double quotes in from/to string
> of translate function in 2.3*/3.1.1 version of hive. Parsing of the same
> query is successful in 2.1.1 version of hive.
> *Steps to reproduce:*
>
> {code:java}
> CREATE TABLE test_table (data string);
> INSERT INTO test_table VALUES("d\"a\"t\"a");
> select translate(data, '"', '') from test_table;
> {code}
>
>
> Parsing fails with the following exception:
> {code:java}
> NoViableAltException(355@[157:5: ( ( Identifier LPAREN )=>
> partitionedTableFunction | tableSource | subQuerySource | virtualTableSource
> )])NoViableAltException(355@[157:5: ( ( Identifier LPAREN )=>
> partitionedTableFunction | tableSource | subQuerySource | virtualTableSource
> )]) at org.antlr.runtime.DFA.noViableAlt(DFA.java:158) at
> org.antlr.runtime.DFA.predict(DFA.java:116) at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource0(HiveParser_FromClauseParser.java:2942)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:2880)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:1451)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1341)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45811)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:39699)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:39951)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:39597)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:38786)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:38674)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2340)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1369) at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:208) at
> org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77) at
> org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70) at
> org.apache.hadoop.hive.ql.Driver.compile(Driver.java:507) at
> org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1388) at
> org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1528) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1308) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1298) at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:276) at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221) at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:465) at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:992) at
> org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:916) at
> org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:795) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.util.RunJar.run(RunJar.java:223) at
> org.apache.hadoop.util.RunJar.main(RunJar.java:136)FAILED: ParseException
> line 1:40 cannot recognize input near 'tt' ';' '<EOF>' in from source
> 0org.apache.hadoop.hive.ql.parse.ParseException: line 1:40 cannot recognize
> input near 'tt' ';' '<EOF>' in from source 0 at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:211) at
> org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77) at
> org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70) at
> org.apache.hadoop.hive.ql.Driver.compile(Driver.java:507) at
> org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1388) at
> org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1528) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1308) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1298) at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:276) at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221) at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:465) at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:992) at
> org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:916) at
> org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:795) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.util.RunJar.run(RunJar.java:223) at
> org.apache.hadoop.util.RunJar.main(RunJar.java:136){code}
>
> Parsing is successful when double quote is escaped.
> The following query works:
>
> {code:java}
> SELECT translate(data, '\"', ' ') from test_table;
> {code}
>
> But the behaviour which looks buggy is when the double quote is not escaped
> in from/to string of translate function, it seems to be interpreting the
> double quote in from/to string of next translate function as ending quote.
> Parsing is successful for the following queries.
>
> {code:java}
> select translate(data, '"', ''), translate(data, '"', '') from test_table;
> select translate(data, '"', ''), translate(data, ' ', '"') from test_table;
> select translate(data, "'", ""), translate(data, "'", "") from test_table;
> select translate(data, "'", ""), translate(data, " ", "'") from test_table;
> select translate(data, '"', '"') from test_table;
> select translate(data, "'", "'") from test_table;
> {code}
>
> The same behaviour exists when we use any other built-in functions or UDFs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
