Re: HQL parser internals

ry. >> >> >> >> Is there any way we can fix this issue? >> >> Could we change the parser in some way to stop the ‘as’ from being >> stripped out? >> >> >> >> Any ideas would be greatly appreciated! >> >> >> >> T

Re: HQL parser internals

ated! > > > > Thanks, > > > > Jay > > > > > > > > > From: *Elliot West* > Date: 19 March 2018 at 21:33 > Subject: Re: HQL parser internals > To: user@hive.apache.org > > Hello again, > > > > We're now testing our system

Re: HQL parser internals

some way to stop the ‘as’ from being stripped out? Any ideas would be greatly appreciated! Thanks, Jay From: Elliot West mailto:tea...@gmail.com>> Date: 19 March 2018 at 21:33 Subject: Re: HQL parser internals To: user@hive.apache.org<mailto:user@hive.apache.org> Hello again

Re: HQL parser internals

Hi Elliot, Yes, the variable substitution is done before the parsing. This make generic query validation much more complicated. As I explained in my previous message, what I did was replacing these ${VARS} by strings "${VARS}" that the HiveParser would agree to parse, and that I could recognize a

Re: HQL parser internals

Hello again, We're now testing our system against a corpus of Hive SQL statements in an effort to quickly highlight edge cases, limitations etc. We're finding that org.apache.hadoop.hive.ql.parse.ParseUtils is stumbling on variables such as ${hiveconf:varname}. Are variable substitutions handled p

Re: HQL parser internals

Thank you all for your rapid responses; some really useful information and pointers in there. We'll keep the list updated with our progress. On 18 February 2018 at 19:00, Dharmesh Kakadia wrote: > +1 for using ParseDriver for this. I also have used it to intercept and > augment query AST. > > A

Re: HQL parser internals

+1 for using ParseDriver for this. I also have used it to intercept and augment query AST. Also, I would echo others sentiment that its quite ugly. It would be great if we can refactor/standardize this. That will make integrating other system a lot easier. Thanks, Dharmesh On Sat, Feb 17, 2018 a

Re: HQL parser internals

Hi Elliot, Actually, I have done quite similar work regarding Hive custom Parsing, you should have a look at my project: https://github.com/flaminem/flamy The Hive parsing related stuff is here: https://github.com/flaminem/flamy/tree/master/src/main/scala/com/flaminem/flamy/parsing/hive A good st

Re: HQL parser internals

> However, ideally we wish to manipulate the original query as delivered by the > user (or as close to it as possible), and we’re finding that the tree has > been modified significantly by the time it hits the hook That's CBO. It takes the Query - > AST -> Calcite Tree -> AST -> hook - the bus

HQL parser internals

Hello, We need to be able to parse and manipulate an HQL query. To date we’ve been intercepting and transforming the parse tree in the SemanticAnalyzerHook. However, ideally we wish to manipulate the original query as delivered by the user (or as close to it as possible), and we’re finding that th