Hi All
Any input on following issue will be useful. Regards Chetan From: Chetan Kothari Sent: Monday, August 27, 2018 1:48 PM To: user@hive.apache.org Subject: RE: Hook is not getting called on running query from beeline Hi Zoltan I changed the configuration to connect to Hive. [test@bdctest-1 hive]$ beeline WARNING: Use "yarn jar" to launch YARN applications. Beeline version 1.2.1000.2.4.2.0-258 by Apache Hive Connected to: Apache Hive (version 1.2.1000.2.4.2.0-258) Driver: Hive JDBC (version 1.2.1000.2.4.2.0-258) Transaction isolation: TRANSACTION_REPEATABLE_READ When I give wrong custom hook name or if hook class is not in classpath, it gives me following error. Error: Error while processing statement: FAILED: Hive Internal Error: java.lang.ClassNotFoundException(test.dc.lineage.listeners.hive.TestPostExecutionHook) (state=08S01,code=12) add jar /u01/bdctest/usr/hdp/2.4.2.0-258/hive/lib/hook-1.0.jar; set hive.exec.post.hooks=test.dc.lineage.listeners.hive.TestPostExecutionHook; I fixed this issue by explicitly setting hook and adding jar files. After setting hook and adding jar file, I don’t set above error, query gets executed but custom hook is not getting called. Regards Chetan -----Original Message----- From: Zoltan Haindrich [mailto:k...@rxd.hu] Sent: Thursday, August 23, 2018 1:14 PM To: HYPERLINK "mailto:user@hive.apache.org"user@hive.apache.org; Chetan Kothari Subject: Re: Hook is not getting called on running query from beeline Hello There is an interesting message after connecting: "Connected to: Spark SQL (version 2.1.0.2.6.0.3-8)". It seems you are not using Hive...please ask about this on a Spark related list. cheers, Zoltan On 08/22/2018 08:43 PM, Chetan Kothari wrote: > It seems --hiveconf is not getting picked from beeline. > > So even if I give wrong hook class name in following command, I am able to > connect and execute queries. > > But hook is not getting called. > > beeline -u > "jdbc:hive2://bdcsceforqa-bdcsce-1.dfml.ucfc2z3b.usdv1.abccloud.com:10 > 001/;transportMode=http;httpPath=cliservice" --hiveconf > hive.exec.post.hooks=abcc.lineage.listeners.TestPostExecutionHook > > Connected to: Spark SQL (version 2.1.0.2.6.0.3-8) > > Driver: Hive JDBC (version 1.2.1000.2.4.2.0-258) > > Transaction isolation: TRANSACTION_REPEATABLE_READ > > Beeline version 1.2.1000.2.4.2.0-258 by Apache Hive > > Regards > > Chetan > > -----Original Message----- > From: Zoltan Haindrich [mailto:k...@rxd.hu] > Sent: Wednesday, August 22, 2018 8:51 PM > To: HYPERLINK "mailto:user@hive.apache.org"user@hive.apache.org; Chetan > Kothari > Subject: Re: Hook is not getting called on running query from beeline > > Hello > > I was wondering about that the property is not getting picked up from > beeline; but it looks ok to me: > > $ beeline -u 'jdbc:hive2://...' --hiveconf hive.exec.post.hooks=asd -e > 'select 1' > > Connected to: Apache Hive (version 1.2.1000.2.6.1.0-129) > > Driver: Hive JDBC (version 1.2.1000.2.6.1.0-129) Transaction > isolation: TRANSACTION_REPEATABLE_READ > > Error: Error while processing statement: FAILED: Hive Internal Error: > java.lang.ClassNotFoundException(asd) (state=08S01,code=12) > > Which version of hive are you using? > > Try first with a non-existent classname. Executing the hooks usually leave > some messages in the logs as well... > > cheers, > > Zoltan > > On 08/22/2018 11:22 AM, Chetan Kothari wrote: > > > Thanks Tanvi for your inputs. > > > > > > Hook is not getting called from beeline/JDBC for any queries. > > > > > > I ensured that hive-site.xml is in classpath and also tried following but >still hook is not getting called. > > > > > > beeline -u "jdbc:hive2://example.com:2181 > > > <http://example.com:2181>,example.com:2181 > > > > <http://example.com:2181>,example:2181/;serviceDiscoveryMode=zooKeeper > > > ;zooKeeperNamespace=hiveserver2" *--hiveconf > > > hive.exec.post.hooks=<your_custom_hook_here>* > > > > > > Regards > > > > > > Chetan > > > > > > *From:*Tanvi Thacker [mailto:tanvithack...@gmail.com] > > > *Sent:* Wednesday, August 22, 2018 9:29 AM > > > *To:* HYPERLINK "mailto:user@hive.apache.org"user@hive.apache.org ><mailto:user@hive.apache.org> > > > *Subject:* Re: Hook is not getting called on running query from > > > beeline > > > > > > Is the hook getting called from beeline/JDBC for queries other than CTAS? > > > can you make sure that your configuration is correctly loaded by > > > beeline or JDBC code? you may check hive-site.xml is in the > classpath > > > or add a property when requesting a connection from DriverManager > or > > > you may also try like this > > > > > > beeline -u "jdbc:hive2://example.com:2181 > > > <http://example.com:2181>,example.com:2181 > > > > <http://example.com:2181>,example:2181/;serviceDiscoveryMode=zooKeeper > > > ;zooKeeperNamespace=hiveserver2" *--hiveconf > > > hive.exec.post.hooks=<your_custom_hook_here>* > > > > > > Regards, > > > > > > Tanvi Thacker > > > > > > On Tue, Aug 21, 2018 at 9:00 AM Chetan Kothari > <HYPERLINK > "mailto:chetan.koth...@oracle.com%20%3cmailto:chetan.koth...@oracle.com%20%3cmailto:chetan.koth...@oracle.com%20%3cmailto:chetan.koth...@oracle.com"chetan.koth...@oracle.com > <mailto:chetan.koth...@oracle.com > <mailto:chetan.koth...@oracle.com%20%3cmailto:chetan.koth...@oracle.com>>> > wrote: > > > > > > Any inputs on following issue will be helpful > > > > > > Regards > > > > > > Chetan > > > > > > *From:* Chetan Kothari > > > *Sent:* Tuesday, August 21, 2018 6:27 PM > > > *To:* HYPERLINK "mailto:user@hive.apache.org"user@hive.apache.org ><mailto:user@hive.apache.org> > <mailto:user@hive.apache.org> > > > *Subject:* Hook is not getting called on running query from > > > beeline > > > > > > Hi > > > > > > I have written custom PostExecutionHook which implements > org.apache.hadoop.hive.ql.hooks.ExecuteWithHookContext to capture > lineage of hive queries like create table as > > > select etc. > > > > > > This hook get's called when I run CTAS query o Hive Shell. But it is >not getting called when I run same query from beeline or programmatically >using JDBC Code. > > > > > > Any inputs on why hook is not getting called when I run query from >beeline or programmatically using JDBC Code. > > > > > > Regards > > > > > > Chetan > > > >
