For the benefit of others: I was able to capture the CreateTable and DropTable events. Got the sample NotificationListener from hcatalog/server-extensions/src/main/java/org/apache/hcatalog/listener/NotificationListener.java - hive - Git at Google , removed unwanted stuff, packaged into a jar and put the jar into /usr/lib/hive/lib. Now restart hive-metastore and hive-server2, changes will take effect (ofcouse hive-site.xml needs an entry for the listener). Any error you can see in /var/log/hive/hive-metastore.log This link has some code samples:http://programcreek.com/java-api-examples/index.php?api=org.apache.hadoop.hive.metastore.api.Table thanks Santhosh hcatalog/server-extensions/src/main/java/org/apache/hcatalog/listener/NotificationListener.ja... Git Sign in apache / hive / 01ca40833d06c035160d8188c116c11981b1d879 / . / hcatalog / server-extensions / src / main / java / org / apache / hcatalog / listener / NotificationListener.java View on apache.googlesource.com Preview by Yahoo From: Sunil N Kumar <suniln.ku...@impetus.co.in> To: Santhosh Thomas <santhosh.tho...@yahoo.com>; "user@hive.apache.org" <user@hive.apache.org> Cc: Raptors <stamford-platf...@impetus.co.in> Sent: Tuesday, May 20, 2014 2:59 PM Subject: RE: How to detect new table creation in hive
Hi Santosh, This class already part of the Hive version 0.12 onwards in following location. $HIVE_HOME/hcatalog/share/hcatalog/hcatalog-server-extensions-0.12.0.jar Just make sure you run Hcatalog server instead of hive metastore. Thanks and Regard. Sunil Kumar From:Santhosh Thomas [mailto:santhosh.tho...@yahoo.com] Sent: Tuesday, May 20, 2014 2:43 PM To: Sunil N Kumar; user@hive.apache.org Cc: Raptors Subject: Re: How to detect new table creation in hive Thanks Sunil. I am already close to it. Where did you put the NotificationListener.class? ~santhosh From:Sunil N Kumar <suniln.ku...@impetus.co.in> To: "user@hive.apache.org" <user@hive.apache.org>; Santhosh Thomas <santhosh.tho...@yahoo.com> Cc: Raptors <stamford-platf...@impetus.co.in> Sent: Tuesday, May 20, 2014 2:22 PM Subject: RE: How to detect new table creation in hive Hi Santhosh, I have used this in the Falcon for creating Hive process to manage Hive tables across multiple Hadoop cluster. Current Hcatalog publish an event to the Message Broker say Active MQ whenever new partition added or deleted in the Hive Metastore. Following setting needs to done at hive-site.xml. This is not documented in the Hive documnetion. But I have used this and it is working as expected. It create topic in the activeMQ as Hcat.<Databasename>.<tablename>. <property> <name>hive.metastore.event.listeners</name> <value>org.apache.hcatalog.listener.NotificationListener</value> <description></description> </property> https://cwiki.apache.org/confluence/display/Hive/HCatalog+Notification HCatalog provides notifications for certain events happening in the system. This way applications such as Oozie can wait for those events and schedule the work that depends on them. The current version of HCatalog supports two kinds of events: · Notification when a new partition is added · Notification when a set of partitions is added From:Santhosh Thomas [mailto:santhosh.tho...@yahoo.com] Sent: Tuesday, May 20, 2014 1:16 PM To: user@hive.apache.org Subject: How to detect new table creation in hive Hello I am writing an application that detects new table creation in hive. I came across attribute ' hive.metastore.event.listeners' in hive-default.xml, but could not find any usage samples. The API docs also does not describe much. Can someone please help? Is there a better way to detect new table creation in hive? thanks Santhosh NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference. NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
