Running Hive JDBC get ClassNotFound: org.apache.hadoop.conf.Configuration

2014-06-13 Thread Néstor Boscán
Hi I recently downloaded the HDP 2.1 Sandbox. I'm trying to create a simple java program that connects to the hive server. I'm using maven with the following dependency: org.apache.hive hive-jdbc 0.13.1 Here is the java program: public class

RE: hbase import..

2014-06-13 Thread Kennedy, Sean C.
Fyi neglected to turn off iptables . From: Kennedy, Sean C. Sent: Thursday, June 12, 2014 8:26 PM To: user@hive.apache.org Subject: hbase import.. Trying to importsv: /hd/hadoop/bin/hadoop jar /hbase/hbase-0.94.15/hbase-0.94.15.jar importtsv '-Dimporttsv.separator=,' -Dimporttsv.column

Re: HCatalog access from a Java app

2014-06-13 Thread Dmitry Vasilenko
BTW, you can also get the Hive schema and partitions (using the code from #1) Table table = hiveMetastoreClient.getTable(databaseName, tableName); List schema = hiveMetastoreClient.getSchema(databaseName, tableName); List partitions = table.getPartitionKeys(); The HCat and Hive APIs for the schem

Re: HCatalog access from a Java app

2014-06-13 Thread Dmitry Vasilenko
Please take a look at http://stackoverflow.com/questions/22630323/hadoop-java-lang-incompatibleclasschangeerror-found-interface-org-apache-hadoo On Fri, Jun 13, 2014 at 9:53 AM, Brian Jeltema < brian.jelt...@digitalenvoy.net> wrote: > Doing this, with the appropriate substitutions for my table

Re: HCatalog access from a Java app

2014-06-13 Thread Brian Jeltema
Doing this, with the appropriate substitutions for my table, jarClass, etc: > 2. To get the table schema... I assume that you are after HCat schema > > > import org.apache.hadoop.conf.Configuration; > import org.apache.hadoop.mapreduce.InputSplit; > import org.apache.hadoop.mapreduce.Job; > im

Re: HCatalog access from a Java app

2014-06-13 Thread Dmitry Vasilenko
I am not sure about java docs... ;-] I have spent the last three years integrating with HCat and to make it work had to go thru the code... So here are some samples that can be helpful to start with. If you are using Hive 0.12.0 I would not bother with the new APIs... I had to create some shim cla

Re: HCatalog access from a Java app

2014-06-13 Thread Brian Jeltema
Version 0.12.0. I’d like to obtain the table’s schema, scan a table partition, and use the schema to parse the rows. I can probably figure this out by looking at the HCatalog source. My concern was that the HCatalog packages in the Hive distributions are excluded in the JavaDoc, which implies

Re: HCatalog access from a Java app

2014-06-13 Thread Dmitry Vasilenko
You should be able to access this information. The exact API depends on the version of Hive/HCat. As you know earlier HCat API is being deprecated and will be removed in Hive 0.14.0. I can provide you with the code sample if you tell me what you are trying to do and what version of Hive you are usi

HCatalog access from a Java app

2014-06-13 Thread Brian Jeltema
I’m experimenting with HCatalog, and would like to be able to access tables and their schema from a Java application (not Hive/Pig/MapReduce). However, the API seems to be hidden, which leads leads me to believe that this is not a supported use case. Is HCatalog use limited to one of the support

Re: Loading xml to hive and fetching unbounded tags

2014-06-13 Thread harish tangella
Hi, We are trying to get the data in the form of rows not in columns ..We are able to get partial data by implementing RecordReader. Logic we have applied is - getting the xml with start and end tag as 'Row' as the result we get only the second row, expected is 2 rows Refering to below xml ,

Re: Loading xml to hive and fetching unbounded tags

2014-06-13 Thread Knowledge gatherer
Are you trying to capture this data in one column and use XPATH with UDF to get the data. On Wed, Jun 11, 2014 at 11:12 AM, harish tangella wrote: > Hi, > > Request you to help. > > Fetching unbounded tags from the xml in hive > > We tried with xpath but unable to get all the unbounded tag

Reduce the amount of logging going into /var/log/hive/userlogs

2014-06-13 Thread Krishna Rao
Last time I looked there wasn't much info available on how to reduce the size of the logs written here (the only suggestions being delete them after a day). Is there anything I can do now to reduce what's logged there in the first place? Cheers, Krishna