Re: Managing Hive logging detail (per Hive statement) in /tmp/${user.name}/

2011-09-12 Thread Adriaan Tijsseling
Should be here: ./conf/hive-exec-log4j.properties ./conf/hive-log4j.properties Cheers, Adriaan On 2011/09/12, at 21:32, Jasper Knulst wrote: > Hi, > > Does anyone know how I can reduce the level of logging of the Hive History > file which are created per Hive statement executed in the (default

Managing Hive logging detail (per Hive statement) in /tmp/${user.name}/

2011-09-12 Thread Jasper Knulst
Hi, Does anyone know how I can reduce the level of logging of the Hive History file which are created per Hive statement executed in the (default) location of /tmp/${user.name}/ hive_job_log_hdfs_201109122109_675419678.txt. The level of detail is very high by default but I can't find a way to tune

RE: Best practices for storing data on Hive

2011-09-12 Thread Steven Wong
I haven't used sorted columns before. I expect sorted columns to give you some query optimization by avoiding some sorting in some cases, but not by pruning input. In this sense, sorted columns give you a very different kind of optimization than partitioned, bucketed, or indexed columns do. ---

Re: CREATE INDEX

2011-09-12 Thread Adriaan Tijsseling
Thanks. I see this: CREATE INDEX index_name ON TABLE base_table_name (col_name, ...) AS index_type [WITH DEFERRED REBUILD] [IDXPROPERTIES (property_name=property_value, ...)] [IN TABLE index_table_name] [ [ ROW FORMAT ...] STORED AS ... | STORED BY ... ] [LOCATION hdfs_path] [TBLPROPERTIES (

Re: Pentaho issue with hive

2011-09-12 Thread Jasper Knulst
Hi *Siddharth, *Did you also install the PHD (Pentaho Distribution for Hadoop) on the cluster side? On my cluster (with PHD installed) there are a lot of Pentaho specific jars in /usr/lib/hadoop-0.20/lib for instance: hive-exec-0.7.0-pentaho-1.0.0.jar hive-jdbc-0.7.0-pentaho-1.0.0.jar hive-met

Re: CREATE INDEX

2011-09-12 Thread Ashutosh Chauhan
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create%2FDropIndexhas DDL commands using which you can create and manipulate indices in Hive. Hope it helps, Ashutosh On Mon, Sep 12, 2011 at 05:46, Adriaan Tijsseling wrote: > Does anyone know the proper docum

Re: Best practices for storing data on Hive

2011-09-12 Thread Mark Grover
Thanks, Steven. So, am I correct in understanding that even the sorting columns (user_id, time in my example) would not be used to optimize the query shown below? On 11-09-09 07:00 PM, Steven Wong wrote: Bucketing only speeds up sampling queries. Hive doesn't know/remember the hash function(

Re: Pentaho issue with hive

2011-09-12 Thread Ashutosh Chauhan
>> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.io.Writable Seems like you have classpath issues. Make sure hadoop-core.jar (or whatever is the name of your hadoop jars) are in classpath Hope it helps, Ashutosh On Mon, Sep 12, 2011 at 09:20, Siddharth Tiwari wrote: > Hi Jasper

RE: Pentaho issue with hive

2011-09-12 Thread Siddharth Tiwari
Hi Jasper I am on Pentaho BI suite 4.0 and Hive 0.7.1 ... please help ** Cheers !!! Siddharth Tiwari Have a refreshing day !!! Date: Mon, 12 Sep 2011 11:47:06 +0200 Subject: Re: Pentaho issue with hive From: jasper.knu...@vlc.nl To: user@hive.apache.org Hi Siddharth,

Re: Loading data into hive table from stdin

2011-09-12 Thread Adriaan Tijsseling
Thanks, that seems to be the proper alternative. Adriaan On 2011/09/12, at 16:05, Vikas Srivastava wrote: > No, > but what you can do is to write a general script to read files from the > folder and store it into list then retrive the file from list and load them > into desire path. > > On Mon

Re: Allow Hive UDF in Scala?

2011-09-12 Thread Zhiyong Xie
btw, it's durable, and I have figured it out. Simply write the UDF in scala with related Java class, then add all dependent jar into Hive, it will work. Joe On Sat, Sep 10, 2011 at 12:41 PM, Zhiyong Xie wrote: > Hi there, > > Anyone has early experience or simple example to write Hive UDF in sca

Re: Loading data into hive table from stdin

2011-09-12 Thread Vikas Srivastava
No, but what you can do is to write a general script to read files from the folder and store it into list then retrive the file from list and load them into desire path. On Mon, Sep 12, 2011 at 7:24 PM, Adriaan Tijsseling wrote: > Sorry, I wasn't clear. What I meant was if the data in path $path

Re: Loading data into hive table from stdin

2011-09-12 Thread Adriaan Tijsseling
Sorry, I wasn't clear. What I meant was if the data in path $path can itself be a script that feeds the data in streaming fashion? Something like "load data using script 'loader.py' into table foo". On 2011/09/12, at 15:36, Vikas Srivastava wrote: > hive -e "load data local in path '$path' into

Hive Hbase integration : totally lost :(

2011-09-12 Thread shazz Ng
Hello ! Since one week I'm trying to integrate Hive with my current HBase instance but whatever the results, all the stuff I follow to build it seems unconsistent... So before I go mad, maybe somebody can help :) What I have : - HBase 0.9.3 running OpenTSDB (http://opentsdb.net/setup-hbase.html u

Re: Error while executing hive queries from script

2011-09-12 Thread Ankit Jain
Hi Praveen, create the table 'social' before running the script. On Mon, Sep 12, 2011 at 9:24 AM, Vikas Srivastava < vikas.srivast...@one97.net> wrote: > remove the ; and check that table is present or not. > > On Mon, Sep 12, 2011 at 5:42 PM, Chalcy Raja < > chalcy.r...@careerbuilder.com> wrot

Re: Loading data into hive table from stdin

2011-09-12 Thread Vikas Srivastava
hive -e "load data local in path '$path' into table $table partition(date='$date')" On Mon, Sep 12, 2011 at 7:04 PM, Adriaan Tijsseling wrote: > Do you have the syntax for the proper hive QL command? > > Thanks! > > On 2011/09/12, at 15:23, Vikas Srivastava wrote: > > > ya you can simple load the

Re: Loading data into hive table from stdin

2011-09-12 Thread Adriaan Tijsseling
Do you have the syntax for the proper hive QL command? Thanks! On 2011/09/12, at 15:23, Vikas Srivastava wrote: > ya you can simple load the data by a simple script without using any api or > anything!! > > On Mon, Sep 12, 2011 at 6:29 PM, Adriaan Tijsseling > wrote: > >> Is it possible to loa

Re: Error while executing hive queries from script

2011-09-12 Thread Vikas Srivastava
remove the ; and check that table is present or not. On Mon, Sep 12, 2011 at 5:42 PM, Chalcy Raja wrote: > Looks like permission issue. Check the permission on the table social. > Run the load script as sudo. > > Hope this helps! > Chalcy > > -Original Message- > From: Praveen Bathala [

Re: Loading data into hive table from stdin

2011-09-12 Thread Vikas Srivastava
ya you can simple load the data by a simple script without using any api or anything!! On Mon, Sep 12, 2011 at 6:29 PM, Adriaan Tijsseling wrote: > Is it possible to load data into a hive table that is fed in using a unix > pipe or streamed via a script? I've only seen streaming options when usin

Loading data into hive table from stdin

2011-09-12 Thread Adriaan Tijsseling
Is it possible to load data into a hive table that is fed in using a unix pipe or streamed via a script? I've only seen streaming options when using SELECT TRANSFORM. Thanks in advance, Adriaan

Re: CREATE INDEX

2011-09-12 Thread rohan monga
yup, I'd be interested in this too. I'd like to know how the indices are applied/used and which queries can they be useful. Thanks, -- Rohan Monga On Mon, Sep 12, 2011 at 6:16 PM, Adriaan Tijsseling wrote: > Does anyone know the proper documentation for using the "CREATE INDEX" > command in h

CREATE INDEX

2011-09-12 Thread Adriaan Tijsseling
Does anyone know the proper documentation for using the "CREATE INDEX" command in hive? The hive homepage doesn't have the information. Cheers, Adriaan

RE: Error while executing hive queries from script

2011-09-12 Thread Chalcy Raja
Looks like permission issue. Check the permission on the table social. Run the load script as sudo. Hope this helps! Chalcy -Original Message- From: Praveen Bathala [mailto:pbatha...@gmail.com] Sent: Saturday, September 10, 2011 10:01 AM To: user@hive.apache.org Subject: Error while e

Re: Pentaho issue with hive

2011-09-12 Thread Jasper Knulst
Hi Siddharth, What versions of Hive and Pentaho are you on ? GR. Jasper 2011/9/12 Siddharth Tiwari > Hi users,I face this issue while connecting to hive > started hive using: HIVE_PORT=1 hive --service hiveserver > Starting hive thrift server > > No information after it > netstat shows: >