Re: different outer join plan between hive 0.9 and hive 0.10

It's a bug (https://issues.apache.org/jira/browse/HIVE-4206). Thanks for reporting it. 2013/3/24 wzc : > Recently we tried to upgrade our hive from 0.9 to 0.10, but found some of > our hive queries almost 7 times slow. One of such query consists multiple > table outer join on the same key. By lo

Re: Setting to make all tables external by default

somebody had answered this before on another mail thread set the external property to default true in metastore and that should solve it On Thu, Mar 28, 2013 at 1:43 AM, Babe Ruth wrote: > > Hello, > > Is there a setting where all tables created will default to external? I > know this isn't ide

Re: hive hbase storage handler fail

Hi Sanjay, thanks for the info. indeed, I did raise up the log level earlier to avoid dumping too much info in app log. I got much the same info and exception after lowering down at INFO. also, I am trying with CDH 4.1.2, except I am running mapreduce rather than YARN. The exception observe on

hive.limit.optimize.fetch.max

Hi I have following settings in the hive-site.xml hive.limit.row.max.size 10 hive.limit.optimize.enable true hive.limit.optimize.fetch.max 11 When I do a select query with WHERE clause it does not LIMIT The results to 10. How do u limit the SELECT query results to 10 rows ? M

Re: hive hbase storage handler fail

If you can run your hive insert data script with debug option u may get some clues /usr/lib/hive/bin/hive -hiveconf hive.root.logger=INFO,console -e "insert into dest select * from some_table_same_structure_as_dest limit 10;" I created a small demo usecase and this is failing for me as well The e

hive hbase storage handler fail

Hi all, I try inserting data into hive table storing by hbase storage handler but fail with the following exception end of mapreduce stage info. however, we didn't have problem accessing HbaseStorageHandler if force accessing YARN rather than mapreduce. (hive: 0.9.0, hbase: 0.92.1). could

Setting to make all tables external by default

Hello, Is there a setting where all tables created will default to external? I know this isn't ideal long term but I am setting up the environment and would rather start with all tables being external then switch them to managed if necessary. Thanks, George

Re: MySQL instance on hadoop name node server in production ?

Thanks Nitin…the mysql replication is a good point Regards sanjay From: Nitin Pawar mailto:nitinpawar...@gmail.com>> Reply-To: "user@hive.apache.org" mailto:user@hive.apache.org>> Date: Wednesday, March 27, 2013 11:36 AM To: "user@hive.apache.org

Re: MySQL instance on hadoop name node server in production ?

few writes understood but many reads will depends on how big your metastore will grow. also what size of hdfs you are looking at that will play a huge factor in this cause namenode will contain the FSImage in memory so you need a larger memory given to namenode. Potentially if its a live cluster t

Re: MySQL instance on hadoop name node server in production ?

Thanks Nitin. The mysql instance is for hive metastore only so few writes, many reads. Regards sanjay From: Nitin Pawar mailto:nitinpawar...@gmail.com>> Reply-To: "user@hive.apache.org" mailto:user@hive.apache.org>> Date: Wednesday, March 27, 2013 11:13 AM To: "user@h

Re: MySQL instance on hadoop name node server in production ?

ideally when you setup namenode and jobtracker in any hadoop cluster on production those are normally dedicated to that purpose only. Depending on how frequently you are going to run queries and how much data the hdfs is going to hold is key factor in deciding this. On Wed, Mar 27, 2013 at 11:32

MySQL instance on hadoop name node server in production ?

Hi all I am planning to install mysql server (as hive metastore) on the same box as my name node. My name node has 16GB RAM and hopefully I can get 2TB Any problems with mysql on the dame node as name node ? Thanks sanjay CONFIDENTIALITY NOTICE == This email message and any

RE: S3/EMR Hive: Load contents of a single file

No problem Keith - it was a worthwhile exercise for me to go back and double check everything was working as expected. -Original Message- From: Keith Wiley [mailto:kwi...@keithwiley.com] Sent: 27 March 2013 17:03 To: user@hive.apache.org Subject: Re: S3/EMR Hive: Load contents of a s

Re: S3/EMR Hive: Load contents of a single file

Okay, I also saw your previous response which analyzed queries into two tables built around two files in the same directory. I guess I was simply wrong in my understanding that a Hive table is fundamentally associated with a directory instead of a file. Turns out, it be can either one. A dire

RE: S3/EMR Hive: Load contents of a single file

A bit more info - do an extended description of the table: $ desc extended gsrc1; And the "location" field is "location:s3://mybucket/path/to/data/src1.txt" Do the same on a table created with a location pointing at the directory and the same info gives (not surprisingly) "location:s3://mybucke

Re: how to make data statistics efficiency in hive?

Hi Jan, You are so great! This expression is just I'm looking for. Thank you very much! Best Regards, Andy Zhou 2013/3/27 Jan Dolinár > Hi Andy, > > I'm not sure if I entirely understood your question, but I think you're > looking for something like this: > > select > concat(date,':',uid)

RE: S3/EMR Hive: Load contents of a single file

Thanks for the reply Keith. > you could have dispensed with the additional "alter table" business and > simply created the original table around the directory in the first place Yep, but I have multiple files in that directory and wanted to create a table based upon one file per table. > Do yo