Re: Configure Hive in Cluster

2013-01-16 Thread Nitin Pawar
looks like a very small cluster with very limited memory to run mapreduce jobs also number of map/reduce slots on nodes are less so at a time only one map is running. but still 15 min is a lot of time for 600MB memory On Thu, Jan 17, 2013 at 12:47 PM, venkatramanan < venkatraman...@smartek21.com

Re: Configure Hive in Cluster

2013-01-16 Thread venkatramanan
Below details are the cluster configuration Configured Capacity : 82.8 GB DFS Used : 1.16 GB Non DFS Used : 31.95 GB DFS Remaining: 49.69 GB DFS Used% : 1.4 % DFS Remaining% : 60.01 % Live Nodes

Re: Configure Hive in Cluster

2013-01-16 Thread nagarjuna kanamarlapudi
What's the size of your cluster .. Number of nodes and their capacity On Thursday, January 17, 2013, Nitin Pawar wrote: > how many number of nodes you have for select query? > whats your select query? > > if its just a select * from table then it does not run any mapreduce job > so its just takin

Re: Configure Hive in Cluster

2013-01-16 Thread Nitin Pawar
how many number of nodes you have for select query? whats your select query? if its just a select * from table then it does not run any mapreduce job so its just taking time to show data on your screen if you are using that query On Thu, Jan 17, 2013 at 12:24 PM, venkatramanan < venkatraman...@s

Re: Configure Hive in Cluster

2013-01-16 Thread venkatramanan
I didnt set any hive parameters and my total table size is 610 MB only On Thursday 17 January 2013 12:11 PM, Nitin Pawar wrote: a bit more details on size of table and select query will help also did you set any hive parameters ? On Thu, Jan 17, 2013 at 12:12 PM, venkatramanan mailto:venkatr

Re: Configure Hive in Cluster

2013-01-16 Thread Nitin Pawar
a bit more details on size of table and select query will help also did you set any hive parameters ? On Thu, Jan 17, 2013 at 12:12 PM, venkatramanan < venkatraman...@smartek21.com> wrote: > Hi All, > > Am Newbie in apache hive. I have create a table and thats points to the > HDFS Folder path a

Configure Hive in Cluster

2013-01-16 Thread venkatramanan
Hi All, Am Newbie in apache hive. I have create a table and thats points to the HDFS Folder path and its takes 15 min to execute the simple "*select*" stmt, Can anyone suggest me for a best practices and performance improvement on hive. Thanks in Advance Venkat

Re: Interaction between Java and Transform Scripts on Hive

2013-01-16 Thread Dean Wampler
The transform scripts (or executables) are run as separate processes, so it sounds like Hive itself is blowing up. That would be consistent with your script working fine outside Hive. The Hive or Hadoop logs might have clues. So, it happens consistently with this one file? I would check to be sure

Interaction between Java and Transform Scripts on Hive

2013-01-16 Thread John Omernik
I am perplexed if I run a transform script on a file by itself, it runs fine, outputs to standard out life is good. If I run the transform script on that same file (with the path and filename being passed into the script via transform so that the python script is doing the exact same thing) I get