RE: What does 'TempStatsStore' do ?

2011-05-10 Thread Raghunath, Ranjith
Is there some advantage to gathering stats in this case? Thank you, Ranjith N. Raghunath From: wd [mailto:w...@wdicc.com] Sent: Tuesday, May 10, 2011 9:54 PM To: user@hive.apache.org Subject: Re: What does 'TempStatsStore' do ? Thanks, I found some new configure options for hive.stats, I'll chan

Re: What does 'TempStatsStore' do ?

2011-05-10 Thread wd
Thanks, I found some new configure options for hive.stats, I'll change them. 2011/5/11 Ning Zhang > TempStatsStore is a derby database for stats gathering (intermediate > stats). You can turn off stats gathering by set > hive.stats.autogather=false. > > On May 10, 2011, at 1:23 PM, Christophe

Re: Strategy for Loading Apache Logs

2011-05-10 Thread hadoopman
You're pretty much going to overwrite the partition every time you want to add data to it. I wish there was an append but there isn't. We're basically doing it the same way you are (looking at your insert statement). The challenge I'm running into (doing the same thing you are) is when I'm b

Re: What does 'TempStatsStore' do ?

2011-05-10 Thread Ning Zhang
TempStatsStore is a derby database for stats gathering (intermediate stats). You can turn off stats gathering by set hive.stats.autogather=false. On May 10, 2011, at 1:23 PM, Christopher, Pat wrote: I don’t know what TempStatsStore is, but derby.log is an artifact of using the default metastore

RE: What does 'TempStatsStore' do ?

2011-05-10 Thread Christopher, Pat
I don't know what TempStatsStore is, but derby.log is an artifact of using the default metastore db, derby. I was unable to figure out a way to get derby.log to not be created or to be created in a particular directory. I also never saw anything useful in that file. I delete it whenever I see

Strategy for Loading Apache Logs

2011-05-10 Thread bichonfrise74
Hi, My end goal is to load the daily Apache logs. I wish to partition it by date and the group has helped me in giving some advices, but it seems that I am still stuck. My "daily Apache logs" can contain dates for 2 days ago, yesterday, and today. So, what I did was I created a staging_weblog tab

What does 'TempStatsStore' do ?

2011-05-10 Thread wd
hi, After upgrade to hive 0.7, I found there will a file named 'derby.log' and directory named 'TempStatsStore' be created every time I run the hive job . What does these files do? Is there a way to prevent them created ?