Most likely, Hadoop's memory settings are too high and Linux starts swapping. You should be able to detect that too using vmstat. Just a guess.
On Mon, Mar 7, 2011 at 10:11 PM, Ajo Fod <ajo....@gmail.com> wrote: > hmm I don't know of such a place ... but if I had to debug, I'd try to > understand the following: > 1) are the underlying files zipped/compressed ... that ususally makes it > slower. > 2) are the files located on the hard drive or hdfs? > 3) are all the cores being used? ... check number of reduce and map tasks. > > -Ajo > > > On Mon, Mar 7, 2011 at 9:24 PM, abhishek pathak < > forever_yours_a...@yahoo.co.in> wrote: > >> I suspected as such.My system is a Core2Duo,1.86 Ghz.I understand that >> map-reduce is not instantaneous, just wanted to confirm that 2200 rows in 4 >> minutes is indeeed not normal behaviour.Could you point me at some places >> where i can get some info on how to tune this up? >> >> Regards, >> Abhishek >> >> ------------------------------ >> *From:* Ajo Fod <ajo....@gmail.com> >> *To:* user@hive.apache.org >> *Sent:* Mon, 7 March, 2011 9:21:51 PM >> *Subject:* Re: Hive too slow? >> >> In my experience, hive is not instantaneous like other DBs, but 4 minutes >> to count 2200 rows seems unreasonable. >> >> For comparison my query of 169k rows one one computer with 4 cores running >> 1Ghz (approx) took 20 seconds. >> >> Cheers, >> Ajo. >> >> On Mon, Mar 7, 2011 at 1:19 AM, abhishek pathak < >> forever_yours_a...@yahoo.co.in> wrote: >> >>> Hi, >>> >>> I am a hive newbie.I just finished setting up hive on a cluster of two >>> servers for my organisation.As a test drill, we operated some simple >>> queries.It took the standard map-reduce algorithm around 4 minutes just to >>> execute this query: >>> >>> count(1) from tablename; >>> >>> The answer returned was around 2200.Clearly, this is not a big number by >>> hadoop standards.My question is whether this is a standard performance or is >>> there some configuration that is not optimised?Will scaling up of data to >>> say,50 times, produce any drastic slowness?I tried reading the documentation >>> but was not clear on these issues, and i would like to have an idea before >>> this setup starts working in a production environment. >>> >>> Thanks in advance, >>> Regards, >>> Abhishek Pathak >>> >>> >>> >>> >> >> >
