Re: No changes or progress status on web UI during mapreduce program running

2011-12-11 Thread Vamshi Krishna
Hi harsh, ya, i no jobs are seen in that jobtracker page, under RUNNING JOBS it is none, under FINISHED JOBS it is none,FAILED JOBS it is none . its just like no job is running. In eclipse i could see during mapreduce program running, as you said "LOcalJobRunner", may be Eclipse is merely launchin

Re: Strange problem while scanning HBase table with PIG

2011-12-11 Thread Stack
On Sat, Dec 10, 2011 at 9:51 AM, Luis A. Sarmento wrote: > Now, this *only happens for one mapper* --> mapper_0 > No matter how I change my scanning parameters - different begin / end > timestamp; more data vs less data; change caching ,etc.. - , it is always > ends up like this: mapper 0 fails,

Re: Aggregations in HBase

2011-12-11 Thread Stack
On Sun, Dec 11, 2011 at 11:56 AM, Suraj Varma wrote: > Coprocessors are available with 0.92 which now has a release candidate (RC0). > > So - you can probably try and build 0.92 RC0 to test this functionality out. > --Suraj > What Suraj said. Look in the coprocessor package. There are some prim

Re: major compaction and regionserver failure...

2011-12-11 Thread Nicolas Spiegelberg
Andy, Some detail on the current compaction algorithm. Major compactions can be triggered in 3 ways: 1) User requested : e.g. From the shell 2) Size based : compact when file <= sum(smaller_files) * 'hbase.hstore.compaction.ratio' This is the normal minor compaction logic. It will upgrade to a

Re: Aggregations in HBase

2011-12-11 Thread Suraj Varma
Coprocessors are available with 0.92 which now has a release candidate (RC0). So - you can probably try and build 0.92 RC0 to test this functionality out. --Suraj On Sun, Dec 11, 2011 at 10:52 AM, Royston Sellman wrote: > I'm a newbie learning HBase using 0.90.4. Got my data bulk loading nicely

RE: major compaction and regionserver failure...

2011-12-11 Thread Andy Sautins
Thanks Mikhail. That's very helpful information. We followed some of the advice that I believe was posted to this list in the past to schedule major compactions manually to run during off-peak times. That has gone very well for us, except when failures happen it seems we end up not majo

Aggregations in HBase

2011-12-11 Thread Royston Sellman
I'm a newbie learning HBase using 0.90.4. Got my data bulk loading nicely into a cluster and now I want to have simple SQL-like aggregations (SUM, AVG, STD, MIN, MAX, MEDIAN, WEIGHTED MEDIAN etc) working. I started off trying to build MR code to do this but stumbled across AggregateProtocol and

Re: major compaction and regionserver failure...

2011-12-11 Thread Mikhail Bautin
Hi Andy, We don't persist manual compaction requests across server restarts, so if e.g. a major compaction was requested manually and the regionserver crashed, the new regionserver that picks up that region will not know about the major compaction request. For automatic compactions, however, compa