Quick question about HRegionServer.addScanner(...)

2011-08-05 Thread lars hofhansl
I see this code in there:     scannerId = rand.nextLong();     String scannerName = String.valueOf(scannerId);     scanners.put(scannerName, s); It would be rare that rand.nextLong() would return the same id while an older scanner is still in progress, but if that happens, it seems the result

Re: Can't disable a table because of a region time out

2011-08-05 Thread Yi Liang
For people who don't want to restart the whole cluster, I have solved the problem by restarting master alone and manually cleaning the table's ZK state. Thanks Jia for the suggestion of restarting master alone. Thanks, Yi On Fri, Aug 5, 2011 at 5:35 PM, Yi Liang wrote: > Looks like the issue o

Re: hbase shell broken in trunk (with simple patch)

2011-08-05 Thread lars hofhansl
Created HBASE-4171 and attached the fix there. - Original Message - From: Gary Helmling To: user@hbase.apache.org; lars hofhansl Cc: Sent: Friday, August 5, 2011 7:04 PM Subject: Re: hbase shell broken in truck (with simple patch) Thanks Lars.  That's a remnant of HBASE-3065. Yes pl

Re: hbase shell broken in truck (with simple patch)

2011-08-05 Thread Gary Helmling
Thanks Lars. That's a remnant of HBASE-3065. Yes please open an issue with the patch. We'll get it in. On Fri, Aug 5, 2011 at 7:01 PM, lars hofhansl wrote: > I noticed in HBase trunk I always get this error in the shell: > > ERROR: undefined method `getRecoverableZookeeper' for > # > > > Thi

Re: hbase shell broken in trunk (with simple patch)

2011-08-05 Thread lars hofhansl
Err... It did say ERROR: undefined method `getZooKeeper' for # Pasted the wrong message. Also fixed subject. - Original Message - From: lars hofhansl To: "user@hbase.apache.org" Cc: Sent: Friday, August 5, 2011 7:01 PM Subject: hbase shell broken in truck (with simple patch) I not

hbase shell broken in truck (with simple patch)

2011-08-05 Thread lars hofhansl
I noticed in HBase trunk I always get this error in the shell: ERROR: undefined method `getRecoverableZookeeper' for # This is apparently caused by HBASE-3065. The fix is trivial: Index: src/main/ruby/hbase/admin.rb === --- src/ma

Re: Quick Question about Bulk loading of HFiles & Timestamps

2011-08-05 Thread Jacques
Perfect. thanks, Jacques On Fri, Aug 5, 2011 at 3:53 PM, Todd Lipcon wrote: > Hi Jacques, > > Yes, the timestamps are set at the time the MR job runs, not the time > they're loaded. So, you'll see the values from the job that wrote its > output most recently. > > You can also specify timestamps

Re: Quick Question about Bulk loading of HFiles & Timestamps

2011-08-05 Thread Todd Lipcon
Hi Jacques, Yes, the timestamps are set at the time the MR job runs, not the time they're loaded. So, you'll see the values from the job that wrote its output most recently. You can also specify timestamps explicitly for each KeyValue, if you prefer. -Todd On Fri, Aug 5, 2011 at 2:10 PM, Jacque

Quick Question about Bulk loading of HFiles & Timestamps

2011-08-05 Thread Jacques
Can someone confirm that bulk loading hfiles keeps cell timestamps from overwriting each other. For example: I run mapreduce A job on Monday. I run mapreduce B job on Tuesday. I then run LoadIncrementalHFiles on job B first, followed by A. Please confirm that at the intersection of outputs A & B

Re: Apparent data loss on 90.4 rc2 after partial zookeeper network partition (on MapR)

2011-08-05 Thread M. C. Srivas
On Fri, Aug 5, 2011 at 11:28 AM, Todd Lipcon wrote: > On Fri, Aug 5, 2011 at 10:21 AM, M. C. Srivas wrote: > > On Fri, Aug 5, 2011 at 9:42 AM, Jean-Daniel Cryans >wrote: > > > >> On Fri, Aug 5, 2011 at 8:52 AM, M. C. Srivas > wrote: > >> > The normal behavior would be for the HMaster to make t

Re: Apparent data loss on 90.4 rc2 after partial zookeeper network partition (on MapR)

2011-08-05 Thread Todd Lipcon
On Fri, Aug 5, 2011 at 10:21 AM, M. C. Srivas wrote: > On Fri, Aug 5, 2011 at 9:42 AM, Jean-Daniel Cryans wrote: > >> On Fri, Aug 5, 2011 at 8:52 AM, M. C. Srivas wrote: >> > The normal behavior would be for the HMaster to make the hlog read-only >> > before processing it very simple fencing

Re: Apparent data loss on 90.4 rc2 after partial zookeeper network partition (on MapR)

2011-08-05 Thread M. C. Srivas
On Fri, Aug 5, 2011 at 9:42 AM, Jean-Daniel Cryans wrote: > On Fri, Aug 5, 2011 at 8:52 AM, M. C. Srivas wrote: > > The normal behavior would be for the HMaster to make the hlog read-only > > before processing it very simple fencing and works on all Posix or > > close-to-Posix systems. Does

Re: Apparent data loss on 90.4 rc2 after partial zookeeper network partition (on MapR)

2011-08-05 Thread Jean-Daniel Cryans
HDFS-1520 was forward ported to trunk by Stack: https://issues.apache.org/jira/browse/HDFS-1948 J-D On Fri, Aug 5, 2011 at 9:45 AM, Ryan Rawson wrote: > The IO fencing was an accidental byproduct of how HDFS-200 was > implemented, so in fact, HBase won't run correctly on HDFS-265 which > does N

Re: Apparent data loss on 90.4 rc2 after partial zookeeper network partition (on MapR)

2011-08-05 Thread Ryan Rawson
The IO fencing was an accidental byproduct of how HDFS-200 was implemented, so in fact, HBase won't run correctly on HDFS-265 which does NOT have that IO fencing, right? On Fri, Aug 5, 2011 at 9:42 AM, Jean-Daniel Cryans wrote: > On Fri, Aug 5, 2011 at 8:52 AM, M. C. Srivas wrote: >> The norma

Re: Apparent data loss on 90.4 rc2 after partial zookeeper network partition (on MapR)

2011-08-05 Thread Jean-Daniel Cryans
On Fri, Aug 5, 2011 at 8:52 AM, M. C. Srivas wrote: > The normal behavior would be for the HMaster to make the hlog read-only > before processing it very simple fencing and works on all Posix or > close-to-Posix systems.  Does that not work on HDFS? I'm sure you know the answer to that :) I

Re: Apparent data loss on 90.4 rc2 after partial zookeeper network partition (on MapR)

2011-08-05 Thread M. C. Srivas
The normal behavior would be for the HMaster to make the hlog read-only before processing it very simple fencing and works on all Posix or close-to-Posix systems. Does that not work on HDFS? On Fri, Aug 5, 2011 at 7:07 AM, M. C. Srivas wrote: > > > On Thu, Aug 4, 2011 at 9:01 PM, Todd Lipc

Re: Apparent data loss on 90.4 rc2 after partial zookeeper network partition (on MapR)

2011-08-05 Thread M. C. Srivas
On Thu, Aug 4, 2011 at 9:01 PM, Todd Lipcon wrote: > On Thu, Aug 4, 2011 at 8:36 PM, lohit wrote: > > 2011/8/4 Ryan Rawson > > > >> Yes, that is what JD is referring to, the so-called IO fence. > >> > >> It works like so: > >> - regionserver is appending to an HLog, continues to do so, hasnt >

Re: Can't disable a table because of a region time out

2011-08-05 Thread Yi Liang
Looks like the issue of HBASE-4064. Hope it will be fixed ASAP. But for now, how can I fix the issue? Due to the region in transition forever, the hbase balancer doesn't work any more. I have tried to kill and restart master alone, but it doesn't help. Do we have to restart the whole cluster for

Re: loading data in HBase table using APIs

2011-08-05 Thread Doug Meil
It's not obvious to a lot of newer folks that an MR job can exist minus the R. On 8/4/11 5:52 PM, "Michael Segel" wrote: > >Uhm Silly question... > >Why would you ever need a reduce step when you're writing to an HBase >table? > >Now I'm sure that there may be some fringe case, but in the p