RE: HBase multi-threaded client consumes a lot of threads

Using HTablePool now, improves the situation a lot. Thanks, Thomas -Original Message- From: Joe Pallas [mailto:joseph.pal...@oracle.com] Sent: Donnerstag, 21. Juli 2011 19:17 To: user@hbase.apache.org Subject: Re: HBase multi-threaded client consumes a lot of threads On Jul 21, 2011, a

Plugs for vendors on the ML (WAS: Monitoring)

I think we should also avoid hijacking threads for that sort of discussion (sorry Joseph!). J-D On Mon, Jul 25, 2011 at 9:08 PM, Stack wrote: > I made a patch for the manual updating our Cloudera text some and > adding in MapR reference.  I did it here: > https://issues.apache.org/jira/browse/HB

Re: Monitoring

I made a patch for the manual updating our Cloudera text some and adding in MapR reference. I did it here: https://issues.apache.org/jira/browse/HBASE-4140 My wordsmithing is not the best so input appreciated. Will just commit tomorrow and push it out if nought said. St.Ack On Mon, Jul 25, 20

Re: Monitoring

On Mon, Jul 25, 2011 at 6:10 PM, Doug Meil wrote: > > Andrew: nicely put. > > Jeff W: I agree with your ordering. > > Stack: I agree with the book change for inclusion, and I agree with the > caveat about 'free as in free beer'. > +1, seems entirely reasonable to list compatible alternatives i

Re: Monitoring

Andrew: nicely put. Jeff W: I agree with your ordering. Stack: I agree with the book change for inclusion, and I agree with the caveat about 'free as in free beer'. Todd/Ryan: regarding the book reference to CDH, I think the CDH reference being 'free as in beer' reference is an important di

Re: Monitoring

> From: Jeff Whiting > If Ted had answered the post along the lines of (I ganked the reply from > Joey): > >     Hadoop and HBase are pretty monitoring tool agnostic. It does provide >     a number of metrics via JMX and a REST interface which you can tie >     into the monitoring tool of your

Re: Monitoring

I agree it's a fine line. As far as vendor specific pronouncements, may I suggest a little goes a long way, quality over quantity. In my opinion it's never sufficient to say only "nonopen product Foo does X" on an open source project's user list. Instead you need to first explain how to do X wi

Re: Monitoring

It seems this is going to have to be something that is a judgment call. It will be hard to define exactly when you should or shouldn't mention something. The general principle is that the ASF option should always be touted first, followed by any other OSS / free options, then if no other optio

Re: Monitoring

On Mon, Jul 25, 2011 at 1:17 PM, Todd Lipcon wrote: > On Mon, Jul 25, 2011 at 1:09 PM, Ryan Rawson wrote: > > > But surely for logical consistency, we should not favor one vendor (as > > we have been for a year now), over another. So would it be correct to > > continue to suggest to users they u

Re: Monitoring

I think it's fair to note which environments you can run HBase on top of. If we disallow that then we will have the tricky bit where there is no ASF release of Hadoop that is suitable to run HBase on top of. And who knows, perhaps the ceph guys, or openstack or might come up with a suitable HDFS

Re: Monitoring

On Mon, Jul 25, 2011 at 1:20 PM, Buttler, David wrote: > However, only two vendors deliver a platform that supports hbase (with > append): Cloudera and MapR.  HortonWorks and ASF do not (to my knowledge). I > am not sure I can count hard to find/compile branches that exist in ASF's > version co

Re: Fanning out hbase queries in parallel

Unfortunately there's no easy patch set to pull coprocessors into any 0.90 HBase version (including CDH3 HBase). The changes are extensive and invasive and include RPC protocol changes. Internally at Trend Micro we run a heavily, heavily patched 0.90-based version of HBase that includes coprocess

Re: Fanning out hbase queries in parallel

Yes. St.Ack On Mon, Jul 25, 2011 at 1:23 PM, Paul Nickerson wrote: > We currently run on the cloudera stack. Would this be something that we can > pull, compile, and plug right into that stack? > > - Original Message - > > From: "Gary Helmling" > To: user@hbase.apache.org > Sent: Monday

Re: Fanning out hbase queries in parallel

We currently run on the cloudera stack. Would this be something that we can pull, compile, and plug right into that stack? - Original Message - From: "Gary Helmling" To: user@hbase.apache.org Sent: Monday, July 25, 2011 2:02:50 PM Subject: Re: Fanning out hbase queries in parallel

RE: Monitoring

However, only two vendors deliver a platform that supports hbase (with append): Cloudera and MapR. HortonWorks and ASF do not (to my knowledge). I am not sure I can count hard to find/compile branches that exist in ASF's version control as "supporting" hbase. MapR and Cloudera both have free v

Re: Monitoring

On Mon, Jul 25, 2011 at 1:09 PM, Ryan Rawson wrote: > But surely for logical consistency, we should not favor one vendor (as > we have been for a year now), over another. So would it be correct to > continue to suggest to users they use CDH? After all, even though it > is ASF2.0 and free, it is

Re: Monitoring

But surely for logical consistency, we should not favor one vendor (as we have been for a year now), over another. So would it be correct to continue to suggest to users they use CDH? After all, even though it is ASF2.0 and free, it is still giving one vendor a leg up over others (including horton

Re: Monitoring

Hey Joe, Hadoop and HBase are pretty monitoring tool agnostic. It does provide a number of metrics via JMX and a REST interface which you can tie into the monitoring tool of your choice. You can enable collection via the REST service by editing $HADOOP_HOME/conf/hadoop-metrics.properties and setti

Re: Monitoring

On Mon, Jul 25, 2011 at 11:55 AM, Ted Dunning wrote: > Todd, > > Good to have you weigh in on this. You provide a good counterweight. > > To take a new hypothetical, suppose that one of the many, many patches that > Cloudera has championed for Hadoop is critical for Hbase operation or makes > Hb

Re: Monitoring

> IMO, an answer that was just "Fixed in CDH, followups off-list" would > be deserving of a yellow card. > > IMO, if the answer was 'No but it is fixed in CDH...', that might be > sufficient (You've answered the question first and then diverted the > user).  If the 'No' and the '.. it is fixed...'

Re: Monitoring

On Mon, Jul 25, 2011 at 11:55 AM, Ted Dunning wrote: > Is it reasonable to answer a question of the form "Is HDFS-xxx fixed?" with > "Fixed in CDH, followups off-list"? > > That seems to be important information for not just the original poster but > others who may have the same problem. > > What

Re: Monitoring

On Mon, Jul 25, 2011 at 12:00 PM, Stack wrote: > I felt you deserved the yellow card because the first response out the > gate was '(Slightly) off topic' and could be read as a plug for a > commercial product. > Yellow accepted. > > Another answer that I want to underscore is "MapR supports Hb

Re: Monitoring

On Mon, Jul 25, 2011 at 11:28 AM, Ted Dunning wrote: > I am very sympathetic here.  Also, somewhat linguistically challenged on > this point since there is a fine line to be walked.  All suggestions are > welcome. > Understood. I'm afraid I'm not known for finesse so its hard to advise navigatin

Re: Monitoring

Todd, Good to have you weigh in on this. You provide a good counterweight. To take a new hypothetical, suppose that one of the many, many patches that Cloudera has championed for Hadoop is critical for Hbase operation or makes Hbase faster. Is it reasonable to answer a question of the form "Is

Re: Monitoring

Let's all resolve not to do that (on-list, particularly). On Mon, Jul 25, 2011 at 11:45 AM, Todd Lipcon wrote: > Then we devolve into an annoying > vendor war which doesn't help anyone. >

Re: Monitoring

On Mon, Jul 25, 2011 at 11:28 AM, Ted Dunning wrote: > I am very sympathetic here. Also, somewhat linguistically challenged on > this point since there is a fine line to be walked. All suggestions are > welcome. > > How should I answer this? The question was "how can I get alerts for my > hbas

Re: Monitoring

I am very sympathetic here. Also, somewhat linguistically challenged on this point since there is a fine line to be walked. All suggestions are welcome. How should I answer this? The question was "how can I get alerts for my hbase cluster"? One answer is definitely MapR. Is there a way to say

Re: Monitoring

On Mon, Jul 25, 2011 at 8:54 AM, Ted Dunning wrote: > Slightly off topic, but MapR runs Hbase very handily (several times faster, > in fact) and provides comprehensive monitoring and alerting out of the box. > Hey Ted: MapR is good stuff indeed but the above can be read as a raw plug for a non-o

Re: Fanning out hbase queries in parallel

Coprocessors are currently only in trunk. They will be in the 0.92 release once we get that out. There's no set date for that, but personally I'll be trying to help get it out sooner than later. On Mon, Jul 25, 2011 at 7:37 AM, Michel Segel wrote: > Which release(s) have coprocessors enabled?

FW: Hbase Shell Throws Runtime Exception

St.Ack, This is indeed a Cloudera distribution. Thanks to Ted Yu I was directed to the source of this error - we had mounted /tmp with "noexec" option for security. I discovered that by removing "noexec" and remounting the partition hbase shell works fine. Cheers, ~Daniel On 7/23/11 3:48 PM, "S

Re: run multiple cluster on same nodes

The short answer is no. It's also not a good idea to even try this. The longer answer is to look at using vmware to split the nodes into two virtual machines... And even then it's not a good idea. Sent from a remote device. Please excuse any typos... Mike Segel On Jul 25, 2011, at 4:09 AM, seve

Re: Fanning out hbase queries in parallel

Which release(s) have coprocessors enabled? Sent from a remote device. Please excuse any typos... Mike Segel On Jul 24, 2011, at 11:03 PM, Sonal Goyal wrote: > Hi Paul, > > Have you taken a look at HBase coprocessors? I think you will find them > useful. > > Best Regards, > Sonal >

Re: Filters for non-Java clients?

The REST API has filter support. Strictly speaking the representation is multilanguage, but only the Java API -- the ScannerModel class, ScannerModel.stringifyFilter -- has support for converting a Java filter tree into a JSON encoded representation of same. However you could do this in Java on

Re: Stargate: Only getting HTTP 200 responses in 0.90.x

Hi Greg, Yes the bug affects any use of the gzip filter.  Best regards,    - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) - Original Message - > From: Greg Cottman > To: "user@hbase.apache.org" > Cc: > Sent: Sunday, July 24, 2011 1

RE: run multiple cluster on same nodes

I believe that it is possible but you need to modify the config to point at different ports / directories for each component. To make sure that you get all of the ports/directories, you will have to go through the hbase-default.xml file and copy each variable that defines a port or directory in

Re: Monitoring

Slightly off topic, but MapR runs Hbase very handily (several times faster, in fact) and provides comprehensive monitoring and alerting out of the box. Contact me off-list for details if you like. On Mon, Jul 25, 2011 at 8:09 AM, Joseph Coleman < joe.cole...@infinitecampus.com> wrote: > Greeting

Monitoring

Greetings, I am relatively new to Hadoop but we now have an 10 node cluster up and running just DFS for now and will be expanding this rapidly as well as adding Hbase. I am looking to find out what people are using for monitoring Hadoop currently. I want to be notified if a node fails, performa

Re: Filters for non-Java clients?

Sounds like a good idea, file a JIRA. -Joey On Sun, Jul 24, 2011 at 10:20 PM, Greg Cottman wrote: > > We are using the REST interface because we have a C++ client, but get > performance complaints arising from the fact that we have to fetch the > entire table for any query. > > Is anyone conside

Re: problem when change zookeeper.znode.parent

Hi, I really appreciate your support. Now I'm waiting for the fixed version. Thanks a lot! 2011/7/25 Ramkrishna S Vasudevan : > Hi > > Filed JIRA - HBASE-4138. > > Will try providing a patch for the same. > > Regards > Ram > > > **

HBQL connection

Hi, Does anyone use HBQL? I found ZooKeeper client connection info every time while executing stmt.executeQuery. HResultSet results = stmt.executeQuery(query); I think once the connection setups up that can be used repeatedly. Any suggestion would be appreciated? Thank you very much. Fleming C

Re: Hbck errors in 0.90.3

Hi Stack, finally the migration worked. We copied table data from 0.20.4 hbase cloud to cdh3u1 cloud (0.90.3 hbase) by using the mozilla approach to copying the files on hdfs level. ( http://blog.mozilla.com/data/2011/02/04/migrating-hbase-in-the-trenches/) As described in the post from mozilla i

Re: HBase Connection error

If it can help you : org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server null for region , row '', but failed after 10 attempts. Exceptions: java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@6bd9e2c7closed java.io

run multiple cluster on same nodes

hi,all I set up one hbase cluster on 8 nodes successfully.Each node has a RegionServer on it. For some reason,i need set up another hbase cluster on the same 8 nodes. But when a start up hbase,and run 'status' on hbase shell, something strange happened!!! The new hbase cluster claim tha

RE: problem when change zookeeper.znode.parent

Hi Filed JIRA - HBASE-4138. Will try providing a patch for the same. Regards Ram *** This e-mail and attachments contain confidential information from HUAWEI, which is intended only for the person or entity w

Re: problem when change zookeeper.znode.parent

Please file a JIRA. Thanks On Jul 25, 2011, at 12:32 AM, Ramkrishna S Vasudevan wrote: > Hi, > > I found the problem why it is continuously hanging when we use a Table > object. > When we use the Admin object first it tries to check the master. > If the zookeeper.znode.parent is not specif

RE: problem when change zookeeper.znode.parent

Hi, I found the problem why it is continuously hanging when we use a Table object. When we use the Admin object first it tries to check the master. If the zookeeper.znode.parent is not specified in the client So it takes the default zookeeper.znode.parent=/hbase and tries to connect to the master