Re: FATAL org.apache.hadoop.hbase.regionserver.wal.HLog: Could not append

2011-10-13 Thread lars hofhansl
Did you do some heavy inserting during that time? I assume you use 0.90.x(?) Might be HBASE-4387. -- Lars - Original Message - From: Stuti Awasthi To: "user@hbase.apache.org" Cc: Sent: Thursday, October 13, 2011 9:44 PM Subject: FATAL org.apache.hadoop.hbase.regionserver.wal.HLog: C

FATAL org.apache.hadoop.hbase.regionserver.wal.HLog: Could not append

2011-10-13 Thread Stuti Awasthi
Hi Friends I am using a mini distributed cluster of 3 nodes. Regionservers are running on all of 3 nodes. This cluster is operational from 2 weeks now without any issue. Today morning I saw one of the region servers are down. I restarted the regionserver dynamically and it started fine and master

Re: Hive+HBase performance is much poorer than Hive+HDFS

2011-10-13 Thread Todd Lipcon
Most of the benchmarks I've seen are about what you're seeing 4-5x overhead reading from HBase vs straight DFS files. Makes sense as we have a whole extra layer involved, plus locking overhead, etc. We can probably do some more optimization and get down to a 2x difference, but we'll never be as fa

Re: HTable.autoFlush not exposed via Thrift?

2011-10-13 Thread Jean-Daniel Cryans
On Thu, Oct 13, 2011 at 1:47 PM, Norbert Burger wrote: > > > BTW, can your app support the semantics of having auto flush off? What's > > your use case like? > > > > By semantics, do you mean the possibility that unflushed data might be lost > if the client goes away? Yep, or if the Thrift serve

Re: HTable.autoFlush not exposed via Thrift?

2011-10-13 Thread Norbert Burger
On Thu, Oct 13, 2011 at 1:05 PM, Jean-Daniel Cryans wrote: > On Thu, Oct 13, 2011 at 8:17 AM, Norbert Burger >wrote: > > > In the ticket, I mentioned setting hbase.client.write.buffer as a > > workaround, but unfortunately it doesn't seem that autoflush (which I > > imagine has a bigger impact on

Re: problem in starting in Hbase

2011-10-13 Thread Jean-Daniel Cryans
Have you done what the exception message tells you to do? It's related to ZooKeeper, not HDFS. J-D On Wed, Oct 12, 2011 at 2:58 PM, Jignesh Patel wrote: > I have following setup in my hbase-site.xml > > > hbase.rootdir > hdfs://localhost:9000/hbase > > > dfs.replication > 1 > > > > but whe

Re: Hive+HBase performance is much poorer than Hive+HDFS

2011-10-13 Thread Jean-Daniel Cryans
Your question is more basic than that, it's actually how much slower is it to sequentially read in HBase compared to HDFS. I'm not sure anyone quantified that, and there's probably a bunch of factors that can influence it, but at least you should try to get the same level of distribution eg since y

Re: HTable.autoFlush not exposed via Thrift?

2011-10-13 Thread Jean-Daniel Cryans
On Thu, Oct 13, 2011 at 8:17 AM, Norbert Burger wrote: > Thanks Ted, I created https://issues.apache.org/jira/browse/HBASE-4586. > > In the ticket, I mentioned setting hbase.client.write.buffer as a > workaround, but unfortunately it doesn't seem that autoflush (which I > imagine has a bigger impa

Re: HTable.autoFlush not exposed via Thrift?

2011-10-13 Thread Norbert Burger
Thanks Ted, I created https://issues.apache.org/jira/browse/HBASE-4586. In the ticket, I mentioned setting hbase.client.write.buffer as a workaround, but unfortunately it doesn't seem that autoflush (which I imagine has a bigger impact on write performance) isn't exposed. Norbert On Wed, Oct 12,

Re: Hive+HBase performance is much poorer than Hive+HDFS

2011-10-13 Thread Weihua JIANG
After set this argument to 1000, I get a result: hive/hbase is 4X slower than hive/hdfs. how much X is the expected slowdown for hive/hbase vs hive/hdfs? Thanks Weihua 2011/10/12 Akash Ashok : > Hi, > To set this parameter you could use "set hbase.client.scanner.caching=500;" > before the execut