https://issues.apache.org/jira/browse/HBASE-7711 ?
This time we checked, and no failed disks on the machine, but same problem
with failing writing to local data node and then it was excluded.
On Wed, Feb 19, 2014 at 1:02 AM, Stack wrote:
> On Mon, Feb 17, 2014 at 1:59 AM, Asaf Mesika
Hi,
I've noticed that in 0.94.7, when you execute a coprocessor, the result
object is converted into a byte buffer, using write() method which is on
the result object.
So, if my result object is 500mb in size, another 500mb is consumed from
the heap, since it is converted to a byte buffer before s
There is a property you can tune to lower default num of retries from 10 to
any number like 2.
On Wednesday, April 9, 2014, kanwal wrote:
> I'm currently running into an issue on my local setup where my application
> is
> unable to connect to the hbase table but I'm successfully able to query th
Bear in mind each region will return its top n, then you will have to run
another top n in your client code. This introduce a numerical error : top
on top.
On Thursday, April 10, 2014, Bogala, Chandra Reddy
wrote:
> Hi,
> I am planning to write endpoint coprocessor to calculate TOP N results for
The Jira says it's enabled by auto. Is there an official explaining this
feature?
On Wednesday, April 9, 2014, Ted Yu wrote:
> Please take a look at http://www.n10k.com/blog/blockcache-101/
>
> For D, hbase.regionserver.global.memstore.size is specified in terms of
> percentage of heap. Unless y
Is it more like Phoenix or Apache Drill?
On Wednesday, March 19, 2014, Tapper, Gunnar wrote:
> Hi,
>
> Sorry it took me so long to respond to this; I got very busy with shipping
> HPDSM. :)
>
> Trafodion has been tested with 0.94.6; 0.94.22 testing is in progress.
>
> You can't download the Traf
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
On Mon, Feb 10, 2014 a
Hi,
We have HBase 0.94.7 deployed in production with 54 Region Servers (Hadoop
1).
Couple of days ago, we had an incident which made our system unusable for
several hours.
HBase started emitting WARN exceptions indefinitely, thus failing any
writes to it. Until stopped this RS, the issue wasn't re
If hot means many requests, then it's only in 0.96 right? 0.94 is only
addressing capacity load on terms of numbers of regions per region server
of the same table.
On Monday, January 20, 2014, Ted Yu wrote:
> bq. under heavy load by serving to hot regions
>
> Did you mean 'two hot regions' ?
> I
Your start rowkey is the prefix it self.
Your end row key is the prefix +1. Increase the last byte by 1. I can send
you the increase function if you need.
No filter needed.
On Wednesday, January 15, 2014, Ramon Wang wrote:
> Hi Folks
>
> We have a table with fixed pattern row key design, the for
You also can't be sure the second rowkey will be in the same region which
is requires you to go on the network to another region server.
On Thursday, January 16, 2014, Amit Sela wrote:
> I have a case where I want to "split" rows with a lot of qualifiers (a very
> small amount of rows < 1%, with
One strong storage efficient feature would be to serialize your Pojo to
binary using Avro (as one example) and saving it as a column. This saves
you on storage since you don't have to write your row key for every column.
If you load the entire Pojo it will save on read time. Also it enables to
supp
There are two projects doing orm for HBase, one named Kundera - check it
out.
On Sunday, January 5, 2014, zhang_xzhi wrote:
> I just read some short introduction of Phoenix these days.
> If I am wrong, pls let me know, thx.
> It seems to me Phoenix will provide a JDBC-driver against hbase.
> But
Couple of notes:
1. When updating to status you essentially add a new rowkey into HBase, I
would give it up all together. The essential requirement seems to point at
retrieving a list of urls in a certain order.
2. Wouldn't salting ruin the sort order required? Priority, date added?
On Friday, Jan
Are the regions scanned in parallel?
On Friday, January 3, 2014, rajeshbabu chintaguntla wrote:
>
> Here are some performance numbers with RLI.
>
> No Region servers : 4
> Data per region: 2 GB
>
> Regions/RS| Total regions| Blocksize(kb) |No#rows matching values| Time
> taken(sec)|
> 50 |
Does Phoenix exposes metrics about its code execution? Network time,
coprocessor time, client time, etc?
On Sunday, December 22, 2013, lars hofhansl wrote:
> You would have to measure the incoming/outgoing traffic on the affected
> machine.
>
> The easiest is to periodically check the output of i
Hi,
Are Filter implementations need to be thread-safe?
Thanks,
Asaf
Hi,
Does anybody have an open source implementation of ColumnRangeFilter but
for multiple ranges?
How do you check in your code you only have one region?
How do you retrieve the HRegion exactly? Since you are running in inside a
coprocessor I presume you have such rpc server per region since Hbase
creates an instance of your coprocessor for each region.
On Tuesday, December 3, 2013, Wukang Lin
In this method, you can get the region's Load per region:
private Map getRegionsLoad() {
try {
Map regionsNameToLoad = new HashMap();
ClusterStatus clusterStatus = hAdmin.getClusterStatus();
for (ServerName serverName : clusterStatus.getServers()) {
I tacked the same problem, and was answerd it was fixed in 0.95
On Thursday, November 28, 2013, Pablo Medina wrote:
> Hi all,
>
> Knowing that replication metrics are global at the region server level in
> HBase 0.94.13, what is the meaning of a metric like sizeOfLogQueue when
> replicating to mo
On our project we store nested record structures with 10-40 fields. We have
decided to save on storage and write throughout by writing a serialized
avro record as value. We place one byte before to allow versioning. We did
it since each column is written with its rowkey, cq, cf and timestamp. Your
thread?
>
> And how to optimize it?
>
> Config BLOCKSIZE/WriteBuffer ...?
>
> I'm a newbie.
> Thanks for help!
>
> ------
>
> jingych
>
> *From:* Asaf Mesika
> *Date:* 2013-11-26 19:20
> *To:* user@hbase.apache.org 'use
The need is too broad. You need to do through the HBase jmx metrics, ands
machine metrics to see what is your bottleneck.
On Tuesday, November 26, 2013, Job Thomas wrote:
>
>
> Hi All,
>
> How can we configure Hbase inorder to perform multythreading/parallel
> query faster .
>
> These are some
How many client threads?
On Tuesday, November 26, 2013, jingych wrote:
> Hi, All!
>
> Recently, I just test the Client write performance with HBase Java API.
>
> I think the performance X-factor is :
>
> 1) client buffer size;
> 2) server memstore size;
>
> But when i did the test, I have the re
bin/hbase shell
In there:
Type help and you'll get along
On Monday, November 25, 2013, ashishkshukladb wrote:
> I want to get the metadata information in Hbase. My basic purpose is to -
>
> 1.get the information about tables like how many tables and name of tables.
>
> 2.get the columnfamilies na
with it, so it may work
> > for you, too.
> >
> > Otis
> > --
> > Performance Monitoring * Log Analytics * Search Analytics
> > Solr & Elasticsearch Support * http://sematext.com/
> >
> >
> > On Wed, Nov 20, 2013 at 1:00 AM, Asaf Mesika
&
oring * Log Analytics * Search Analytics
> Solr & Elasticsearch Support * http://sematext.com/
>
>
> On Wed, Nov 20, 2013 at 1:00 AM, Asaf Mesika
> wrote:
> > Thanks for clearing that out.
> > I'm using your message to ping anyone who assist as to it appears the use
; But I agree that in general with this kind of workload, we should handle
> WAL files more intelligently and free up those WAL files which don't have
> any dependency (that is, all their entries are already flushed) when
> archiving. We do that in trunk but not in any released version,
The crashing sounds odd. What's your exception?
On Monday, November 11, 2013, Andrea wrote:
> Hi, I have HBase above Hadoop with one master node, one zookeeper node and
> four regionservers. When I turn up everything, on two nodes (or sometime
> the
> same node) are created the tables .META. and
Are you reading using HBase client or do you have an inputFormat for
reading HFiles?
On Wednesday, November 13, 2013, Amit Sela wrote:
> Hi,
>
> We do something like that programmatically.
> Read blobbed HBase data (qualifiers represent cross-sections such as
> country_product and blob data such
My "dream" is to get to your point. I was always stopped before reaching
network limit. My premise was the WAL synchronize was the key bottleneck in
write.
How much data are you inserting? How much client threads? Batch size in
client?
Share some more info on your cluster and test setup.
On Thurs
Using java, you can use HBaseAdmin to get cluster load, there, for each
region you can get write and read request. Just sum up all regions to get
total read and total write request.
On Thursday, November 14, 2013, Sandeep L wrote:
> Is it possible to get from api instead of hbase_metrics.
>
> Tha
e.
>
> Hope this helps,
> Mike
>
>
> On Sat, Nov 16, 2013 at 1:16 AM, Ted Yu wrote:
>
> > bq. all regions of that customer
> >
> > Since the rowkey starts with , any single customer would only
> > span few regions (normally 1 region), right ?
> >
&
bucket.
On Sat, Nov 16, 2013 at 8:16 AM, Ted Yu wrote:
> bq. all regions of that customer
>
> Since the rowkey starts with , any single customer would only
> span few regions (normally 1 region), right ?
>
>
> On Fri, Nov 15, 2013 at 9:56 PM, Asaf Mesika
> wrote:
>
>
ince you
> query
> > on it
> >
> > Have you looked at this API in Scan ?
> >
> > public Scan setTimeRange(long minStamp, long maxStamp)
> >
> >
> > Cheers
> >
> >
> > On Fri, Nov 15, 2013 at 1:28 PM, Asaf Mesika
> > wrote:
> &g
gt; Have you looked at this API in Scan ?
>
> public Scan setTimeRange(long minStamp, long maxStamp)
>
>
> Cheers
>
>
> On Fri, Nov 15, 2013 at 1:28 PM, Asaf Mesika
> wrote:
>
> > The problem is that I do know my rowkey design, and it follows people's
> >
region"
>
> Cheers
> Ramon
>
>
> On Thu, Nov 14, 2013 at 8:47 PM, Asaf Mesika
> wrote:
>
> > It's from the same table.
> > The thing is that some simply have less data saved in HBase,
> > while others have x50 (max) data.
> > I'm tryin
We are using both of the following
properties: hbase.regionserver.dns.interface, base.master.dns.interface.
Both set to the interface name we want. We have two interfaces as you
described - one for inner communication and one for external.
What exactly is not working for you?
On Wed, Nov 13, 20
or different table, you may consider to combine some
> cold regions for some tables.
>
> Thanks
> Ramon
>
>
> On Thu, Nov 14, 2013 at 4:59 PM, Asaf Mesika
> wrote:
>
> > Hi,
> >
> > Have anyone ran into a case where a Region Server is hosting regions, in
>
Hi,
Have anyone ran into a case where a Region Server is hosting regions, in
which some regions are getting lots of write requests, and the rest gets
maye 1/1000 of the rate of write requests?
This leads to a situation where the HLog queue reaches its maxlogs limit
since, those HLogs containing t
Hi,
I see that https://issues.apache.org/jira/browse/HBASE-3680 is talking
about adding MSLAB metrics (size of mslab for region, etc).
I see that it is fixed in 0.92.3, but when looking at the source code of
0.94.7 I can't find it in RegionServerMetrics.java, or any other java file.
Am I missing
Also note that due to HDFS placement policy the store file is written
locally - we'll first replica of each of its blocks so it's dispersed
amongst the drives you have onto on the data node which is also the region
server.
On Thursday, November 7, 2013, Ted Yu wrote:
> The store file is flushed t
Both are created when you declare the table and not in runtime so in
shouldn't matter to you anyway
On Thursday, November 7, 2013, Nasron Cheong wrote:
> Why is that? Afaik everything is just a byte sequence, what prevents
> non-printable chars from being used in CF/table names?
>
> - Nasron
>
>
On this note:
If I hammer a region server with scans that has let's say 100 rows caching
- is there an OOME protections which lowers the caching in case ther server
is stressed in memory? Or will it fail the region server with OOME?
On Monday, November 4, 2013, Anoop John wrote:
> Have u tested t
ly. The
> question is, where the bytes are compressed? Directly in the map-function
> (If no, is it possible to compress it there with lzo?!) or in the region
> server?
>
> kind regards
>
>
> 2013/11/2 Asaf Mesika
>
> > If mean, if you take all those bytes if the bit
you know an implementation?
>
> kind regards
>
>
> 2013/11/2 Asaf Mesika >
>
> > I would try to compress this bit set.
> >
> > On Nov 2, 2013, at 2:43 PM, John >
> wrote:
> >
> > > Hi,
> > >
> > > thanks for your answe
I would try to compress this bit set.
On Nov 2, 2013, at 2:43 PM, John wrote:
> Hi,
>
> thanks for your answer! I increase the "Map Task Maximum Heap Size" to 2gb
> and it seems to work. The OutOfMemoryEroror is gone. But the HBase Region
> server are now crashing all the time :-/ I try to stor
Can you please explain why is this suspicious?
On Monday, October 7, 2013, Jean-Daniel Cryans wrote:
> This line:
>
> [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74,
> real=88.77
> secs]
>
> Is suspicious. Are you swapping?
>
> J-D
>
>
> On Mon, Oct 7, 2013 at 8:34 AM, prak
Hi,
I've been reading here that HBase 0.94.x has been working in production for
a few folks here with Java7.
I also read that CDH4.4 is not recommended to work with Java 7 in
production.
Anybody has any idea why?
How many Parallel GC were you using?
Regarding block cache - just to see I understood this right: if your are
doing a massive read in HBase it's better to turn off block caching through
the Scan attribute?
On Thursday, October 10, 2013, Otis Gospodnetic wrote:
> Hi Ramu,
>
> I think I saw mentio
Bucket seems like a rather good name for it. The method for generating
could be Hash, running sequence modded, etc. So HashBucket,
RoundRobinBucket, etc.
On Tuesday, October 22, 2013, James Taylor wrote:
> One thing I neglected to mention is that the table is pre-split at the
> "prepending-row-ke
Can you give an example of your query?
On Friday, November 1, 2013, Nasron Cheong wrote:
> Hi,
>
> I'm trying to determine the best way to serialize a sequence of
> integers/strings that represent a hierarchy for a column qualifier, which
> would be compatible with the ColumnPrefixFilters, and Bi
Check through HDFS UI that your cluster haven't reached maximum disk
capacity
On Thursday, October 24, 2013, Vimal Jain wrote:
> Hi Ted/Jean,
> Can you please help here ?
>
>
> On Tue, Oct 22, 2013 at 10:29 PM, Vimal Jain >
> wrote:
>
> > Hi Ted,
> > Yes i checked namenode and datanode logs and i
I couldn't get the Row Value Constructor feature.
Do you perhaps have a real world use case to demonstrate this?
On Friday, October 25, 2013, James Taylor wrote:
> The Phoenix team is pleased to announce the immediate availability of
> Phoenix 2.1 [1].
> More than 20 individuals contributed to th
That seems like too much client threads. How much mb/sec did you on that 1
RS?
On Friday, October 25, 2013, Vladimir Rodionov wrote:
> You can not saturate region server with one client (unless you probably
> use hbase-async) if all data is cached in RAM.
> In our performance tests we have run 10
If your query (scan) needs a region on the failed region server, the client
will fail and silently retry about 10 times. The sleep time increase as
each retry fails and can reach 10min. On the server side, the master takes
3min to realize the RS failed thus issue a region move, which might take a
f
How much data do you that you need NoSQL db?
On Monday, October 28, 2013, saiprabhur wrote:
> Hi Folks,
>
> New to NOSQL designing data model for primary care system. i have
> normalized
> sample DB relationship model e.g. HBASE-0.94.0
>
> Patient table:
>
> 1) Patient_id - PK
> 2) A
Hi,
I'm writing an Endpoint Coprocessor for HBase v0.94.6 (cdh4.3.1).
I'm trying to understand how the CoprocessorProtocol and
CoprocessorEndPoint implementation versioning works, both backwards and
forward.
For instance, I have my protocol as:
public interface MyQueryProtocol extends Coprocess
We did the same but on the client side, without any issue
On Monday, August 26, 2013, Olle Mårtensson wrote:
> Hi,
>
> I have developed a coprocessor that is extending BaseRegionObserver and
> implements the
> postPut method. The postPut method scans the columns of the row that the
> put was issu
Any Cloudera release for that as well?
On Saturday, August 24, 2013, Aaron T. Myers wrote:
> Hello,
>
> Please see below for the official announcement of a serious security
> vulnerability which has been discovered and subsequently fixed in Apache
> HBase releases.
>
> Best,
> Aaron
>
> -BEGI
Ted, can you elaborate a little bit why this issue boosts performance?
I couldn't figure out from the issue comments if they execCoprocessor scans
the entire .META. table or and entire table, to understand the actual
improvement.
Thanks!
On Fri, Aug 9, 2013 at 8:44 AM, Ted Yu wrote:
> I thin
If you can mark a row by adding a column qualifier which will be used as
your flag by its existence, and its name will be lexicographically first,
then it won't be slow as you said about filters below.
On Monday, August 12, 2013, ccalugaru wrote:
> Hi all,
> I have the following hbase use case:
>
Please list down your jvm switches for hbase
On Wednesday, August 7, 2013, Vimal Jain wrote:
> Hi Ted,
> I am using centOS.
> I could not get output of "ps aux | grep pid" as currently the hbase/hadoop
> is down in production due to some internal reasons.
>
> Can you please help me in figuring ou
Yep. That's a confusing one.
When running /hbase stop master, it sets the shutdown flag in ZK. RS listen
in on this flag, and once they see it set, they shut them selfs down. Once
they are all down, the master goes down as well.
On Saturday, August 3, 2013, Jean-Daniel Cryans wrote:
> Ah then doi
Cool!
Do you guys store your data twice then? Once on HBase and another on ES?
And then add index data?
On Tuesday, August 6, 2013, Alex Newman wrote:
> Based on the previous work using async libraries to index HBase into
> elastic search, I've created.
>
> https://github.com/posix4e/Elasticsearc
We have also developed a fabric customization for HBase stack and all our
components.
Anyone had experience with Ambari relative to Fabric?
On Tuesday, August 6, 2013, Jay Vyas wrote:
> thanks .. !
>
> well there certainly are a few customized ones on github but they are
> varying degrees of mai
Hi,
Is there a way in MRv2 to direct a reducer to run in the RS it will write
its Puts, thus achieve write locality?
Thanks!
Asaf
Very cute.
On Monday, July 15, 2013, anil gupta wrote:
> Hey Swarnim,
>
> Really nice work. Keep on going!
>
> ~Anil
>
>
>
> On Sun, Jul 14, 2013 at 5:59 PM, kulkarni.swar...@gmail.com <
> kulkarni.swar...@gmail.com > wrote:
>
> > Hello all,
> >
> > In order to help understand the intricacies of
If you get KeyValue object by the client, then you have its size for free
(it's the size of its internal byte array) - so why is it time consuming?
On Fri, Jul 5, 2013 at 2:23 AM, Bikash Agrawal wrote:
> Hi ,
>
> Need to calculate data size in HBase. I can do it by using KV length. But
> it is t
To my knowledge, scan is not parallel, hence the speed of queries of
Impala, Phoenix, and other similar projects.
On Saturday, July 13, 2013, ch huang wrote:
> hi ted ,for example i have a table with 10 regions, if i offer the
> condition hit the data of 8 regions,is it different do it use oragin
gt; On Thu, Jul 11, 2013 at 9:11 PM, Asaf Mesika
> >
> wrote:
>
> > Do you think prefix compression can also be utilized here? In our use
> case
> > we sent a list of Put of counters in which the key is quite long and the
> > keys are quite similar to one another.
Dump the jar in the lib directory in hbase , for ever region server.
On Friday, July 12, 2013, ch huang wrote:
> ATT
>
do is just load it into my test table ,only for
> the regions of the table
>
> On Fri, Jul 12, 2013 at 12:07 PM, Asaf Mesika
> wrote:
>
> > The only way to register endpoint coprocessor jars is by placing them in
> > lib dir if hbase and modifying hbase-site.xml to point
You need to see the jvm crash in .out log file and see if maybe its the .so
native Hadoop code that making the problem. In our case we
Downgraded from jvm 1.6.0-37 to 33 and it solved the issue.
On Friday, July 12, 2013, David Koch wrote:
> Hello,
>
> NOTE: I posted the same message in the the C
k at http://hbase.apache.org/book.html#d2617e13654 and section
> J.4.3.2
>
> On Thu, Jul 11, 2013 at 9:01 PM, Asaf Mesika
> >
> wrote:
>
> > I thought that in 0.95 ProtoBuf provides RPC compression, no?
> >
> > On Friday, July 12, 2013, Alok Singh Mahor wrote:
> >
The only way to register endpoint coprocessor jars is by placing them in
lib dir if hbase and modifying hbase-site.xml to point to it under a
property name I forgot at the moment.
What you described is a way to register an Observer type coprocessor.
On Friday, July 12, 2013, ch huang wrote:
> i
I thought that in 0.95 ProtoBuf provides RPC compression, no?
On Friday, July 12, 2013, Alok Singh Mahor wrote:
> To Jean :
> Thanks for replying. well could you please elaborate your answer..and by
> that 'query' ..i meant can anyone clear my doubt :-)
>
> To Doug:
> Thanks for replying. but the
Great! Waiting for the videos as it looks like a very interesting
conference.
On Wednesday, July 10, 2013, Azuryy Yu wrote:
> Hi dear all,
>
> HBase con 2013 slides are available now.
>
> http://www.hbasecon.com/schedule/
>
> Just share information here.
>
This means you can safely run Hadoop and Hbase on jvm 7?
We were just considering switching in production to java 7.
On Thursday, July 11, 2013, Azuryy Yu wrote:
> Otis,
>
> I will do this test, maybe on the end of this month. because I haven't big
> memory server for test now util the end of thi
What the hdfs data locality metric?
And remote read and local read?
On Thursday, July 4, 2013, Viral Bajaria wrote:
> Currently datanode shows a lot of clienttrace logs for DFSClient. I did a
> quick command line check to see how many clienttrace do I get per active
> RegionServer and it seems th
I think there is a metric in HBase and HDFS (JMX) reflecting that.
If you find it and find it useful, do tell...
On Thursday, July 4, 2013, Viral Bajaria wrote:
> Hi,
>
> If I have enabled shortcircuit reads, should I ever be seeing clienttrace
> logs in the datanode for the regionserver DFSClien
Seems right. You can make it more efficient by creating your result array
in advance and then fill it.
Regarding time filtering. Have you see that in Scan you can set start time
and end time?
On Wednesday, July 3, 2013, Flavio Pompermaier wrote:
> All my enums produce positive integers so I don't
Did you somehow turned the Security flag on for HBase since your exception
is Security related.
On Wednesday, July 3, 2013, SamSalman wrote:
> Hello Experts,
>
> I am quite new to Hbase and Hadoop and above all new to java too. Recently
> started working on Hbase and Java. I have successfully ins
Do you have only 5 static author names?
Keep in mind the column family name is defined when creating the table.
Regarding tall vs wide debate:
HBase is first and for most a Key Value database thus reads and writes in
the column-value level. So it doesn't really care about rows.
But it's not entire
Yep. Other DBs like
Mongo may have the stuff you need out of the box.
Another option is to encode the whole class using Avro, and writing a
filter on top of that.
You basically use one column and store it there.
Yes, you pay the penalty of loading your entire class and extract the
fields you need t
rt of the code.
> >
> > I have no good explanation then, and also no good suggestion about how
> to improve this.
> >
> >
> >
> >
> > From: Asaf Mesika
> > To: "user@hbase.apache.org" ; lars hofhansl <
> la...@apach
in the local cluster.
>
> Also my first reply - upon re-reading it - sounded a bit rough, that was
> not intended.
>
> -- Lars
>
>
> - Original Message -
> From: Asaf Mesika
> To: "user@hbase.apache.org" ; lars hofhansl <
> la...@apache.org>
&
Did you guys thought about adding coprocessors hooks to the replication,
like preReplicateLogEntries, or something like that? I mean, in his case,
perhaps utilizing it could have changed the table name before running
through the replication process at the sink RS.
On Fri, Jun 21, 2013 at 1:48 AM
You shouldn't tweak any Configuration at all. It simply works - setting up
both mini HDFS and mini HBase.
On Fri, Jun 21, 2013 at 2:36 AM, Varun Sharma wrote:
> Hi Christopher, I am looking into getting maven 3.0.4 installed before i
> start playing with it.
>
> Elliot, does the HBaseTestingUt
On Thu, Jun 20, 2013 at 9:42 PM, yun peng wrote:
> Thanks Asaf, I made the response inline.
>
> On Thu, Jun 20, 2013 at 9:32 AM, Asaf Mesika
> wrote:
>
> > On Thu, Jun 20, 2013 at 12:59 AM, yun peng
> wrote:
> >
> > > Thanks for the reply. The id
e time it takes to run the
HTable.batch call on the slave RS, right? If I enable compression somehow
(hack HBase code to test drive it), I will only speed up transfer time of
the batch to the slave RS, but still wait on the insertion of this batch
into the slave cluster.
> -- Lars
>
lly, but this under
utilizing my cluster hardware, no?
> As for your other question, more threads can lead to better interleaving
> of CPU and IO, thus leading to better throughput (this relationship is not
> linear, though).
>
>
>
> -- Lars
>
>
>
> - O
if (!admin.tableExists(tableName)){
Shouldn't it be:
if (admin.tableExists(tableName)){
?
On Thu, Jun 20, 2013 at 6:42 PM, Aji Janis wrote:
> I have the following function for adding column family but its not working
> for some reason. I am able to get proper configuration and establish the
>
bably includes HDFS replication. There's much I can do about that
though.
We should note that the network capacity is not the issue, since it was
measured 30MB/sec Receive and 20MB/sec Transmit, thus it's far from the
measured max bandwidth of 111MB/sec (measured by running nc - netcat).
gion mapping.
>
Why do you need the sequential key approach? Let's say you have a group
data correlated in some way but is scattered in 2-3 RS. You can always
write a coprocessor to run some logic close to the data, and then run it
again on the merged data in the client side, right?
>
>
Hi,
I've been conducting lots of benchmarks to test the maximum throughput of
replication in HBase.
I've come to the conclusion that HBase replication is not suited for write
intensive application. I hope that people here can show me where I'm wrong.
*My setup*
*Cluster (*Master and slave are al
omize such policy on hmaster? Or there is a similiar
> way as what CoProcessor does on region servers...
>
>
> On Wed, Jun 19, 2013 at 4:58 PM, Asaf Mesika
> >
> wrote:
>
> > The new splitted region might be moved due to load balancing. Aren't you
> > experienci
The new splitted region might be moved due to load balancing. Aren't you
experiencing the classic hot spotting? Only 1 RS getting all write traffic?
Just place a preceding byte before the time stamp and round robin each put
on values 1-num of region servers.
On Wednesday, June 19, 2013, yun peng w
:09 AM, Asaf Mesika
> wrote:
>
> > Does HBase Region Server arbitrarily chooses which one to publish to its
> >
>
> The metrics reports the last source to report in. So for recovered edits
> since it will probably have higher throughput than the live log, you'll
>
1 - 100 of 214 matches
Mail list logo
