Thanks for the explanation Qian!
I think being able to balance empty regions is important and the preferred
result to me.
The best way so far is to manually 'balance' the regions if we need to add
pre-splits dynamically.
Jianshi
On Tue, Sep 23, 2014 at 11:35 AM, Qiang Tian wrote:
> Hello,
Cycling previous bits (w.r.t. adjusting weights for table skew):
http://search-hadoop.com/m/DHED4CWSqW1/snapshot+timeout+problem&subj=Re+snapshot+timeout+problem
Cheers
On Mon, Sep 22, 2014 at 8:35 PM, Qiang Tian wrote:
> Hello, I happened to got balancer related issues 2 months ago and looked
considering 80-20 rule(does it apply to hbase tables?), what about
specifying balance property(per-table vs non per-table) on a table
basis(default is non per-table), then those few large tables could be
balanced well with acceptable total balance cost?
On Tue, Sep 23, 2014 at 11:35 AM, Qiang Tia
Hello, I happened to got balancer related issues 2 months ago and looked at
that part, below is a summary:
1)by default, hbase balancer(StochasticLoadBalancer by default) does not
balance regions per table. i.e. all regions are considered as 1 table. so
if you have many tables, especially some tab
https://issues.apache.org/jira/browse/HBASE-11368 related to the original
issue too.
On Mon, Sep 22, 2014 at 10:18 AM, Ted Yu wrote:
> As you noted in the FIXME, there're some factors which should be tackled by
> balancer / assignment manager.
>
> Please continue digging up master log so that we
As you noted in the FIXME, there're some factors which should be tackled by
balancer / assignment manager.
Please continue digging up master log so that we can find the cause for
balancer not fulfilling your goal.
Cheers
On Mon, Sep 22, 2014 at 10:09 AM, Jianshi Huang
wrote:
> Ok, I fixed this
Ok, I fixed this by manually reassign region servers to newly created ones.
def reassignRegionServer(admin: HBaseAdmin, regions: Seq[HRegionInfo],
regionServers: Seq[ServerName]): Unit = {
val rand = new Random()
regions.foreach { r =>
val idx = rand.nextInt(regionServers.size)
Hmm...any workaround? I only want to do this:
Rebalance the new regions *evenly* to all servers after manually adding
splits, so later bulk insertions won't cause contention.
P.S.
Looks like two of the region servers which had majority of the regions were
down during Major compaction... I guess i
Yes, I have access to Master UI, however logs/*.log cannot be opened or
downloaded, must be some security restrictions in the proxy...
Jianshi
On Tue, Sep 23, 2014 at 12:06 AM, Ted Yu wrote:
> Do you have access to Master UI ?
>
> :60010/logs/ would show you list of log files.
>
> The you can v
Do you have access to Master UI ?
:60010/logs/ would show you list of log files.
The you can view :60010/logs/hbase--master-XXX.log
Cheers
On Mon, Sep 22, 2014 at 9:00 AM, Jianshi Huang
wrote:
> Ah... I don't have access to HMaster logs... I need to ask the admin.
>
> Jianshi
>
> On Mon, Sep
Ah... I don't have access to HMaster logs... I need to ask the admin.
Jianshi
On Mon, Sep 22, 2014 at 11:49 PM, Ted Yu wrote:
> bq. assign per-table balancer class
>
> No that I know of.
> Can you pastebin master log involving output from balancer ?
>
> Cheers
>
> On Mon, Sep 22, 2014 at 8:29 A
bq. assign per-table balancer class
No that I know of.
Can you pastebin master log involving output from balancer ?
Cheers
On Mon, Sep 22, 2014 at 8:29 AM, Jianshi Huang
wrote:
> Hi Ted,
>
> I moved setBalancerRunning before balancer and run them twice. However I
> still got highly skewed regi
Hi Ted,
I moved setBalancerRunning before balancer and run them twice. However I
still got highly skewed region distribution.
I guess it's because of the StochasticLoadBalancer, can I assign per-table
balancer class in HBase?
Jianshi
On Mon, Sep 22, 2014 at 9:50 PM, Ted Yu wrote:
> admin.set
admin.setBalancerRunning() call should precede the call to admin.balancer().
You can inspect master log to see whether regions are being moved off the
heavily loaded server.
Cheers
On Mon, Sep 22, 2014 at 1:42 AM, Jianshi Huang
wrote:
> Hi Ted and others,
>
> I did the following after adding s
Hi Ted and others,
I did the following after adding splits (without data) to my table, however
the region is still very imbalanced (one region server has 221 regions and
other 50 region servers have about 4~8 regions each).
admin.balancer()
admin.setBalancerRunning(true, true)
The ba
Yes. See the following method in HBaseAdmin:
public boolean balancer()
On Fri, Sep 5, 2014 at 9:38 AM, Jianshi Huang
wrote:
> Thanks Ted!
>
> Didn't know I still need to run the 'balancer' command.
>
> Is there a way to do it programmatically?
>
> Jianshi
>
>
>
> On Sat, Sep 6, 2014 at 12:29
Thanks Ted!
Didn't know I still need to run the 'balancer' command.
Is there a way to do it programmatically?
Jianshi
On Sat, Sep 6, 2014 at 12:29 AM, Ted Yu wrote:
> After splitting the region, you may need to run balancer to spread the new
> regions out.
>
> Cheers
>
>
> On Fri, Sep 5, 20
Hi Steven,
I did 1) and 2) and the error was during LoadIncrementalHFiles.
I can't do 3) because that CF is mostly used for mapreduce inputs, so a
continuous rowkey is preferred.
Jianshi
On Sat, Sep 6, 2014 at 12:29 AM, Magana-zook, Steven Alan <
maganazo...@llnl.gov> wrote:
> Jianshi,
>
> I
After splitting the region, you may need to run balancer to spread the new
regions out.
Cheers
On Fri, Sep 5, 2014 at 9:25 AM, Jianshi Huang
wrote:
> Hi Shahab,
>
> I see, that seems to be the right way...
>
>
> On Sat, Sep 6, 2014 at 12:21 AM, Shahab Yunus
> wrote:
>
> > Shahab
>
>
>
>
>
> -
Jianshi,
I have seen many solutions to importing this kind of data:
1. Pre-splitting regions (I did not try this)
2. Using a map reduce job to create HFiles instead of putting individual
rows into the database
(instructions here: http://hbase.apache.org/book/arch.bulk.load.html
3. Modifying the
Hi Shahab,
I see, that seems to be the right way...
On Sat, Sep 6, 2014 at 12:21 AM, Shahab Yunus
wrote:
> Shahab
--
Jianshi Huang
LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/
For monotonically increasing data: can you try to do pre-splitting of the
destination table? That can help in avoiding one region getting overloaded
at the time of bulkimport.
Regards,
Shahab
On Fri, Sep 5, 2014 at 12:14 PM, Jianshi Huang
wrote:
> Thanks Ted, I'll try to do a major compact.
>
Thanks Ted, I'll try to do a major compact.
Hi Steven,
Yes, most of my rows are hashed to make it randomly distributed, but one
column family has monotonically increasing rowkeys, and it's used for
recording sequence of events.
Do you have a solution how to bulk import this kind of data?
Jiansh
Hi Jianshi,
What are the field(s) in your row key? If your row key is monotonically
increasing then you will be sending all of your requests to one region
server. Even after the region splits, all new entries will keep punishing
one server (the region responsible for the split containing the new k
See HBASE-10882 where Victor's workaround was to do major compaction before
the bulkload.
Cheers
On Fri, Sep 5, 2014 at 8:15 AM, Jianshi Huang
wrote:
> I'm importing 2TB of generated HFiles to HBase and I constantly get the
> following errors:
>
> Caused by:
>
> org.apache.hadoop.hbase.ipc.Rem
If you files are on the same cluster, then this is not the issue. Do you
have the command line you run to launch your bulk load?
2014-09-05 11:54 GMT-04:00 Jianshi Huang :
> Hi JM,
>
> What do you mean by the 'destination cluster'? The files are in the same
> Hadoop/HDFS cluster where HBase is
Hi JM,
What do you mean by the 'destination cluster'? The files are in the same
Hadoop/HDFS cluster where HBase is running.
Do you mean do the bulk importing on HBase Master node?
Jianshi
On Fri, Sep 5, 2014 at 11:18 PM, Jean-Marc Spaggiari <
jean-m...@spaggiari.org> wrote:
> Hi Jianshi,
>
>
Hi Jianshi,
You might want to upload the file on the destination cluster first and then
re-run your bulk load from there. That way the transfer time will not be
taken into consideration for the timeout size the files will be local.
JM
2014-09-05 11:15 GMT-04:00 Jianshi Huang :
> I'm importing
I'm importing 2TB of generated HFiles to HBase and I constantly get the
following errors:
Caused by:
org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.RegionTooBusyException):
org.apache.hadoop.hbase.RegionTooBusyException: failed to get a lock in
6 ms.
regionName=g
29 matches
Mail list logo
