2016-03-18 13:14 GMT-04:00 Frank Luo <[email protected]>: > >Compactions should not hold the writes except if there is to many stores > files. > > I believe too many store files (9 I think?) is indeed the trigger for > compact. I always do MajorCompact before inserting. >
This is not required. 9 is the blocking. But compactions should start at 3. MajorCompactions are expensive for IO... If you region is already 4GB and to add 10MB to it, you will force to rewrite a 4GB file for nothing. Don't think it's a good idea. > > > > What does your heap size look like? > > 36GB allocated to a region server. > that's a lot. No issues onthe Garbage collection side? Any GC pause anywhere? Which JDK are you running? > > > Do you have anything on the logs when this happen? > > Haven't really looked the log files. I got most of information from the > web console. > Might be interesting to look on the log side next time you face that... JMS > > > Also, have you think about eventually merging some of the tables > together? > > Haven't thought about it. I might go there if no other options. > > -----Original Message----- > From: Jean-Marc Spaggiari [mailto:[email protected]] > Sent: Friday, March 18, 2016 12:02 PM > To: user > Subject: Re: is it a good idea to disable tables not currently hot? > > Strange. Compactions should not hold the writes except if there is to many > stores files. What does your heap size look like? Do you have anything on > the logs when this happen? Also, have you think about eventually merging > some of the tables together? > > JMS > > 2016-03-18 12:58 GMT-04:00 Frank Luo <[email protected]>: > > > There are two reasons I am hesitating going that route. > > > > One is that most of tables are fairly small. Going to 10GB will force > > tables to shrink to some nodes but not evenly distributed around the > > cluster, hence discouraging parallelism. But I think I can manage this > > issue if the second is resolved. > > > > The second issue, which I have battled with for two years now, is that > > I am doing online puts, which occasionally triggers compacts when a > > region is heavily inserted, and whenever it happens, all subsequent > > read/write are all on hold and I can see time out error on the client > > side. A typical compact runs for 4 minutes now and I have to increase > > timeout on a number of places to accommodate that. So if I increase > > the size to 10 GB, will compact time double? > > > > -----Original Message----- > > From: Jean-Marc Spaggiari [mailto:[email protected]] > > Sent: Friday, March 18, 2016 11:34 AM > > To: user > > Subject: Re: is it a good idea to disable tables not currently hot? > > > > So you can safely increase your maximum region size to 10GB, which > > will divide the number of regions by 2. When you will be on 1.1.2 you > > can also do online merge to reduce this number of regions. The might > help too. > > > > JMS > > > > 2016-03-18 12:32 GMT-04:00 Frank Luo <[email protected]>: > > > > > 0.98 on hdp 2.2 currently. > > > > > > Soon will be on hdp2.3.4, which has HBase 1.1.2. > > > > > > -----Original Message----- > > > From: Jean-Marc Spaggiari [mailto:[email protected]] > > > Sent: Friday, March 18, 2016 11:29 AM > > > To: user > > > Subject: Re: is it a good idea to disable tables not currently hot? > > > > > > Hi Frank, > > > > > > It might be doable. > > > > > > What HBase version are you running? > > > > > > JMS > > > > > > 2016-03-18 12:25 GMT-04:00 Frank Luo <[email protected]>: > > > > > > > No one has experience disabling tables? > > > > > > > > -----Original Message----- > > > > From: Frank Luo [mailto:[email protected]] > > > > Sent: Thursday, March 17, 2016 4:51 PM > > > > To: [email protected] > > > > Subject: is it a good idea to disable tables not currently hot? > > > > > > > > We have a multi tenants environment and each client occupies x > > > > number of hbase regions. We currently have about 500 regions per > > > > region server and I understand the guideline is less than 200. So > > > > we need to reduce the region counts. Increasing region file size > > > > is no more an option because we are already at 5G and I don’t want > > > > to go > > higher. > > > > > > > > Due to our unique use cases, all clients are running for a few > > > > hours in a day, then being quiet for the rest of time. So I am > > > > thinking whether it is a good idea to disable all quiet tables and > > > > only enable them when they are ready to run. Does anyone have > > > > experience on > > that? > > > > > > > > One thing I worry about is the Balancer. I am pretty sure the > > > > balancer will be confused when regions come and go. And I cannot > > > > afford not to have it running in case of region server crashes and > > > > come back. So doesn’t anyone have good ideas how to handle it? > > > > > > > > I already doing compact myself so that is not an issue. > > > > > > > > Another related question, if a region is enabled but not active > > > > read/write, how much resources it takes in terms of region server? > > > > > > > > Thanks! > > > > > > > > Frank Luo > > > > > > > > > > Merkle was named a leader in Customer Insights Services Providers by > > > Forrester Research < > > > http://www.merkleinc.com/who-we-are-customer-relationship-marketing- > > > ag > > > ency/awards-recognition/merkle-named-leader-forrester?utm_source=ema > > > il footer&utm_medium=email&utm_campaign=2016MonthlyEmployeeFooter > > > > > > > > > > Forrester Research report names 500friends, a Merkle Company, a > > > leader in customer Loyalty Solutions for Midsize Organizations< > > > http://www.merkleinc.com/who-we-are-customer-relationship-marketing- > > > ag > > > ency/awards-recognition/500friends-merkle-company-named?utm_source=e > > > ma ilfooter&utm_medium=email&utm_campaign=2016MonthlyEmployeeFooter > > > > > > > This email and any attachments transmitted with it are intended for > > > use by the intended recipient(s) only. If you have received this > > > email in error, please notify the sender immediately and then delete > > > it. If you are not the intended recipient, you must not keep, use, > > > disclose, copy or distribute this email without the author’s prior > permission. > > > We take precautions to minimize the risk of transmitting software > > > viruses, but we advise you to perform your own virus checks on any > > > attachment to this message. We cannot accept liability for any loss > > > or damage caused by software viruses. The information contained in > > > this communication may be confidential and may be subject to the > > attorney-client privilege. > > > > > Merkle was named a leader in Customer Insights Services Providers by > > Forrester Research < > > http://www.merkleinc.com/who-we-are-customer-relationship-marketing-ag > > ency/awards-recognition/merkle-named-leader-forrester?utm_source=email > > footer&utm_medium=email&utm_campaign=2016MonthlyEmployeeFooter > > > > > > > Forrester Research report names 500friends, a Merkle Company, a leader > > in customer Loyalty Solutions for Midsize Organizations< > > http://www.merkleinc.com/who-we-are-customer-relationship-marketing-ag > > ency/awards-recognition/500friends-merkle-company-named?utm_source=ema > > ilfooter&utm_medium=email&utm_campaign=2016MonthlyEmployeeFooter > > > > > This email and any attachments transmitted with it are intended for > > use by the intended recipient(s) only. If you have received this email > > in error, please notify the sender immediately and then delete it. If > > you are not the intended recipient, you must not keep, use, disclose, > > copy or distribute this email without the author’s prior permission. > > We take precautions to minimize the risk of transmitting software > > viruses, but we advise you to perform your own virus checks on any > > attachment to this message. We cannot accept liability for any loss or > > damage caused by software viruses. The information contained in this > > communication may be confidential and may be subject to the > attorney-client privilege. > > > Merkle was named a leader in Customer Insights Services Providers by > Forrester Research > < > http://www.merkleinc.com/who-we-are-customer-relationship-marketing-agency/awards-recognition/merkle-named-leader-forrester?utm_source=emailfooter&utm_medium=email&utm_campaign=2016MonthlyEmployeeFooter > > > > Forrester Research report names 500friends, a Merkle Company, a leader in > customer Loyalty Solutions for Midsize Organizations< > http://www.merkleinc.com/who-we-are-customer-relationship-marketing-agency/awards-recognition/500friends-merkle-company-named?utm_source=emailfooter&utm_medium=email&utm_campaign=2016MonthlyEmployeeFooter > > > This email and any attachments transmitted with it are intended for use by > the intended recipient(s) only. If you have received this email in error, > please notify the sender immediately and then delete it. If you are not the > intended recipient, you must not keep, use, disclose, copy or distribute > this email without the author’s prior permission. We take precautions to > minimize the risk of transmitting software viruses, but we advise you to > perform your own virus checks on any attachment to this message. We cannot > accept liability for any loss or damage caused by software viruses. The > information contained in this communication may be confidential and may be > subject to the attorney-client privilege. >
