[
https://issues.apache.org/jira/browse/HBASE-17338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15882303#comment-15882303
]
Anoop Sam John commented on HBASE-17338:
----------------------------------------
Thanks for the reviews Stack & Ram.. I have to fix the failed test as those
were having assert on heapOverhead size after cell addition to Memstore. Now we
track heapSize not overhead. Will add more comments as Stack asked
Regarding Ram's comment.
I have different opinion. I dont think we need change this way. We track
dataSize (irrespective of cell data in on heap or off heap area).. This
dataSize been used at Segment level for in memory flush decision, Region level
for on disk flushes and globally to force flush some regions. At the 1st 2
levels, it is not doubt that we have to track all the cell data size together.
Now the point Ram says is when we have off heap configured and max off heap
global size is say 12 GB, once the data size globally reaches this level, we
will force flush some regions. So his point is for this tracking, we have to
consider only off heap Cells and on heap Cell's data size should not get
accounted in the data size but only in the heapSize. (At global level. But at
region and segment level it has to get applied). 2 reasons why I am not in
favor of this
1. This makes the impl so complex. We need to add isOffheap check down the
layers. Also at 2 layers we have to consider these on heap cell data size and
one level not.
2. When off heap is enabled, (We have the MSLAB pool off heap in place), we
will end up in having on heap Cells when the pool is out of BBs. We will
create on demand LABs on heap. If we dont consider those cell's data size at
global level, we may reach forced flush level a bit late. That is the only
gain. But here the on demand LAB creation is a sign that the write load is so
high and delaying the flush will add more pressure to the MSLAB and more and
more on demand BBs (2 MB sized) need to get created. One aim of the off heap
work is to reduce the max heap space need for the servers.
So lets consider the cell data size globally also (how we do now) and make
global flushes.
Now even if MSLAB is used, the append/increment use cases wont use MSLAB. The
cells will be on heap then. But for such a use case, enabling MSLAB (and pool)
is totally waste. That is a mis configuration.
More and more on demand BB creation, when MSLAB pool is a bad sign. We have a
WARN log for just one time as of now.. May be we should repeat this log at
certain intervals. (Like for every 100th pool miss or so.)
We should be able to turn MSLAB usage ON/OFF per table also. Now this is
possible? Am not sure.
These 2 things need to be checked and done in another jiras IMO.
> Treat Cell data size under global memstore heap size only when that Cell can
> not be copied to MSLAB
> ---------------------------------------------------------------------------------------------------
>
> Key: HBASE-17338
> URL: https://issues.apache.org/jira/browse/HBASE-17338
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
> Affects Versions: 2.0.0
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 2.0.0
>
> Attachments: HBASE-17338.patch, HBASE-17338_V2.patch,
> HBASE-17338_V2.patch, HBASE-17338_V4.patch
>
>
> We have only data size and heap overhead being tracked globally. Off heap
> memstore works with off heap backed MSLAB pool. But a cell, when added to
> memstore, not always getting copied to MSLAB. Append/Increment ops doing an
> upsert, dont use MSLAB. Also based on the Cell size, we sometimes avoid
> MSLAB copy. But now we track these cell data size also under the global
> memstore data size which indicated off heap size in case of off heap
> memstore. For global checks for flushes (against lower/upper watermark
> levels), we check this size against max off heap memstore size. We do check
> heap overhead against global heap memstore size (Defaults to 40% of xmx) But
> for such cells the data size also should be accounted under the heap overhead.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
