[
https://issues.apache.org/jira/browse/HBASE-17338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893768#comment-15893768
]
Anoop Sam John commented on HBASE-17338:
----------------------------------------
bq.Thanks. So, datasize vs heapsize, have we written these up? datasize is
serialized size... whether serialized on rpc or in blockcache and heapsize is
how large a 'live' Cell in java heap is? If the Cell data is offheap, it is the
the onheap Cell proxy object?
Yes
bq.How does the global threshold work in the case where we are doing offheap
accounting too. The global check will look at the onheap limit and the offheap
limit and if we hit the offheap limit before we hit the onheap limit, we'll
flush?
In case of off heap MSLAB in place, the global memstore size is specified using
a new config as u know. So we will have this check against the aggregated
dataSize . Also we will be having a max possible on heap memstore global size
(Defualt 40% of xmx) This check also done. Any one of the condition met, we
will do some forced flush. (What we do now also) with blocking writes.
bq.Should be easy enough to do?
Should be IMO
> Treat Cell data size under global memstore heap size only when that Cell can
> not be copied to MSLAB
> ---------------------------------------------------------------------------------------------------
>
> Key: HBASE-17338
> URL: https://issues.apache.org/jira/browse/HBASE-17338
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
> Affects Versions: 2.0.0
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 2.0.0
>
> Attachments: HBASE-17338.patch, HBASE-17338_V2.patch,
> HBASE-17338_V2.patch, HBASE-17338_V4.patch, HBASE-17338_V5.patch
>
>
> We have only data size and heap overhead being tracked globally. Off heap
> memstore works with off heap backed MSLAB pool. But a cell, when added to
> memstore, not always getting copied to MSLAB. Append/Increment ops doing an
> upsert, dont use MSLAB. Also based on the Cell size, we sometimes avoid
> MSLAB copy. But now we track these cell data size also under the global
> memstore data size which indicated off heap size in case of off heap
> memstore. For global checks for flushes (against lower/upper watermark
> levels), we check this size against max off heap memstore size. We do check
> heap overhead against global heap memstore size (Defaults to 40% of xmx) But
> for such cells the data size also should be accounted under the heap overhead.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
