[
https://issues.apache.org/jira/browse/HBASE-11425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-11425:
--------------------------
Attachment: heap.png
Have you fellas run this for a while? I seem to OOME easy enough. I tried with
a small heap, 4G, but it OOME'd then had to keep going back up to 16G again
though I had a big offheap. I tried to capture the increasing use of heap but
this diagram is best I got... I've not done heap analysis.. but in the diagram
you can see heap use start to rise and then plummet... now I am crawling doing
Full GCs every couple of seconds.
The last meaningful log was this in regionserver log:
2015-10-14 16:51:10,058 DEBUG [main-BucketCacheWriter-2] bucket.BucketCache:
This block 3f0157e7daee45fdb25202c496c95c46_1649898813 is still referred by 1
readers. Can not be freed now
It started complaining 50minutes ago...
> Cell/DBB end-to-end on the read-path
> ------------------------------------
>
> Key: HBASE-11425
> URL: https://issues.apache.org/jira/browse/HBASE-11425
> Project: HBase
> Issue Type: Umbrella
> Components: regionserver, Scanners
> Affects Versions: 0.99.0
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Attachments: BenchmarkTestCode.zip, Benchmarks_Tests.docx,
> HBASE-11425-E2E-NotComplete.patch, HBASE-11425.patch, Offheap reads in HBase
> using BBs_V2.pdf, Offheap reads in HBase using BBs_final.pdf, gc.png,
> gets.png, heap.png, load.png, median.png
>
>
> Umbrella jira to make sure we can have blocks cached in offheap backed cache.
> In the entire read path, we can refer to this offheap buffer and avoid onheap
> copying.
> The high level items I can identify as of now are
> 1. Avoid the array() call on BB in read path.. (This is there in many
> classes. We can handle class by class)
> 2. Support Buffer based getter APIs in cell. In read path we will create a
> new Cell with backed by BB. Will need in CellComparator, Filter (like SCVF),
> CPs etc.
> 3. Avoid KeyValue.ensureKeyValue() calls in read path - This make byte copy.
> 4. Remove all CP hooks (which are already deprecated) which deal with KVs.
> (In read path)
> Will add subtasks under this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
