[
https://issues.apache.org/jira/browse/HIVE-17344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134858#comment-16134858
]
Janos Gub commented on HIVE-17344:
----------------------------------
bq. it's either allocated exactly to size
If it is allocated to size, then the bb.remaining will be 0 right?
bq. read from disk the same way
Then here it is 0 also?
The weigher in LocalCache sums up the getMemoryUsage of the TailAndFileData.
If the remaining size is ALWAYS 0, how HIVE-16133 is limiting the maximum size
of the cache? (Or how is it different from setting the maxsize of the cache to
limit the number of elements in it)
Isn't HIVE-16133 about restricting the total memory usage of cache?
cc [~hagleitn]
> LocalCache element memory usage is not calculated properly.
> -----------------------------------------------------------
>
> Key: HIVE-17344
> URL: https://issues.apache.org/jira/browse/HIVE-17344
> Project: Hive
> Issue Type: Bug
> Reporter: Janos Gub
> Assignee: Janos Gub
> Attachments: HIVE-17344.patch
>
>
> Orc footer cache has a calculation of memory usage:
> {code:java}
> public int getMemoryUsage() {
> return bb.remaining() + 100; // 100 is for 2 longs, BB and java overheads
> (semi-arbitrary).
> }
> {code}
> ByteBuffer.remaining returns the remaining space in the bytebuffer, thus
> allowing this cache have elements MAXWEIGHT/100 of arbitrary size. I think
> the correct solution would be bb.capacity.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
