Yuxing Yao created HIVE-15290:
---------------------------------
Summary: Stripe size smaller than specified.
Key: HIVE-15290
URL: https://issues.apache.org/jira/browse/HIVE-15290
Project: Hive
Issue Type: Bug
Components: ORC
Affects Versions: 2.0.1, 2.1.0, 2.0.0, 1.2.1, 1.2.0
Reporter: Yuxing Yao
In Hive-1.2.0, the real stripe size of output orc file will be very small if
most of table data are empty, result in too many Column Statistics objects
consumes most of the memory.
I found it become better in Hive-2.0.1, but the stripe size still much smaller
than expected.
I saw there's a Jira item: https://issues.apache.org/jira/browse/HIVE-13232
moved the compressed = null out of if block, this changes helps a lot, but for
completely fix this, another change is needed in `OutStream.getBufferSize()`
I've created the PR:
https://github.com/apache/hive/pull/118
Please take a look.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
