Peter Toth created HIVE-23014:
---------------------------------
Summary: ORC reading performance
Key: HIVE-23014
URL: https://issues.apache.org/jira/browse/HIVE-23014
Project: Hive
Issue Type: Bug
Components: ORC
Reporter: Peter Toth
Spark 3 adds support for using Hive 2.3.6 besides the old Hive 1.2.1 version.
Some of the ORC reading benchmark shows that there is a huge performance
difference in ORC reading between the 2 versions. I measured that
{{org.apache.hadoop.hive.ql.io.orc.ReaderImpl}} in hive-exec-2.3.6-core.jar is
~3-5 times slower than in hive-exec-1.2.1.spark2.jar.
I'm not sure if more recent Hive versions still suffer from this performance
regression.
Please see some details here: SPARK-30565
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
