iBenny created HIVE-25794:
-----------------------------
Summary: CombineHiveRecordReader: log statements in a loop leads
to memory pressure
Key: HIVE-25794
URL: https://issues.apache.org/jira/browse/HIVE-25794
Project: Hive
Issue Type: Bug
Components: Logging
Reporter: iBenny
Similar to HIVE-16150, a huge string will be built in a loop, even the log
level is INFO. That leads to memory pressure when processing a big number of
split files.
>From
>[CombineHiveRecordReader.java|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveRecordReader.java#L116],
> the following needs to be fixed.
LOG.debug("Found spec for " + path + " " + otherPart + " from " +
pathToPartInfo);
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
