[
https://issues.apache.org/jira/browse/HIVE-24581?focusedWorklogId=533010&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-533010
]
ASF GitHub Bot logged work on HIVE-24581:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Jan/21 13:31
Start Date: 08/Jan/21 13:31
Worklog Time Spent: 10m
Work Description: kuczoram commented on a change in pull request #1826:
URL: https://github.com/apache/hive/pull/1826#discussion_r553944336
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java
##########
@@ -1866,92 +1870,6 @@ private static boolean isDirUsable(Path child, long
visibilityTxnId, List<Path>
return true;
}
- public static class HdfsFileStatusWithoutId implements HdfsFileStatusWithId {
- private final FileStatus fs;
-
- public HdfsFileStatusWithoutId(FileStatus fs) {
- this.fs = fs;
- }
-
- @Override
- public FileStatus getFileStatus() {
- return fs;
- }
-
- @Override
- public Long getFileId() {
- return null;
- }
- }
-
- /**
- * Find the original files (non-ACID layout) recursively under the partition
directory.
- * @param fs the file system
- * @param dir the directory to add
- * @return the list of original files
- * @throws IOException
- */
- public static List<HdfsFileStatusWithId> findOriginals(FileSystem fs, Path
dir, Ref<Boolean> useFileIds,
- boolean ignoreEmptyFiles, boolean recursive) throws IOException {
- List<HdfsFileStatusWithId> originals = new ArrayList<>();
- List<HdfsFileStatusWithId> childrenWithId =
tryListLocatedHdfsStatus(useFileIds, fs, dir, hiddenFileFilter);
- if (childrenWithId != null) {
- for (HdfsFileStatusWithId child : childrenWithId) {
- if (child.getFileStatus().isDirectory()) {
- if (recursive) {
- originals.addAll(findOriginals(fs,
child.getFileStatus().getPath(), useFileIds,
- ignoreEmptyFiles, true));
- }
- } else {
- if (!ignoreEmptyFiles || child.getFileStatus().getLen() > 0) {
- originals.add(child);
- }
- }
- }
- } else {
- List<FileStatus> children = HdfsUtils.listLocatedStatus(fs, dir,
hiddenFileFilter);
- for (FileStatus child : children) {
- if (child.isDirectory()) {
- if (recursive) {
- originals.addAll(findOriginals(fs, child.getPath(), useFileIds,
ignoreEmptyFiles, true));
- }
- } else {
- if (!ignoreEmptyFiles || child.getLen() > 0) {
Review comment:
As far as I see, the ignoreEmptyFiles parameter is removed and it is not
present in the newly added methods. Is this parameter not used any more? (Just
want to make sure that removing it won't have any side effect.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 533010)
Time Spent: 20m (was: 10m)
> Remove AcidUtils call from OrcInputformat for non transactional tables
> ----------------------------------------------------------------------
>
> Key: HIVE-24581
> URL: https://issues.apache.org/jira/browse/HIVE-24581
> Project: Hive
> Issue Type: Improvement
> Reporter: Peter Varga
> Assignee: Peter Varga
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently the split generation in OrcInputformat is tightly coupled with acid
> and AcidUtils.getAcidState is called even if the table is not transactional.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
