xiarixiaoyao edited a comment on issue #4600:
URL: https://github.com/apache/hudi/issues/4600#issuecomment-1013838991
@gubinjie
Are there only log files in your current Hudi table?
now hive cannot read log only files, This is hive's problem. Hive will
filter out all the files which start by ".", so log files will be filter out.
you can try follow patch code for hive, it should be worked as we have
already use it in our produce env:
modify org.apache.hadoop.hive.common.FileUtils
**public static final PathFilter HIDDEN_FILES_PATH_FILTER = new
PathFilter() {
@Override
public boolean accept(Path p) {
String name = p.getName();
boolean isHudiMeta = name.startsWith(".hoodie");
boolean isHudiLog = false;
Pattern LOG_FILE_PATTERN =
Pattern.compile("\\.(.*)_(.*)\\.(.*)\\.([0-9]*)(_(([0-9]*)-([0-9]*)-([0-9]*)))?");
Matcher matcher = LOG_FILE_PATTERN.matcher(name);
if (matcher.find()) {
isHudiLog = true;
}
boolean isHudiFile = isHudiLog || isHudiMeta;
return (!name.startsWith("_") && !name.startsWith(".")) || isHudiFile;
}
};**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
