> On Feb. 18, 2014, 7:02 p.m., Xuefu Zhang wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/JarCache.java, line 91 > > <https://reviews.apache.org/r/18200/diff/5/?file=494786#file494786line91> > > > > cool. But what about the old cached files? Does HDFS clean them up > > automatically?
The files will have to be periodically cleaned up by the user or admin. Often times admins have policies that delete files which have not been accessed for a long time like a year. We update the access time on files once per day for this purpose. In reality most users won't be using thousands of jars so they won't have to clean them up. - Brock ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18200/#review34740 ----------------------------------------------------------- On Feb. 18, 2014, 3:36 a.m., Brock Noland wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/18200/ > ----------------------------------------------------------- > > (Updated Feb. 18, 2014, 3:36 a.m.) > > > Review request for hive. > > > Bugs: HIVE-860 > https://issues.apache.org/jira/browse/HIVE-860 > > > Repository: hive-git > > > Description > ------- > > Caches auxiliary jars and remote runtime jars in /user/$user/.hiveJars by > their sha1 hash. This results in: > > 1) faster queries > 2) less distributed cache churn > 3) a smaller/cleaner hive-exec jar > > > Diffs > ----- > > bin/hive 3bd949f > packaging/src/main/assembly/bin.xml a97ef7d > ql/pom.xml 53d0b9e > ql/src/java/org/apache/hadoop/hive/ql/exec/HiveAuxClasspathBuilder.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 288da8e > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/JarCache.java PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java 326654f > shims/aggregator/pom.xml 7aa8c4c > > Diff: https://reviews.apache.org/r/18200/diff/ > > > Testing > ------- > > Tested manually on a cluster. > > > Thanks, > > Brock Noland > >
