fix is in t disable auditing, which is now the default https://issues.apache.org/jira/browse/HADOOP-18094
everything is OK for apps which retain the same fs instances for the life of the app, but not for Hive... will do a better fix ASAP where in exchange for loss of auditing after a GC event, only weak refs are held in maps private to the auditor. i will put that in hadoop common as i would want to use the same code in thread-levek IOStatistics tracking. there we;d demand create an IOStatistics snapshot per thread, short lived worker threads for stream io would still update the stats of the thread the stream was created in. this will let lus collect stats on store io through the orc/paquet readers for each thread doing work for a job, and include them in job stats. and how would that be useful? well. look at this coimparison of job/task commit performance with the manifest committer https://gist.github.com/steveloughran/7dc1e68220db67327b781b345b42c0b8
