Re: DFSOutputStream.Packet retention even if close() called when IOE encountered

2012-10-01 Thread Todd Lipcon
Hey Benoit, Sorry for the slow response on this. The attachment seems to have not come through. Would you mind filing an HDFS JIRA and attach the reproducer case there? Thanks -Todd On Wed, Sep 5, 2012 at 12:28 AM, Benoit Perroud wrote: > Hi All, > > I experience some memory retention while co

DFSOutputStream.Packet retention even if close() called when IOE encountered

2012-09-05 Thread Benoit Perroud
Hi All, I experience some memory retention while copying data into HDFS when a IOExeption is thrown. My use case is the following: I have multiple threads sharing a FileSystem object, all uploading files. At some point quota is exceeded in one thread and I get a DSQuotaExceededException (subclass