This is likely some artifact hive leaves behind. Our filecrush tool has piece called Clean.clean
https://github.com/edwardcapriolo/filecrush I use it to delete anything in hdfs /tmp older then N seconds. Edward On Fri, Sep 7, 2012 at 11:28 AM, Sam Darwin <samuel.d.dar...@gmail.com> wrote: > Hi, > > I am seeing like one million of these files on our hadoop cluster. > > 1005717 files like /tmp/hdfs/hdfs_2012082902171088341605155583849.pipeout > 1005742 files like /tmp/hdfs/hive_job_log_hdfs_201208290217_1000376604.txt > > My questions are: > > 1. What is a .pipeout file, and can they be deleted at any time? > What might happen if a pipeout file is removed that shouldn't be > removed? > > 2. Is it entirely up the admin to log rotate these? Why aren't > they rotated by default when you install the packages? > > Thanks, > Sam
