Can you share your query and use case? ~Aniket
On Tue, Jul 10, 2012 at 9:39 AM, Harsh J <ha...@cloudera.com> wrote: > This appears to be a Hive issue (something probably called FS.close() > too early?). Redirecting to the Hive user lists as they can help > better with this. > > On Tue, Jul 10, 2012 at 9:59 PM, 안의건 <ahneui...@gmail.com> wrote: > > Hello. I have a problem with the filesystem closing. > > > > The filesystem was closed when the hive query is running. > > It is 'select' query and the data size is about 1TB. > > I'm using hadoop-0.20.2 and hive-0.7.1. > > > > The error log is telling that tmp file is not deleted, or the tmp path > > exception is occurred. > > > > Is there any hadoop configuration I'm missing? > > > > Thank you > > > > [stderr logs] > > org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: > > Filesystem closed > > at > > > org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:454) > > at > > > org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:636) > > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:557) > > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) > > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) > > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) > > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) > > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) > > at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:193) > > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) > > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) > > at org.apache.hadoop.mapred.Child.main(Child.java:170) > > Caused by: java.io.IOException: Filesystem closed > > at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:226) > > at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:617) > > at > > > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:453) > > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:648) > > at org.apache.hadoop.fs.FileSystem.deleteOnExit(FileSystem.java:615) > > at > > > org.apache.hadoop.hive.shims.Hadoop20Shims.fileSystemDeleteOnExit(Hadoop20Shims.java:68) > > at > > > org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:451) > > ... 12 more > > > > -- > Harsh J > -- "...:::Aniket:::... Quetzalco@tl"
