[ https://issues.apache.org/jira/browse/HIVE-14784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15497969#comment-15497969 ]
Naveen Gangam commented on HIVE-14784: -------------------------------------- A simple change to re-create the parent directory before attempting to create the operation log will avoid such exceptions in the logs. > Operation logs are disabled automatically if the parent directory does not > exist. > --------------------------------------------------------------------------------- > > Key: HIVE-14784 > URL: https://issues.apache.org/jira/browse/HIVE-14784 > Project: Hive > Issue Type: Bug > Components: HiveServer2 > Affects Versions: 1.1.0 > Reporter: Naveen Gangam > Assignee: Naveen Gangam > > Operation logging is disabled automatically for the query if for some reason > the parent directory (named after the hive session id) that gets created when > the session is established gets deleted (for any reason). For ex: if the > operation logdir is /tmp which automatically can get purged at a configured > interval by the OS. > Running a query from that session leads to > {code} > 2016-09-15 15:09:16,723 WARN org.apache.hive.service.cli.operation.Operation: > Unable to create operation log file: > /tmp/hive/operation_logs/b8809985-6b38-47ec-a49b-6158a67cd9fc/d35414f7-2418-426c-8489-c6f643ca4599 > java.io.IOException: No such file or directory > at java.io.UnixFileSystem.createFileExclusively(Native Method) > at java.io.File.createNewFile(File.java:1012) > at > org.apache.hive.service.cli.operation.Operation.createOperationLog(Operation.java:195) > at > org.apache.hive.service.cli.operation.Operation.beforeRun(Operation.java:237) > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:255) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:398) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:385) > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:271) > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:490) > at > org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313) > at > org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:692) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {code} > This later leads to errors like (more prominent when using HUE as HUE does > not close hive sessions and attempts to retrieve the operations logs days > after they were created). > {code} > WARN org.apache.hive.service.cli.thrift.ThriftCLIService: Error fetching > results: > org.apache.hive.service.cli.HiveSQLException: Couldn't find log associated > with operation handle: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=d35414f7-2418-426c-8489-c6f643ca4599] > at > org.apache.hive.service.cli.operation.OperationManager.getOperationLogRowSet(OperationManager.java:259) > at > org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:701) > at > org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:451) > at > org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:676) > at > org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1553) > at > org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1538) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:692) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)