[
https://issues.apache.org/jira/browse/HIVE-14301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387560#comment-15387560
]
Steve Loughran commented on HIVE-14301:
---------------------------------------
I worry about the code assuming that the return code of rename() is relevant.
It's one of the key bits of API ambiguity, where HDFS sometimes behaves
differently from Posix filesystems —nobody knows quite what to do. I'd rely on
the filesystems throwing an exception if something serious went wrong, and just
logging at warn here.
> insert overwrite fails for nonpartitioned tables in s3
> ------------------------------------------------------
>
> Key: HIVE-14301
> URL: https://issues.apache.org/jira/browse/HIVE-14301
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Priority: Minor
> Attachments: HIVE-14301.1.patch
>
>
> {noformat}
> hive> insert overwrite table s3_2 select * from default.test2;
> Query ID = hrt_qa_20160719164737_90fb1f30-0ade-4a64-ab65-a6a7550be25a
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (Executing on YARN cluster with App id
> application_1468941549982_0010)
> --------------------------------------------------------------------------------
> VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED
> KILLED
> --------------------------------------------------------------------------------
> Map 1 .......... SUCCEEDED 1 1 0 0 0
> 0
> --------------------------------------------------------------------------------
> VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 11.90 s
>
> --------------------------------------------------------------------------------
> Loading data to table default.s3_2
> Failed with exception java.io.IOException: rename for src path:
> s3a://test-ks/test2/.hive-staging_hive_2016-07-19_16-47-37_787_4725676452829013403-1/-ext-10000/000000_0.deflate
> to dest path:s3a://test-ks/test2/000000_0.deflate returned false
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask
> 2016-07-19 16:43:46,244 ERROR [main]: exec.Task
> (SessionState.java:printError(948)) - Failed with exception
> java.io.IOException: rename for src path:
> s3a://test-ks/testing/.hive-staging_hive_2016-07-19_16-42-20_739_1716954454570249450-1/-ext-10000/000000_0.deflate
> to dest path:s3a://test-ks/testing/000000_0.deflate returned false
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: rename
> for src path:
> s3a://test-ks/testing/.hive-staging_hive_2016-07-19_16-42-20_739_1716954454570249450-1/-ext-10000/000000_0.deflate
> to dest path:s3a://test-ks/testing/000000_0.deflate returned false
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2856)
> at org.apache.hadoop.hive.ql.metadata.Hive.replaceFiles(Hive.java:3113)
> at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:1700)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:328)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1726)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1472)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1271)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1138)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1128)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.io.IOException: rename for src path:
> s3a://test-ks/testing/.hive-staging_hive_2016-07-19_16-42-20_739_1716954454570249450-1/-ext-10000/000000_0.deflate
> to dest path:s3a://test-ks/testing/000000_0.deflate returned false
> at org.apache.hadoop.hive.ql.metadata.Hive$3.call(Hive.java:2836)
> at org.apache.hadoop.hive.ql.metadata.Hive$3.call(Hive.java:2825)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
