Ryan, I wonder why setting 'hive.merge.mapfiles=false' could solve the issue. The issue seems to be a metastore related (drop table could not find default.test_table). This is probably due to the database support newly introduced in 0.6 (see JIRA HIVE-675).
On Dec 7, 2010, at 10:21 AM, Ryan LeCompte wrote: Sorry for all of the messages! Stupid gmail client. It turns out that I was able to resolve the failures by setting "hive.merge.mapfiles=false" (by default it is true). I think this is because I'm running Hadoop version 0.20.1, r810220. This thread is what lead me to try this: http://www.mail-archive.com/hive-u...@hadoop.apache.org/msg03955.html On Tue, Dec 7, 2010 at 12:26 PM, Ryan LeCompte <lecom...@gmail.com<mailto:lecom...@gmail.com>> wrote: Digging even further, here's what I see: NOTE: We have a table in Hive called "test_table" but this seems to look for "default.test_table" ? ) 2010-12-07 00:52:24,600 ERROR metadata.Hive (Hive.java:getTable(357)) - NoSuchObjectException(message:default.test_table table not found) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:399) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:432) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:354) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:333) at org.apache.hadoop.hive.ql.exec.DDLTask.dropTable(DDLTask.java:1088) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:129) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:99) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:64) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:582) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:462) at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:324) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123) On Tue, Dec 7, 2010 at 12:19 PM, Ryan LeCompte <lecom...@gmail.com<mailto:lecom...@gmail.com>> wrote: I just put the Hive log4j config file on DEBUG, and here is the error that I'm seeing: 2010-12-07 12:16:50,281 WARN mapred.JobClient (JobClient.java:configureCommandLineOptions(539)) - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 2010-12-07 12:17:26,399 DEBUG conf.Configuration (Configuration.java:<init>(210)) - java.io.IOException: config() at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:210) at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:197) at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:429) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:230) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) 2010-12-07 12:17:26,405 DEBUG conf.Configuration (Configuration.java:<init>(210)) - java.io.IOException: config() at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:210) at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:197) at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:172) at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:449) at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:430) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:230) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) On Tue, Dec 7, 2010 at 12:12 PM, Ryan LeCompte <lecom...@gmail.com<mailto:lecom...@gmail.com>> wrote: Hello all, This morning we upgraded our cluster to use Hive 0.6.0 from Hive 0.5.0. I ran the Derby DB migration script before upgrading. All of our queries are running fine, however our INSERT OVERWRITE queries are failing immediately. I am not sure what the problem is. When we submit an INSERT OVERWRITE query, the query fails with the maps in "Pending". Here is the failure from the web interface: HTTP ERROR 500 java.lang.ArrayIndexOutOfBoundsException: 0 at org.apache.hadoop.mapred.JobInProgress.getTaskInProgress(JobInProgress.java:2523) at org.apache.hadoop.mapred.taskdetails_jsp._jspService(taskdetails_jsp.java:115) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522) Does anyone have any ideas? Thanks, Ryan
