Hi,
We just pushed Hive .7 and using Mysql has the HiveMetaStore backend into
our ad-hoc cluster. There are large number of queries that are submitted in
parallel against the Hive backend . Some of the Hive jobs seems to fail
when the definition of the tables are not found. When we re-run the same
query again, the job works fine. We checked the Mysql database and the load
is minimal with only 80 connections at peak time.
Is there something I should tune in Mysql or Hive to reduce this type of
errors from happening?
Thanks,
-ray
10/11/28 11:04:41 INFO parse.ParseDriver: Parse Completed
10/11/28 11:04:41 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
10/11/28 11:04:41 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic
Analysis
10/11/28 11:04:41 INFO parse.SemanticAnalyzer: Get metadata for source
tables
10/11/28 11:04:42 INFO metastore.HiveMetaStore: 0: Opening raw store with
implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
10/11/28 11:04:42 INFO metastore.ObjectStore: ObjectStore, initialize called
10/11/28 11:04:42 ERROR DataNucleus.Plugin: Bundle "org.eclipse.jdt.core"
requires "org.eclipse.core.resources" but it cannot be resolved.
10/11/28 11:04:42 ERROR DataNucleus.Plugin: Bundle "org.eclipse.jdt.core"
requires "org.eclipse.core.runtime" but it cannot be resolved.
10/11/28 11:04:42 ERROR DataNucleus.Plugin: Bundle "org.eclipse.jdt.core"
requires "org.eclipse.text" but it cannot be resolved.
10/11/28 11:04:45 INFO metastore.ObjectStore: Initialized ObjectStore
10/11/28 11:04:48 INFO metastore.HiveMetaStore: 0: get_table : db=default
tbl=f_adv_stats
10/11/28 11:04:49 ERROR metadata.Hive:
NoSuchObjectException(message:default.f_adv_stats table not found)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$15.run(HiveMetaStore.java:953)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$15.run(HiveMetaStore.java:948)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:234)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:948)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:581)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:711)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:671)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:755)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:6381)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:341)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:645)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199)
at
org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:220)
at
org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:227)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:323)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
10/11/28 11:04:49 ERROR parse.SemanticAnalyzer:
org.apache.hadoop.hive.ql.parse.SemanticException: line 40:5 Table not found
f_adv_stats
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:757)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:6381)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:341)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:645)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199)
at
org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:220)
at
org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:227)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:323)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
FAILED: Error in semantic analysis: line 40:5 Table not found f_adv_stats
10/11/28 11:04:49 ERROR ql.Driver: FAILED: Error in semantic analysis: line
40:5 Table not found f_adv_stats
org.apache.hadoop.hive.ql.parse.SemanticException: line 40:5 Table not found
f_adv_stats
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:881)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:6381)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:341)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:645)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199)
at
org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:220)
at
org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:227)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:323)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.hadoop.hive.ql.parse.SemanticException: line 40:5
Table not found f_adv_stats
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:757)
... 14 more
