Thanks for the logs. Couple of things here, 1. Based on the logs,the HiveServer2 seem to be down. This is an issue if you use Hue/Beeline 2. If you are using Hive CLI, you can still test it, where you have to add the (fs.s3a.access.key,fs.s3.secret.key)s3 keys in hdfs advanced configuration snippet for core-site.xml and hdfs-site.xml via CM.Also for location it should be s3a:// not s3://
On Wed, Nov 7, 2018, 2:58 PM Garry Chen <g...@cornell.edu wrote: > Hi Suresh, > > I am using Hive 1.1.0-cdh5.14.4 and hive server log as below. > > > > 2018-11-07 19:43:16,581 WARN [main]: server.HiveServer2 > (HiveServer2.java:startHiveServer2(581)) - Error starting HiveServer2 on > attempt 1, will retry in 60000ms > > java.lang.RuntimeException: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.RuntimeException: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient > > at > org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:840) > > at > org.apache.hadoop.hive.ql.session.SessionState.getAuthorizationMode(SessionState.java:1686) > > at > org.apache.hadoop.hive.ql.session.SessionState.isAuthorizationModeV2(SessionState.java:1697) > > at > org.apache.hadoop.hive.ql.session.SessionState.applyAuthorizationPolicy(SessionState.java:1745) > > at > org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:125) > > at org.apache.hive.service.cli.CLIService.init(CLIService.java:111) > > at > org.apache.hive.service.CompositeService.init(CompositeService.java:59) > > at > org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:125) > > at > org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:542) > > at > org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:89) > > at > org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:793) > > at > org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:666) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at org.apache.hadoop.util.RunJar.run(RunJar.java:226) > > at org.apache.hadoop.util.RunJar.main(RunJar.java:141) > > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.RuntimeException: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient > > at > org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthorizeProviderManager(HiveUtils.java:391) > > at > org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:817) > > ... 17 more > > Caused by: java.lang.RuntimeException: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient > > at > org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProviderBase.setConf(HiveAuthorizationProviderBase.java:114) > > at > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) > > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) > > at > org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthorizeProviderManager(HiveUtils.java:388) > > ... 18 more > > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient > > at > org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:220) > > at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:338) > > at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:299) > > at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:274) > > at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:256) > > at > org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.init(DefaultHiveAuthorizationProvider.java:29) > > at > org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProviderBase.setConf(HiveAuthorizationProviderBase.java:112) > > ... 21 more > > Caused by: java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient > > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1638) > > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:67) > > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:82) > > at > org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3411) > > at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3430) > > at > org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3655) > > at > org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:231) > > at > org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:215) > > ... 27 more > > Caused by: java.lang.reflect.InvocationTargetException > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1636) > > ... 34 more > > Caused by: MetaException(message:Could not connect to meta store using any > of the URIs provided. Most recent failure: > org.apache.thrift.transport.TTransportException: java.net.ConnectException: > Connection refused (Connection refused) > > at org.apache.thrift.transport.TSocket.open(TSocket.java:226) > > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:464) > > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:244) > > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1636) > > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:67) > > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:82) > > at > org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3411) > > at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3430) > > at > org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3655) > > at > org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:231) > > at > org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:215) > > at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:338) > > at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:299) > > at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:274) > > at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:256) > > at > org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.init(DefaultHiveAuthorizationProvider.java:29) > > at > org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProviderBase.setConf(HiveAuthorizationProviderBase.java:112) > > at > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) > > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) > > at > org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthorizeProviderManager(HiveUtils.java:388) > > at > org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:817) > > at > org.apache.hadoop.hive.ql.session.SessionState.getAuthorizationMode(SessionState.java:1686) > > at > org.apache.hadoop.hive.ql.session.SessionState.isAuthorizationModeV2(SessionState.java:1697) > > at > org.apache.hadoop.hive.ql.session.SessionState.applyAuthorizationPolicy(SessionState.java:1745) > > at > org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:125) > > at org.apache.hive.service.cli.CLIService.init(CLIService.java:111) > > at > org.apache.hive.service.CompositeService.init(CompositeService.java:59) > > at > org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:125) > > at > org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:542) > > at > org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:89) > > at > org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:793) > > at > org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:666) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at org.apache.hadoop.util.RunJar.run(RunJar.java:226) > > at org.apache.hadoop.util.RunJar.main(RunJar.java:141) > > Caused by: java.net.ConnectException: Connection refused (Connection > refused) > > at java.net.PlainSocketImpl.socketConnect(Native Method) > > at > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) > > at > java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) > > at > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) > > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > > at java.net.Socket.connect(Socket.java:589) > > at org.apache.thrift.transport.TSocket.open(TSocket.java:221) > > ... 42 more > > ) > > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:512) > > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:244) > > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) > > > > *From: *Suresh Kumar Sethuramaswamy <rock...@gmail.com> > *Reply-To: *"user@hive.apache.org" <user@hive.apache.org> > *Date: *Wednesday, November 7, 2018 at 2:50 PM > *To: *"user@hive.apache.org" <user@hive.apache.org> > *Subject: *Re: Create external table with s3 location error > > > > Are you using EMR or Apache hadoop open source? > > Can you share your hive megastore logs? > > > > > > On Wed, Nov 7, 2018, 2:19 PM Garry Chen <g...@cornell.edu wrote: > > hi All, > > I am try to create a external table using s3 as location > but failed. I add my access key and security key in hive-site.xml and > reboot the server. Any suggestion? > > > > hive> create external table kv (key int, values string) location > 's3://cu-iclick/test'; > > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. > MetaException(message:java.lang.NullPointerException) > > > > Garry > >
