Thanks for your input. We are able to install Kylin 1.6 successfully on CDH 5.9 and able to execute the queries.
We have created a separate folder and hbase metadata for the kylin 1.6. Is it ok to copy the Kylin 1.5 meta-data and hbase tables on Kylin 1.6 folders? By doing so I don’t have to rebuild the cubes in kylin 1.6. Please confirm if this approach will work. Thanks, Manoj Jain > On Jan 27, 2017, at 8:37 AM, [email protected] wrote: > > > > For item a), correct. In your case, I think this is the root cause. > > For item b), if the latest kylin could not work for the newer CDH, the older > one could not work either definetly. The dev team has no resource to verify > CDH5.9 yet. There is no special APIs used for CDH5.7+. So the CDH release > should work for you. > > > > > > Yiming Liu > > > > > > > 发件人: Manoj Jain > > 发送时间: 1月27日 星期五 20:48 > > 主题: Re: URGENT !!! Help needed - Issue executing query in Kylin > > 收件人: [email protected] > > 抄送: [email protected], Sudeep Dey, Raja Tallapaneni, Sanjeev Singh > > > > Thanks for your quick response. Need few clarification: > > > > a) "When you upgrade, please follow the upgrade instructuon, such as > redeploy the coprocessor jar.” - I believe you are saying when we upgrade the > kylin version, we need to redeploy the coprocessor jar. Confirm my > understanding. > > > > b) We plan to deploy the newer version of the Kylin. However, the new Kylin > version does not mention compatibility with CDH 5.9. > > > I need your advice whether we upgrade to newer version of Kylin or apply some > patch that will resolve the current issue. > > > > http://kylin.apache.org/download/ > > > 1.6.0 > > > In this version Kylin has better support for Kafka as data source, read blog > New NRT Streaming in Apache Kylin.Release NotesSource download: > apache-kylin-1.6.0-src.tar.gz [asc] [md5]Binary download:for HBase 0.98 > (includes HDP 2.2, CDH 5.2/5.3) - apache-kylin-1.6.0-bin.tar.gz [asc] > [md5]for HBase 1.x (includes HDP 2.3/2.4, EMR 5.0) - > apache-kylin-1.6.0-hbase1.x-bin.tar.gz [asc] [md5]for CDH 5.5/5.7/5.8 - > apache-kylin-1.6.0-cdh5.7-bin.tar.gz [asc] [md5] > > > Thanks, > > > Manoj Jain > > > > On Jan 26, 2017, at 11:59 PM, [email protected] wrote: > > > > When you upgrade, please follow the upgrade instructuon, such as redeploy the > coprocessor jar. > > > > Yiming Liu > > > > > > > > On Fri, Jan 27, 2017 at 12:33 PM +0800, "Manoj Jain" <[email protected]> wrote: > > > > > > > > > We were running klyin 1.5 on CDH 5.7 and was working fine. > > > > Post upgrade to CDH 5.9, we are seeing an exception while running the queries > in Kylin. > > > > I have restarted the Kylin 1.5 instance and also build the test cube from > scratch. The cube built was successful but seeing an exception in the > Kylin.log when running the queries (in the Kyiin ‘Insight’ tab). > > > > Please help us to resolve this issue at the earliest. Let us know if you > need any other information. > > > > Kylin Version: 1.5.2 > > > > Excerpts from the Server Environment > > > HBASE_LOG_DIR = /var/log/hbase > > > HBASE_HOME = /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hbase/bin/.. > > > HBASE_OPTS = -Xmx268435456 -XX:+HeapDumpOnOutOfMemoryError > -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode > -Djava.net.preferIPv4Stack=true > -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf > -Dhbase.log.dir=/var/log/hbase -Dhbase.log.file=hbase.log > -Dhbase.home.dir=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hbase/bin/.. > -Dhbase.id.str=hbase -Dhbase.root.logger=INFO,console > -Djava.library.path=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hadoop/lib/native:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hbase/bin/../lib/native/Linux-amd64-64 > -Dhbase.security.logger=INFO,NullAppender > > > HBASE_ENV_INIT = true > > > hbase_dependency = > /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hbase/bin/../lib/hbase-common-1.2.0-cdh5.9.0.jar > > > > > > Kylin log exceprts > > > > 2017-01-26 18:33:50,419 INFO [http-bio-10101-exec-7] v2.CubeHBaseRPC:271 : > Visiting hbase table KYLIN_XXXXXXXXX: cuboid exact match, from 2 to 2 Start: > \x00\x00\x00\x00\x00\x0 > > > 0\x00\x00\x00\x02\x00 (\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00) Stop: > \x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\xFF\x00 > (\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\xFF\x00), > > > No Fuzzy Key > > > 2017-01-26 18:33:50,419 DEBUG [http-bio-10101-exec-7] > v2.CubeHBaseEndpointRPC:319 : Submitting rpc to 1 shards starting from shard > 0, scan range count 1 > > > 2017-01-26 18:33:50,419 INFO [http-bio-10101-exec-7] > v2.CubeHBaseEndpointRPC:102 : Timeout for ExpectedSizeIterator is: 198000 > > > 2017-01-26 18:33:50,420 DEBUG [http-bio-10101-exec-7] > enumerator.OLAPEnumerator:127 : return TupleIterator... > > > 2017-01-26 18:33:50,430 ERROR [pool-7-thread-5] > util.LoggableCachedThreadPool:44 : Execution exception when running task in > pool-7-thread-5 > > > 2017-01-26 18:33:50,431 ERROR [pool-7-thread-5] > util.LoggableCachedThreadPool:54 : Caught exception in thread > pool-7-thread-5: > > > java.lang.RuntimeException: <sub-thread for GTScanRequest 1c42a42b> Error > when visiting cubes by endpoint > > > at > org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1.run(CubeHBaseEndpointRPC.java:344) > > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > > at java.lang.Thread.run(Thread.java:745) > > > Caused by: org.apache.hadoop.hbase.DoNotRetryIOException: > org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.NoSuchMethodError: > org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion()Lorg/apache/hadoop/hbase/regionserver/HRegion; > > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2212) > > > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109) > > > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185) > > > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165) > > > Caused by: java.lang.NoSuchMethodError: > org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion()Lorg/apache/hadoop/hbase/regionserver/HRegion; > > > at > org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService.visitCube(CubeVisitService.java:177) > > > at > org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService.callMethod(CubeVisitProtos.java:4164) > > > at > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7797) > > > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1982) > > > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1964) > > > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33652) > > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170) > > > ... 3 more > > > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > > > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > > > at > org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106) > > > at > org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95) > > > at > org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:327) > > > at > org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1631) > > > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:92) > > > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:89) > > > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126) > > > at > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:95) > > > at > org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:56) > > > at > org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:4225) > > > at > org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$2.call(CubeHBaseEndpointRPC.java:392) > > > at > org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$2.call(CubeHBaseEndpointRPC.java:388) > > > at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1797) > > > ... 4 more > > > Caused by: > org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException): > org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.NoSuchMethodError: > org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion()Lorg/apache/hadoop/hbase/regionserver/HRegion; > > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2212) > > > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109) > > > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185) > > > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165) > > > Caused by: java.lang.NoSuchMethodError: > org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion()Lorg/apache/hadoop/hbase/regionserver/HRegion; > > > at > org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService.visitCube(CubeVisitService.java:177) > > > at > org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService.callMethod(CubeVisitProtos.java:4164) > > > at > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7797) > > > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1982) > > > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1964) > > > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33652) > > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170) > > > ... 3 more > > > > at > org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1269) > > > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:226) > > > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:331) > > > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.execService(ClientProtos.java:34118) > > > at > org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1627) > > > ... 13 more > > > > > > Thanks, > > > Manoj Jain > > > >
