> We were running klyin 1.5 on CDH 5.7 and was working fine.
>
> Post upgrade to CDH 5.9, we are seeing an exception while running the queries
> in Kylin.
>
> I have restarted the Kylin 1.5 instance and also build the test cube from
> scratch. The cube built was successful but seeing an exception in the
> Kylin.log when running the queries (in the Kyiin ‘Insight’ tab).
>
> Please help us to resolve this issue at the earliest. Let us know if you
> need any other information.
>
> Kylin Version: 1.5.2
>
> Excerpts from the Server Environment
> HBASE_LOG_DIR = /var/log/hbase
> HBASE_HOME = /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hbase/bin/..
> HBASE_OPTS = -Xmx268435456 -XX:+HeapDumpOnOutOfMemoryError
> -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -Djava.net
> <http://djava.net/>.preferIPv4Stack=true
> -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf
> -Dhbase.log.dir=/var/log/hbase -Dhbase.log.file=hbase.log
> -Dhbase.home.dir=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hbase/bin/..
> -Dhbase.id.str=hbase -Dhbase.root.logger=INFO,console
> -Djava.library.path=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hadoop/lib/native:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hbase/bin/../lib/native/Linux-amd64-64
> -Dhbase.security.logger=INFO,NullAppender
> HBASE_ENV_INIT = true
> hbase_dependency =
> /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hbase/bin/../lib/hbase-common-1.2.0-cdh5.9.0.jar
>
>
>
> Kylin log exceprts
>
> 2017-01-26 18:33:50,419 INFO [http-bio-10101-exec-7] v2.CubeHBaseRPC:271 :
> Visiting hbase table KYLIN_XXXXXXXXX: cuboid exact match, from 2 to 2 Start:
> \x00\x00\x00\x00\x00\x0
> 0\x00\x00\x00\x02\x00 (\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00) Stop:
> \x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\xFF\x00
> (\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\xFF\x00),
> No Fuzzy Key
> 2017-01-26 18:33:50,419 DEBUG [http-bio-10101-exec-7]
> v2.CubeHBaseEndpointRPC:319 : Submitting rpc to 1 shards starting from shard
> 0, scan range count 1
> 2017-01-26 18:33:50,419 INFO [http-bio-10101-exec-7]
> v2.CubeHBaseEndpointRPC:102 : Timeout for ExpectedSizeIterator is: 198000
> 2017-01-26 18:33:50,420 DEBUG [http-bio-10101-exec-7]
> enumerator.OLAPEnumerator:127 : return TupleIterator...
> 2017-01-26 18:33:50,430 ERROR [pool-7-thread-5]
> util.LoggableCachedThreadPool:44 : Execution exception when running task in
> pool-7-thread-5
> 2017-01-26 18:33:50,431 ERROR [pool-7-thread-5]
> util.LoggableCachedThreadPool:54 : Caught exception in thread
> pool-7-thread-5:
> java.lang.RuntimeException: <sub-thread for GTScanRequest 1c42a42b> Error
> when visiting cubes by endpoint
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1.run(CubeHBaseEndpointRPC.java:344)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.DoNotRetryIOException:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion()Lorg/apache/hadoop/hbase/regionserver/HRegion;
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2212)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion()Lorg/apache/hadoop/hbase/regionserver/HRegion;
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService.visitCube(CubeVisitService.java:177)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService.callMethod(CubeVisitProtos.java:4164)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7797)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1982)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1964)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33652)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)
> ... 3 more
>
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:327)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1631)
> at
> org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:92)
> at
> org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:89)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
> at
> org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:95)
> at
> org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:56)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:4225)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$2.call(CubeHBaseEndpointRPC.java:392)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$2.call(CubeHBaseEndpointRPC.java:388)
> at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1797)
> ... 4 more
> Caused by:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException):
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion()Lorg/apache/hadoop/hbase/regionserver/HRegion;
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2212)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion()Lorg/apache/hadoop/hbase/regionserver/HRegion;
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService.visitCube(CubeVisitService.java:177)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService.callMethod(CubeVisitProtos.java:4164)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7797)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1982)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1964)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33652)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)
> ... 3 more
>
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1269)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:226)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:331)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.execService(ClientProtos.java:34118)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1627)
> ... 13 more
>
>
>
> Thanks,
> Manoj Jain
