Hi,
Above issue is resolved.
Thanks and Regards,
Arjun S
On Wed, 16 Aug 2023 at 01:34, arjun s <arjunjoice...@gmail.com> wrote:
> Hello, I appreciate your assistance. The solution you offered successfully
> resolved the previous issue. However, when running the system afterward, I
> encountered the following error.
> 2023-08-15 19:57:29.057 ERROR jobRequest(LINKISCLI_techuser_hive_2)
> execute failed,21304, Task is Failed,errorMsg: errCode: 12003 ,desc:
> linkis-instance:9101_2 Failed to async get EngineNode AMErrorException:
> errCode: 30002 ,desc: ServiceInstance(linkis-cg-engineconn,
> linkis-instance:34063) ticketID: eaeaf65c-db6e-4841-b5f8-4e907021f5bf
> Failed to initialize engine, reason: ServiceInstance(linkis-cg-engineconn,
> linkis-instance:34063): log dir:
> /appcom/tmp/techuser/20230815/hive/eaeaf65c-db6e-4841-b5f8-4e907021f5bf/logs,java.lang.NoClassDefFoundError:
> com/fasterxml/jackson/jaxrs/json/JacksonJaxbJsonProvider
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at
> org.apache.hadoop.yarn.util.timeline.TimelineUtils.<clinit>(TimelineUtils.java:60)
> at
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:200)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at org.apache.tez.client.TezYarnClient.init(TezYarnClient.java:49)
> at
> org.apache.tez.client.TezClient.startFrameworkClient(TezClient.java:487)
> at org.apache.tez.client.TezClient.start(TezClient.java:384)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:433)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:368)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:245)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:236)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:678)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:586)
> at
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.getSessionState(HiveEngineConnFactory.scala:135)
> at
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.doCreateHiveSession(HiveEngineConnFactory.scala:120)
> at
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConnSession(HiveEngineConnFactory.scala:105)
> at
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConnSession(HiveEngineConnFactory.scala:60)
> at
> org.apache.linkis.manager.engineplugin.common.creation.AbstractEngineConnFactory.createEngineConn(EngineConnFactory.scala:47)
> at
> org.apache.linkis.manager.engineplugin.common.creation.AbstractEngineConnFactory.createEngineConn$(EngineConnFactory.scala:45)
> at
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConn(HiveEngineConnFactory.scala:60)
> at
> org.apache.linkis.engineconn.core.engineconn.DefaultEngineConnManager.createEngineConn(EngineConnManager.scala:47)
> at
> org.apache.linkis.engineconn.launch.EngineConnServer$.main(EngineConnServer.scala:88)
> at
> org.apache.linkis.engineconn.launch.EngineConnServer.main(EngineConnServer.scala)
> Caused by: java.lang.ClassNotFoundException:
> com.fasterxml.jackson.jaxrs.json.JacksonJaxbJsonProvider
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 34 more
> ,ip: linkis-instance ,port: 9101 ,serviceKind: linkis-cg-linkismanager
> ,ip: linkis-instance ,port: 9104 ,serviceKind: linkis-cg-entrance
>
>
> I've ensured that the relevant JAR is present in both
> {LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/dist/3.1.3/lib and
> {LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/plugin/3.1.3. Despite
> these efforts, the issue remains unresolved. Could you please provide
> guidance on this matter?
>
> Thanks and Regards,
> Arjun S
>
> On Tue, 15 Aug 2023 at 08:19, peacewong <peacew...@apache.org> wrote:
>
>> Hi Arjun,
>> Linkis is compatible with hive on tez and requires the following steps:
>> - You need to copy Tez-related dependencies to
>> {LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/dist/3.1.3/lib is the
>> dist
>> not plugin directory [1]. You can also modify hive ec pom to add tez
>> dependency compile
>> - vim
>>
>> {LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/dist/3.1.3/conf/linkis-engineconn.properties
>> and update linkis.hive.engine.type=tez
>> - sh {LINKIS_HOME}/sbin/linkis-daemon.sh restart linkis-cg-manager
>>
>>
>> 1. https://linkis.apache.org/docs/latest/deployment/install-engineconn/
>>
>>
>> Best Regards,
>> Peace Wong.
>>
>> arjun s <arjunjoice...@gmail.com> 于2023年8月15日周二 02:40写道：
>>
>> > Hi team,
>> > I'm new to Apache Linkis. I've successfully set up Apache Linkis version
>> > 1.4.1 and I'm currently attempting to integrate it with Hive version
>> 3.1.3.
>> > However, I've encountered an issue while executing the following
>> command:
>> > sh ./bin/linkis-cli -engineType hive-3.1.3 -codeType hql -code "select *
>> > from load1" -submitUser techuser -proxyUser techuser
>> >
>> > I am getting below error
>> > 2023-08-14 11:53:20.053 INFO Background is starting a new engine for
>> > you,execId TaskID_10_otJobId_astJob_9_codeExec_9 mark id is mark_12, it
>> may
>> > take several seconds, please wait
>> > 2023-08-14 11:53:42.053 ERROR jobRequest(LINKISCLI_techuser_hive_7)
>> > execute failed,21304, Task is Failed,errorMsg: errCode: 12003 ,desc:
>> > linkis-instance:9101_16 Failed to async get EngineNode
>> AMErrorException:
>> > errCode: 30002 ,desc: ServiceInstance(linkis-cg-engineconn,
>> > linkis-instance:41969) ticketID: e04ac7a2-fc39-43fb-87a6-d88d4e084f13
>> > Failed to initialize engine, reason:
>> ServiceInstance(linkis-cg-engineconn,
>> > linkis-instance:41969): log dir:
>> >
>> >
>> /appcom/tmp/techuser/20230814/hive/e04ac7a2-fc39-43fb-87a6-d88d4e084f13/logs,java.lang.NoClassDefFoundError:
>> > org/apache/tez/dag/api/TezConfiguration
>> > at
>> >
>> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:661)
>> > at
>> >
>> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:586)
>> > at
>> >
>> >
>> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.getSessionState(HiveEngineConnFactory.scala:135)
>> > at
>> >
>> >
>> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.doCreateHiveSession(HiveEngineConnFactory.scala:120)
>> > at
>> >
>> >
>> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConnSession(HiveEngineConnFactory.scala:105)
>> > at
>> >
>> >
>> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConnSession(HiveEngineConnFactory.scala:60)
>> > at
>> >
>> >
>> org.apache.linkis.manager.engineplugin.common.creation.AbstractEngineConnFactory.createEngineConn(EngineConnFactory.scala:47)
>> > at
>> >
>> >
>> org.apache.linkis.manager.engineplugin.common.creation.AbstractEngineConnFactory.createEngineConn$(EngineConnFactory.scala:45)
>> > at
>> >
>> >
>> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConn(HiveEngineConnFactory.scala:60)
>> > at
>> >
>> >
>> org.apache.linkis.engineconn.core.engineconn.DefaultEngineConnManager.createEngineConn(EngineConnManager.scala:47)
>> > at
>> >
>> >
>> org.apache.linkis.engineconn.launch.EngineConnServer$.main(EngineConnServer.scala:88)
>> > at
>> >
>> >
>> org.apache.linkis.engineconn.launch.EngineConnServer.main(EngineConnServer.scala)
>> > Caused by: java.lang.ClassNotFoundException:
>> > org.apache.tez.dag.api.TezConfiguration
>> > at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
>> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
>> > ... 12 more
>> > ,ip: linkis-instance ,port: 9101 ,serviceKind: linkis-cg-linkismanager
>> > ,ip: linkis-instance ,port: 9104 ,serviceKind: linkis-cg-entrance
>> >
>> >
>> > To provide context, I'm using Hive on Tez, and I've copied all the Tez
>> JAR
>> > files to {LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/plugin/3.1.3.
>> > However, despite this, the error persists. My concern is whether the new
>> > version of Linkis supports Hive on Tez, or if there are any additional
>> > steps I need to follow to enable Hive on Tez integration.
>> >
>> > Thanks in Advance
>> >
>>
>
