Hello, I appreciate your assistance. The solution you offered successfully
resolved the previous issue. However, when running the system afterward, I
encountered the following error.
2023-08-15 19:57:29.057 ERROR jobRequest(LINKISCLI_techuser_hive_2)
execute failed,21304, Task is Failed,errorMsg: errCode: 12003 ,desc:
linkis-instance:9101_2 Failed to async get EngineNode AMErrorException:
errCode: 30002 ,desc: ServiceInstance(linkis-cg-engineconn,
linkis-instance:34063) ticketID: eaeaf65c-db6e-4841-b5f8-4e907021f5bf
Failed to initialize engine, reason: ServiceInstance(linkis-cg-engineconn,
linkis-instance:34063): log dir:
/appcom/tmp/techuser/20230815/hive/eaeaf65c-db6e-4841-b5f8-4e907021f5bf/logs,java.lang.NoClassDefFoundError:
com/fasterxml/jackson/jaxrs/json/JacksonJaxbJsonProvider
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.<clinit>(TimelineUtils.java:60)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:200)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.client.TezYarnClient.init(TezYarnClient.java:49)
at
org.apache.tez.client.TezClient.startFrameworkClient(TezClient.java:487)
at org.apache.tez.client.TezClient.start(TezClient.java:384)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:433)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:368)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:245)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:236)
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:678)
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:586)
at
org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.getSessionState(HiveEngineConnFactory.scala:135)
at
org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.doCreateHiveSession(HiveEngineConnFactory.scala:120)
at
org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConnSession(HiveEngineConnFactory.scala:105)
at
org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConnSession(HiveEngineConnFactory.scala:60)
at
org.apache.linkis.manager.engineplugin.common.creation.AbstractEngineConnFactory.createEngineConn(EngineConnFactory.scala:47)
at
org.apache.linkis.manager.engineplugin.common.creation.AbstractEngineConnFactory.createEngineConn$(EngineConnFactory.scala:45)
at
org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConn(HiveEngineConnFactory.scala:60)
at
org.apache.linkis.engineconn.core.engineconn.DefaultEngineConnManager.createEngineConn(EngineConnManager.scala:47)
at
org.apache.linkis.engineconn.launch.EngineConnServer$.main(EngineConnServer.scala:88)
at
org.apache.linkis.engineconn.launch.EngineConnServer.main(EngineConnServer.scala)
Caused by: java.lang.ClassNotFoundException:
com.fasterxml.jackson.jaxrs.json.JacksonJaxbJsonProvider
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 34 more
,ip: linkis-instance ,port: 9101 ,serviceKind: linkis-cg-linkismanager
,ip: linkis-instance ,port: 9104 ,serviceKind: linkis-cg-entrance
I've ensured that the relevant JAR is present in both
{LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/dist/3.1.3/lib and
{LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/plugin/3.1.3. Despite
these efforts, the issue remains unresolved. Could you please provide
guidance on this matter?
Thanks and Regards,
Arjun S
On Tue, 15 Aug 2023 at 08:19, peacewong <peacew...@apache.org> wrote:
> Hi Arjun,
> Linkis is compatible with hive on tez and requires the following steps:
> - You need to copy Tez-related dependencies to
> {LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/dist/3.1.3/lib is the dist
> not plugin directory [1]. You can also modify hive ec pom to add tez
> dependency compile
> - vim
>
> {LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/dist/3.1.3/conf/linkis-engineconn.properties
> and update linkis.hive.engine.type=tez
> - sh {LINKIS_HOME}/sbin/linkis-daemon.sh restart linkis-cg-manager
>
>
> 1. https://linkis.apache.org/docs/latest/deployment/install-engineconn/
>
>
> Best Regards,
> Peace Wong.
>
> arjun s <arjunjoice...@gmail.com> 于2023年8月15日周二 02:40写道：
>
> > Hi team,
> > I'm new to Apache Linkis. I've successfully set up Apache Linkis version
> > 1.4.1 and I'm currently attempting to integrate it with Hive version
> 3.1.3.
> > However, I've encountered an issue while executing the following command:
> > sh ./bin/linkis-cli -engineType hive-3.1.3 -codeType hql -code "select *
> > from load1" -submitUser techuser -proxyUser techuser
> >
> > I am getting below error
> > 2023-08-14 11:53:20.053 INFO Background is starting a new engine for
> > you,execId TaskID_10_otJobId_astJob_9_codeExec_9 mark id is mark_12, it
> may
> > take several seconds, please wait
> > 2023-08-14 11:53:42.053 ERROR jobRequest(LINKISCLI_techuser_hive_7)
> > execute failed,21304, Task is Failed,errorMsg: errCode: 12003 ,desc:
> > linkis-instance:9101_16 Failed to async get EngineNode AMErrorException:
> > errCode: 30002 ,desc: ServiceInstance(linkis-cg-engineconn,
> > linkis-instance:41969) ticketID: e04ac7a2-fc39-43fb-87a6-d88d4e084f13
> > Failed to initialize engine, reason:
> ServiceInstance(linkis-cg-engineconn,
> > linkis-instance:41969): log dir:
> >
> >
> /appcom/tmp/techuser/20230814/hive/e04ac7a2-fc39-43fb-87a6-d88d4e084f13/logs,java.lang.NoClassDefFoundError:
> > org/apache/tez/dag/api/TezConfiguration
> > at
> >
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:661)
> > at
> >
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:586)
> > at
> >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.getSessionState(HiveEngineConnFactory.scala:135)
> > at
> >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.doCreateHiveSession(HiveEngineConnFactory.scala:120)
> > at
> >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConnSession(HiveEngineConnFactory.scala:105)
> > at
> >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConnSession(HiveEngineConnFactory.scala:60)
> > at
> >
> >
> org.apache.linkis.manager.engineplugin.common.creation.AbstractEngineConnFactory.createEngineConn(EngineConnFactory.scala:47)
> > at
> >
> >
> org.apache.linkis.manager.engineplugin.common.creation.AbstractEngineConnFactory.createEngineConn$(EngineConnFactory.scala:45)
> > at
> >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConn(HiveEngineConnFactory.scala:60)
> > at
> >
> >
> org.apache.linkis.engineconn.core.engineconn.DefaultEngineConnManager.createEngineConn(EngineConnManager.scala:47)
> > at
> >
> >
> org.apache.linkis.engineconn.launch.EngineConnServer$.main(EngineConnServer.scala:88)
> > at
> >
> >
> org.apache.linkis.engineconn.launch.EngineConnServer.main(EngineConnServer.scala)
> > Caused by: java.lang.ClassNotFoundException:
> > org.apache.tez.dag.api.TezConfiguration
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> > ... 12 more
> > ,ip: linkis-instance ,port: 9101 ,serviceKind: linkis-cg-linkismanager
> > ,ip: linkis-instance ,port: 9104 ,serviceKind: linkis-cg-entrance
> >
> >
> > To provide context, I'm using Hive on Tez, and I've copied all the Tez
> JAR
> > files to {LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/plugin/3.1.3.
> > However, despite this, the error persists. My concern is whether the new
> > version of Linkis supports Hive on Tez, or if there are any additional
> > steps I need to follow to enable Hive on Tez integration.
> >
> > Thanks in Advance
> >
>
