Hi Arjun,
Thank you.
arjun s <arjunjoice...@gmail.com> 于2023年8月21日周一 18:08写道：
> Hi,
> Sure, I will submit PR to Linkis Project
>
> Thanks and Regards,
> Arjun S
>
> On Thu, 17 Aug 2023 at 09:05, peacewong <peacew...@apache.org> wrote:
>
> > Hello,
> >
> > Can you submit the relevant PR to the Linkis project, and add these
> > dependencies by default?
> >
> > Best Regards,
> > Peace Wong
> >
> > arjun s <arjunjoice...@gmail.com> 于2023年8月16日周三 23:04写道：
> >
> > > Hi,
> > > I omitted to restart the service after placing the appropriate JAR file
> > in
> > > the directory.
> > >
> > > Thanks and Regards,
> > > Arjun S
> > >
> > > On Wed, 16 Aug 2023 at 12:59, peacewong <peacew...@apache.org> wrote:
> > >
> > > > Thank you, what is the reason?
> > > >
> > > > arjun s <arjunjoice...@gmail.com> 于2023年8月16日周三 15:00写道：
> > > >
> > > > > Hi,
> > > > > Above issue is resolved.
> > > > >
> > > > > Thanks and Regards,
> > > > > Arjun S
> > > > >
> > > > >
> > > > > On Wed, 16 Aug 2023 at 01:34, arjun s <arjunjoice...@gmail.com>
> > wrote:
> > > > >
> > > > > > Hello, I appreciate your assistance. The solution you offered
> > > > > successfully
> > > > > > resolved the previous issue. However, when running the system
> > > > afterward,
> > > > > I
> > > > > > encountered the following error.
> > > > > > 2023-08-15 19:57:29.057 ERROR
> jobRequest(LINKISCLI_techuser_hive_2)
> > > > > > execute failed,21304, Task is Failed,errorMsg: errCode: 12003
> > ,desc:
> > > > > > linkis-instance:9101_2 Failed to async get EngineNode
> > > > AMErrorException:
> > > > > > errCode: 30002 ,desc: ServiceInstance(linkis-cg-engineconn,
> > > > > > linkis-instance:34063) ticketID:
> > eaeaf65c-db6e-4841-b5f8-4e907021f5bf
> > > > > > Failed to initialize engine, reason:
> > > > > ServiceInstance(linkis-cg-engineconn,
> > > > > > linkis-instance:34063): log dir:
> > > > > >
> > > > >
> > > >
> > >
> >
> /appcom/tmp/techuser/20230815/hive/eaeaf65c-db6e-4841-b5f8-4e907021f5bf/logs,java.lang.NoClassDefFoundError:
> > > > > > com/fasterxml/jackson/jaxrs/json/JacksonJaxbJsonProvider
> > > > > > at java.lang.ClassLoader.defineClass1(Native Method)
> > > > > > at
> java.lang.ClassLoader.defineClass(ClassLoader.java:756)
> > > > > > at
> > > > > >
> > > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> > > > > > at
> > > java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
> > > > > > at
> > java.net.URLClassLoader.access$100(URLClassLoader.java:74)
> > > > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
> > > > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
> > > > > > at java.security.AccessController.doPrivileged(Native
> > Method)
> > > > > > at
> > java.net.URLClassLoader.findClass(URLClassLoader.java:362)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> > > > > > at
> > > > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.yarn.util.timeline.TimelineUtils.<clinit>(TimelineUtils.java:60)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:200)
> > > > > > at
> > > > > >
> > > >
> > org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> > > > > > at
> > > > > org.apache.tez.client.TezYarnClient.init(TezYarnClient.java:49)
> > > > > > at
> > > > > >
> > > >
> > org.apache.tez.client.TezClient.startFrameworkClient(TezClient.java:487)
> > > > > > at
> > org.apache.tez.client.TezClient.start(TezClient.java:384)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:433)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:368)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:245)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:236)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:678)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:586)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.getSessionState(HiveEngineConnFactory.scala:135)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.doCreateHiveSession(HiveEngineConnFactory.scala:120)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConnSession(HiveEngineConnFactory.scala:105)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConnSession(HiveEngineConnFactory.scala:60)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.linkis.manager.engineplugin.common.creation.AbstractEngineConnFactory.createEngineConn(EngineConnFactory.scala:47)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.linkis.manager.engineplugin.common.creation.AbstractEngineConnFactory.createEngineConn$(EngineConnFactory.scala:45)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConn(HiveEngineConnFactory.scala:60)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineconn.core.engineconn.DefaultEngineConnManager.createEngineConn(EngineConnManager.scala:47)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineconn.launch.EngineConnServer$.main(EngineConnServer.scala:88)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineconn.launch.EngineConnServer.main(EngineConnServer.scala)
> > > > > > Caused by: java.lang.ClassNotFoundException:
> > > > > > com.fasterxml.jackson.jaxrs.json.JacksonJaxbJsonProvider
> > > > > > at
> > java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> > > > > > at
> > > > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> > > > > > ... 34 more
> > > > > > ,ip: linkis-instance ,port: 9101 ,serviceKind:
> > > linkis-cg-linkismanager
> > > > > > ,ip: linkis-instance ,port: 9104 ,serviceKind: linkis-cg-entrance
> > > > > >
> > > > > >
> > > > > > I've ensured that the relevant JAR is present in both
> > > > > > {LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/dist/3.1.3/lib
> and
> > > > > > {LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/plugin/3.1.3.
> > > Despite
> > > > > > these efforts, the issue remains unresolved. Could you please
> > provide
> > > > > > guidance on this matter?
> > > > > >
> > > > > > Thanks and Regards,
> > > > > > Arjun S
> > > > > >
> > > > > > On Tue, 15 Aug 2023 at 08:19, peacewong <peacew...@apache.org>
> > > wrote:
> > > > > >
> > > > > >> Hi Arjun,
> > > > > >> Linkis is compatible with hive on tez and requires the
> following
> > > > > steps:
> > > > > >> - You need to copy Tez-related dependencies to
> > > > > >> {LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/dist/3.1.3/lib
> is
> > > the
> > > > > >> dist
> > > > > >> not plugin directory [1]. You can also modify hive ec pom to add
> > tez
> > > > > >> dependency compile
> > > > > >> - vim
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> {LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/dist/3.1.3/conf/linkis-engineconn.properties
> > > > > >> and update linkis.hive.engine.type=tez
> > > > > >> - sh {LINKIS_HOME}/sbin/linkis-daemon.sh restart
> > linkis-cg-manager
> > > > > >>
> > > > > >>
> > > > > >> 1.
> > > > https://linkis.apache.org/docs/latest/deployment/install-engineconn/
> > > > > >>
> > > > > >>
> > > > > >> Best Regards,
> > > > > >> Peace Wong.
> > > > > >>
> > > > > >> arjun s <arjunjoice...@gmail.com> 于2023年8月15日周二 02:40写道：
> > > > > >>
> > > > > >> > Hi team,
> > > > > >> > I'm new to Apache Linkis. I've successfully set up Apache
> Linkis
> > > > > version
> > > > > >> > 1.4.1 and I'm currently attempting to integrate it with Hive
> > > version
> > > > > >> 3.1.3.
> > > > > >> > However, I've encountered an issue while executing the
> following
> > > > > >> command:
> > > > > >> > sh ./bin/linkis-cli -engineType hive-3.1.3 -codeType hql -code
> > > > > "select *
> > > > > >> > from load1" -submitUser techuser -proxyUser techuser
> > > > > >> >
> > > > > >> > I am getting below error
> > > > > >> > 2023-08-14 11:53:20.053 INFO Background is starting a new
> engine
> > > for
> > > > > >> > you,execId TaskID_10_otJobId_astJob_9_codeExec_9 mark id is
> > > mark_12,
> > > > > it
> > > > > >> may
> > > > > >> > take several seconds, please wait
> > > > > >> > 2023-08-14 11:53:42.053 ERROR
> > > jobRequest(LINKISCLI_techuser_hive_7)
> > > > > >> > execute failed,21304, Task is Failed,errorMsg: errCode: 12003
> > > > ,desc:
> > > > > >> > linkis-instance:9101_16 Failed to async get EngineNode
> > > > > >> AMErrorException:
> > > > > >> > errCode: 30002 ,desc: ServiceInstance(linkis-cg-engineconn,
> > > > > >> > linkis-instance:41969) ticketID:
> > > > e04ac7a2-fc39-43fb-87a6-d88d4e084f13
> > > > > >> > Failed to initialize engine, reason:
> > > > > >> ServiceInstance(linkis-cg-engineconn,
> > > > > >> > linkis-instance:41969): log dir:
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> /appcom/tmp/techuser/20230814/hive/e04ac7a2-fc39-43fb-87a6-d88d4e084f13/logs,java.lang.NoClassDefFoundError:
> > > > > >> > org/apache/tez/dag/api/TezConfiguration
> > > > > >> > at
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:661)
> > > > > >> > at
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:586)
> > > > > >> > at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.getSessionState(HiveEngineConnFactory.scala:135)
> > > > > >> > at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.doCreateHiveSession(HiveEngineConnFactory.scala:120)
> > > > > >> > at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConnSession(HiveEngineConnFactory.scala:105)
> > > > > >> > at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConnSession(HiveEngineConnFactory.scala:60)
> > > > > >> > at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.linkis.manager.engineplugin.common.creation.AbstractEngineConnFactory.createEngineConn(EngineConnFactory.scala:47)
> > > > > >> > at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.linkis.manager.engineplugin.common.creation.AbstractEngineConnFactory.createEngineConn$(EngineConnFactory.scala:45)
> > > > > >> > at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConn(HiveEngineConnFactory.scala:60)
> > > > > >> > at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineconn.core.engineconn.DefaultEngineConnManager.createEngineConn(EngineConnManager.scala:47)
> > > > > >> > at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineconn.launch.EngineConnServer$.main(EngineConnServer.scala:88)
> > > > > >> > at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> org.apache.linkis.engineconn.launch.EngineConnServer.main(EngineConnServer.scala)
> > > > > >> > Caused by: java.lang.ClassNotFoundException:
> > > > > >> > org.apache.tez.dag.api.TezConfiguration
> > > > > >> > at
> > > > java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> > > > > >> > at
> java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> > > > > >> > at
> > > > > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> > > > > >> > at
> java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> > > > > >> > ... 12 more
> > > > > >> > ,ip: linkis-instance ,port: 9101 ,serviceKind:
> > > > > linkis-cg-linkismanager
> > > > > >> > ,ip: linkis-instance ,port: 9104 ,serviceKind:
> > linkis-cg-entrance
> > > > > >> >
> > > > > >> >
> > > > > >> > To provide context, I'm using Hive on Tez, and I've copied all
> > the
> > > > Tez
> > > > > >> JAR
> > > > > >> > files to
> > {LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/plugin/
> > > > > 3.1.3.
> > > > > >> > However, despite this, the error persists. My concern is
> whether
> > > the
> > > > > new
> > > > > >> > version of Linkis supports Hive on Tez, or if there are any
> > > > additional
> > > > > >> > steps I need to follow to enable Hive on Tez integration.
> > > > > >> >
> > > > > >> > Thanks in Advance
> > > > > >> >
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
