shenbinglife opened a new issue, #1952: URL: https://github.com/apache/incubator-linkis/issues/1952
### Search before asking - [X] I searched the [issues](https://github.com/apache/incubator-linkis/issues) and found no similar issues. ### Linkis Component linkis-cg-engineConnplugin ### What happened + What you expected to happen 1. Exec python in Scripts ```python print("asdf") ``` 2. page return failed， notice ``` Job with execId-IDE_root_spark_1 and subJobId : 12 from orchestrator completed with state ErrorExecuteResponse(21304, Task is Failed,errorMsg: errCode: 12003 ,desc: cq-test-dtp-cdh-10-3-104-43:9101_2 Failed to async get EngineNode AMErrorException: errCode: 30002 ,desc: ServiceInstance(linkis-cg-engineconn, cq-test-dtp-cdh-10-3-104-43:34171) ticketID:ee7f2405-9e34-4e36-a1d4-0fa0fc8942d2 初始化引擎失败,原因: Failed to start EngineConn, reason: ECMErrorException: errCode: 10000 ,desc: Failed to init DefaultEngineConn(ShuttingDown, ee7f2405-9e34-4e36-a1d4-0fa0fc8942d2, org.apache.linkis.manager.engineplugin.common.resource.UserNodeResource@1bd1844d, [[key: userCreator, value: {"creator":"IDE","user":"root"}, str: root-IDE], [key: engineType, value: {"engineType":"spark","version":"2.4.3"}, str: spark-2.4.3]], EngineConnCreationDescImpl(dssmark_id: mark_2,null,{wds.linkis.rm.yarnqueue.memory.max=300G, wds.linkis.rm.client.core.max=10, wds.linkis.rm.instance=10, wds.linkis.rm .yarnqueue.instance.max=30, wds.linkis.rm.yarnqueue.cores.max=150, wds.linkis.rm.client.memory.max=20G, wds.linkis.rm.yarnqueue=default}), org.apache.linkis.ecm.core.engineconn.EngineConnInfo@33e362f3, null, org.apache.linkis.ecm.core.launch.EngineConnLaunchRunnerImpl@6bcd8032, ServiceInstance(linkis-cg-engineconn, cq-test-dtp-cdh-10-3-104-43:34171), null), status shutting down ,ip: cq-test-dtp-cdh-10-3-104-43 ,port: 9102 ,serviceKind: linkis-cg-engineconnmanager You can go to this path(/appcom/tmp/root/workDir/ee7f2405-9e34-4e36-a1d4-0fa0fc8942d2/logs) to find the reason or ask the administrator for help ,ip: cq-test-dtp-cdh-10-3-104-43 ,port: 9101 ,serviceKind: linkis-cg-linkismanager ,ip: cq-test-dtp-cdh-10-3-104-43 ,port: 9104 ,serviceKind: linkis-cg-entrance,null) ``` 3. see workDir log , find guava conflict ``` The following method did not exist: com.google.common.cache.CacheBuilder.recordStats()Lcom/google/common/cache/CacheBuilder; The method's class, com.google.common.cache.CacheBuilder, is available from the following locations: jar:file:/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/jars/guava-11.0.2.jar!/com/google/common/cache/CacheBuilder.class jar:file:/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/spark/hive/hive-exec-2.1.1-cdh6.3.0.jar!/com/google/common/cache/CacheBuilder.class The class hierarchy was loaded from the following locations: com.google.common.cache.CacheBuilder: file:/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/jars/guava-11.0.2.jar ``` 4. see workDir engineConnExec.sh, And why it's a spark job? I am running python ```shell spark-submit --master yarn --deploy-mode client --name Linkis-EngineConn-Spark_IDE --jars ,${UDF_JARS} --conf spark.driver.extraJavaOptions=" -server -XX:+UseG1GC -XX:MaxPermSize=250m -XX:PermSize=128m -Xloggc:${LOG_DIRS}/gc.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Dwds.linkis.server.conf=linkis-engineconn.properties -Dwds.linkis.gateway.url=http://****:9201 -Dlogging.file=log4j2-engineconn.xml -DTICKET_ID=7de696ac-ae00-4478-bd49-62a2b4f78a76 -Djava.io.tmpdir=${TEMP_DIRS} -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=${RANDOM_PORT}" --driver-memory 2g --driver-class-path :${CLASSPATH} --driver-cores 1 --executor-memory 4g --executor-cores 2 --num-executors 3 --queue default --class org.apache.linkis.engineconn.launch.EngineConnServer 1> ${LOG_DIRS}/stdout 2>> ${LOG_DIRS}/stderr ${PWD}/lib/linkis-engineplugin-spark-1.0.3.jar --engineconn-conf wds.linkis.rm.instance=10 --engineconn-conf label.userCreator=root-IDE --engineconn-conf tick etId=7de696ac-ae00-4478-bd49-62a2b4f78a76 --engineconn-conf wds.linkis.rm.yarnqueue.memory.max=300G --engineconn-conf label.engineType=spark-2.4.3 --engineconn-conf wds.linkis.rm.yarnqueue.instance.max=30 --engineconn-conf wds.linkis.rm.client.memory.max=20G --engineconn-conf wds.linkis.rm.client.core.max=10 --engineconn-conf wds.linkis.rm.yarnqueue.cores.max=150 --engineconn-conf user=root --engineconn-conf wds.linkis.rm.yarnqueue=default --spring-conf eureka.client.serviceUrl.defaultZone=http://****:20303/eureka/ --spring-conf logging.config=classpath:log4j2-engineconn.xml --spring-conf spring.profiles.active=engineconn --spring-conf server.port=46287 --spring-conf spring.application.name=linkis-cg-engineconn ``` ### Relevent platform DSS 1.0.1 Linkis 1.0.3 CDH6.3.0 ### Reproduction script see before ### Anything else _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@linkis.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@linkis.apache.org For additional commands, e-mail: dev-h...@linkis.apache.org
