Hi Tengting Xu Thanks for your help, it works! I download spark3.2.1 and modify SPARK_HOME=`path in .bashrc to spark 3.1.1`, now I can import hive tables via kylin web interface. So is there any way to keep spark3.2.1 run on cluster and just keep kylin run on spark3.1.1?
Thanks! 发件人: Mukvin <[email protected]> 发送时间: 2022年5月20日 9:22 收件人: [email protected]; [email protected] 主题: Re:回复: Re:回复: Re:回复: File does not exist ....... fairscheduler.xml Hi Gao Michael, I mean you can export SPARK_HOME=`path to spark 3.1.1` for KYLIN, do not need to downgrade 3.2.1. You can keep them both. -- Best regards. Tengting Xu At 2022-05-19 22:47:14, "Gao Michael" <[email protected]<mailto:[email protected]>> wrote: Hi Tengting Xu: I already deployed spark3.2.1 on my cluster, you mean I should degrade it to 3.1.1? 发件人: Mukvin <[email protected]<mailto:[email protected]>> 发送时间: 2022年5月19日 22:36 收件人: [email protected]<mailto:[email protected]>; [email protected]<mailto:[email protected]> 主题: Re:回复: Re:回复: File does not exist ....... fairscheduler.xml Hi Gao Michael, As I checked the log file, I found two errors in it. "controller.TableController:131 : Failed to load Hive Table java.lang.StackOverflowError" and "File not found exception". They are related to spark. As you have mentioned your env is with spark 3.2.1, but currently, kylin-4.0.1-bin-spark3.tar.gz only supports spark 3.1.1 as Kylin's official site mentioned, so you may need to download spark 3.1.1 and export the path to be `spark home`. Then you can try to restart Kylin again and check. -- Best regards. Tengting Xu At 2022-05-19 22:15:15, "Gao Michael" <[email protected]<mailto:[email protected]>> wrote: Hi Mukin The log file too long and I only can only send pats of it as an attachment. I import table via kylin web interface, and watch log file by command “tail -f”, The log file watch terminal was rolling continuous in the process of import. Attachment cover a complete loading process Thanks 发件人: Mukvin <[email protected]<mailto:[email protected]>> 发送时间: 2022年5月19日 21:28 收件人: [email protected]<mailto:[email protected]>; [email protected]<mailto:[email protected]> 主题: Re:回复: File does not exist ....... fairscheduler.xml Hi Gao Michael, Could you give a full log about kylin.log located in $KYLIN_HOME/logs/kylin.log. Let me get the full trace of this error. -- Best regards. Tengting Xu At 2022-05-19 19:41:20, "Gao Michael" <[email protected]<mailto:[email protected]>> wrote: Hi all, Who can Help? 发件人: Gao Michael <[email protected]<mailto:[email protected]>> 发送时间: 2022年5月18日 19:23 收件人: [email protected]<mailto:[email protected]> 主题: File does not exist ....... fairscheduler.xml Hi all I got follow error when I try to add table to my kylin project via web interface: 2022-05-18 18:48:37,904 INFO [Thread-7] ui.SparkUI:57 : Bound SparkUI to 0.0.0.0, and started at http://hadoop-cluster-001:4040 2022-05-18 18:48:38,681 ERROR [Thread-7] scheduler.FairSchedulableBuilder:94 : Error while building the fair scheduler pools java.io.FileNotFoundException: File does not exist: /home/michael/opt/modules/kylin/apache-kylin-4.0.1-bin-spark3/conf/fairscheduler.xml and I’m sure file exist! michael@hadoop-cluster-001:~/opt/modules/kylin/apache-kylin-4.0.1-bin-spark3$<mailto:michael@hadoop-cluster-001:~/opt/modules/kylin/apache-kylin-4.0.1-bin-spark3$> ls conf/ fairscheduler.xml kylin_job_conf_inmem.xml kylin.properties setenv.sh spark-executor-log4j.properties kylin_hive_conf.xml kylin_job_conf.xml kylin-server-log4j.properties setenv-tool.sh kylin_job_conf_cube_merge.xml kylin-kafka-consumer.xml kylin-tools-log4j.properties spark-driver-log4j.properties What’s the problem and how to solve? Thanks!
