Dear: Wechat group "Apache Linkis community Development group "chat records are as follows: 微信群"Apache Linkis 社区开发群"的聊天记录如下:
————— 2022-10-10 ————— Mr Flash 16:19 The Dolphin Scheduling cluster version was successfully installed in one try I was lost in thought... peacewong@WDS 16:35 Linkis relies not only on the commands of the underlying engines, but also on the jar packages of the related engines, so there are some conflicts. The following optimization should try to use the local classpath of the machine Mr. Lightning 16:38 Using native dependencies is one way to do this, but I also have a very awkward situation. kylin, for example, uses native dependencies. The last time I tested a version, it didn't boot and loaded hundreds of jars. I know there's a packet conflict, but I don't know where to start Mr. Lightning 16:39 kylin wrote several scripts to collect locally dependent jars Zhi-yuan liu ephod The dolphin's startup script is to load the local jars into the classpatch Zhi-yuan liu 16:41 It's actually loaded at run time... Zhi-yuan liu calm But it doesn't seem to depend much on the environment itself, it's basically just a matter of having the jdk connected to zk and mysql Sargent Ti 17:43 It is true that pom dolphin's main service does not have too many special dependencies, unlike linkis, scala, hadoop, spring cloud, etc. [Facepalm] we are also looking at how to streamline the future Zhi-yuan liu and He plugins the component connection spi, so he doesn't really care about the connection. Mr. Flash 18:30 [wiping sweat] I was just saying that when I realized I couldn't connect to hive, I might have to compile it myself Zhi-yuan liu when It is hive version 2 by default Zhi-yuan liu when Kerberos authentication doesn't seem to have much to say Mr. Flash 18:32 I hive is 1.1.0... Mr. Flash 18:33 Let's compile it tomorrow. It's an inevitable problem. You have to compile it yourself Zhi-yuan liu would have Oh, it's the same as mine Zhi-yuan liu 18:44 Change the hive.jdbc.version in the outermost pom Zhi-yuan liu 18:44 But it's version 3.0.1. If it's version 3.1.0, it looks like there are two changes 闪电先生 16:19 尝试了一下 一次就成功安装海豚调度集群版本 我陷入了沉思…… peacewong@WDS 16:35 Linkis因为不止依赖了各个底层引擎的命令，还依赖了相关的引擎的jar包所以确实存在一些冲突的问题，后续得优化下尽量使用机器本地的classpath 闪电先生 16:38 使用本机的依赖是个办法，不过这种我也碰到很尴尬的情况。 比如说kylin，就是用本机的依赖，我上次测试某个版本，启动不了，加载了几百个jar。 知道是有包冲突，但是无从下手 闪电先生 16:39 kylin写了几个脚本去收集本地依赖的jar 刘志远 16:40 海豚的启动脚本就是把本地jar加载到classpatch 刘志远 16:41 其实是运行的时候加载的。。。 刘志远 16:42 不过它本身好像就不太依赖环境，基本上只要有jdk连的上zk和mysql就行了 Sargent Ti 17:43 看pom dolphin的主服务确实并没有太多特别的依赖，不像linkis，scala、hadoop、spring cloud等等[捂脸]我们也在看后继怎么精简 刘志远 18:26 他是把组件连接spi插件化了，所以其实他是不太管能不能连上。 闪电先生 18:30 [擦汗]我才刚说完 就发现连不上hive可能还是要自己编译 刘志远 18:31 他默认是2版本的hive 刘志远 18:31 Kerberos认证好像没有太多说明 闪电先生 18:32 我hive是1.1.0…… 闪电先生 18:33 明天编译吧 这种问题确实无法避免 都要自己编译 刘志远 18:43 咦，跟我的一样呀 刘志远 18:44 就把最外面的pom里面的hive.jdbc.version版本改一下 刘志远 18:44 不过是3.0.1版本的，如果是3.1.0的话好像得改两个地方
