I have put the testing code at github https://github.com/guxiaobo/gxb-testing/blob/main/calcite-testing/src/test/java/com/xsmartware/testing/calcite/Test1.java
------------------ Original ------------------ From: "xiaobo ";<guxiaobo1...@qq.com>; Send time: Friday, Feb 25, 2022 9:05 PM To: "dev"<dev@calcite.apache.org>; Subject: Re: calcite multi-threading problem Hi, Now we come to the problem similar as the one at https://github.com/apache/calcite/pull/2433#issuecomment-860024076, a simple sql "select max(id) from s.t1" caused the following exception, users report upgrading janino to 3.1.6 can fix this problem, but as we discussed before janino 3.1.6 is not fully compatible with calcite 1.29.0 because of the dependency issue. at org.apache.calcite.avatica.Helper.wrap(Helper.java:37) ~[avatica-core-1.20.0.jar:1.20.0] at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:130) ~[calcite-core-1.29.0.jar:1.29.0] at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1130) ~[calcite-core-1.29.0.jar:1.29.0] at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepare_(CalcitePrepareImpl.java:1032) ~[calcite-core-1.29.0.jar:1.29.0] at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepareRel(CalcitePrepareImpl.java:988) ~[calcite-core-1.29.0.jar:1.29.0] at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:668) ~[calcite-core-1.29.0.jar:1.29.0] at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:513) ~[calcite-core-1.29.0.jar:1.29.0] at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:483) ~[calcite-core-1.29.0.jar:1.29.0] at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:249) ~[calcite-core-1.29.0.jar:1.29.0] at org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:229) ~[calcite-core-1.29.0.jar:1.29.0] ... 13 common frames omitted Caused by: org.codehaus.commons.compiler.InternalCompilerException: Compiling "Baz" in File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 1, Column 1: File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 41, Column 45: Compiling "bind(org.apache.calcite.DataContext root)": File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 60, Column 64 at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:369) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.access$000(UnitCompiler.java:231) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$1.visitCompilationUnit(UnitCompiler.java:333) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$1.visitCompilationUnit(UnitCompiler.java:330) ~[janino-3.1.4.jar:na] at org.codehaus.janino.Java$CompilationUnit.accept(Java.java:367) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:330) ~[janino-3.1.4.jar:na] at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:245) ~[janino-3.1.4.jar:na] at org.codehaus.janino.ClassBodyEvaluator.cook(ClassBodyEvaluator.java:294) ~[janino-3.1.4.jar:na] at org.codehaus.janino.ClassBodyEvaluator.cook(ClassBodyEvaluator.java:288) ~[janino-3.1.4.jar:na] at org.codehaus.janino.ClassBodyEvaluator.cook(ClassBodyEvaluator.java:267) ~[janino-3.1.4.jar:na] at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:52) ~[commons-compiler-3.1.4.jar:na] at org.codehaus.janino.ClassBodyEvaluator.createInstance(ClassBodyEvaluator.java:428) ~[janino-3.1.4.jar:na] at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.getBindable(EnumerableInterpretable.java:166) ~[calcite-core-1.29.0.jar:1.29.0] at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:127) ~[calcite-core-1.29.0.jar:1.29.0] ... 21 common frames omitted Caused by: org.codehaus.commons.compiler.InternalCompilerException: File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 41, Column 45: Compiling "bind(org.apache.calcite.DataContext root)": File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 60, Column 64 at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3222) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1379) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1352) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:800) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:412) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:231) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:391) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:386) ~[janino-3.1.4.jar:na] at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java.java:1692) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:386) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:359) ~[janino-3.1.4.jar:na] ... 34 common frames omitted Caused by: java.lang.RuntimeException: File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 60, Column 64 at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2639) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.access$2700(UnitCompiler.java:231) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$6.visitLocalVariableDeclarationStatement(UnitCompiler.java:1539) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$6.visitLocalVariableDeclarationStatement(UnitCompiler.java:1523) ~[janino-3.1.4.jar:na] at org.codehaus.janino.Java$LocalVariableDeclarationStatement.accept(Java.java:3840) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1523) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1607) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3531) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3218) ~[janino-3.1.4.jar:na] ... 44 common frames omitted Caused by: org.codehaus.commons.compiler.InternalCompilerException: File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 60, Column 80: Compiling "new org.apache.calcite.adapter.enumerable.BasicAggregateLambdaFactory(new org.apache.calcite.linq4j.function.Function0() { ... }, accumulatorAdders)": File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 61, Column 5: Compiling "new org.apache.calcite.linq4j.function.Function0() { ... }": File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 62, Column 21: Compiling "apply()": File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 70, Column 17: org.codehaus.commons.compiler.InternalCompilerException: Operand stack underflow at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5731) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2622) ~[janino-3.1.4.jar:na] ... 52 common frames omitted Caused by: org.codehaus.commons.compiler.InternalCompilerException: File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 61, Column 5: Compiling "new org.apache.calcite.linq4j.function.Function0() { ... }": File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 62, Column 21: Compiling "apply()": File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 70, Column 17: org.codehaus.commons.compiler.InternalCompilerException: Operand stack underflow at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5731) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.invokeConstructor(UnitCompiler.java:8297) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5468) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.access$9700(UnitCompiler.java:231) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$15.visitNewClassInstance(UnitCompiler.java:4643) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$15.visitNewClassInstance(UnitCompiler.java:4604) ~[janino-3.1.4.jar:na] at org.codehaus.janino.Java$NewClassInstance.accept(Java.java:5560) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4604) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5729) ~[janino-3.1.4.jar:na] ... 53 common frames omitted Caused by: org.codehaus.commons.compiler.InternalCompilerException: File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 62, Column 21: Compiling "apply()": File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 70, Column 17: org.codehaus.commons.compiler.InternalCompilerException: Operand stack underflow at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3222) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1379) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1352) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:800) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:977) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:947) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.access$200(UnitCompiler.java:231) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$2.visitAnonymousClassDeclaration(UnitCompiler.java:389) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$2.visitAnonymousClassDeclaration(UnitCompiler.java:386) ~[janino-3.1.4.jar:na] at org.codehaus.janino.Java$AnonymousClassDeclaration.accept(Java.java:1375) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:386) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5571) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.access$9400(UnitCompiler.java:231) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$15.visitNewAnonymousClassInstance(UnitCompiler.java:4640) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$15.visitNewAnonymousClassInstance(UnitCompiler.java:4604) ~[janino-3.1.4.jar:na] at org.codehaus.janino.Java$NewAnonymousClassInstance.accept(Java.java:5608) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4604) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5729) ~[janino-3.1.4.jar:na] ... 61 common frames omitted Caused by: org.codehaus.commons.compiler.InternalCompilerException: File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 70, Column 17: org.codehaus.commons.compiler.InternalCompilerException: Operand stack underflow at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2393) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:231) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1530) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1523) ~[janino-3.1.4.jar:na] at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:3198) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1523) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1607) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3531) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3218) ~[janino-3.1.4.jar:na] ... 78 common frames omitted Caused by: org.codehaus.commons.compiler.InternalCompilerException: Operand stack underflow at org.codehaus.janino.StackMap.peekOperand(StackMap.java:94) ~[janino-3.1.4.jar:na] at org.codehaus.janino.CodeContext.popOperand(CodeContext.java:1349) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.putfield(UnitCompiler.java:12363) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileSet2(UnitCompiler.java:6498) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.access$13400(UnitCompiler.java:231) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$20.visitFieldAccess(UnitCompiler.java:6480) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$20.visitFieldAccess(UnitCompiler.java:6475) ~[janino-3.1.4.jar:na] at org.codehaus.janino.Java$FieldAccess.accept(Java.java:4668) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileSet(UnitCompiler.java:6475) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileSet2(UnitCompiler.java:6489) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.access$13200(UnitCompiler.java:231) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$20.visitAmbiguousName(UnitCompiler.java:6478) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$20.visitAmbiguousName(UnitCompiler.java:6475) ~[janino-3.1.4.jar:na] at org.codehaus.janino.Java$AmbiguousName.accept(Java.java:4574) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compileSet(UnitCompiler.java:6475) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3992) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.access$6000(UnitCompiler.java:231) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$12.visitAssignment(UnitCompiler.java:3944) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler$12.visitAssignment(UnitCompiler.java:3924) ~[janino-3.1.4.jar:na] at org.codehaus.janino.Java$Assignment.accept(Java.java:4835) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3924) ~[janino-3.1.4.jar:na] at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2391) ~[janino-3.1.4.jar:na] ... 86 common frames omitted With janino 3.1.4 ------------------ Original ------------------ From: "xiaobo ";<guxiaobo1...@qq.com>; Send time: Saturday, Feb 12, 2022 9:14 PM To: "dev"<dev@calcite.apache.org>; Subject: Re: calcite multi-threading problem 3.1.4 of janino and commons-compiler works now. ------------------ Original ------------------ From: "xiaobo ";<guxiaobo1...@qq.com>; Send time: Saturday, Feb 12, 2022 8:04 PM To: "dev"<dev@calcite.apache.org>; Subject: Re: calcite multi-threading problem but strange enough, the same code works in a single thread. ------------------ Original ------------------ From: "xiaobo ";<guxiaobo1...@qq.com>; Send time: Saturday, Feb 12, 2022 8:02 PM To: "dev"<dev@calcite.apache.org>; Subject: Re: calcite multi-threading problem we checked the janino release note page http://janino-compiler.github.io/janino/changelog.html Fixed issue #141: Unable to find org.codehaus.commons.compiler.properties in java 11: ICompilerFactories were loaded through the current thread's "context class loader", which is a bad choice in some environments. Thus, the methods "getDefaultCompilerFactory()", "getAllCompilerFactories()" and "getCompilerFactory(className)" were duplicated and augmented with a "classLoader" parameter. what does this mean? the getDefaultCompilerFactory method with a classloader parameter was removed intentionally? but it seems calcite still needs it , so what version of janino should we use for the latest calcite-core? ------------------ Original ------------------ From: "xiaobo ";<guxiaobo1...@qq.com>; Send time: Saturday, Feb 12, 2022 12:01 PM To: "dev"<dev@calcite.apache.org>; Subject: Re: calcite multi-threading problem In our use case, the Java class which opens calcite connection is contained in a external Java jar file, which is loaded dynamicly through classloaders other than the default systemloader like this: try { ClassLoader loader = URLClassLoader.newInstance( new URL[] { new URL("jar:file:" + path + "!/") }, getClass().getClassLoader()); Class<?> clazz = Class.forName(classPath, true, loader); we mention this because the error message shows a classloader parameter. ------------------ Original ------------------ From: "xiaobo ";<guxiaobo1...@qq.com>; Send time: Saturday, Feb 12, 2022 11:56 AM To: "dev"<dev@calcite.apache.org>; Subject: Re: calcite multi-threading problem And if we remove calcite from our project, janino is not in "resolved depencies" library list of pom.xml, So we think it not the janino conflic problem. ------------------ Original ------------------ From: "xiaobo ";<guxiaobo1...@qq.com>; Send time: Saturday, Feb 12, 2022 9:49 AM To: "dev"<dev@calcite.apache.org>; Subject: Re: calcite multi-threading problem can you share the solution for the problem, we are using the following libraries, and it seems janino 3.1.6 is only dependent by calcite-core. <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <maven.compiler.encoding>UTF-8</maven.compiler.encoding> <java.version>9</java.version> <maven.compiler.source>9</maven.compiler.source> <maven.compiler.target>9</maven.compiler.target> <log4j2.version>2.17.1</log4j2.version> <drools.version>7.64.0.Final</drools.version> </properties> <groupId>org.apache.calcite</groupId> <artifactId>calcite-core</artifactId> <version>1.29.0</version> </dependency> <dependency> <groupId>org.apache.calcite.avatica</groupId> <artifactId>avatica-core</artifactId> <version>1.20.0</version> </dependency> <dependency> <groupId>org.drools</groupId> <artifactId>drools-core</artifactId> <version>${drools.version}</version> </dependency> <dependency> <groupId>org.drools</groupId> <artifactId>drools-compiler</artifactId> <version>${drools.version}</version> </dependency> <dependency> <groupId>org.drools</groupId> <artifactId>drools-decisiontables</artifactId> <version>${drools.version}</version> </dependency> ------------------ Original ------------------ From: "xiong duan";<nobigo...@gmail.com>; Send time: Saturday, Feb 12, 2022 8:43 AM To: "dev"<dev@calcite.apache.org>; Subject: Re: calcite multi-threading problem Yes, It is a dependency issue. I have met the same problem. +1 for Dmitry. Dmitry Sysolyatin <dm.sysolya...@gmail.com> 于2022年2月11日周五 21:18写道： > Actually, I had the same problem with spark. Spark 3.2.1 uses the old > version of janino, calcite 1.29 uses the new version of janino and they are > not compatible. > If I downgrade janino version to version which spark uses I got an error > not method found exception > If I upgrade the janino version to the version which calcite uses I get the > same error but with another method. > > As a result, I built a separate calcite jar package where I shaded janino > dependency > > > > On Fri, Feb 11, 2022 at 2:39 PM stanilovsky evgeny < > estanilovs...@gridgain.com> wrote: > > > for example on a current calcite branch: > > grep janino gradle.properties > > janino.version=3.1.6 > > > > > > > yes, we use drools in the same project, and drools uses janino > > > too, is there a version list of janino which calcite supports? > > > > > > > > > > > > > > > ---Original--- > > > From: "Dmitry Sysolyatin"<dm.sysolya...@gmail.com> > > > Date: Fri, Feb 11, 2022 18:30 PM > > > To: "dev"<dev@calcite.apache.org>; > > > Subject: Re: calcite multi-threading problem > > > > > > > > > Hi! > > > > > > It looks like a dependency issue, not like a multi-threading issue. Most > > > > > > likely you have some dependency that use another version of janino but > > > calcite requires another one > > > > > > > > > On Fri, Feb 11, 2022 at 12:18 PM xiaobo <guxiaobo1...@qq.com.invalid > > > wrote: > > > > > > > Hi, > > > > > > > > > > > > > > > we open a calcite connection with ReflectiveSchema in one thread, and > > > > > > > execute sqls in other threads against the connection, but failed with > > > > this error > > > > > > > > > > > > > > > > > > > > com.google.common.util.concurrent.ExecutionError: > > > > java.lang.NoSuchMethodError: > > > > > > > > > > > org.codehaus.commons.compiler.CompilerFactoryFactory.getDefaultCompilerFactory(Ljava/lang/ClassLoader;) > > > > > > > > Lorg/codehaus/commons/compiler/ICompilerFactory; > > > > > > > > > > > > > > > do we miss anything regarding to multi-threading with calcite? > > >
