Thanks for help; I digged into it and the issue turned out to be the
version of Janino:
flink-table has pinned Janino's version to 3.011 as that is the version
Calcite is using; However due to other dependencies in my project, at
runtime application code had ended up using a newer version of Janino:
3.1.6.
I could come up with a fix in my pom and other dependencies and let Flink
pick its desired version. That fixed the issue.
A question here is: What if an application which has a dependency on Flink,
also has dependency on 'org.codehaus.janino' with a different version? Does
it make sense for flink-table to shade Janino's dependency so that it
won't get overridden by other dependencies in a larger project?
Thanks,
Pouria
On Thu, May 19, 2022 at 5:31 AM Xuyang <xyzhong...@163.com> wrote:
> Hi, can you provide the DDL of your source table?
> I test your query in my idea and it works. Here is my code.
>
> create temporary table myTable(
> userid int,
> pageid int,
> p_userid string,
> rowtime as proctime()
> ) with (
> 'connector' = 'datagen'
> );
>
>
> SELECT window_start, window_end, userid, count(pageid) AS pgcnt FROM TABLE
> (TUMBLE(TABLE myTable, DESCRIPTOR(rowtime), INTERVAL '5' SECONDS)) WHERE
> (p_userid <> 'User_6') GROUP BY window_start, window_end, userid
>
>
>
> --
> Best！
> Xuyang
>
>
> At 2022-05-19 08:53:03, "Pouria Pirzadeh" <pouria.pirza...@gmail.com>
> wrote:
>
> I am running a Flink application in Java that performs window aggregation.
> The query runs successfully on Flink 1.14.4. However, after upgrading to
> Flink 1.15.0 and switching the code to use Windowing TVF, it fails with a
> runtime error as planner can not compile and instantiate window Aggs
> Handler code.
> It seems the generated table program code is invalid and can not be
> compiled by janino's SimpleCompiler to get executable runtime code.
>
> Is there any specific change required to go from Flink 1.14.4 to 1.15.0,
> beside upgrading maven dependencies?
>
> Here is the query:
>
> tableEnv.createTemporaryView("myTable", table);
> String sql = " SELECT window_start, window_end, userid, count(pageid) AS
> pgcnt FROM TABLE(TUMBLE(TABLE myTable, DESCRIPTOR(rowtime), INTERVAL '5'
> SECONDS)) WHERE (p_userid <> 'User_6') GROUP BY window_start, window_end,
> userid";
> final Table result = tableEnv.sqlQuery(sql);
> result.execute().print();
>
> Here is the error stack trace:
>
> Caused by: java.lang.RuntimeException: Could not instantiate generated
> class 'LocalWindowAggsHandler$34'
> at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(
> GeneratedClass.java:74)
> at org.apache.flink.table.runtime.operators.aggregate.window.combines.
> LocalAggCombiner$Factory.createRecordsCombiner(LocalAggCombiner.java:127)
> at org.apache.flink.table.runtime.operators.aggregate.window.buffers.
> RecordsWindowBuffer$LocalFactory.create(RecordsWindowBuffer.java:204)
> at org.apache.flink.table.runtime.operators.aggregate.window.
> LocalSlicingWindowAggOperator.open(LocalSlicingWindowAggOperator.java:101)
> at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain
> .initializeStateAndOpenOperators(RegularOperatorChain.java:107)
> at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(
> StreamTask.java:700)
> at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1
> .call(StreamTaskActionExecutor.java:55)
> at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(
> StreamTask.java:676)
> at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask
> .java:643)
> at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(
> Task.java:948)
> at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:
> 917)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: org.apache.flink.util.FlinkRuntimeException:
> org.apache.flink.api.common.InvalidProgramException: Table program cannot
> be compiled. This is a bug. Please file an issue.
> at org.apache.flink.table.runtime.generated.CompileUtils.compile(
> CompileUtils.java:94)
> at org.apache.flink.table.runtime.generated.GeneratedClass.compile(
> GeneratedClass.java:101)
> at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(
> GeneratedClass.java:68)
> ... 13 more
> Caused by:
> org.apache.flink.shaded.guava30.com.google.common.util.concurrent.
> UncheckedExecutionException: org.apache.flink.api.common.
> InvalidProgramException: Table program cannot be compiled. This is a bug.
> Please file an issue.
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.get(LocalCache.java:2051)
> at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache.get(
> LocalCache.java:3962)
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$LocalManualCache.get(LocalCache.java:4859)
> at org.apache.flink.table.runtime.generated.CompileUtils.compile(
> CompileUtils.java:92)
> ... 15 more
> Caused by: org.apache.flink.api.common.InvalidProgramException: Table
> program cannot be compiled. This is a bug. Please file an issue.
> at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
> CompileUtils.java:107)
> at org.apache.flink.table.runtime.generated.CompileUtils.lambda$compile$0(
> CompileUtils.java:92)
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$LocalManualCache$1.load(LocalCache.java:4864)
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.loadSync(LocalCache.java:2278)
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.get(LocalCache.java:2045)
> ... 18 more
> Caused by: org.codehaus.commons.compiler.InternalCompilerException:
> Compiling "LocalWindowAggsHandler$34" in Line 2, Column 22: Line 33,
> Column 23: Compiling "accumulate(org.apache.flink.table.data.RowData
> accInput)": Line 87, Column 13: org.codehaus.commons.compiler.
> InternalCompilerException: Operand stack underflow
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:369)
> at org.codehaus.janino.UnitCompiler.access$000(UnitCompiler.java:231)
> at org.codehaus.janino.UnitCompiler$1.visitCompilationUnit(UnitCompiler
> .java:333)
> at org.codehaus.janino.UnitCompiler$1.visitCompilationUnit(UnitCompiler
> .java:330)
> at org.codehaus.janino.Java$CompilationUnit.accept(Java.java:367)
> at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:330)
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:245)
> at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler
> .java:473)
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:223)
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:209)
> at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:82)
> at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:77)
> at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
> CompileUtils.java:104)
> ... 24 more
> Caused by: org.codehaus.commons.compiler.InternalCompilerException: Line
> 33, Column 23: Compiling "accumulate(org.apache.flink.table.data.RowData
> accInput)": Line 87, Column 13: org.codehaus.commons.compiler.
> InternalCompilerException: Operand stack underflow
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3222)
> at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler
> .java:1379)
> at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler
> .java:1352)
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:800)
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:412)
> at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:231)
> at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(
> UnitCompiler.java:391)
> at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(
> UnitCompiler.java:386)
> at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java
> .java:1692)
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:386)
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:359)
> ... 36 more
> Caused by: org.codehaus.commons.compiler.InternalCompilerException: Line
> 87, Column 13: org.codehaus.commons.compiler.InternalCompilerException:
> Operand stack underflow
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2393)
> at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:231)
> at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(
> UnitCompiler.java:1530)
> at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(
> UnitCompiler.java:1523)
> at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:3198)
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1523)
> at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:
> 1607)
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3531)
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3218)
> ... 46 more
> Caused by: org.codehaus.commons.compiler.InternalCompilerException:
> Operand stack underflow
> at org.codehaus.janino.StackMap.peekOperand(StackMap.java:95)
> at org.codehaus.janino.CodeContext.popOperand(CodeContext.java:1349)
> at org.codehaus.janino.UnitCompiler.putfield(UnitCompiler.java:12363)
> at org.codehaus.janino.UnitCompiler.compileSet2(UnitCompiler.java:6498)
> at org.codehaus.janino.UnitCompiler.access$13400(UnitCompiler.java:231)
> at org.codehaus.janino.UnitCompiler$20.visitFieldAccess(UnitCompiler.java:
> 6480)
> at org.codehaus.janino.UnitCompiler$20.visitFieldAccess(UnitCompiler.java:
> 6475)
> at org.codehaus.janino.Java$FieldAccess.accept(Java.java:4668)
> at org.codehaus.janino.UnitCompiler.compileSet(UnitCompiler.java:6475)
> at org.codehaus.janino.UnitCompiler.compileSet2(UnitCompiler.java:6489)
> at org.codehaus.janino.UnitCompiler.access$13200(UnitCompiler.java:231)
> at org.codehaus.janino.UnitCompiler$20.visitAmbiguousName(UnitCompiler
> .java:6478)
> at org.codehaus.janino.UnitCompiler$20.visitAmbiguousName(UnitCompiler
> .java:6475)
> at org.codehaus.janino.Java$AmbiguousName.accept(Java.java:4574)
> at org.codehaus.janino.UnitCompiler.compileSet(UnitCompiler.java:6475)
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3992)
> at org.codehaus.janino.UnitCompiler.access$6000(UnitCompiler.java:231)
> at org.codehaus.janino.UnitCompiler$12.visitAssignment(UnitCompiler.java:
> 3944)
> at org.codehaus.janino.UnitCompiler$12.visitAssignment(UnitCompiler.java:
> 3924)
> at org.codehaus.janino.Java$Assignment.accept(Java.java:4835)
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3924)
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2391)
> ... 54 more
>
>
