I am running a Flink application in Java that performs window aggregation.
The query runs successfully on Flink 1.14.4. However, after upgrading to
Flink 1.15.0 and switching the code to use Windowing TVF, it fails with a
runtime error as planner can not compile and instantiate window Aggs
Handler code.
It seems the generated table program code is invalid and can not be
compiled by janino's SimpleCompiler to get executable runtime code.
Is there any specific change required to go from Flink 1.14.4 to 1.15.0,
beside upgrading maven dependencies?
Here is the query:
tableEnv.createTemporaryView("myTable", table);
String sql = " SELECT window_start, window_end, userid, count(pageid) AS
pgcnt FROM TABLE(TUMBLE(TABLE myTable, DESCRIPTOR(rowtime), INTERVAL '5'
SECONDS)) WHERE (p_userid <> 'User_6') GROUP BY window_start, window_end,
userid";
final Table result = tableEnv.sqlQuery(sql);
result.execute().print();
Here is the error stack trace:
Caused by: java.lang.RuntimeException: Could not instantiate generated class
'LocalWindowAggsHandler$34'
at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(
GeneratedClass.java:74)
at org.apache.flink.table.runtime.operators.aggregate.window.combines.
LocalAggCombiner$Factory.createRecordsCombiner(LocalAggCombiner.java:127)
at org.apache.flink.table.runtime.operators.aggregate.window.buffers.
RecordsWindowBuffer$LocalFactory.create(RecordsWindowBuffer.java:204)
at org.apache.flink.table.runtime.operators.aggregate.window.
LocalSlicingWindowAggOperator.open(LocalSlicingWindowAggOperator.java:101)
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain
.initializeStateAndOpenOperators(RegularOperatorChain.java:107)
at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(
StreamTask.java:700)
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(
StreamTaskActionExecutor.java:55)
at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(
StreamTask.java:676)
at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask
.java:643)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(
Task.java:948)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:917)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.flink.util.FlinkRuntimeException:
org.apache.flink.api.common.InvalidProgramException: Table program cannot
be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.compile(
CompileUtils.java:94)
at org.apache.flink.table.runtime.generated.GeneratedClass.compile(
GeneratedClass.java:101)
at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(
GeneratedClass.java:68)
... 13 more
Caused by:
org.apache.flink.shaded.guava30.com.google.common.util.concurrent.
UncheckedExecutionException: org.apache.flink.api.common.
InvalidProgramException: Table program cannot be compiled. This is a bug.
Please file an issue.
at org.apache.flink.shaded.guava30.com.google.common.cache.
LocalCache$Segment.get(LocalCache.java:2051)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache.get(
LocalCache.java:3962)
at org.apache.flink.shaded.guava30.com.google.common.cache.
LocalCache$LocalManualCache.get(LocalCache.java:4859)
at org.apache.flink.table.runtime.generated.CompileUtils.compile(
CompileUtils.java:92)
... 15 more
Caused by: org.apache.flink.api.common.InvalidProgramException: Table
program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
CompileUtils.java:107)
at org.apache.flink.table.runtime.generated.CompileUtils.lambda$compile$0(
CompileUtils.java:92)
at org.apache.flink.shaded.guava30.com.google.common.cache.
LocalCache$LocalManualCache$1.load(LocalCache.java:4864)
at org.apache.flink.shaded.guava30.com.google.common.cache.
LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
at org.apache.flink.shaded.guava30.com.google.common.cache.
LocalCache$Segment.loadSync(LocalCache.java:2278)
at org.apache.flink.shaded.guava30.com.google.common.cache.
LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
at org.apache.flink.shaded.guava30.com.google.common.cache.
LocalCache$Segment.get(LocalCache.java:2045)
... 18 more
Caused by: org.codehaus.commons.compiler.InternalCompilerException:
Compiling "LocalWindowAggsHandler$34" in Line 2, Column 22: Line 33, Column
23: Compiling "accumulate(org.apache.flink.table.data.RowData accInput)":
Line 87, Column 13: org.codehaus.commons.compiler.InternalCompilerException:
Operand stack underflow
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:369)
at org.codehaus.janino.UnitCompiler.access$000(UnitCompiler.java:231)
at org.codehaus.janino.UnitCompiler$1.visitCompilationUnit(UnitCompiler
.java:333)
at org.codehaus.janino.UnitCompiler$1.visitCompilationUnit(UnitCompiler
.java:330)
at org.codehaus.janino.Java$CompilationUnit.accept(Java.java:367)
at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:330)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:245)
at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler
.java:473)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:223)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:209)
at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:82)
at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:77)
at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
CompileUtils.java:104)
... 24 more
Caused by: org.codehaus.commons.compiler.InternalCompilerException: Line 33,
Column 23: Compiling "accumulate(org.apache.flink.table.data.RowData
accInput)": Line 87, Column 13: org.codehaus.commons.compiler.
InternalCompilerException: Operand stack underflow
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3222)
at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler
.java:1379)
at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler
.java:1352)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:800)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:412)
at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:231)
at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(
UnitCompiler.java:391)
at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(
UnitCompiler.java:386)
at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java.java:
1692)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:386)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:359)
... 36 more
Caused by: org.codehaus.commons.compiler.InternalCompilerException: Line 87,
Column 13: org.codehaus.commons.compiler.InternalCompilerException: Operand
stack underflow
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2393)
at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:231)
at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler
.java:1530)
at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler
.java:1523)
at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:3198)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1523)
at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1607
)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3531)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3218)
... 46 more
Caused by: org.codehaus.commons.compiler.InternalCompilerException: Operand
stack underflow
at org.codehaus.janino.StackMap.peekOperand(StackMap.java:95)
at org.codehaus.janino.CodeContext.popOperand(CodeContext.java:1349)
at org.codehaus.janino.UnitCompiler.putfield(UnitCompiler.java:12363)
at org.codehaus.janino.UnitCompiler.compileSet2(UnitCompiler.java:6498)
at org.codehaus.janino.UnitCompiler.access$13400(UnitCompiler.java:231)
at org.codehaus.janino.UnitCompiler$20.visitFieldAccess(UnitCompiler.java:
6480)
at org.codehaus.janino.UnitCompiler$20.visitFieldAccess(UnitCompiler.java:
6475)
at org.codehaus.janino.Java$FieldAccess.accept(Java.java:4668)
at org.codehaus.janino.UnitCompiler.compileSet(UnitCompiler.java:6475)
at org.codehaus.janino.UnitCompiler.compileSet2(UnitCompiler.java:6489)
at org.codehaus.janino.UnitCompiler.access$13200(UnitCompiler.java:231)
at org.codehaus.janino.UnitCompiler$20.visitAmbiguousName(UnitCompiler.java:
6478)
at org.codehaus.janino.UnitCompiler$20.visitAmbiguousName(UnitCompiler.java:
6475)
at org.codehaus.janino.Java$AmbiguousName.accept(Java.java:4574)
at org.codehaus.janino.UnitCompiler.compileSet(UnitCompiler.java:6475)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3992)
at org.codehaus.janino.UnitCompiler.access$6000(UnitCompiler.java:231)
at org.codehaus.janino.UnitCompiler$12.visitAssignment(UnitCompiler.java:
3944)
at org.codehaus.janino.UnitCompiler$12.visitAssignment(UnitCompiler.java:
3924)
at org.codehaus.janino.Java$Assignment.accept(Java.java:4835)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3924)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2391)
... 54 more
