Glad to see you find the root cause.
I think we can shade the janino dependency if it influences the usage.
WDYT, godfrey?
Best,
Shengkai
Pouria Pirzadeh <pouria.pirza...@gmail.com> 于2022年5月21日周六 00:59写道：
> Thanks for help; I digged into it and the issue turned out to be the
> version of Janino:
> flink-table has pinned Janino's version to 3.011 as that is the version
> Calcite is using; However due to other dependencies in my project, at
> runtime application code had ended up using a newer version of Janino:
> 3.1.6.
> I could come up with a fix in my pom and other dependencies and let Flink
> pick its desired version. That fixed the issue.
>
> A question here is: What if an application which has a dependency on
> Flink, also has dependency on 'org.codehaus.janino' with a different
> version? Does it make sense for flink-table to shade Janino's dependency so
> that it won't get overridden by other dependencies in a larger project?
>
> Thanks,
> Pouria
>
> On Thu, May 19, 2022 at 5:31 AM Xuyang <xyzhong...@163.com> wrote:
>
>> Hi, can you provide the DDL of your source table?
>> I test your query in my idea and it works. Here is my code.
>>
>> create temporary table myTable(
>> userid int,
>> pageid int,
>> p_userid string,
>> rowtime as proctime()
>> ) with (
>> 'connector' = 'datagen'
>> );
>>
>>
>> SELECT window_start, window_end, userid, count(pageid) AS pgcnt FROM
>> TABLE(TUMBLE(TABLE myTable, DESCRIPTOR(rowtime), INTERVAL '5' SECONDS))
>> WHERE (p_userid <> 'User_6') GROUP BY window_start, window_end, userid
>>
>>
>>
>> --
>> Best！
>> Xuyang
>>
>>
>> At 2022-05-19 08:53:03, "Pouria Pirzadeh" <pouria.pirza...@gmail.com>
>> wrote:
>>
>> I am running a Flink application in Java that performs window aggregation.
>> The query runs successfully on Flink 1.14.4. However, after upgrading to
>> Flink 1.15.0 and switching the code to use Windowing TVF, it fails with a
>> runtime error as planner can not compile and instantiate window Aggs
>> Handler code.
>> It seems the generated table program code is invalid and can not be
>> compiled by janino's SimpleCompiler to get executable runtime code.
>>
>> Is there any specific change required to go from Flink 1.14.4 to 1.15.0,
>> beside upgrading maven dependencies?
>>
>> Here is the query:
>>
>> tableEnv.createTemporaryView("myTable", table);
>> String sql = " SELECT window_start, window_end, userid, count(pageid) AS
>> pgcnt FROM TABLE(TUMBLE(TABLE myTable, DESCRIPTOR(rowtime), INTERVAL '5'
>> SECONDS)) WHERE (p_userid <> 'User_6') GROUP BY window_start, window_end,
>> userid";
>> final Table result = tableEnv.sqlQuery(sql);
>> result.execute().print();
>>
>> Here is the error stack trace:
>>
>> Caused by: java.lang.RuntimeException: Could not instantiate generated
>> class 'LocalWindowAggsHandler$34'
>> at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(
>> GeneratedClass.java:74)
>> at org.apache.flink.table.runtime.operators.aggregate.window.combines.
>> LocalAggCombiner$Factory.createRecordsCombiner(LocalAggCombiner.java:127)
>> at org.apache.flink.table.runtime.operators.aggregate.window.buffers.
>> RecordsWindowBuffer$LocalFactory.create(RecordsWindowBuffer.java:204)
>> at org.apache.flink.table.runtime.operators.aggregate.window.
>> LocalSlicingWindowAggOperator.open(LocalSlicingWindowAggOperator.java:101
>> )
>> at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain
>> .initializeStateAndOpenOperators(RegularOperatorChain.java:107)
>> at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(
>> StreamTask.java:700)
>> at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1
>> .call(StreamTaskActionExecutor.java:55)
>> at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(
>> StreamTask.java:676)
>> at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask
>> .java:643)
>> at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(
>> Task.java:948)
>> at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:
>> 917)
>> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741)
>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563)
>> at java.base/java.lang.Thread.run(Thread.java:834)
>> Caused by: org.apache.flink.util.FlinkRuntimeException:
>> org.apache.flink.api.common.InvalidProgramException: Table program
>> cannot be compiled. This is a bug. Please file an issue.
>> at org.apache.flink.table.runtime.generated.CompileUtils.compile(
>> CompileUtils.java:94)
>> at org.apache.flink.table.runtime.generated.GeneratedClass.compile(
>> GeneratedClass.java:101)
>> at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(
>> GeneratedClass.java:68)
>> ... 13 more
>> Caused by:
>> org.apache.flink.shaded.guava30.com.google.common.util.concurrent.
>> UncheckedExecutionException: org.apache.flink.api.common.
>> InvalidProgramException: Table program cannot be compiled. This is a
>> bug. Please file an issue.
>> at org.apache.flink.shaded.guava30.com.google.common.cache.
>> LocalCache$Segment.get(LocalCache.java:2051)
>> at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache.get
>> (LocalCache.java:3962)
>> at org.apache.flink.shaded.guava30.com.google.common.cache.
>> LocalCache$LocalManualCache.get(LocalCache.java:4859)
>> at org.apache.flink.table.runtime.generated.CompileUtils.compile(
>> CompileUtils.java:92)
>> ... 15 more
>> Caused by: org.apache.flink.api.common.InvalidProgramException: Table
>> program cannot be compiled. This is a bug. Please file an issue.
>> at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
>> CompileUtils.java:107)
>> at org.apache.flink.table.runtime.generated.CompileUtils
>> .lambda$compile$0(CompileUtils.java:92)
>> at org.apache.flink.shaded.guava30.com.google.common.cache.
>> LocalCache$LocalManualCache$1.load(LocalCache.java:4864)
>> at org.apache.flink.shaded.guava30.com.google.common.cache.
>> LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
>> at org.apache.flink.shaded.guava30.com.google.common.cache.
>> LocalCache$Segment.loadSync(LocalCache.java:2278)
>> at org.apache.flink.shaded.guava30.com.google.common.cache.
>> LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
>> at org.apache.flink.shaded.guava30.com.google.common.cache.
>> LocalCache$Segment.get(LocalCache.java:2045)
>> ... 18 more
>> Caused by: org.codehaus.commons.compiler.InternalCompilerException:
>> Compiling "LocalWindowAggsHandler$34" in Line 2, Column 22: Line 33,
>> Column 23: Compiling "accumulate(org.apache.flink.table.data.RowData
>> accInput)": Line 87, Column 13: org.codehaus.commons.compiler.
>> InternalCompilerException: Operand stack underflow
>> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:369)
>> at org.codehaus.janino.UnitCompiler.access$000(UnitCompiler.java:231)
>> at org.codehaus.janino.UnitCompiler$1.visitCompilationUnit(UnitCompiler
>> .java:333)
>> at org.codehaus.janino.UnitCompiler$1.visitCompilationUnit(UnitCompiler
>> .java:330)
>> at org.codehaus.janino.Java$CompilationUnit.accept(Java.java:367)
>> at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:330)
>> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:245)
>> at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler
>> .java:473)
>> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:223)
>> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:209)
>> at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:82)
>> at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:77)
>> at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
>> CompileUtils.java:104)
>> ... 24 more
>> Caused by: org.codehaus.commons.compiler.InternalCompilerException: Line
>> 33, Column 23: Compiling "accumulate(org.apache.flink.table.data.RowData
>> accInput)": Line 87, Column 13: org.codehaus.commons.compiler.
>> InternalCompilerException: Operand stack underflow
>> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3222)
>> at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler
>> .java:1379)
>> at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler
>> .java:1352)
>> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:800)
>> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:412)
>> at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:231)
>> at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(
>> UnitCompiler.java:391)
>> at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(
>> UnitCompiler.java:386)
>> at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java
>> .java:1692)
>> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:386)
>> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:359)
>> ... 36 more
>> Caused by: org.codehaus.commons.compiler.InternalCompilerException: Line
>> 87, Column 13: org.codehaus.commons.compiler.InternalCompilerException:
>> Operand stack underflow
>> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2393)
>> at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:231)
>> at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(
>> UnitCompiler.java:1530)
>> at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(
>> UnitCompiler.java:1523)
>> at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:3198)
>> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1523)
>> at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:
>> 1607)
>> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3531)
>> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3218)
>> ... 46 more
>> Caused by: org.codehaus.commons.compiler.InternalCompilerException:
>> Operand stack underflow
>> at org.codehaus.janino.StackMap.peekOperand(StackMap.java:95)
>> at org.codehaus.janino.CodeContext.popOperand(CodeContext.java:1349)
>> at org.codehaus.janino.UnitCompiler.putfield(UnitCompiler.java:12363)
>> at org.codehaus.janino.UnitCompiler.compileSet2(UnitCompiler.java:6498)
>> at org.codehaus.janino.UnitCompiler.access$13400(UnitCompiler.java:231)
>> at org.codehaus.janino.UnitCompiler$20.visitFieldAccess(UnitCompiler
>> .java:6480)
>> at org.codehaus.janino.UnitCompiler$20.visitFieldAccess(UnitCompiler
>> .java:6475)
>> at org.codehaus.janino.Java$FieldAccess.accept(Java.java:4668)
>> at org.codehaus.janino.UnitCompiler.compileSet(UnitCompiler.java:6475)
>> at org.codehaus.janino.UnitCompiler.compileSet2(UnitCompiler.java:6489)
>> at org.codehaus.janino.UnitCompiler.access$13200(UnitCompiler.java:231)
>> at org.codehaus.janino.UnitCompiler$20.visitAmbiguousName(UnitCompiler
>> .java:6478)
>> at org.codehaus.janino.UnitCompiler$20.visitAmbiguousName(UnitCompiler
>> .java:6475)
>> at org.codehaus.janino.Java$AmbiguousName.accept(Java.java:4574)
>> at org.codehaus.janino.UnitCompiler.compileSet(UnitCompiler.java:6475)
>> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3992)
>> at org.codehaus.janino.UnitCompiler.access$6000(UnitCompiler.java:231)
>> at org.codehaus.janino.UnitCompiler$12.visitAssignment(UnitCompiler.java:
>> 3944)
>> at org.codehaus.janino.UnitCompiler$12.visitAssignment(UnitCompiler.java:
>> 3924)
>> at org.codehaus.janino.Java$Assignment.accept(Java.java:4835)
>> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3924)
>> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2391)
>> ... 54 more
>>
>>
