Viet Pham created FLINK-16589:
---------------------------------
Summary: Flink Table SQL fails/crashes with big queries with lots
of fields
Key: FLINK-16589
URL: https://issues.apache.org/jira/browse/FLINK-16589
Project: Flink
Issue Type: Bug
Components: Table SQL / API
Affects Versions: 1.10.0
Reporter: Viet Pham
Hi,
My use case is a streaming application with a few streaming tables.
I was trying to build a SELECT query (and registering it as a temporary view)
with about 200 fields/expressions out of another streaming table. The
application is successfully submitted to Flink cluster. However the worker
processes keep crashing, with the exception as quoted below.
It clearly mentioned in the log that this is a bug, so I fire this ticket. By
the way, if I lower the number of fields down to 100 then it works nicely.
Please advice.
Thanks a lot for all the efforts bring Flink up. It is really amazing!
{code:java}
java.lang.RuntimeException: Could not instantiate generated class
'GroupAggsHandler$9687' at
org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:57)
at
org.apache.flink.table.runtime.operators.aggregate.MiniBatchGroupAggFunction.open(MiniBatchGroupAggFunction.java:136)
at
org.apache.flink.table.runtime.operators.bundle.AbstractMapBundleOperator.open(AbstractMapBundleOperator.java:84)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.initializeStateAndOpen(StreamTask.java:1007)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:454)
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:94)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:449)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:461)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:707) at
org.apache.flink.runtime.taskmanager.Task.run(Task.java:532) at
java.lang.Thread.run(Thread.java:748)Caused by:
org.apache.flink.util.FlinkRuntimeException:
org.apache.flink.api.common.InvalidProgramException: Table program cannot be
compiled. This is a bug. Please file an issue. at
org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:68)
at
org.apache.flink.table.runtime.generated.GeneratedClass.compile(GeneratedClass.java:78)
at
org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:52)
... 10 moreCaused by:
org.apache.flink.shaded.guava18.com.google.common.util.concurrent.UncheckedExecutionException:
org.apache.flink.api.common.InvalidProgramException: Table program cannot be
compiled. This is a bug. Please file an issue. at
org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2203)
at
org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache.get(LocalCache.java:3937)
at
org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4739)
at
org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:66)
... 12 moreCaused by: org.apache.flink.api.common.InvalidProgramException:
Table program cannot be compiled. This is a bug. Please file an issue. at
org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:81)
at
org.apache.flink.table.runtime.generated.CompileUtils.lambda$compile$1(CompileUtils.java:66)
at
org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4742)
at
org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
at
org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
at
org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
at
org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
... 15 moreCaused by: org.codehaus.janino.InternalCompilerException:
Compiling "GroupAggsHandler$9687": Code of method
"retract(Lorg/apache/flink/table/dataformat/BaseRow;)V" of class
"GroupAggsHandler$9687" grows beyond 64 KB at
org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:382) at
org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237) at
org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:465)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:216) at
org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207) at
org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80) at
org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75) at
org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:78)
... 21 moreCaused by: org.codehaus.janino.InternalCompilerException: Code
of method "retract(Lorg/apache/flink/table/dataformat/BaseRow;)V" of class
"GroupAggsHandler$9687" grows beyond 64 KB at
org.codehaus.janino.CodeContext.makeSpace(CodeContext.java:1009) at
org.codehaus.janino.CodeContext.write(CodeContext.java:901) at
org.codehaus.janino.CodeContext.writeShort(CodeContext.java:1026) at
org.codehaus.janino.UnitCompiler.writeConstantLongInfo(UnitCompiler.java:12274)
at org.codehaus.janino.UnitCompiler.pushConstant(UnitCompiler.java:10679)
at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4936) at
org.codehaus.janino.UnitCompiler.access$8400(UnitCompiler.java:215) at
org.codehaus.janino.UnitCompiler$16.visitUnaryOperation(UnitCompiler.java:4414)
at
org.codehaus.janino.UnitCompiler$16.visitUnaryOperation(UnitCompiler.java:4394)
at org.codehaus.janino.Java$UnaryOperation.accept(Java.java:4719) at
org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4394) at
org.codehaus.janino.UnitCompiler.fakeCompile(UnitCompiler.java:3719) at
org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5569) at
org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2580) at
org.codehaus.janino.UnitCompiler.access$2700(UnitCompiler.java:215) at
org.codehaus.janino.UnitCompiler$6.visitLocalVariableDeclarationStatement(UnitCompiler.java:1503)
at
org.codehaus.janino.UnitCompiler$6.visitLocalVariableDeclarationStatement(UnitCompiler.java:1487)
at
org.codehaus.janino.Java$LocalVariableDeclarationStatement.accept(Java.java:3511)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1487) at
org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1567)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3388) at
org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1357)
at
org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1330)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:822) at
org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:432) at
org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:215) at
org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:411)
at
org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:406)
at
org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java.java:1414)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:406) at
org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:378) ... 28
more
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
