Hi. I open a ticket about upgrading the version[1]. Maybe it is worth a try.
Best, Shengkai [1] https://issues.apache.org/jira/browse/FLINK-27995 Benenson, Michael <mikhail_benen...@intuit.com> 于2022年6月10日周五 04:51写道： > Hi, David > > > > Hard to tell for sure, but yes, [1] could also indicate some problems with > Janio. > > Both Flink 1.14.3 & 1.15.0 use <janino.version>3.0.11</janino.version> > > The difference is that Flink 1.14.3 has been compiled & run on jvm-8, but > Flink 1.15.0 has been compiled & run on jvm-11 > > > > In the revision history for Janio at [2] one comment for release 3.0.11 is > > - At last, the "jdk-9" implementation works. > > Probably, we could imply that jdk-11 does not work in Janio 3.0.11, so > switch to the latest version of Janio 3.1.7 > <https://mvnrepository.com/artifact/org.codehaus.janino/janino/3.1.7> could > help. > > > > Do you know, how to make nightly flink build with janio 3.1.7 > <https://mvnrepository.com/artifact/org.codehaus.janino/janino/3.1.7>? > > > > [1] https://lists.apache.org/thread/9tw165cgpdqz4ron76b1ckmwm9hy4qfd > > [2] https://janino-compiler.github.io/janino/changelog.html > > > > > > > > *From: *David Anderson <dander...@apache.org> > *Date: *Thursday, June 9, 2022 at 12:05 PM > *To: *Benenson, Michael <mikhail_benen...@intuit.com> > *Cc: *user@flink.apache.org <user@flink.apache.org>, Deshpande, Omkar < > omkar_deshpa...@intuit.com>, Waghulde, Suraj <suraj_waghu...@intuit.com> > *Subject: *Re: Fink 15: InvalidProgramException: Table program cannot be > compiled. This is a bug > > This email is from an external sender. > > > > Sorry; I guess I jumped to conclusions and got it wrong. Let's dig in > deeper. > > > > This sounds similar to what was reported in this thread [1] where someone > else ran into problems with Janino after upgrading from 1.14 to 1.15. Could > this be another instance of the same issue (i.e., Flink ends up using the > wrong version of Janino)? > > > > [1] https://lists.apache.org/thread/9tw165cgpdqz4ron76b1ckmwm9hy4qfd > > > > On Thu, Jun 9, 2022 at 8:18 PM Benenson, Michael < > mikhail_benen...@intuit.com> wrote: > > Hi, David > > > > I have tried CREATE TABLE … without proc_time, but it does not help, I see > the same exception. > > > And this is a new issue for Flink 1.15.0, for Flink 1.14.3 it works fine, > even with both event_time & proc_time in CREATE TABLE statement. > > Now I use > > CREATE OR REPLACE TABLE input ( > event_header ROW(topic_name STRING), > `timestamp` STRING NOT NULL, > event_time AS TO_TIMESTAMP(fix_instant(`timestamp`), > 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''), > properties ROW(company_id STRING NOT NULL, scope_area STRING, action STRING) > NOT NULL, > WATERMARK FOR event_time AS event_time - INTERVAL '5' SECOND > ) WITH ( > 'connector' = 'kafka', > 'topic' = 'mb-1644796800-qbo', > 'properties.bootstrap.servers' = 'localhost:9092', > 'format' = 'json', > 'scan.startup.mode' = 'latest-offset', > 'json.ignore-parse-errors' = 'true', > 'json.fail-on-missing-field' = 'false' > ); > > And got the same exception > > > java.lang.RuntimeException: Could not instantiate generated class > 'WatermarkGenerator$0' > > at > org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:74) > ~[flink-table-runtime-1.15.0.jar:1.15.0] > … > Caused by: org.apache.flink.util.FlinkRuntimeException: > org.apache.flink.api.common.InvalidProgramException: Table program cannot > be compiled. This is a bug. Please file an issue. > > at > org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:94) > ~[flink-table-runtime-1.15.0.jar:1.15.0] > > … > > Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column > 75: Cannot determine simple type name "org" > > at > org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211) > ~[flink-table-runtime-1.15.0.jar:1.15.0] > > > > *From: *David Anderson <dander...@apache.org> > *Date: *Thursday, June 9, 2022 at 5:34 AM > *To: *Benenson, Michael <mikhail_benen...@intuit.com> > *Cc: *user@flink.apache.org <user@flink.apache.org>, Deshpande, Omkar < > omkar_deshpa...@intuit.com>, Waghulde, Suraj <suraj_waghu...@intuit.com> > *Subject: *Re: Fink 15: InvalidProgramException: Table program cannot be > compiled. This is a bug > > This email is from an external sender. > > > > A Table can have at most one time attribute. In your Table the proc_time > column is a processing time attribute, and when you define a watermark on > the event_time column then that column becomes an event-time attribute. > > > > If you want to combine event time and processing time, you can use > the PROCTIME() function in your queries without having a processing time > attribute as one of the columns in the table. > > > > Best, > > David > > > > On Wed, Jun 8, 2022 at 9:46 PM Benenson, Michael < > mikhail_benen...@intuit.com> wrote: > > Hi, folks > > > > *Short description*: > > > > I use Flink 1.15.0 sql-client and Java User Define Function in CREATE > TABLE … statement to get Timestamp. > > It works OK, if I do not use Timestamp in Watermark, but if used in > Watermark it causes > > java.lang.RuntimeException: Could not instantiate generated class > 'WatermarkGenerator$0' > > … > > Caused by: org.apache.flink.api.common.InvalidProgramException: Table > program cannot be compiled. This is a bug. Please file an issue. > > at org.apache.flink.table.runtime.generated.CompileUtils.doCompile( > CompileUtils.java:107) > > … > > Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column > 75: Cannot determine simple type name "org" > > at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java: > 12211) > > > > > > *Details*: > > java -version > > openjdk version "11.0.14.1" 2022-02-08 LTS > > OpenJDK Runtime Environment Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS) > > OpenJDK 64-Bit Server VM Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS, > mixed mode) > > > > Flink 1.15.0, flink-1.15.0/bin/sql-client.sh > > > > SET 'sql-client.execution.result-mode' = 'tableau'; > > SET 'table.exec.sink.not-null-enforcer' = 'drop'; > > > > CREATE TEMPORARY FUNCTION default_catalog.default_database.fix_instant > > AS 'com.intuit.data.strmprocess.udf.FixInstant' LANGUAGE JAVA; > > > > CREATE OR REPLACE TABLE input ( > > event_header ROW(topic_name STRING), > > `timestamp` STRING NOT NULL, > > event_time AS TO_TIMESTAMP(fix_instant(`timestamp`), > 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''), > > properties ROW(company_id STRING NOT NULL, scope_area STRING, > action STRING) NOT NULL, > > WATERMARK FOR event_time AS event_time - INTERVAL '5' SECOND, > > proc_time AS PROCTIME() > > ) WITH ( > > 'connector' = 'kafka', > > 'topic' = 'mb-1644796800-qbo', > > 'properties.bootstrap.servers' = 'localhost:9092', > > 'format' = 'json', > > 'scan.startup.mode' = 'latest-offset', > > 'json.ignore-parse-errors' = 'true', > > 'json.fail-on-missing-field' = 'false' > > ); > > > > SELECT `timestamp`, event_time, event_header.topic_name AS topic, > properties.company_id as company FROM input > > LIMIT 10 > > ; > > > > Works fine, if I comment WATERMARK FOR event_time … > Causes an error, if WATERMARK FOR event_time is used: > > 2022-06-08 12:16:32 > > java.lang.RuntimeException: Could not instantiate generated class > 'WatermarkGenerator$0' > > at org.apache.flink.table.runtime.generated.GeneratedClass > .newInstance(GeneratedClass.java:74) > > at org.apache.flink.table.runtime.generated. > GeneratedWatermarkGeneratorSupplier.createWatermarkGenerator( > GeneratedWatermarkGeneratorSupplier.java:62) > > at org.apache.flink.streaming.api.operators.source. > ProgressiveTimestampsAndWatermarks.createMainOutput( > ProgressiveTimestampsAndWatermarks.java:104) > > at org.apache.flink.streaming.api.operators.SourceOperator > .initializeMainOutput(SourceOperator.java:426) > > at org.apache.flink.streaming.api.operators.SourceOperator > .emitNextNotReading(SourceOperator.java:402) > > at org.apache.flink.streaming.api.operators.SourceOperator.emitNext( > SourceOperator.java:387) > > at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput > .emitNext(StreamTaskSourceInput.java:68) > > at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor > .processInput(StreamOneInputProcessor.java:65) > > at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput( > StreamTask.java:519) > > at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor > .runMailboxLoop(MailboxProcessor.java:203) > > at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop( > StreamTask.java:804) > > at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke( > StreamTask.java:753) > > at org.apache.flink.runtime.taskmanager.Task > .runWithSystemExitMonitoring(Task.java:948) > > at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task > .java:927) > > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741) > > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) > > at java.base/java.lang.Thread.run(Thread.java:829) > > Caused by: org.apache.flink.util.FlinkRuntimeException: > org.apache.flink.api.common.InvalidProgramException: Table program cannot > be compiled. This is a bug. Please file an issue. > > at org.apache.flink.table.runtime.generated.CompileUtils.compile( > CompileUtils.java:94) > > at org.apache.flink.table.runtime.generated.GeneratedClass.compile( > GeneratedClass.java:101) > > at org.apache.flink.table.runtime.generated.GeneratedClass > .newInstance(GeneratedClass.java:68) > > ... 16 more > > Caused by: > org.apache.flink.shaded.guava30.com.google.common.util.concurrent. > UncheckedExecutionException: org.apache.flink.api.common. > InvalidProgramException: Table program cannot be compiled. This is a bug. > Please file an issue. > > at org.apache.flink.shaded.guava30.com.google.common.cache. > LocalCache$Segment.get(LocalCache.java:2051) > > at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache. > get(LocalCache.java:3962) > > at org.apache.flink.shaded.guava30.com.google.common.cache. > LocalCache$LocalManualCache.get(LocalCache.java:4859) > > at org.apache.flink.table.runtime.generated.CompileUtils.compile( > CompileUtils.java:92) > > ... 18 more > > Caused by: org.apache.flink.api.common.InvalidProgramException: Table > program cannot be compiled. This is a bug. Please file an issue. > > at org.apache.flink.table.runtime.generated.CompileUtils.doCompile( > CompileUtils.java:107) > > at org.apache.flink.table.runtime.generated.CompileUtils > .lambda$compile$0(CompileUtils.java:92) > > at org.apache.flink.shaded.guava30.com.google.common.cache. > LocalCache$LocalManualCache$1.load(LocalCache.java:4864) > > at org.apache.flink.shaded.guava30.com.google.common.cache. > LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) > > at org.apache.flink.shaded.guava30.com.google.common.cache. > LocalCache$Segment.loadSync(LocalCache.java:2278) > > at org.apache.flink.shaded.guava30.com.google.common.cache. > LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) > > at org.apache.flink.shaded.guava30.com.google.common.cache. > LocalCache$Segment.get(LocalCache.java:2045) > > ... 21 more > > Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column > 75: Cannot determine simple type name "org" > > at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java: > 12211) > > at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler > .java:6833) > > at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler > .java:6594) > > at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler > .java:6607) > > at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler > .java:6607) > > at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler > .java:6607) > > at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler > .java:6607) > > at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler > .java:6607) > > at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler > .java:6607) > > at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6573) > > at org.codehaus.janino.UnitCompiler.access$13900(UnitCompiler.java:215 > ) > > at org.codehaus.janino.UnitCompiler$22$1.visitReferenceType( > UnitCompiler.java:6481) > > at org.codehaus.janino.UnitCompiler$22$1.visitReferenceType( > UnitCompiler.java:6476) > > at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3928) > > at org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java: > 6476) > > at org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java: > 6469) > > at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3927) > > at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469) > > at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:7121) > > at org.codehaus.janino.UnitCompiler.access$17000(UnitCompiler.java:215 > ) > > at org.codehaus.janino.UnitCompiler$22$2.visitNewClassInstance( > UnitCompiler.java:6529) > > at org.codehaus.janino.UnitCompiler$22$2.visitNewClassInstance( > UnitCompiler.java:6490) > > at org.codehaus.janino.Java$NewClassInstance.accept(Java.java:5190) > > at org.codehaus.janino.UnitCompiler$22.visitRvalue(UnitCompiler.java: > 6490) > > at org.codehaus.janino.UnitCompiler$22.visitRvalue(UnitCompiler.java: > 6469) > > at org.codehaus.janino.Java$Rvalue.accept(Java.java:4116) > > at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469) > > at org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable( > UnitCompiler.java:9237) > > at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9123 > ) > > at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9025 > ) > > at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5062 > ) > > at org.codehaus.janino.UnitCompiler.access$9100(UnitCompiler.java:215) > > at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation( > UnitCompiler.java:4423) > > at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation( > UnitCompiler.java:4396) > > at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073) > > at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396) > > at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java: > 5662) > > at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3783) > > at org.codehaus.janino.UnitCompiler.access$5900(UnitCompiler.java:215) > > at org.codehaus.janino.UnitCompiler$13.visitMethodInvocation( > UnitCompiler.java:3762) > > at org.codehaus.janino.UnitCompiler$13.visitMethodInvocation( > UnitCompiler.java:3734) > > at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073) > > at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3734) > > at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2360) > > at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:215) > > at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement( > UnitCompiler.java:1494) > > at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement( > UnitCompiler.java:1487) > > at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:2874) > > at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1487) > > at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler > .java:1567) > > at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3388) > > at org.codehaus.janino.UnitCompiler.compileDeclaredMethods( > UnitCompiler.java:1357) > > at org.codehaus.janino.UnitCompiler.compileDeclaredMethods( > UnitCompiler.java:1330) > > at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:822) > > at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:432) > > at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:215) > > at org.codehaus.janino.UnitCompiler$2 > .visitPackageMemberClassDeclaration(UnitCompiler.java:411) > > at org.codehaus.janino.UnitCompiler$2 > .visitPackageMemberClassDeclaration(UnitCompiler.java:406) > > at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java > .java:1414) > > at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:406) > > at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:378) > > at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237) > > at org.codehaus.janino.SimpleCompiler.compileToClassLoader( > SimpleCompiler.java:465) > > at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:216) > > at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207) > > at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80) > > at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75) > > at org.apache.flink.table.runtime.generated.CompileUtils.doCompile( > CompileUtils.java:104) > > ... 27 more > > > > > > Java code: > > > > package com.intuit.data.strmprocess.udf; > > import org.apache.flink.table.functions.ScalarFunction; > > > > > > > > > > > > > */** Usage in SQL: CREATE TEMPORARY FUNCTION > default_catalog.default_database.fix_instant AS > 'com.intuit.data.strmprocess.udf.FixInstant' LANGUAGE JAVA; CREATE OR > REPLACE TABLE input ( `timestamp` STRING NOT NULL, event_time AS > TO_TIMESTAMP(fix_instant(`timestamp`), 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''), > */ *public class FixInstant extends ScalarFunction > { > private static final long *serialVersionUID *= -3115461377254640072L; > > public String eval(String s) { > if (s == null) return null; > else if (s.indexOf('.') > 0) return s; > else if (s.endsWith("Z")) return s.replace("Z", ".000Z"); > else return null; > } > } > > > > Compiled for Java 8 or Java 11, no differences > > > > Any ideas? > > > >
