Re: Fink 15: InvalidProgramException: Table program cannot be compiled. This is a bug

Hi. I open a ticket about upgrading the version[1]. Maybe it is worth a try. Best, Shengkai [1] https://issues.apache.org/jira/browse/FLINK-27995 Benenson, Michael 于2022年6月10日周五 04:51写道： > Hi, David > > > > Hard to tell for sure, but yes, [1] could also indicate some problems with > Janio. >

RE: Recover watermark from savepoint

Hi Sweta, It is actually a sound idea to implement a dedicated process function for this purpose, as David suggests. Especially if you are in a situation where waiting for a valid natural watermark after a restore from savepoint is not sufficient. We had a situation with input streams of differ

Re:Flink SQL JDBC connector for Postgres can't handle UUID datatype

Unfortunately, it seems that flink currently doesn't support the type UUID for jdbc conector. I think you can create a new issue on jira and restart the dicussion about this. -- Best！ Xuyang At 2022-06-09 22:40:45, "Aaron Weiss" wrote: Hey there, We have just started to use t

How to schedule stateful function in flink

I have a use case in which I have to schedule a stateful function for a certain frequency in a day , so that I can perform business functions on all the data stored in state till data time. Thanks.

flink-ml onlinekmeans

Hi, I have a couple of questions about the onlineKmeans algorithm. I am running OnlikeKmeans on a small dataset (36k records) with k=2 and testing varying decay rates. The features are consumed in flink from kafka topic. Sample feature: [0., 5.0, 1.0, 0.0, 0.0, 0.0, 0.0] Is the implementation

Re: Fink 15: InvalidProgramException: Table program cannot be compiled. This is a bug

Sorry; I guess I jumped to conclusions and got it wrong. Let's dig in deeper. This sounds similar to what was reported in this thread [1] where someone else ran into problems with Janino after upgrading from 1.14 to 1.15. Could this be another instance of the same issue (i.e., Flink ends up using

Re: [External] Re: Source vs SourceFunction and testing

Hi Carlos, You might want to join the discussion about FLIP-238[1] to share your thoughts with us. Thanks! Best regards, Jing [1] https://lists.apache.org/thread/7gjxto1rmkpff4kl54j8nlg5db2rqhkt On Thu, Jun 9, 2022 at 2:13 PM Sanabria, Carlos < carlos.sanab...@accenture.com> wrote: > Thanks f

Flink SQL JDBC connector for Postgres can't handle UUID datatype

Hey there, We have just started to use the Flink SQL JDBC connector to do some writes to Postgres. Our Postgres table has a UUID column we need to write to through Flink SQL. However, it doesn't appear that the connector supports that type in any way. UUID isn't listed in the Postgres datatypes

Exception: class java.sql.Timestamp cannot be cast to class java.time.LocalDateTime | pyflink 1.15.0

Hi, I'd like to ask for help regarding the java exception: Caused by: java.lang.ClassCastException: class java.sql.Timestamp cannot be cast to class java.time.LocalDateTime (java.sql.Timestamp is in module java.sql of loader 'platform'; java.time.LocalDateTime is in module java.base of loader 'boo

Re: Fink 15: InvalidProgramException: Table program cannot be compiled. This is a bug

A Table can have at most one time attribute. In your Table the proc_time column is a processing time attribute, and when you define a watermark on the event_time column then that column becomes an event-time attribute. If you want to combine event time and processing time, you can use the PROCTIME

Flink Shaded dependencies and extending Flink APIs

Hi all, I'm working on an integration project trying to write some library code that will allow us at the Wikimedia Foundation to use Flink with our 'Event Platform '. Specifically, I'm trying to write a reusable step near the end of a pipeline t

RE: [External] Re: Source vs SourceFunction and testing

Thanks for your quick response! Yes, this is exactly what we were looking for! Seems like a really nice feature. Even better than the FromElementsSource we were asking for, because it allows to generate the events dynamically. Is there any way we can vote for the FLIP-238 to be accepted? -O

Re:Re: How to handle deletion of items using PyFlink SQL?

Hi, Dian Fu. I think John's requirement is like a cdc source that the source needs the ability to know which of datas should be deleted and then notify the framework, and that is why I recommendation John to use the UDTF. And hi, John. I'm not sure this doc [1] is enough. BTW,

Re: [External] Re: Source vs SourceFunction and testing

Hi Carlos, FLIP-238 [1] is proposing a FLIP-27-based data generator source and I think this is what you are looking for. This FLIP was created just days ago so it may take some time to get accepted and released. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-238%3A+Introduce+FLIP-27-

Flink JobManager node JVM Metaspace is too high

When using FinkUI to upload the Flink task Jar, and the task fails to start, I re-upload the Flink task Jar. At this time, the JVM Metaspace will increase abnormally. What is the reason?

RE: [External] Re: Source vs SourceFunction and testing

Hi everyone! Sorry for reopening the thread, but I am having some problems related to this case while migrating our code from Flink 1.12 to Flink 1.15. We have a base project that encapsulates a ton of common code and configurations. One of the abstractions we have is an AbstractDataStreamJob

Re: Recover watermark from savepoint

Sweta, Flink does not include watermarks in savepoints, nor are they included in aligned checkpoints. For what it's worth, I believe that with unaligned checkpoints in-flight watermarks are included in checkpoints, but I don't believe that would solve the problem, since the watermark strategy's st