Could not retrieve submitted JobGraph from state handle

2021-11-15 Thread Alexey Trenikhun
Hello, We are using Kubernetes HA and Azure Blob storage and in rare cases I see following error: Could not retrieve submitted JobGraph from state handle under jobGraph-. This indicates that the retrieved state handle is broken. Try cleaning the state handle stor

Re: Noob question

2021-11-15 Thread Caizhi Weng
Hi! Please make sure to always cc the user mailing list. Otherwise other people will not see your mail. Can the checkpoint be manual trigger too? No you can't. Checkpoint can only be triggered by time. This is one of the main difference between checkpoint and savepoint [1]. What's your use cas

Re: PyFlink / Table API Exception: class "BatchExecCalc$7746" grows beyond 64 KB

2021-11-15 Thread Caizhi Weng
Hi! In Flink 1.14 we've introduced the new code splitting mechanism which will solve this issue. Please consider upgrading to Flink 1.14. Schmid Christian 于2021年11月16日周二 上午12:43写道: > Hi all > > > > I ran into an exception while trying to execute a large sql query, which > basically runs "over a

Re: PyFlink SQL window aggregation data not written out to file

2021-11-15 Thread Guoqin Zheng
Hi Roman, Thanks for your quick response! Yes, it does seem to be the window closing problem. So if I change the tumble window on eventTime, which is column 'requestTime', it works fine. I guess the EOF of the test data file kicks in a watermark of Long.MAX_VALUE. But my application code needs

Re: PyFlink SQL window aggregation data not written out to file

2021-11-15 Thread Roman Khachatryan
Hi Guoqin, I think the problem might be related to watermarks and checkpointing: - if the file is too small, the only watermark will be the one after fully reading the file - in exactly once mode, sink waits for a checkpoint completion before committing the files Recently, there were some improve

Re: Fabric8 does not support EC keys

2021-11-15 Thread Nicolás Ferrario
Hi Yang, I tried that and *-Dkubernetes.certs.client.key.algo=EC* ( https://github.com/fabric8io/kubernetes-client/blob/278ca235dc4ab5653e82dbe2960004ab62f021e4/kubernetes-client/src/main/java/io/fabric8/kubernetes/client/Config.java#L79) but none seems to work :( I'm launching flink with this: /f

PyFlink SQL window aggregation data not written out to file

2021-11-15 Thread Guoqin Zheng
Hi all, I am reposting my stackoverflow question here. So I would like to do a local test of my pyflink sql application. The sql query does a very simple job: read from source

Re:

2021-11-15 Thread Roman Khachatryan
Hi, to unsubscribe, please send an email to user-unsubscr...@flink.apache.org Regards, Roman On Mon, Nov 15, 2021 at 10:01 AM xm lian wrote: > > Unsubscribe

Re:

2021-11-15 Thread Roman Khachatryan
Hi Uday, to unsubscribe, please send an email to user-unsubscr...@flink.apache.org Regards, Roman On Mon, Nov 15, 2021 at 12:19 PM Uday Garikipati wrote: > > Unsubscribe

[no subject]

2021-11-15 Thread Uday Garikipati
Unsubscribe

Re: pipeline are not started sporadically

2021-11-15 Thread Caizhi Weng
Hi! I'm not familiar with BigQuery so I'm afraid I can't provide too much help on this. Also Flink does not have a built in BigQuery connector. Are you using some custom connectors? If only BigQuery jobs will cause the problem it is very likely that there are issues in the BigQuery connector. akk

[no subject]

2021-11-15 Thread xm lian
Unsubscribe