Re: Job manager crash

2021-09-05 Thread Yangze Guo
Hi, The root cause is not "java.lang.NoClassDefFound". The job has been running but could not edit the config map "myJob--jobmanager-leader" and it seems finally disconnected with the API server. Is there another job with the same cluster id (myJob) ? I would also

Re: pyflink table to datastream

2021-09-05 Thread Caizhi Weng
Hi! I don't quite understand this question, but I suppose you first run the table program and then run the data stream program and you want the results of the two programs to be identical? If this is the case, the job will run twice as Flink will not cache the result of a job, so in each run the

Re: Job manager crash

2021-09-05 Thread Caizhi Weng
Hi! There is a message saying "java.lang.NoClassDefFound Error: org/apache/hadoop/hdfs/HdfsConfiguration" in your log file. Are you visiting HDFS in your job? If yes it seems that your Flink distribution or your cluster is lacking hadoop classes. Please make sure that there are hadoop jars in the

Re: Obtaining Flink Conf in User Code

2021-09-05 Thread Arvid Heise
I think there is no easy way to access the configuration. There is GlobalConfiguration but it's marked Internal and I think it's even discouraged inside core Flink. However, usually you have full access to the Configuraiton inside your main and you can simply pass it or the desired option to your