Re: Did the semantics of Kafka's earliest offset change with the new source API?

2022-07-17 Thread Alexis Sarda-Espinosa
Hi David, thanks for the info, indeed I had misunderstood our old configuration, we didn't use earliest before, we just used the default. Regards, Alexis. Am Fr., 15. Juli 2022 um 14:46 Uhr schrieb David Anderson < dander...@apache.org>: > What did change was the default starting position when

Re: PyFlink and parallelism

2022-07-17 Thread Dian Fu
Hi John, You could try to use `filtered_stream._j_data_stream.getTransformation().setParallelism(int(table_env.get_config().get('my.custom.parallelism', 1)))`. Usually you don't need to do this, however, when converting Table to DataStream, it returns a Java DataStream object which has no setPara

Re: PyFlink and parallelism

2022-07-17 Thread Juntao Hu
It's not the issue with Python-Java object conversion, you get a DataStream rather than SingleOutputStreamOperator underlying the Python DataStream wrapper after calling `to_data_stream`, and `setParallelism` is only available on SingleOutputStreamOperator. To work around this, change `set_parallel

Re: Metrics & Monitoring in Flink SQL

2022-07-17 Thread yuxia
I'm afraid of there's no document for it. But there's a FLIP[1] define the metrics that the connector (source / sink ) should expose. And I think the offical connectors will cover most of these metrics if possbile. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-33%3A+Standardize+Con