[DISCUSS][FLINK-26014]Document how to use the working directory for faster local recoveries

After read FLIP-198 & FLIP-201，I confuse that this feature can be used on Yarn mode and how to confige a deterministic `taskmanager.resource-id` ?Or just suit for k8s mode.

Re: standalone mode support in the kubernetes operator (FLIP-25)

Hi all, Does 'standalone mode support in the kubernetes operator' means: Using flink-k8s-operator to manage jobs deployed in a standalone cluster? What is the advantag doing so. Yang Wang 于2022年7月14日周四 10:55写道： > > I think the standalone mode support is expected to be done in the version > 1.2.0

Flink application mode, multiple jobs

Hi all, Is there any limitation on the number of jobs you can deploy together within the same Flink application? We are noticing some exceptions related to task slots at job startup. It typically recovers after 10-20 minutes. What are some of the recommended configurations that we can tune to all

Re: [DISCUSS] Contribution of Multi Cluster Kafka Source

Hi all, Circling back on this--I have created a first draft document in confluence: https://cwiki.apache.org/confluence/display/FLINK/FLIP-246%3A+Multi+Cluster+Kafka+Source . Looking forward to hear all your feedback in this email thread! Best, Mason On Thu, Jun 30, 2022 at 6:57 AM Thomas Weise

Re: Making Kafka source respect offset changed externally

Hi Yaroslav, The test I did was just using earliest, I'll test with committed offset again, thanks. Regards, Alexis. On Thu, 14 Jul 2022, 20:49 Yaroslav Tkachenko, wrote: > Hi Alexis, > > Do you use OffsetsInitializer.committedOffsets() to specify your Kafka > consumer offsets? In this case, i

Re: Issues with watermark alignment in Flink 1.15

Found the reason: it does not work together with .withIdleness(Duration.ofSeconds(1)) Isn't this a valid scenario: one subtask has multiple input streams/channels where some are idle, others have large watermark skews? In addition, do we expect that the watermark update interval in: .withWaterm

Re: Making Kafka source respect offset changed externally

Hi Alexis, Do you use OffsetsInitializer.committedOffsets() to specify your Kafka consumer offsets? In this case, it should get the offsets from Kafka and not the state. On Thu, Jul 14, 2022 at 11:18 AM Alexis Sarda-Espinosa < sarda.espin...@gmail.com> wrote: > Hello, > > Regarding the new Kafka

Making Kafka source respect offset changed externally

Hello, Regarding the new Kafka source (configure with a consumer group), I found out that if I manually change the group's offset with Kafka's admin API independently of Flink (while the job is running), the Flink source will ignore that and reset it to whatever it stored internally. Is there any

Re: Flink running same task on different Task Manager

-> If so, I think you can set Task1 and Task2 to the same parallelism and set them in the same slot sharing group. In this way, Task1 and Task2 will be deployed into the same slot(That is, the same task manager). *Updating task details * *Task1- Source some static data over HTTPS and keep it in me

Re: Unit test have Error "could not find implicit value for evidence parameter"

Thanks for the tip! On Wed, Jul 13, 2022 at 10:33 AM Jing Ge wrote: > Hi, > > you don't have to do that. Next time you can try "Invalidate Caches..." > under the File menu in Intellij Idea. > > Best regards, > Jing > > On Wed, Jul 13, 2022 at 7:21 PM Min Tu via user > wrote: > >> Thanks a lot !

Re: Kubernetes Operator - Logging

Biao and Matyas, Thanks for the responses. Biao, I tried what you suggested, but maybe I'm not clear on this step Use -Dlogback.configurationFile= to specify the logback.xml(mounted or packaged in the image) in spec.flinkConfiguration I added my logback-console.xml to /opt/flink/logback-console.