Re: Env Vars in the Flink Web UI

I'm inclined to go with d), removing it entirely. I must admit that I liked the idea behind the change; exposing more information about what might impact Flink's behavior is a good thing, although I'm irked that the statement in the FLIP about env variables already being exposed in the logs ju

Re: Env Vars in the Flink Web UI

I am not opposed to removing this completely based on Chesnay's reasoning. In general I agree that this feature probably does more harm than good. Gyula On Wed, Nov 16, 2022 at 9:13 AM Chesnay Schepler wrote: > I'm inclined to go with d), removing it entirely. > > I must admit that I liked the

Re: [DISCUSS] Release Flink 1.15.3

Thanks, everyone, for offering your help. Best, Fabian On Wed, Nov 16, 2022 at 12:01 AM Danny Cranmer wrote: > > Hey Fabian, > > I am out this week, I can take a look Monday if still required. > > Thanks, > Danny > > On Tue, Nov 15, 2022 at 8:37 PM Martijn Visser > wrote: > > > Hi Fabian, > > >

Re: [VOTE] Release 1.15.3, release candidate #1

Thank you Fabian for the release. +1 (non-binding) - Verified that the source release can be built successfully. - Verified that the checksum and gpg files match the corresponding source release files, binary release files, and maven artifacts. - Verified that the source archives do not contain a

[jira] [Created] (FLINK-30039) Insert overwrite show excetion that query and insert schema do not match

Hang HOU created FLINK-30039: Summary: Insert overwrite show excetion that query and insert schema do not match Key: FLINK-30039 URL: https://issues.apache.org/jira/browse/FLINK-30039 Project: Flink

[jira] [Created] (FLINK-30040) Add more PyFlink API examples

Huang Xingbo created FLINK-30040: Summary: Add more PyFlink API examples Key: FLINK-30040 URL: https://issues.apache.org/jira/browse/FLINK-30040 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-30041) Setup conjars https mirror

Chesnay Schepler created FLINK-30041: Summary: Setup conjars https mirror Key: FLINK-30041 URL: https://issues.apache.org/jira/browse/FLINK-30041 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-30042) Create/Use new CI image with Maven 3.8.6

Chesnay Schepler created FLINK-30042: Summary: Create/Use new CI image with Maven 3.8.6 Key: FLINK-30042 URL: https://issues.apache.org/jira/browse/FLINK-30042 Project: Flink Issue Type:

[jira] [Created] (FLINK-30043) Some example sqls in flink table store rescale-bucket doucument are incorrect.

Hang HOU created FLINK-30043: Summary: Some example sqls in flink table store rescale-bucket doucument are incorrect. Key: FLINK-30043 URL: https://issues.apache.org/jira/browse/FLINK-30043 Project: Flink

[jira] [Created] (FLINK-30044) Deduplicate plugin parser loops

Chesnay Schepler created FLINK-30044: Summary: Deduplicate plugin parser loops Key: FLINK-30044 URL: https://issues.apache.org/jira/browse/FLINK-30044 Project: Flink Issue Type: Technical

[jira] [Created] (FLINK-30045) FromClasspathEntryClassInformationProvider too eager to verify MainClass

Nico Kruber created FLINK-30045: --- Summary: FromClasspathEntryClassInformationProvider too eager to verify MainClass Key: FLINK-30045 URL: https://issues.apache.org/jira/browse/FLINK-30045 Project: Flink

Argo CD health check for FlinkDeployment

Hi all, We are exploring Argo CD to manage `FlinkDeployment` resources but noticed that the health checking for it doesn't work properly. To give you some context, Argo CD uses Lua scripts to check some state-related fields and map them to three status values: "Healthy", "Progressing" and "Degrad

[jira] [Created] (FLINK-30046) Cannot use digest in Helm chart to reference image

Tobias Hofer created FLINK-30046: Summary: Cannot use digest in Helm chart to reference image Key: FLINK-30046 URL: https://issues.apache.org/jira/browse/FLINK-30046 Project: Flink Issue Type

Re: Argo CD health check for FlinkDeployment

Hi Xingcan! If you are looking for checking the health of the deployed Flink jobs, status.jobStatus.state is a good place to start. At any given time that should represent the Flink Job Status. RUNNING means it's processing data other states mean that it is doing something else (restarting, failin

Re: Argo CD health check for FlinkDeployment

Hi Gyula, Thanks for the explanation! The distinction between Flink jobs and FlinkDeployments makes sense! I'll try to make some changes to Argo CD and hopefully can get some review from you or other Flink-K8s-op contributors then. Best, Xingcan On Wed, Nov 16, 2022 at 10:40 AM Gyula Fóra wrot

Re: [DISCUSS] FLIP-271: Autoscaling

Thanks for the update! Please see comments inline. On Tue, Nov 15, 2022 at 11:46 PM Maximilian Michels wrote: > Of course! Let me know if your concerns are addressed. The wiki page has > been updated. > > >It will be great to add this in the FLIP so that reviewers can understand > how the source

Re: [DISCUSS] FLIP-271: Autoscaling

Hi Dong, Let me address your comments. Time for scale / backlog processing time derivation: We can add some more details to the Flip but at this point the implementation is actually much simpler than the algorithm to describe it. I would not like to add more equations etc because it just overcomp

Re: [DISCUSS] Allow sharing (RocksDB) memory between slots

> I think not being able to isolate all kinds of memory does not mean we > should give up the isolation on all kinds of memory. And I believe "managed > memory is isolated and others are not" is much easier for the users to > understand compared to "part of the managed memory is isolated and others

[jira] [Created] (FLINK-30047) getLastSavepointStatus should return null when there is never savepoint completed or pending

Clara Xiong created FLINK-30047: --- Summary: getLastSavepointStatus should return null when there is never savepoint completed or pending Key: FLINK-30047 URL: https://issues.apache.org/jira/browse/FLINK-30047

Re: Env Vars in the Flink Web UI

Hi everyone, thanks a lot for your feedback so far. Right now, we have pretty much a consensus to not show environment variables at all in the Web UI going forward. I think, we can address this in 1.16.1, as I consider this a vulnerability that should be addressed in a patch release rather than w

Re: Env Vars in the Flink Web UI

+1 for d) don't print any env vars in the web UI (at least by default) There could be an option to allow-list printing of env vars but it should be off by default. Generally I think that those that should be able to see env vars probably can get there by other means, like kubetcl exec Thanks, Th

Stack Overflow Question re compressing broadcast state

Hi all, Just posted this question on SO: How to enable compression for Flink broadcast state checkpoints Basically it doesn’t look like broadcast state respects the compressed state (checkpoints/savepoints) setting, but I might be reading the c

Re: [DISCUSS] Allow sharing (RocksDB) memory between slots

Agreed that the documentation regarding managed memory could be improved. Just to clarify, breaking isolation is just one of the concerns. I think my biggest concern is the extra complexity in managed memory calculations. I've been reached out by many users, online or offline, asking about the man

[jira] [Created] (FLINK-30048) MapState.remove(UK key) is better to return the old value.

xljtswf created FLINK-30048: --- Summary: MapState.remove(UK key) is better to return the old value. Key: FLINK-30048 URL: https://issues.apache.org/jira/browse/FLINK-30048 Project: Flink Issue Type:

[jira] [Created] (FLINK-30049) CsvBulkWriter is unsupported for S3 FileSystem

Jingsong Lee created FLINK-30049: Summary: CsvBulkWriter is unsupported for S3 FileSystem Key: FLINK-30049 URL: https://issues.apache.org/jira/browse/FLINK-30049 Project: Flink Issue Type: Bu

Can we make flink operator using golang?

Hi, I found we already have Flink operator implemented by java. But I have two questions: 1. If we can implement Flink operator using golang? Is there some hidden difficult traps? 2. We can submit Java jar jobs or sql jobs, can we submit golang jobs? Thank you.