Re: How can we exclude operator level metrics from getting reported

2024-05-13 Thread Biao Geng
Hi Sachin, Your setting looks fine to me. If you want to verify that, one way is to set the log level to 'trace' and check if logs like 'Ignoring metric {}.{} for reporter #{} due to filter rules.' is printed. Best, Biao Geng Sachin Mittal 于2024年5月12日周日 02:51写道: > Hi > I have a following metr

Flink autoscaler with AWS ASG: checkpoint access issue

2024-05-13 Thread Chetas Joshi
Hello, Set up I am running my Flink streaming jobs (upgradeMode = stateless) on an AWS EKS cluster. The node-type for the pods of the streaming jobs belongs to a node-group that has an AWS ASG (auto scaling group). The streaming jobs are the FlinkDeployments managed by the flink-k8s-operator (1.8

Proper way to modify log4j config file for kubernetes-session

2024-05-13 Thread Vararu, Vadim
Hi, Trying to configure loggers in the log4j-console.properties file (that is mounted from the host where the kubernetes-session.sh is invoked and referenced by the TM processes via - Dlog4j.configurationFile). Is there a proper (documented) way to do that, meaning to append/modify the log4j c

Best Practices? Fault Isolation for Processing Large Number of Same-Shaped Input Kafka Topics in a Big Flink Job

2024-05-13 Thread Kevin Lam via user
Hi everyone, I'm currently prototyping on a project where we need to process a large number of Kafka input topics (say, a couple of hundred), all of which share the same DataType/Schema. Our objective is to run the same Flink SQL on all of the input topics, but I am concerned about doing this in