Failed to start remote bundle - Beam on Flink Runner

Hello Users! One of our Beam jobs on Flink Runner abruptly failed with the following exception. From logs and metrics, everything seemed right. Any reason why this would happen ? org.apache.flink.util.SerializedThrowable: Failed to start remote bundle at org.apache.beam.runners.flink.translat

SDK Worker availability metrics

Hi Users! Is there a recommended approach to publish metrics on the number of sdk workers available/running as a gauge ? [1] https://github.com/apache/beam/blob/master/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/DefaultJobBundleFactory.java#L267 [2] htt

Re: SDK Worker availability metrics

to use? > > On Sun, Aug 7, 2022 at 1:02 PM aryan m wrote: > >> Hi Users! >> Is there a recommended approach to publish metrics on the number of >> sdk workers available/running as a gauge ? >> >> >> [1] >> https://github.com/apache/beam/blob/ma

Re: SDK Worker availability metrics

values don't change through > the lifetime of the cluster and/or job. > > On Mon, Aug 8, 2022 at 4:59 PM aryan m wrote: > >> Hi Luke! >> Thanks !! We use the Flink Runner and run SDK workers as processes >> [1] within a k8s pod. Can you please share broad s