I'll try these out, thanks!! On Fri, Oct 4, 2024 at 2:01 PM Robert Burke <rob...@frantil.com> wrote:
> ProcessBundleProgressRequest/ProcessBundleProgressResponse are usable for > this purpose, and are recommended for gauging bundle progress anyway. > > > > https://github.com/apache/beam/blob/master/model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto#L154 > > > Same for the WorkerStatus request which should provide information on the > worker. This has the advantage of being out of the ControlService path. > > > https://github.com/apache/beam/blob/master/model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto#L1278 > > I think a heartbeat was considered, but honestly we are either sending > bundles/ and rpcs anyway, or would want regular updates on the worker level > health from the status message, so I don't think there's value in another > RPC. > > Grpc possibly has something built in as well, if only for connection > monitoring. > > On Fri, Oct 4, 2024, 10:46 AM Joey Tran <joey.t...@schrodinger.com> wrote: > >> Hey all, >> >> I have a python runner I've written and I'm debugging the case when a SDK >> worker crashes. Currently my driver/ runner starts the SDK worker and then >> pushes data / instructions through the data and control connections, but >> this just silently waits forever if the worker has actually crashed for >> whatever reason. >> >> Is there a way to poll SDK workers to see if they're even alive? >> >> Cheers, >> Joey >> >
