Hey Everyone, I am working on a new Rust based portable Beam Runner and I'm at pipeline execution phase where the Rust runner side needs to communicate with the worker sdk harness(Java) and execute the stages( stages are nothing but a set of fused transforms, formed using the greedy fusion approach from Beam Java utils, rewritten in Rust for the runner)
For the stage to run on a worker, the runner needs to register the stage information with the worker and then send a run request via grpc channels. The worker will execute the transforms in the stage and send the output back to runner via data grpc channel in the form of Elements [1] Elements contain the output data as raw bytes which the runner needs to decode to get the actual data like String, Int or POJO. Typically other runners do it with Beam's defined coders for encoding and decoding. But for Rust there isn't Beam coders implementation. Curious if anyone previously worked on coders for cross languages, or similar things, and how did you implement in general. Thanks, Ganesh. [1] - https://github.com/apache/beam/blob/55eb624e5cd00e546ab19fc411281a0e5f596142/model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto#L724
