I created an example project with my attempts to get something working: https://github.com/gonzojive/beam-go-k8s
The good news is I got something working with Beam, Go, Bazel, Flink, and Kubernetes (minikube). The working version doesn't use the operator and only runs in a single pod. I would still like to figure out how to use the Flink operator with a Go Beam job. I'm not yet sure how the operator should work together with the "beam_flink1.14_job_server" and the Go binary. On Fri, May 20, 2022 at 5:17 PM Red Daly <redd...@gmail.com> wrote: > Hi, > > This is a request for documentation to walk through using the Flink > Kubernetes operator with the Go Beam SDK. I will plan to update the thread > as I investigate. I hope this feedback is useful. > > A few observations: > > 1) The official "Quick Start guide > <https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/try-flink-kubernetes-operator/quick-start/>" > for the Kubernetes operator is great. It would be great if there were a > followup guide for how to use this operator to run a Beam job, especially a > Beam Go job. > > 2) The closest to a walkthrough I have found is > github.com/sambvfx/beam-flink-k8s, but that doesn't seem to use the > operator at all. > > 3) There are 3 different Flink Kubernetes operators. (Lyft, > GoogleCloudPlatform, and the Apache one.) The github repos for these don't > have information about the other projects in their READMEs, from what I can > tell. So it is a bit confusing. > > 4) There are a lot of concepts between Beam and Flink. To be a bit blunt, > I don't want to learn many concepts up front. I just want to run a few > commands and be up and running, then invest the time to learn more concepts > later. > > - Red >
