Hello Everyone, In order to finalize https://github.com/apache/beam/pull/25171, I would need to work *closely* with someone who has deeper knowledge of how jenkins was set up, particularly its gcloud and kubectl configuration. If anyone with this knowledge would like to work on this with me, please let me know. *Until then, I will no longer work on this PR or https://github.com/apache/beam/issues/19465 <https://github.com/apache/beam/issues/19465> I will leave the PR open until 2/1 5 PM PST and convert it to draft state, thereafter*.
I was successful at refactoring the Kafka cluster outside service spec from LoadBalancer to NodePort along with the Kafka broker configuration and validated in my own Google Kubernetes Engine that I can connect via private network access using Apache Kafka getting started. It was challenging to debug environment variables that the jenkins job required. Due to the nuances of the jenkins setup and its particulars with gcloud and kubectl, I was not able to test and replicate this in my own environment. Best, Damon On Wed, Jan 25, 2023 at 12:12 PM Damon Douglas <damondoug...@google.com> wrote: > Hello Everyone, > > Thank you for your feedback. I started a pull request > https://github.com/apache/beam/pull/25171 to address this issue. The PR > description contains a summary list of the proposed changes. > > Best, > > Damon > > On Mon, Jan 23, 2023 at 10:38 AM Chamikara Jayalath <chamik...@google.com> > wrote: > >> +1 for reducing access to what's required by tests. >> >> Thanks, >> Cham >> >> On Mon, Jan 23, 2023 at 10:30 AM Yi Hu via dev <dev@beam.apache.org> >> wrote: >> >>> Hi Damon, >>> >>> Thanks for the proposal! Our k8s infrastructure has long been under >>> maintained. I agree Public IP exposure is not necessary and it has >>> triggered security alerts quite often... Would like to help if needed. >>> >>> Best, >>> Yi >>> >>> On Mon, Jan 23, 2023 at 1:25 PM Damon Douglas via dev < >>> dev@beam.apache.org> wrote: >>> >>>> Hello Everyone, >>>> >>>> I would like to share with the community a proposal [1] to refactor a >>>> Kubernetes Kafka External Load Balancer dependency in tests. It fixes >>>> [2]. The referenced document summarizes the situation, background, >>>> assessment, and recommendation. >>>> >>>> Best, >>>> >>>> Damon >>>> >>>> *References* >>>> >>>> 1.[Public facing][Beam][issue/25119] Refactor Kubernetes Kafka >>>> External load balancer dependency in tests >>>> <https://docs.google.com/document/d/1U9aTAwL_YzrrPh4IAkpMQ6MXAwVumTg9JpPCAxxgJug/edit?usp=sharing&resourcekey=0-8vzToz5LJiqPBlvCKd-SOQ> >>>> 2. https://github.com/apache/beam/issues/25119 >>>> >>>
