Hi Alexander, Thanks so much for the info!
It sounds like the best path forward is for us to develop outside of Apache while, in parallel, working to gain committer status. Our goal will be to eventually move anything we build under the Apache umbrella once we're more plugged in to the community. As for migrating the existing Pub/Sub connector to the new Source API, we actually have somebody currently building a new Pub/Sub connector from scratch (using the new Source API). Once that is ready, we will make sure to get that new implementation moved under Apache and help with the migration effort. Thanks again for the response and I'm sure we will be chatting soon! Best, Claire On Wed, Feb 14, 2024 at 7:36 AM Alexander Fedulov < alexander.fedu...@gmail.com> wrote: > Hi Claire, > > Thanks for reaching out. It's great that there is interest from Google > in spearheading the development of the respective Flink connectors. > > As of now,there is only one GCP-specific connector developed directly as > part > of ASF Flink, namely the Pub/Sub one. It has already been externalized here > [1]. > Grouping further connectors under apache/flink-connectors-gcp makes sense, > but > it would be nice to first understand which GCP connectors you plan to add > before we create this new umbrella project. > > I do not think establishing a dedicated workgroup to help with the > GCP-specific > development is a realistic goal, though. The development will most probably > take > place on the regular ASF best effort basis (which involves mailing list > discussions, > reaching out to people for reviews, etc.) until your developers gain > committer status > and can work more independently. > > One immediate open item where the Flink community would definitely > appreciate your > help is with the migration of the existing Pub/Sub connector to the new > Source API. > As you can see here [2], it is one of the two remaining connectors where we > have not > yet made progress, and it seems like a great place to start the > collaboration. > Flink 2.0 aims to remove the SourceFunction API, which the current Pub/Sub > connector > relies on. It would be great if your colleagues could assist with this > effort [3]. > > Best, > Alexander Fedulov > > [1] https://github.com/apache/flink-connector-gcp-pubsub > [2] https://issues.apache.org/jira/browse/FLINK-28045 > [3] https://issues.apache.org/jira/browse/FLINK-32673 > > > > On Tue, 13 Feb 2024 at 17:25, Claire McCarthy > <clairemccar...@google.com.invalid> wrote: > > > Hi Devs! > > > > I’d like to kick off a discussion on setting up a repo for a new fleet of > > Google Cloud connectors. > > > > A bit of context: > > > > - > > > > We have a team of Google engineers who are looking to build/maintain > > 5-10 GCP connectors for Flink. > > - > > > > We are wondering if it would make sense to host our connectors under > the > > ASF umbrella following a similar repo structure as AWS ( > > https://github.com/apache/flink-connector-aws). In our case: > > apache/flink-connectors-gcp. > > - > > > > Currently, we have no Flink committers on our team. We are actively > > involved in the Apache Beam community and have a number of ASF members > > on > > the team. > > > > > > We saw that one of the original motivations for externalizing connectors > > was to encourage more activity and contributions around connectors by > > easing the contribution overhead. We understand that the decision was > > ultimately made to host the externalized connector repos under the ASF > > organization. For the same reasons (release infra, quality assurance, > > integration with the community, etc.), we would like all GCP connectors > to > > live under the ASF organization. > > > > We want to ask the Flink community what you all think of this idea, and > > what would be the best way for us to go about contributing something like > > this. We are excited to contribute and want to learn and follow your > > practices. > > > > A specific issue we know of is that our changes need approval from Flink > > committers. Do you have a suggestion for how best to go about a new > > contribution like ours from a team that does not have committers? Is it > > possible, for example, to partner with a committer (or a small cohort) > for > > tight engagement? We also know about ASF voting and release process, but > > that doesn't seem to be as much of a potential hurdle. > > > > Huge thanks in advance for sharing your thoughts! > > > > > > Claire > > >