Hi,
I see the stable core Flink API as a prerequisite for modularity. And
for connectors it is not just the source and sink API (source being
stable as of 1.14), but everything that is required to build and
maintain a connector downstream, such as the test utilities and
infrastructure.
Without th
Hi Konstantin,
> the connectors need to be adopted and require at least one release per
Flink minor release.
However, this will make the releases of connectors slower, e.g. maintain
features for multiple branches and release multiple branches.
I think the main purpose of having an external connect
Hi everyone,
regarding the stability of the APIs. I think everyone agrees that
connector APIs which are stable across minor versions (1.13->1.14) are the
mid-term goal. But:
a) These APIs are still quite young, and we shouldn't make them @Public
prematurely either.
b) Isn't this *mostly* orthogo
Hi,
I think Thomas raised very good questions and would like to know your
opinions if we want to move connectors out of flink in this version.
(1) is the connector API already stable?
> Separate releases would only make sense if the core Flink surface is
> fairly stable though. As evident from Ic
Could you clarify what release cadence you're thinking of? There's quite
a big range that fits "more frequent than Flink" (per-commit, daily,
weekly, bi-weekly, monthly, even bi-monthly).
On 19/10/2021 14:15, Martijn Visser wrote:
Hi all,
I think it would be a huge benefit if we can achieve m
TBH I think you're overestimating how much work it is to create a
non-Flink release. Having done most of the flink-shaded releases, I
really don't see an issue of even doing weekly releases with that process.
We can not reduce the number of votes AFAIK; the ASF seems very clear on
that matter
Hey all,
I don't have much to add to the general discussion. Just a single
comment on:
that we could adjust the bylaws for the connectors such that we need
fewer PMCs to approve a release. Would it be enough to have one PMC
vote per connector release?
I think it's not an option. This
Thank you, Arvid & team, for working on this.
I would also favor one connector repository under the ASF. This will
already force us to provide better tools and more stable APIs, which
connectors developed outside of Apache Flink will benefit from, too.
Besides simplifying the formal release proce
Okay I think it is clear that the majority would like to keep connectors
under the Apache Flink umbrella. That means we will not be able to have
per-connector repositories and project management, automatic dependency
bumping with Dependabot, or semi-automatic releases.
So then I'm assuming the dir
Hi all,
I think it would be a huge benefit if we can achieve more frequent releases
of connectors, which are not bound to the release cycle of Flink itself. I
agree that in order to get there, we need to have stable interfaces which
are trustworthy and reliable, so they can be safely used by those
Thanks for initiating this discussion.
There are definitely a few things that are not optimal with our
current management of connectors. I would not necessarily characterize
it as a "mess" though. As the points raised so far show, it isn't easy
to find a solution that balances competing requiremen
Generally, the issues are reproducibility and control.
Stuffs completely broken on the Flink side for a week? Well then so are
the connector repos.
(As-is) You can't go back to a previous version of the snapshot. Which
also means that checking out older commits can be problematic because
you'd
I think you're misinterpreting my comment.
Independent from the repo split we should only keep the connectors in
the Flink project that we actively maintain.
The rest we might as well just drop.
If some external people are interested in maintaining these connectors
then there's nothing stoppin
Hi folks,
thanks for joining the discussion. I'd like to give some ideas on how
certain concerns are going to be addressed:
Ingo:
> In general I think breaking up the big repo would be a good move with many
> benefits (which you have outlined already). One concern would be how to
> proceed with o
Hi, all
I understand very well that the maintainers of the community want to move the
connector to an external system. Indeed, the development and maintenance of the
connector requires a lot of energy, and these do not involve the Flink core
framework, which can reduce the maintenance pressure
We are mostly talking about the freedom this would bring to the connector
authors, but we still don't have answers for the important topics:
- How exactly are we going to maintain the high quality standard of the
connectors?
- How would the connector release cycle to look like? Is this going to
af
Thanks for driving this discussion Arvid! I think this will be one giant leap
for Flink community. Externalizing connectors would give connector developers
more freedom in developing, releasing and maintaining, which can attract more
developers for contributing their connectors and expand the Fl
My opinion of splitting the Flink repositories hasn't changed; I'm still
in favor of it.
While it would technically be possible to release individual connectors
even if they are part of the Flink repo,
it is quite a hassle to do so and error prone due to the current branch
structure.
A split
Hi Arvid,
In general I think breaking up the big repo would be a good move with many
benefits (which you have outlined already). One concern would be how to
proceed with our docs / examples if we were to really separate out all
connectors.
1. More real-life examples would essentially now depend o
Dear community,
Today I would like to kickstart a series of discussions around creating an
external connector repository. The main idea is to decouple the release
cycle of Flink with the release cycles of the connectors. This is a common
approach in other big data analytics projects and seems to s
20 matches
Mail list logo
