We should have release branches which are in sync with the release branches in the main repository. Connectors should be compatible across minor releases. The versioning could be of the form "flinkversion-connectorversion", e.g. 0.10-connector1.
>The pluggable architecture is great! (why Don't we call it Flink plugins? my 2 >cents) We can still change the name. IMHO "Plugins" is a bit broad since this is currently only targeted at the connectors included in Flink. >Would we loose test coverage by putting the connectors into a separate >repository/maven project? Not necessarily. Two possibilities: 1) Run a connectors test jar during the normal Travis tests in the main repository 2) Trigger a Travis test run at the connectors repository upon a commit into the main repository Option 1 seems like the better alternative because we would immediately see if a change breaks the connectors. Of course, if changes are made in the connectors repository, we would also run tests with the main repository. On Thu, Dec 10, 2015 at 11:00 PM, jun aoki <ja...@apache.org> wrote: > The pluggable architecture is great! (why Don't we call it Flink plugins? > my 2 cents) > It will be nice if we come up with an idea of what directory structure > should look like before start dumping connectors (plugins). > Also wonder what to do with versioning. > At some point, for example, Twitter v1 connector could be compatible with > flink 0.10 but Flume v2 connector could be compatible with trunk, etc. It > should be taken consideration either in the directory structure or > branching strategy. > > On Thu, Dec 10, 2015 at 7:12 AM, Aljoscha Krettek <aljos...@apache.org> > wrote: > >> We would need to have a stable interface between the connectors and flink >> and have very good checks that ensure that we don’t inadvertently break >> things. >> >> > On 10 Dec 2015, at 15:45, Fabian Hueske <fhue...@gmail.com> wrote: >> > >> > Sounds like a good idea to me. >> > >> > +1 >> > >> > Fabian >> > >> > 2015-12-10 15:31 GMT+01:00 Maximilian Michels <m...@apache.org>: >> > >> >> Hi squirrels, >> >> >> >> By this time, we have numerous connectors which let you insert data >> >> into Flink or output data from Flink. >> >> >> >> On the streaming side we have >> >> >> >> - RollingSink >> >> - Flume >> >> - Kafka >> >> - Nifi >> >> - RabbitMQ >> >> - Twitter >> >> >> >> On the batch side we have >> >> >> >> - Avro >> >> - Hadoop compatibility >> >> - HBase >> >> - HCatalog >> >> - JDBC >> >> >> >> >> >> Many times we would have liked to release updates to the connectors or >> >> even create new ones in between Flink releases. This is currently not >> >> possible because the connectors are part of the main repository. >> >> >> >> Therefore, I have created a new repository at >> >> https://git-wip-us.apache.org/repos/asf/flink-connectors.git. The idea >> >> is to externalize the connectors to this repository. We can then >> >> update and release them independently of the main Flink repository. I >> >> think this will give us more flexibility in the development process. >> >> >> >> What do you think about this idea? >> >> >> >> Cheers, >> >> Max >> >> >> >> > > > -- > -jun
