Yes, absolutely. Setting up another repository for Flink ML would be no problem.
On Sat, Dec 12, 2015 at 1:52 AM, Henry Saputra <henry.sapu...@gmail.com> wrote: > I had small chat with Till about how to help manage Flink ML Libraries > contributions, which use Flink ML as dependencies. > > I suppose if this approached is the way to go for Flink connectors, > could we do the same for Flink ML libraries? > > > - Henry > > On Fri, Dec 11, 2015 at 1:33 AM, Maximilian Michels <m...@apache.org> wrote: >> We should have release branches which are in sync with the release >> branches in the main repository. Connectors should be compatible >> across minor releases. The versioning could be of the form >> "flinkversion-connectorversion", e.g. 0.10-connector1. >> >>>The pluggable architecture is great! (why Don't we call it Flink plugins? my >>>2 cents) >> >> We can still change the name. IMHO "Plugins" is a bit broad since this >> is currently only targeted at the connectors included in Flink. >> >>>Would we loose test coverage by putting the connectors into a separate >>>repository/maven project? >> >> Not necessarily. Two possibilities: >> >> 1) Run a connectors test jar during the normal Travis tests in the >> main repository >> 2) Trigger a Travis test run at the connectors repository upon a >> commit into the main repository >> >> Option 1 seems like the better alternative because we would >> immediately see if a change breaks the connectors. Of course, if >> changes are made in the connectors repository, we would also run tests >> with the main repository. >> >> On Thu, Dec 10, 2015 at 11:00 PM, jun aoki <ja...@apache.org> wrote: >>> The pluggable architecture is great! (why Don't we call it Flink plugins? >>> my 2 cents) >>> It will be nice if we come up with an idea of what directory structure >>> should look like before start dumping connectors (plugins). >>> Also wonder what to do with versioning. >>> At some point, for example, Twitter v1 connector could be compatible with >>> flink 0.10 but Flume v2 connector could be compatible with trunk, etc. It >>> should be taken consideration either in the directory structure or >>> branching strategy. >>> >>> On Thu, Dec 10, 2015 at 7:12 AM, Aljoscha Krettek <aljos...@apache.org> >>> wrote: >>> >>>> We would need to have a stable interface between the connectors and flink >>>> and have very good checks that ensure that we don’t inadvertently break >>>> things. >>>> >>>> > On 10 Dec 2015, at 15:45, Fabian Hueske <fhue...@gmail.com> wrote: >>>> > >>>> > Sounds like a good idea to me. >>>> > >>>> > +1 >>>> > >>>> > Fabian >>>> > >>>> > 2015-12-10 15:31 GMT+01:00 Maximilian Michels <m...@apache.org>: >>>> > >>>> >> Hi squirrels, >>>> >> >>>> >> By this time, we have numerous connectors which let you insert data >>>> >> into Flink or output data from Flink. >>>> >> >>>> >> On the streaming side we have >>>> >> >>>> >> - RollingSink >>>> >> - Flume >>>> >> - Kafka >>>> >> - Nifi >>>> >> - RabbitMQ >>>> >> - Twitter >>>> >> >>>> >> On the batch side we have >>>> >> >>>> >> - Avro >>>> >> - Hadoop compatibility >>>> >> - HBase >>>> >> - HCatalog >>>> >> - JDBC >>>> >> >>>> >> >>>> >> Many times we would have liked to release updates to the connectors or >>>> >> even create new ones in between Flink releases. This is currently not >>>> >> possible because the connectors are part of the main repository. >>>> >> >>>> >> Therefore, I have created a new repository at >>>> >> https://git-wip-us.apache.org/repos/asf/flink-connectors.git. The idea >>>> >> is to externalize the connectors to this repository. We can then >>>> >> update and release them independently of the main Flink repository. I >>>> >> think this will give us more flexibility in the development process. >>>> >> >>>> >> What do you think about this idea? >>>> >> >>>> >> Cheers, >>>> >> Max >>>> >> >>>> >>>> >>> >>> >>> -- >>> -jun
