Thanks a lot for your response, and sorry for my late response. how do you intend downstream CIs in flink / flink-statefun to be using > this? > Would there be released artifacts from `flink-project-utils` to expose > each tool (e.g. the `LicenseChecker`)?
My idea was that flink and flink-statefun would just clone the flink-project-utils repository, build the tool locally and run it, without releasing the artifacts anywhere. The build takes just a few seconds, and the maintenance overhead is a little lower with this approach. I also considered building a maven plugin, but I guess forking the maven-shade-plugin is the only feasible option. Adding a custom maven-shade-transformer is not feasible, since we won't have access to the required data. All this sounded too complicated, so I discarded the idea. Thanks a lot for the list of copied utilities. I guess if we are adding more sister projects to Flink, or splitting the main repo in the future, we need to look into coming up with a properly written set of shared release utilities. I have the feeling that currently the effort for creating a separate repository is not justified for just checking one module in flink-statefun. Let's revisit this in the future. On Fri, Nov 6, 2020 at 4:28 AM Tzu-Li (Gordon) Tai <tzuli...@apache.org> wrote: > Hi Robert, > > I think this could be useful in flink-statefun. > > StateFun currently has two modules that bundles dependencies, most > importantly the `flink-statefun-distribution` module which currently > bundles some Flink dependencies as well as Flink connectors (Kafka, > Kinesis). > Upgrading the Flink version in StateFun typically involves a bulk update on > the NOTICE of that module, so some automatic validation in CI could help > with that. > The other module that bundles dependencies is the StateFun examples, which > we've been thinking about stopping to release Maven artifacts for. > > On Thu, Nov 5, 2020 at 9:54 PM Robert Metzger <rmetz...@apache.org> wrote: > > > 1. Is this relevant for flink-statefun? > > > > So, really there is only one module that would benefit from this tool > (which could possibly be enough already for sharing to make sense). > To justify the efforts for sharing this nice utility, I'd like to have a > better idea of: how do you intend downstream CIs in flink / flink-statefun > to be using this? Would there be released artifacts from > `flink-project-utils` to expose each tool (e.g. the `LicenseChecker`)? > It almost looks as if it would be easiest to reuse this tool if it was > provided as a Maven plugin, though I'm not sure how possible that is and > probably out-of-scope for this discussion. > > > > > > 2. For the repository name, what do you think about > "flink-project-utils" ? > > I'd like to use a generic name so that we can potentially share other > > internal utilities. > > > > I like the idea of sharing internal utilities in general across the two > repos. > > To gauge how useful this repo would be in the end, here's some info on what > StateFun has copied already to flink-statefun: > > - About to copy checking for dead links in docs [1] > - Several release-related scripts for creating source bundles, deploying > staging jars, updating branch version, etc. [2]. However, these have > somewhat evolved in StateFun to tailor it for flink-statefun, so I > guess it > would not make sense to share release related tooling. > - Utility around building the documentation (currently flink and > flink-statefun share the same Jekyll setup). > > Best, > Gordon > > [1] https://github.com/apache/flink-statefun/pull/171 > [2] https://github.com/apache/flink-statefun/tree/master/tools/releasing >
