I did a release check for license issues - all in all, we need a new RC. The only blocker I found was the missing jquery license file.
Another somewhat critical thing is that "statefun-flink-distribution" bundles many unwanted dependencies. - Because the shading merges the notice files, this is not a legal issue. - Because Flinks inverted classloading still uses "parent-first" for all "org.apache.flink.*" classes, this does not break the system But it is unwanted behavior and makes the artifacts unnecessarily large. I opened FLINK-16891 - FLINK-16897 for the issues I found. All issues are fixed in this PR: https://github.com/apache/flink-statefun/pull/85 On Tue, Mar 31, 2020 at 7:17 PM Stephan Ewen <se...@apache.org> wrote: > I have found a few things, am preparing a joint PR to fix them. > > So far, only the missing jquery license would have been a release blocker. > > On Tue, Mar 31, 2020 at 6:24 PM Chesnay Schepler <ches...@apache.org> > wrote: > >> The jquery license is in fact missing from the master/release-1.10 >> branches. https://issues.apache.org/jira/browse/FLINK-16888 >> >> >> On 31/03/2020 12:18, Chesnay Schepler wrote: >> > For Kafka we traditionally exclude the NOTICE file since as far as we >> > can tell it is misleading anyway, see the flink-sql-connector-kafka >> > modules. >> > >> > @Robert for the Flink project the jquery license is in the source at >> > licenses/LICENSE.jquery >> > >> > I'm a bit concerned just how many licensing issues are showing up in >> > these RCs. I would suggest to do a proper scan of the licensing before >> > opening another RC. >> > >> > And yes, the missing MIT license is grounds for cancellation, hence, -1. >> > >> > On 31/03/2020 11:56, Robert Metzger wrote: >> >> Thanks a lot Gordon! >> >> >> >> Checked: >> >> - files in the staging repository seem to be ok (no unexpected files, >> >> versions set correctly, quickstart archetype looks ok) >> >> - statefun-ridesharing-example-simulator-2.0.0.jar (and >> >> >> /org/apache/flink/statefun-flink-distribution/2.0.0/statefun-flink-distribution-2.0.0.jar) >> >> >> >> >> contains a NOTICE file in the root which seems to come from Apache >> >> Kafka. >> >> The file states >> >> >> >>> This distribution has a binary dependency on jersey, which is >> available >> >>> under the CDDL >> >>> License. The source code of jersey can be found at >> >>> https://github.com/jersey/jersey/. >> >> This text is not mentioned in our NOTICE file (which is located in >> >> META-INF/NOTICE). >> >> I'm not a lawyer, but the NOTICE file situation might be confusing in >> >> that >> >> jar. The first NOTICE file you see is from Kafka. If we argue that >> >> this is >> >> not the right file, >> >> because that one is located in META-INF/NOTICE, then we might be at >> >> risk of >> >> not having properly forwarded Kafka's NOTICE file. >> >> I believe this is okay, as we somehow include all the necessary >> >> information, but we should address this in the next release (or if >> >> this RC >> >> gets cancelled again). >> >> I'm also curious to hear the opinion of others on this. >> >> >> >> - The source release contains "docs/page/js/jquery.min.js", which is >> MIT >> >> licensed. The MIT license requires us to ship a copy of the license >> with >> >> each copy of the source. >> >> apache/flink also has this file: >> >> https://github.com/apache/flink/blob/master/docs/page/js/jquery.min.js, >> >> >> but >> >> it ships the jquery license in the "licenses/" folder (even though this >> >> file is not in git, I guess it's added during release generation?!) >> >> >> >> I believe we have to cancel this RC because of the missing license >> >> file in >> >> the source distribution? I'm not voting on this RC, in case I have >> >> overlooked something and we can continue. >> >> >> >> >> >> On Tue, Mar 31, 2020 at 9:31 AM Tzu-Li (Gordon) Tai >> >> <tzuli...@apache.org> >> >> wrote: >> >> >> >>> ======= NOTICE ======= >> >>> >> >>> For your testing, please continue to use this staging area for the >> >>> Maven >> >>> artifacts: >> >>> >> https://repository.apache.org/content/repositories/orgapacheflink-1344/ >> >>> >> >>> The only difference between this staging repo and the original repo >> >>> posted >> >>> in this thread ( >> >>> >> https://repository.apache.org/content/repositories/orgapacheflink-1343/) >> >>> >> >>> is that a few unintended source release distributions have been >> removed >> >>> from the Maven repo staging area. >> >>> Those should not be built and published by Maven, since we use our own >> >>> tools to build the source distributions (staged at >> >>> >> https://dist.apache.org/repos/dist/dev/flink/flink-statefun-2.0.0-rc4/). >> >>> >> >>> >> >>> Since this does not affect any code in the project, and the staged >> >>> Maven >> >>> artifacts are still built with the same commit hash as the source >> >>> distribution, >> >>> this RC vote will continue to run until the original vote end time. >> >>> >> >>> All previous votes in this thread will still be accounted for. >> >>> >> >>> On Tue, Mar 31, 2020 at 2:57 PM Tzu-Li (Gordon) Tai >> >>> <tzuli...@apache.org> >> >>> wrote: >> >>> >> >>>> Sounds good, I'll post a new link to this vote thread, which will >> have >> >>> the >> >>>> problem fixed in a new maven staging repository. >> >>>> >> >>>> On Tue, Mar 31, 2020 at 2:51 PM Robert Metzger <rmetz...@apache.org> >> >>>> wrote: >> >>>> >> >>>>> Thank you for looking into this. >> >>>>> >> >>>>> I'm fine with keeping this RC open, but re-vote on a new maven >> >>>>> staging >> >>>>> repository. >> >>>>> >> >>>>> On Tue, Mar 31, 2020 at 8:42 AM Tzu-Li (Gordon) Tai < >> >>> tzuli...@apache.org> >> >>>>> wrote: >> >>>>> >> >>>>>> Found the culprit: >> >>>>>> >> >>>>>> The Stateful Functions project uses the Apache POM as the parent >> >>>>>> POM, >> >>>>> and >> >>>>>> uses the `apache-release` build profile to build the staging jars. >> >>>>>> >> >>>>>> The problem arises because the `apache-release` build profile >> itself >> >>>>>> bundles a source release distribution to be released to Maven. >> >>>>>> This should be disabled specifically for us, because we use our own >> >>>>> tooling >> >>>>>> (tools/releasing/create_source_release.sh) to create the source >> >>> tarballs >> >>>>>> which does correctly exclude all those unexpected files Robert >> >>>>>> found. >> >>>>>> >> >>>>>> Will rebuild the RC. I think in this case, it's completely fine to >> >>> keep >> >>>>>> with the original voting end time, since nothing is really touched, >> >>> only >> >>>>>> excluding some files from the staging Maven repository. >> >>>>>> >> >>>>>> On Tue, Mar 31, 2020 at 2:29 PM Tzu-Li (Gordon) Tai < >> >>>>> tzuli...@apache.org> >> >>>>>> wrote: >> >>>>>> >> >>>>>>> Hi Robert, >> >>>>>>> >> >>>>>>> I think you're right. There should be no tarballs / jars packaged >> >>> for >> >>>>>>> statefun-parent actually, only the pom file since that's the >> parent >> >>>>>> module >> >>>>>>> which only has pom packaging. >> >>>>>>> I'm looking into it. >> >>>>>>> >> >>>>>>> On Tue, Mar 31, 2020 at 2:23 PM Robert Metzger < >> rmetz...@apache.org >> >>>>>>> wrote: >> >>>>>>> >> >>>>>>>> While checking the release, I found a 77 >> >>>>>>>> MB statefun-parent-2.0.0-source-release.zip file in the maven >> >>> staging >> >>>>>>>> repo: >> >>>>>>>> >> >>>>>>>> >> >>> >> https://repository.apache.org/content/repositories/orgapacheflink-1343/org/apache/flink/statefun-parent/2.0.0/ >> >>> >> >>>>>>>> It seems that the file contains all ruby dependencies in docs/ >> >>>>>>>> from >> >>>>>> jekyll >> >>>>>>>> for the docs (in >> >>> "statefun-parent-2.0.0/docs/.rubydeps/ruby/2.5.0"). >> >>>>> I >> >>>>>>>> don't think we want to publish these files as part of the release >> >>> to >> >>>>>> maven >> >>>>>>>> central? >> >>>>>>>> (It also contains python venv files in >> "statefun-python-sdk/venv") >> >>>>>>>> >> >>>>>>>> I guess this is a reason to cancel the RC? >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> On Tue, Mar 31, 2020 at 6:10 AM Tzu-Li (Gordon) Tai < >> >>>>>> tzuli...@apache.org> >> >>>>>>>> wrote: >> >>>>>>>> >> >>>>>>>>> +1 (binding) >> >>>>>>>>> >> >>>>>>>>> ** Legal ** >> >>>>>>>>> - checksums and GPG files match corresponding release files >> >>>>>>>>> - Source distribution does not contain binaries, contents are >> >>> sane >> >>>>> (no >> >>>>>>>>> .git* / .travis* / generated html content files) >> >>>>>>>>> - Bundled source LICENSEs and NOTICE looks good. Mentions >> bundled >> >>>>>>>>> font-awesome dependency in docs and copied sources from fastutil >> >>> ( >> >>>>>>>>> http://fastutil.di.unimi.it/) >> >>>>>>>>> - Bundled LICENSEs and NOTICE files for Maven artifacts looks >> >>> good. >> >>>>>>>>> Artifacts that do bundle dependencies are: >> >>>>>> statefun-flink-distribution, >> >>>>>>>>> statefun-ridesharing-example-simulator, statefun-flink-core >> >>> (copied >> >>>>>>>>> sources). >> >>>>>>>>> - Python SDK distributions (source and wheel) contain ASLv2 >> >>> LICENSE >> >>>>>> and >> >>>>>>>>> NOTICE files (no bundled dependencies) >> >>>>>>>>> - All POMs / README / Python SDK setup.py / Dockerfiles / doc >> >>>>> configs >> >>>>>>>> point >> >>>>>>>>> to same version “2.0.0” >> >>>>>>>>> - README looks good >> >>>>>>>>> >> >>>>>>>>> ** Functional ** >> >>>>>>>>> - Building from source dist with end-to-end tests enabled (mvn >> >>>>> clean >> >>>>>>>> verify >> >>>>>>>>> -Prun-e2e-tests) passes (JDK 8) >> >>>>>>>>> - Generated quickstart from archetype looks good (correct POM / >> >>>>>>>> Dockerfile >> >>>>>>>>> / service file) >> >>>>>>>>> - Examples run: Java Greeter / Java Ridesharing / Python Greeter >> >>> / >> >>>>>>>> Python >> >>>>>>>>> SDK Walkthrough >> >>>>>>>>> - Flink Harness works in IDE >> >>>>>>>>> - Test remote functions deployment mode with AWS ecosystem: >> >>> remote >> >>>>>>>> Python >> >>>>>>>>> functions running in AWS Lambda behind AWS API Gateway, Java >> >>>>> embedded >> >>>>>>>>> functions running in AWS ECS >> >>>>>>>>> >> >>>>>>>>> On Tue, Mar 31, 2020 at 12:09 PM Tzu-Li (Gordon) Tai < >> >>>>>>>> tzuli...@apache.org> >> >>>>>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>>> FYI - I've also updated the website Downloads page to include >> >>>>> this >> >>>>>>>>> release. >> >>>>>>>>>> Please also consider that for your reviews: >> >>>>>>>>>> https://github.com/apache/flink-web/pull/318 >> >>>>>>>>>> >> >>>>>>>>>> On Tue, Mar 31, 2020 at 3:42 AM Konstantin Knauf < >> >>>>>>>>> konstan...@ververica.com> >> >>>>>>>>>> wrote: >> >>>>>>>>>> >> >>>>>>>>>>> Hi Gordon, >> >>>>>>>>>>> >> >>>>>>>>>>> +1 (non-binding) >> >>>>>>>>>>> >> >>>>>>>>>>> * Maven build from source...check >> >>>>>>>>>>> * Python build from source...check >> >>>>>>>>>>> * Went through Walkthrough based on local builds...check >> >>>>>>>>>>> >> >>>>>>>>>>> Cheers, >> >>>>>>>>>>> >> >>>>>>>>>>> Konstantin >> >>>>>>>>>>> >> >>>>>>>>>>> On Mon, Mar 30, 2020 at 5:52 AM Tzu-Li (Gordon) Tai < >> >>>>>>>>> tzuli...@apache.org> >> >>>>>>>>>>> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>>> Hi everyone, >> >>>>>>>>>>>> >> >>>>>>>>>>>> Please review and vote on the *release candidate #4* for the >> >>>>>>>> version >> >>>>>>>>>>> 2.0.0 >> >>>>>>>>>>>> of Apache Flink Stateful Functions, >> >>>>>>>>>>>> as follows: >> >>>>>>>>>>>> [ ] +1, Approve the release >> >>>>>>>>>>>> [ ] -1, Do not approve the release (please provide specific >> >>>>>>>> comments) >> >>>>>>>>>>>> **Testing Guideline** >> >>>>>>>>>>>> >> >>>>>>>>>>>> You can find here [1] a doc that we can use for >> >>> collaborating >> >>>>>>>> testing >> >>>>>>>>>>>> efforts. >> >>>>>>>>>>>> The listed testing tasks in the doc also serve as a >> >>> guideline >> >>>>> in >> >>>>>>>> what >> >>>>>>>>> to >> >>>>>>>>>>>> test for this release. >> >>>>>>>>>>>> If you wish to take ownership of a testing task, simply put >> >>>>> your >> >>>>>>>> name >> >>>>>>>>>>> down >> >>>>>>>>>>>> in the "Checked by" field of the task. >> >>>>>>>>>>>> >> >>>>>>>>>>>> **Release Overview** >> >>>>>>>>>>>> >> >>>>>>>>>>>> As an overview, the release consists of the following: >> >>>>>>>>>>>> a) Stateful Functions canonical source distribution, to be >> >>>>>>>> deployed to >> >>>>>>>>>>> the >> >>>>>>>>>>>> release repository at dist.apache.org >> >>>>>>>>>>>> b) Stateful Functions Python SDK distributions to be >> >>> deployed >> >>>>> to >> >>>>>>>> PyPI >> >>>>>>>>>>>> c) Maven artifacts to be deployed to the Maven Central >> >>>>> Repository >> >>>>>>>>>>>> **Staging Areas to Review** >> >>>>>>>>>>>> >> >>>>>>>>>>>> The staging areas containing the above mentioned artifacts >> >>>>> are as >> >>>>>>>>>>> follows, >> >>>>>>>>>>>> for your review: >> >>>>>>>>>>>> * All artifacts for a) and b) can be found in the >> >>>>> corresponding >> >>>>>> dev >> >>>>>>>>>>>> repository at dist.apache.org [2] >> >>>>>>>>>>>> * All artifacts for c) can be found at the Apache Nexus >> >>>>>> Repository >> >>>>>>>> [3] >> >>>>>>>>>>>> All artifacts are singed with the >> >>>>>>>>>>>> key 1C1E2394D3194E1944613488F320986D35C33D6A [4] >> >>>>>>>>>>>> >> >>>>>>>>>>>> Other links for your review: >> >>>>>>>>>>>> * JIRA release notes [5] >> >>>>>>>>>>>> * source code tag "release-2.0.0-rc4" [6] [7] >> >>>>>>>>>>>> >> >>>>>>>>>>>> **Extra Remarks** >> >>>>>>>>>>>> >> >>>>>>>>>>>> * Part of the release is also official Docker images for >> >>>>> Stateful >> >>>>>>>>>>>> Functions. This can be a separate process, since the >> >>> creation >> >>>>> of >> >>>>>>>> those >> >>>>>>>>>>>> relies on the fact that we have distribution jars already >> >>>>>> deployed >> >>>>>>>> to >> >>>>>>>>>>>> Maven. I will follow-up with this after these artifacts are >> >>>>>>>> officially >> >>>>>>>>>>>> released. >> >>>>>>>>>>>> In the meantime, there is this discussion [8] ongoing about >> >>>>> where >> >>>>>>>> to >> >>>>>>>>>>> host >> >>>>>>>>>>>> the StateFun Dockerfiles. >> >>>>>>>>>>>> * The Flink Website and blog post is also being worked on >> >>> (by >> >>>>>>>> Marta) >> >>>>>>>>> as >> >>>>>>>>>>>> part of the release, to incorporate the new Stateful >> >>> Functions >> >>>>>>>>> project. >> >>>>>>>>>>> We >> >>>>>>>>>>>> can follow up with a link to those changes afterwards in >> >>> this >> >>>>>> vote >> >>>>>>>>>>> thread, >> >>>>>>>>>>>> but that would not block you to test and cast your votes >> >>>>> already. >> >>>>>>>>>>>> * Since the Flink website changes are still being worked on, >> >>>>> you >> >>>>>>>> will >> >>>>>>>>>>> not >> >>>>>>>>>>>> yet be able to find the Stateful Functions docs from there. >> >>>>> Here >> >>>>>>>> are >> >>>>>>>>> the >> >>>>>>>>>>>> links [9] [10]. >> >>>>>>>>>>>> >> >>>>>>>>>>>> **Vote Duration** >> >>>>>>>>>>>> >> >>>>>>>>>>>> Since this RC only fixes licensing issues from previous RCs, >> >>>>>>>>>>>> and the code itself has not been touched, >> >>>>>>>>>>>> I'd like to stick with the original vote ending time. >> >>>>>>>>>>>> >> >>>>>>>>>>>> The vote will be open for at least 72 hours starting Monday >> >>>>>>>>>>>> *(target end date is Wednesday, April 1st).* >> >>>>>>>>>>>> It is adopted by majority approval, with at least 3 PMC >> >>>>>> affirmative >> >>>>>>>>>>> votes. >> >>>>>>>>>>>> Thanks, >> >>>>>>>>>>>> Gordon >> >>>>>>>>>>>> >> >>>>>>>>>>>> [1] >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>> >> https://docs.google.com/document/d/1P9yjwSbPQtul0z2AXMnVolWQbzhxs68suJvzR6xMjcs/edit?usp=sharing >> >>> >> >>>>>>>>>>>> [2] >> >>>>> >> https://dist.apache.org/repos/dist/dev/flink/flink-statefun-2.0.0-rc4/ >> >>>>> >> >>>>>>>>>>>> [3] >> >>>>>>>>>>>> >> >>> >> https://repository.apache.org/content/repositories/orgapacheflink-1343/ >> >>>>>>>>>>>> [4] https://dist.apache.org/repos/dist/release/flink/KEYS >> >>>>>>>>>>>> [5] >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>> >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12346878 >> >>> >> >>>>>>>>>>>> [6] >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>> >> https://gitbox.apache.org/repos/asf?p=flink-statefun.git;a=commit;h=5d5d62fca2dbe3c75e8157b7ce67d4d4ce12ffd9 >> >>> >> >>>>>>>>>>>> [7] >> >>>>>>>> https://github.com/apache/flink-statefun/tree/release-2.0.0-rc4 >> >>>>>>>>>>>> [8] >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>> >> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Creating-a-new-repo-to-host-Stateful-Functions-Dockerfiles-td39342.html >> >>> >> >>>>>>>>>>>> [9] >> >>>>>>>> https://ci.apache.org/projects/flink/flink-statefun-docs-master/ >> >>>>>>>>>>>> [10] >> >>>>> >> https://ci.apache.org/projects/flink/flink-statefun-docs-release-2.0/ >> >>>>>>>>>>>> TIP: You can create a `settings.xml` file with these >> >>> contents: >> >>>>>>>>>>>> """ >> >>>>>>>>>>>> <settings> >> >>>>>>>>>>>> <activeProfiles> >> >>>>>>>>>>>> <activeProfile>flink-statefun-2.0.0</activeProfile> >> >>>>>>>>>>>> </activeProfiles> >> >>>>>>>>>>>> <profiles> >> >>>>>>>>>>>> <profile> >> >>>>>>>>>>>> <id>flink-statefun-2.0.0</id> >> >>>>>>>>>>>> <repositories> >> >>>>>>>>>>>> <repository> >> >>>>>>>>>>>> <id>flink-statefun-2.0.0</id> >> >>>>>>>>>>>> <url> >> >>>>>>>>>>>> >> >>> >> https://repository.apache.org/content/repositories/orgapacheflink-1343/ >> >>>>>>>>>>>> </url> >> >>>>>>>>>>>> </repository> >> >>>>>>>>>>>> <repository> >> >>>>>>>>>>>> <id>archetype</id> >> >>>>>>>>>>>> <url> >> >>>>>>>>>>>> >> >>> >> https://repository.apache.org/content/repositories/orgapacheflink-1343/ >> >>>>>>>>>>>> </url> >> >>>>>>>>>>>> </repository> >> >>>>>>>>>>>> </repositories> >> >>>>>>>>>>>> </profile> >> >>>>>>>>>>>> </profiles> >> >>>>>>>>>>>> </settings> >> >>>>>>>>>>>> """ >> >>>>>>>>>>>> >> >>>>>>>>>>>> And reference that in you maven commands via `--settings >> >>>>>>>>>>>> path/to/settings.xml`. >> >>>>>>>>>>>> This is useful for creating a quickstart based on the staged >> >>>>>>>> release >> >>>>>>>>> and >> >>>>>>>>>>>> for building against the staged jars. >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> -- >> >>>>>>>>>>> >> >>>>>>>>>>> Konstantin Knauf | Head of Product >> >>>>>>>>>>> >> >>>>>>>>>>> +49 160 91394525 >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> Follow us @VervericaData Ververica < >> >>> https://www.ververica.com/> >> >>>>>>>>>>> >> >>>>>>>>>>> -- >> >>>>>>>>>>> >> >>>>>>>>>>> Join Flink Forward <https://flink-forward.org/> - The Apache >> >>>>> Flink >> >>>>>>>>>>> Conference >> >>>>>>>>>>> >> >>>>>>>>>>> Stream Processing | Event Driven | Real Time >> >>>>>>>>>>> >> >>>>>>>>>>> -- >> >>>>>>>>>>> >> >>>>>>>>>>> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany >> >>>>>>>>>>> >> >>>>>>>>>>> -- >> >>>>>>>>>>> Ververica GmbH >> >>>>>>>>>>> Registered at Amtsgericht Charlottenburg: HRB 158244 B >> >>>>>>>>>>> Managing Directors: Timothy Alexander Steinert, Yip Park Tung >> >>>>>> Jason, >> >>>>>>>> Ji >> >>>>>>>>>>> (Tony) Cheng >> >>>>>>>>>>> >> > >> > >> >>
