By now, I'm reasonably sure that the test instabilities on the end-to-end test are only instabilities. I pushed changes to increase timeouts to make the tests more stable. As in any project, there will always be bugs but I think we could release this RC4 and be reasonably sure that it works well.
Now, we only need to have the required number of PMC votes. On Wed, Mar 27, 2019, at 07:22, Congxian Qiu wrote: > +1 （non-binding) > > • checked signature and checksum ok > • mvn clean package -DskipTests ok > • Run job on yarn ok > • Test state migration with POJO type (both heap and rocksdb) ok > • - 1.6 -> 1.8 > • - 1.7 -> 1.8 > • - 1.8 -> 1.8 > > > Best, Congxian > On Mar 27, 2019, 10:26 +0800, vino yang <yanghua1...@gmail.com>, wrote: > > +1 (non-binding) > > > > - checked JIRA release note > > - ran "mvn package -DskipTests" > > - checked signature and checksum > > - started a cluster locally and ran some examples in binary > > - checked web site announcement's PR > > > > Best, > > Vino > > > > > > Xiaowei Jiang <xiaow...@gmail.com> 于2019年3月26日周二 下午8:20写道： > > > > > +1 (non-binding) > > > > > > - checked checksums and GPG files > > > - build from source successfully- run end-to-end precommit tests > > > successfully- run end-to-end nightly tests successfully > > > Xiaowei > > > On Tuesday, March 26, 2019, 8:09:19 PM GMT+8, Yu Li <car...@gmail.com> > > > wrote: > > > > > > +1 (non-binding) > > > > > > - Checked release notes: OK > > > - Checked sums and signatures: OK > > > - Source release > > > - contains no binaries: OK > > > - contains no 1.8-SNAPSHOT references: OK > > > - build from source: OK (8u101) > > > - mvn clean verify: OK (8u101) > > > - Binary release > > > - no examples appear to be missing > > > - started a cluster; WebUI reachable, example ran successfully > > > - end-to-end test (all but K8S and docker ones): OK (8u101) > > > - Repository appears to contain all expected artifacts > > > > > > Best Regards, > > > Yu > > > > > > > > > On Tue, 26 Mar 2019 at 14:28, Kurt Young <ykt...@gmail.com> wrote: > > > > > > > +1 (non-binding) > > > > > > > > Checked items: > > > > - checked checksums and GPG files > > > > - verified that the source archives do not contains any binaries > > > > - checked that all POM files point to the same version > > > > - build from source successfully > > > > > > > > Best, > > > > Kurt > > > > > > > > > > > > On Tue, Mar 26, 2019 at 10:57 AM Shaoxuan Wang <wshaox...@gmail.com> > > > > wrote: > > > > > > > > > +1 (non-binding) > > > > > > > > > > I tested RC4 with the following items: > > > > > - Maven Central Repository contains all artifacts > > > > > - Built the source with Maven (ensured all source files have Apache > > > > > headers), and executed built-in tests via "mvn clean verify" > > > > > - Manually executed the tests in IntelliJ IDE > > > > > - Verify that the quickstarts for Scala and Java are working with the > > > > > staging repository in IntelliJ > > > > > - Checked the benchmark results. The perf regression of > > > > > tuple-key-by/statebackend/tumblingWindow are gone, but the regression > > > on > > > > > serializer still exists. > > > > > > > > > > Regards, > > > > > Shaoxuan > > > > > > > > > > On Tue, Mar 26, 2019 at 8:06 AM jincheng sun <sunjincheng...@gmail.com > > > > > > > > > wrote: > > > > > > > > > > > Hi Aljoscha, I think you are right, increase the timeout config will > > > > fix > > > > > > this issue. this depends on the resource of Travis. I would like > > > share > > > > > > some phenomenon during my test (not the flink problem) as follows: > > > :-) > > > > > > > > > > > > During my testing, `mvn clean verify` and `nightly end-to-end test ` > > > > both > > > > > > consume a lot of machine resources (especially memory/network), and > > > the > > > > > > network bandwidth requirements of `nightly end-to-end test ` are > > > > > > also > > > > > very > > > > > > high. In China, need to use VPN acceleration (100~200Kb before > > > > > > acceleration, 3~4Mb after acceleration), I have encountered: [Avro > > > > > > Confluent Schema Registry nightly end-to-end test' failed after 18 > > > > > minutes > > > > > > and 15 seconds! Test exited with exit Code 1] takes more than 18 > > > > minutes, > > > > > > the download failed because the network bandwidth is not enough. and > > > it > > > > > > runs smoothly when using VPN acceleration. The overall end-to-end > > > > > > run > > > > was > > > > > > passed twice. The Docker resource configuration (CUPs 7, Mem: 28.7G, > > > > > Swap: > > > > > > 3.5G). See detail log here > > > > > > < > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1CcyTCyZmMmP57pkKv4drjSuxW61_u78HR3q1fJJODMw/edit?usp=sharing > > > > > > > > > > > > > . > > > > > > > > > > > > Just now, I had checked the Travis for your last commit (Increase > > > > startup > > > > > > timeout in end-to-end tests), in addition to the Cleanup phase, > > > > > > other > > > > > > phases are successful. here > > > > > > <https://travis-ci.org/apache/flink/builds/511071777> > > > > > > > > > > > > In order to verify that our speculation is accurate, I can help with > > > 10 > > > > > and > > > > > > 20 seconds timeout config on my repo verification to see if 100% > > > > > recurring > > > > > > timeout problem. It is already running, we are waiting for the > > > result. > > > > > > 10seconds < > > > https://travis-ci.org/sunjincheng121/flink/builds/511235749 > > > > > > > > > > > 20seconds < > > > https://travis-ci.org/sunjincheng121/flink/builds/511235598 > > > > > > > > > > > > > > > > > Best, > > > > > > Jincheng > > > > > > > > > > > > Aljoscha Krettek <aljos...@apache.org> 于2019年3月26日周二 上午1:04写道： > > > > > > > > > > > > > Thanks for the testing done so far! > > > > > > > > > > > > > > There has been quite some flakiness on Travis lately, see here: > > > > > > > https://travis-ci.org/apache/flink/branches < > > > > > > > https://travis-ci.org/apache/flink/branches>. I’m a bit hesitant > > > to > > > > > > > release in this state. Looking at the tests you can see that all > > > > > > > of > > > > the > > > > > > > end-to-end tests fail because waiting for the dispatcher to come > > > > > > > up > > > > > times > > > > > > > out. I also noticed that this usually takes about 5-8 seconds on > > > > > Travis, > > > > > > so > > > > > > > a 10 second timeout might be a bit low. I pushed commits to > > > increase > > > > > that > > > > > > > to 20 secs. Let’s see what will happen. > > > > > > > > > > > > > > I’ll keep you posted! > > > > > > > Aljoscha > > > > > > > > > > > > > > > On 25. Mar 2019, at 13:13, jincheng sun < > > > sunjincheng...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > > Great thanks for preparing the RC4 of Flink 1.8.0, Aljoscha! > > > > > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > > > > > I checked the functional things as follows(Without performance > > > > > > > > verification): > > > > > > > > > > > > > > > > 1. Checking Artifacts： > > > > > > > > > > > > > > > > 1). Download the release source code - SUCCESS > > > > > > > > 2). Check Source release flink-1.8.0-src.tgz.sha512 - SUCCESS > > > > > > > > 3). Download the released JAR - SUCCESS > > > > > > > > 4). Check if checksums and GPG files match the corresponding > > > > > release > > > > > > > > files - SUCCESS. > > > > > > > > 5). Verify that the source archives do not contain any > > > binaries > > > > - > > > > > > > > SUCCESS. > > > > > > > > 6). Build the source with `mvn clean verify -DskipTests` to > > > > ensure > > > > > > all > > > > > > > > source files have Apache headers - SUCCESS > > > > > > > > 7). Check that all POM files point to the same version - > > > SUCCESS > > > > > > > > 8). Read the `README.md` file to ensure there is nothing > > > > > unexpected > > > > > > - > > > > > > > > SUCCESS > > > > > > > > > > > > > > > > 2. Testing Larger Setups > > > > > > > > > > > > > > > > Cluster Environment：7 nodes, jm 1024m, tm 4096m > > > > > > > > Testing Jobs: WordCount(Batch&Streaming), > > > > > > DataStreamAllroundTestProgram > > > > > > > > > > > > > > > > 1). Use local&hdfs file systems for checkpoints - SUCCESS > > > > > > > > 2). Use hdfs file systems for input/output -SUCCESS > > > > > > > > 3). Run examples on YARN(with or without session) - SUCCESS > > > > > > > > 4). Test failover and recovery. - SUCCESS > > > > > > > > 5). Test incremental&non-incremental checkpoint - SUCCESS > > > > > > > > 6). Test connector - kafka -SUCCESS > > > > > > > > > > > > > > > > 3. Testing Functionality > > > > > > > > > > > > > > > > 1). Built-in tests(linux&mac os) > > > > > > > > - `mvn cealn verify` (some test timeout error and test case > > > > bug > > > > > > see > > > > > > > > FLINK-12001 <https://issues.apache.org/jira/browse/FLINK-12001>, > > > > all > > > > > > of > > > > > > > > them are not the blocker) > > > > > > > > - build for scala 2.11（mvn clean install -P scala-2.11 > > > > > > -DskipTests） > > > > > > > > - SUCCESS > > > > > > > > - Run the scripted nightly end-to-end test - SUCCESS > > > > > > > > > > > > > > > > 2). Quickstarts > > > > > > > > - Verify that the quickstarts for Scala with the staging > > > > > > repository > > > > > > > > in IntelliJ - SUCCESS > > > > > > > > - Verify that the quickstarts for Java with the staging > > > > > repository > > > > > > > in > > > > > > > > IntelliJ - SUCCESS > > > > > > > > > > > > > > > > 3). Simple Starter Experience and Use Cases > > > > > > > > > > > > > > > > - run all examples from IntelliJ IDE - SUCCESS > > > > > > > > - Start a local cluster and verify that the processes - > > > > SUCCESS > > > > > > > > a. Examine the *.out files (should be empty) and the log > > > > files > > > > > > > > (should contain no exceptions) > > > > > > > > b. Test for Linux, MacOS > > > > > > > > c. Shutdown and verify there are no exceptions in the log > > > > > output > > > > > > > > (after shutdown) > > > > > > > > > > > > > > > > - Verify that the examples are running from both ./bin/flink > > > > and > > > > > > > from > > > > > > > > the web-based job submission tool(following items) - SUCCESS > > > > > > > > a. Start multiple task managers in the local cluster > > > > > > > > b. Change the flink-conf.yml to define more than one task > > > > slot > > > > > > (2) > > > > > > > > c. Run the examples with a parallelism > 1 > > > > > > > > d. Examine the log output - no error messages should be > > > > > > > encountered > > > > > > > > > > > > > > > > 4. Review the PR > > > > > > > > - [Add 1.8 Release Blog Post] - Just a reminder, updated the > > > > > > release > > > > > > > > date to correct date before merging. > > > > > > > > > > > > > > > > Cheers, > > > > > > > > Jincheng > > > > > > > > > > > > > > > > Piotr Nowojski <pi...@ververica.com> 于2019年3月25日周一 下午4:11写道： > > > > > > > > > > > > > > > > > +1 from my side. Previously spotted performance regression > > > > > > > > > seems > > > > to > > > > > be > > > > > > > > > gone, or mostly gone. > > > > > > > > > > > > > > > > > > Piotrek > > > > > > > > > > > > > > > > > > > On 21 Mar 2019, at 17:52, Aljoscha Krettek < > > > aljos...@apache.org> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > Please review and vote on the release candidate 4 for Flink > > > > 1.8.0, > > > > > as > > > > > > > > > follows: > > > > > > > > > > [ ] +1, Approve the release > > > > > > > > > > [ ] -1, Do not approve the release (please provide specific > > > > > comments) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The complete staging area is available for your review, > > > > > > > > > > which > > > > > > includes: > > > > > > > > > > * JIRA release notes [1], > > > > > > > > > > * the official Apache source release and binary convenience > > > > > releases > > > > > > to > > > > > > > > > be deployed to dist.apache.org [2], which are signed with the > > > key > > > > > > with > > > > > > > > > fingerprint F2A67A8047499BBB3908D17AA8F4FD97121D7293 [3], > > > > > > > > > > * all artifacts to be deployed to the Maven Central > > > > > > > > > > Repository > > > > [4], > > > > > > > > > > * source code tag "release-1.8.0-rc4" [5], > > > > > > > > > > * website pull request listing the new release [6] > > > > > > > > > > * website pull request adding announcement blog post [7]. > > > > > > > > > > > > > > > > > > > > The vote will be open for at least 72 hours. It is adopted > > > > > > > > > > by > > > > > > majority > > > > > > > > > approval, with at least 3 PMC affirmative votes. > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Aljoscha > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12344274 > > > > > > > > > > [2] > > > > https://dist.apache.org/repos/dist/dev/flink/flink-1.8.0-rc4/ > > > > > > > > > > [3] https://dist.apache.org/repos/dist/release/flink/KEYS > > > > > > > > > > [4] > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapacheflink-1215 > > > > > > > > > > [5] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://gitbox.apache.org/repos/asf?p=flink.git;a=tag;h=c650befc10c8bb6cc4b007ae250b7b2173046145 > > > > > > > > > > [6] https://github.com/apache/flink-web/pull/180 < > > > > > > > > > https://github.com/apache/flink-web/pull/180> > > > > > > > > > > [7] https://github.com/apache/flink-web/pull/179 < > > > > > > > > > https://github.com/apache/flink-web/pull/179> > > > > > > > > > > > > > > > > > > > > P.S. The difference to the previous RCs is small, you can > > > > > > > > > > fetch > > > > the > > > > > > > tags > > > > > > > > > and do a "git log release-1.8.0-rc1..release-1.8.0-rc4” to see > > > the > > > > > > > > > difference in commits. Its fixes for the issues that led to > > > > > > > > > the > > > > > > > > > cancellation of the previous RCs plus smaller fixes. Most > > > > > > > > > verification/testing that was carried out should apply as is > > > > > > > > > to > > > > this > > > > > > RC. > > > > > > > > > Any functional verification that you did on previous RCs > > > > > > > > > should > > > > > > > therefore > > > > > > > > > easily carry over to this one. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
