Great Job on Release 1.6! I just checked it out and still I can see v.1.6-SNAPSHOT on the title of https://ci.apache.org/projects/flink/flink-docs-release-1.6/ <https://ci.apache.org/projects/flink/flink-docs-release-1.6/>
and when I click on any options, it redirects me to master docs 1.7-SNAPSHOT. I opened this ticket https://issues.apache.org/jira/browse/FLINK-10112 <https://issues.apache.org/jira/browse/FLINK-10112> Also I don’t see v1.6 on “Pick Docs Version" drop down Cheers, Yaz > On Aug 8, 2018, at 3:24 PM, Timo Walther <twal...@apache.org> wrote: > > +1 > > - successfully run `mvn clean verify` locally > - successfully run end-to-end tests locally (except for SQL Client end-to-end > test) > > Found a bug in the class loading of SQL JAR files. This is not a blocker but > a bug that we should fix soon. As an easy workaround user should not use > different Kafka versions as SQL Client dependencies. > > Regards, > Timo > > Am 08.08.18 um 18:10 schrieb Dawid Wysakowicz: >> +1 >> >> - verified compilation, tests >> - verified checksum and gpg files >> - verified sbt templates (g8, quickstart) - run assemblies on local cluster >> >> - I could not execute the nightly-tests.sh though. The tests that were >> failing most often are: >> - test_streaming_file_sink.sh >> - test_streaming_elasticsearch.sh >> >> Those are connectors though and it might be only tests flakiness so I >> think it should not block the release. >> >> On 08/08/18 16:36, Chesnay Schepler wrote: >>> I did not use the tools/list_deps.py script as I wasn't aware that it >>> existed. >>> >>> Even if I were I wouldn't have used it and in fact would advocate for >>> removing it. >>> It manually parses and constructs dependency information which is >>> utterly unnecessary as maven already provides this functionality, with >>> the added bonus of also accounting for dependencyManagement and >>> transitive dependencies which we obviously have to take into account. >>> >>> I used this one-liner instead: >>> |mvn dependency:list | ||grep| |":.*:.*:.*"| || ||grep| |-||v| |-e >>> ||"Finished at"| |-e ||"Some problems"| || ||cut| |-d] -f2- | ||sed| >>> |'s/:[a-z]*$//g'| || ||sort| |-u >>> >>> |which I have documented here: >>> https://cwiki.apache.org/confluence/display/FLINK/Dependencies >>> >>> On 08.08.2018 15:06, Aljoscha Krettek wrote: >>>> +1 >>>> >>>> - verified checksum and gpg files >>>> - verified LICENSE and NOTICE: NOTICE didn't change from 1.5, LICENSE >>>> had one unnecessary part removed >>>> >>>> Side comment: I'm not sure whether the "Verify that the LICENSE and >>>> NOTICE file is correct for the binary and source releases" part is >>>> valid anymore because we only have one LICENSE and NOTICE file. also >>>> "The LICENSE and NOTICE files in flink-dist/src/main/flink-bin refer >>>> to the binary distribution and mention all of Flink's Maven >>>> dependencies as well" can be dropped because we don't have them anymore. >>>> >>>> I came to the same conclusion on dependencies. I used >>>> tools/list_deps.py and diff'ed the output for 1.5 and 1.6, that's >>>> probably what Chesnay also did ... :-) >>>> >>>>> On 8. Aug 2018, at 14:43, Chesnay Schepler <ches...@apache.org> wrote: >>>>> >>>>> +1 >>>>> >>>>> - verified source release contains no binaries >>>>> - verified correct versions in source release >>>>> - verified compilation, tests and E2E-tests pass (on travis) >>>>> - verified checksum and gpg files >>>>> >>>>> New dependencies (excluding dependencies where we simply depend on a >>>>> different version now): >>>>> Apache licensed: >>>>> io.confluent:common-utils:jar:3.3.1 >>>>> io.confluent:kafka-schema-registry-client:jar:3.3.1 >>>>> io.prometheus:simpleclient_pushgateway:jar:0.3.0 >>>>> various Apache Nifi dependencies >>>>> various Apache Parquet dependencies >>>>> various ElasticSearch dependencies >>>>> CDDL: >>>>> javax.ws.rs:javax.ws.rs-api:jar:2.1 >>>>> Bouncycastle (MIT-like): >>>>> org.bouncycastle:bcpkix-jdk15on:jar:1.59 >>>>> org.bouncycastle:bcprov-jdk15on:jar:1.59 >>>>> MIT: >>>>> org.projectlombok:lombok:jar:1.16.20 >>>>> >>>>> On 08.08.2018 13:28, Till Rohrmann wrote: >>>>>> Thanks for reporting these problems Chesnay. The usage string in >>>>>> `standalone-job.sh` is out dated and should be updated. The same >>>>>> applies to >>>>>> the typo. >>>>>> >>>>>> When calling `standalone-job.sh start --job-classname foobar.Job` >>>>>> please >>>>>> make sure that the user code jar is contained in the classpath (e.g. >>>>>> putting the jar in the lib directory). Documenting this behaviour >>>>>> is part >>>>>> of the pending issue FLINK-10001. >>>>>> >>>>>> We should fix all of these issues. They are, however, no release >>>>>> blockers. >>>>>> >>>>>> Cheers, >>>>>> Till >>>>>> >>>>>> On Wed, Aug 8, 2018 at 11:31 AM Chesnay Schepler >>>>>> <ches...@apache.org> wrote: >>>>>> >>>>>>> I found some issues with the standalone-job.sh script. >>>>>>> >>>>>>> I ran "./bin/standalone-job.sh start" as described by the usage >>>>>>> string. >>>>>>> >>>>>>> 2018-08-08 09:22:34,385 ERROR >>>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - >>>>>>> Could not parse command line arguments [--configDir, >>>>>>> /home/zento/svn/flink-1.6.0/flink-1.6.0/conf]. >>>>>>> org.apache.flink.runtime.entrypoint.FlinkParseException: >>>>>>> Failed to >>>>>>> parse the command line arguments. >>>>>>> at >>>>>>> >>>>>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52) >>>>>>> >>>>>>> at >>>>>>> >>>>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143) >>>>>>> >>>>>>> Caused by: org.apache.commons.cli.MissingOptionException: >>>>>>> Missing >>>>>>> required option: j >>>>>>> at >>>>>>> >>>>>>> org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199) >>>>>>> >>>>>>> at >>>>>>> org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130) >>>>>>> at >>>>>>> org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81) >>>>>>> at >>>>>>> >>>>>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50) >>>>>>> >>>>>>> ... 1 more >>>>>>> >>>>>>> The script should fail earlier if no jar is provided, with a better >>>>>>> error message. >>>>>>> It is also undocumented, and the usage instructions don't appear >>>>>>> correct. >>>>>>> >>>>>>> Passing a jar with the -j option leads to a ClassNotFoundException: >>>>>>> "./bin/standalone-job.sh start -j examples/streaming/WordCount.jar" >>>>>>> >>>>>>> 2018-08-08 09:26:30,562 ERROR >>>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - >>>>>>> Cluster initialization failed. >>>>>>> java.lang.reflect.UndeclaredThrowableException >>>>>>> at >>>>>>> >>>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1854) >>>>>>> >>>>>>> at >>>>>>> >>>>>>> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) >>>>>>> >>>>>>> at >>>>>>> >>>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:189) >>>>>>> >>>>>>> at >>>>>>> >>>>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:158) >>>>>>> >>>>>>> Caused by: org.apache.flink.util.FlinkException: Could not >>>>>>> load the >>>>>>> provied entrypoint class. >>>>>>> at >>>>>>> >>>>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:92) >>>>>>> >>>>>>> at >>>>>>> >>>>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.retrieveJobGraph(StandaloneJobClusterEntryPoint.java:75) >>>>>>> >>>>>>> at >>>>>>> >>>>>>> org.apache.flink.runtime.entrypoint.JobClusterEntrypoint.createDispatcher(JobClusterEntrypoint.java:107) >>>>>>> >>>>>>> at >>>>>>> >>>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startClusterComponents(ClusterEntrypoint.java:353) >>>>>>> >>>>>>> at >>>>>>> >>>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:232) >>>>>>> >>>>>>> at >>>>>>> >>>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:190) >>>>>>> >>>>>>> at >>>>>>> java.security.AccessController.doPrivileged(Native Method) >>>>>>> at javax.security.auth.Subject.doAs(Subject.java:422) >>>>>>> at >>>>>>> >>>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) >>>>>>> >>>>>>> ... 3 more >>>>>>> Caused by: java.lang.ClassNotFoundException: >>>>>>> examples/streaming/WordCount.jar >>>>>>> at >>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:381) >>>>>>> at >>>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>>>>>> at >>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338) >>>>>>> at >>>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>>> at >>>>>>> >>>>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:89) >>>>>>> >>>>>>> ... 11 more >>>>>>> >>>>>>> So this seems to not work at all, but maybe I'm using it wrong? >>>>>>> >>>>>>> (There's also typo in "Could not load the provied entrypoint class") >>>>>>> >>>>>>> On 08.08.2018 10:33, Piotr Nowojski wrote: >>>>>>>> +1 from my side >>>>>>>> >>>>>>>> I’ve spent some time playing around with various examples (batching, >>>>>>> streaming and SQL) on EMR 6 nodes cluster with yarn deployment, with >>>>>>> different configuration options (number of task >>>>>>> managers/memory/Flip6/credit base flow control/metrics) and >>>>>>> everything >>>>>>> looks now fine (after fixing >>>>>>> https://issues.apache.org/jira/browse/FLINK-9969 < >>>>>>> https://issues.apache.org/jira/browse/FLINK-9969> ). >>>>>>>> Piotrek >>>>>>>> >>>>>>>>> On 7 Aug 2018, at 17:17, Till Rohrmann <trohrm...@apache.org> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hi everyone, >>>>>>>>> Please review and vote on the release candidate #4 for the version >>>>>>> 1.6.0, >>>>>>>>> as follows: >>>>>>>>> [ ] +1, Approve the release >>>>>>>>> [ ] -1, Do not approve the release (please provide specific >>>>>>>>> comments) >>>>>>>>> >>>>>>>>> >>>>>>>>> The complete staging area is available for your review, which >>>>>>>>> includes: >>>>>>>>> * JIRA release notes [1], >>>>>>>>> * the official Apache source release and binary convenience >>>>>>>>> releases to >>>>>>> be >>>>>>>>> deployed to dist.apache.org [2], which are signed with the key with >>>>>>>>> fingerprint 1F302569A96CFFD5 [3], >>>>>>>>> * all artifacts to be deployed to the Maven Central Repository [4], >>>>>>>>> * source code tag "release-1.6.0-rc4" [5], >>>>>>>>> * website pull request listing the new release and adding >>>>>>>>> announcement >>>>>>> blog >>>>>>>>> post [6]. >>>>>>>>> >>>>>>>>> Please use this document for coordinating testing efforts: [7] >>>>>>>>> >>>>>>>>> The vote will be shortened since we only adde a minor fix on top >>>>>>>>> of the >>>>>>> RC >>>>>>>>> 3. It will close on Wednesday 6:30pm CET. It is adopted by majority >>>>>>>>> approval, with at least 3 PMC affirmative votes. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Your friendly Release Manager >>>>>>>>> >>>>>>>>> [1] >>>>>>>>> >>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12342760 >>>>>>> >>>>>>>>> [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.6.0/ >>>>>>>>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS >>>>>>>>> [4] >>>>>>> https://repository.apache.org/content/repositories/orgapacheflink-1178 >>>>>>> >>>>>>>>> [5] https://github.com/apache/flink/tree/release-1.6.0-rc4 >>>>>>>>> [6] https://github.com/apache/flink-web/pull/117 >>>>>>>>> [7] >>>>>>>>> >>>>>>> https://docs.google.com/document/d/1upBFZQ7tbaSkYvDiLqfUFXKg8Xxs-lVheEfb66e4jpo/edit?usp=sharing >>>>>>> >>>>>>>>> Pro-tip: you can create a settings.xml file with these contents: >>>>>>>>> >>>>>>>>> <settings> >>>>>>>>> <activeProfiles> >>>>>>>>> <activeProfile>flink-1.6.0</activeProfile> >>>>>>>>> </activeProfiles> >>>>>>>>> <profiles> >>>>>>>>> <profile> >>>>>>>>> <id>flink-1.6.0</id> >>>>>>>>> <repositories> >>>>>>>>> <repository> >>>>>>>>> <id>flink-1.6.0</id> >>>>>>>>> <url> >>>>>>>>> >>>>>>>>> https://repository.apache.org/content/repositories/orgapacheflink-1178/ >>>>>>>>> >>>>>>>>> </url> >>>>>>>>> </repository> >>>>>>>>> <repository> >>>>>>>>> <id>archetype</id> >>>>>>>>> <url> >>>>>>>>> >>>>>>>>> https://repository.apache.org/content/repositories/orgapacheflink-1178/ >>>>>>>>> >>>>>>>>> </url> >>>>>>>>> </repository> >>>>>>>>> </repositories> >>>>>>>>> </profile> >>>>>>>>> </profiles> >>>>>>>>> </settings> >>>>>>>>> >>>>>>>>> And reference that in you maven commands via --settings >>>>>>>>> path/to/settings.xml. This is useful for creating a quickstart >>>>>>>>> based on >>>>>>> the >>>>>>>>> staged release and for building against the staged jars. >>> >
