I found some issues with the standalone-job.sh script.
I ran "./bin/standalone-job.sh start" as described by the usage string.
2018-08-08 09:22:34,385 ERROR
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
Could not parse command line arguments [--configDir,
/home/zento/svn/flink-1.6.0/flink-1.6.0/conf].
org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to
parse the command line arguments.
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
at
org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
Caused by: org.apache.commons.cli.MissingOptionException: Missing
required option: j
at
org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
at
org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
at
org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
... 1 more
The script should fail earlier if no jar is provided, with a better
error message.
It is also undocumented, and the usage instructions don't appear correct.
Passing a jar with the -j option leads to a ClassNotFoundException:
"./bin/standalone-job.sh start -j examples/streaming/WordCount.jar"
2018-08-08 09:26:30,562 ERROR
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
Cluster initialization failed.
java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1854)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:189)
at
org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:158)
Caused by: org.apache.flink.util.FlinkException: Could not load the
provied entrypoint class.
at
org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:92)
at
org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.retrieveJobGraph(StandaloneJobClusterEntryPoint.java:75)
at
org.apache.flink.runtime.entrypoint.JobClusterEntrypoint.createDispatcher(JobClusterEntrypoint.java:107)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startClusterComponents(ClusterEntrypoint.java:353)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:232)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:190)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
... 3 more
Caused by: java.lang.ClassNotFoundException:
examples/streaming/WordCount.jar
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at
org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:89)
... 11 more
So this seems to not work at all, but maybe I'm using it wrong?
(There's also typo in "Could not load the provied entrypoint class")
On 08.08.2018 10:33, Piotr Nowojski wrote:
+1 from my side
I’ve spent some time playing around with various examples (batching, streaming and
SQL) on EMR 6 nodes cluster with yarn deployment, with different configuration
options (number of task managers/memory/Flip6/credit base flow control/metrics) and
everything looks now fine (after fixing
https://issues.apache.org/jira/browse/FLINK-9969
<https://issues.apache.org/jira/browse/FLINK-9969> ).
Piotrek
On 7 Aug 2018, at 17:17, Till Rohrmann <trohrm...@apache.org> wrote:
Hi everyone,
Please review and vote on the release candidate #4 for the version 1.6.0,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
The complete staging area is available for your review, which includes:
* JIRA release notes [1],
* the official Apache source release and binary convenience releases to be
deployed to dist.apache.org [2], which are signed with the key with
fingerprint 1F302569A96CFFD5 [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag "release-1.6.0-rc4" [5],
* website pull request listing the new release and adding announcement blog
post [6].
Please use this document for coordinating testing efforts: [7]
The vote will be shortened since we only adde a minor fix on top of the RC
3. It will close on Wednesday 6:30pm CET. It is adopted by majority
approval, with at least 3 PMC affirmative votes.
Thanks,
Your friendly Release Manager
[1]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12342760
[2] https://dist.apache.org/repos/dist/dev/flink/flink-1.6.0/
[3] https://dist.apache.org/repos/dist/release/flink/KEYS
[4] https://repository.apache.org/content/repositories/orgapacheflink-1178
[5] https://github.com/apache/flink/tree/release-1.6.0-rc4
[6] https://github.com/apache/flink-web/pull/117
[7]
https://docs.google.com/document/d/1upBFZQ7tbaSkYvDiLqfUFXKg8Xxs-lVheEfb66e4jpo/edit?usp=sharing
Pro-tip: you can create a settings.xml file with these contents:
<settings>
<activeProfiles>
<activeProfile>flink-1.6.0</activeProfile>
</activeProfiles>
<profiles>
<profile>
<id>flink-1.6.0</id>
<repositories>
<repository>
<id>flink-1.6.0</id>
<url>
https://repository.apache.org/content/repositories/orgapacheflink-1178/
</url>
</repository>
<repository>
<id>archetype</id>
<url>
https://repository.apache.org/content/repositories/orgapacheflink-1178/
</url>
</repository>
</repositories>
</profile>
</profiles>
</settings>
And reference that in you maven commands via --settings
path/to/settings.xml. This is useful for creating a quickstart based on the
staged release and for building against the staged jars.
