GitHub user NicoK opened a pull request:
https://github.com/apache/flink/pull/4951
[FLINK-7989][yarn] do not deploy the dist.jar file twice
## What is the purpose of the change
We always add the dist.jar ourselves, but it could also be inside a shipped
folder such as the "lib/" folder and was then distributed multiple times.
This PR is based on #4939 which changed the artefact uploads a bit.
## Brief change log
- also exclude the `flink-dist*.jar` from any (recursive) upload of a
shipped directory, e.g. `lib`
## Verifying this change
This change is already covered by existing tests, such as any test
deploying a program on YARN (if it didn't include the `flink-dist*.jar` file,
it would fail). Additionally, a manual deployment showed that `flink-dist*.jar`
is only uploaded once.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (JavaDocs)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NicoK/flink flink-7989
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4951.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4951
----
commit 6d40434b90d654df55dff3d267721743adc4f586
Author: Nico Kruber <n...@data-artisans.com>
Date: 2016-11-09T20:04:50Z
[FLINK-4228][yarn/s3] fix for yarn staging with s3a defaultFs
+ includes a new unit tests for recursive uploads to hfds:// targets
+ add a unit test for recursive file uploads to s3:// via s3a
commit df9f9b855185bafe89ac09f4accbd603807e3f56
Author: Nico Kruber <n...@data-artisans.com>
Date: 2017-11-03T14:29:07Z
[FLINK-4228][yarn/s3] turn the dependencies around
Instead of having flink-s3-fs-hadoop depend on flink-yarn_<scala_version>,
let flink-yarn depend on the s3 filesystem and implement the test there.
This is safer with regards to the scala-independent flink-s3-fs-hadoop
project.
commit e5488ef719573da0fe86b8355a8b997dd3e3d80a
Author: Nico Kruber <n...@data-artisans.com>
Date: 2017-11-03T20:19:34Z
[FLINK-7989][yarn] do not add the dist.jar file twice
We always add the dist.jar ourselves, but it could also be inside a shipped
folder such as the "lib/" folder and was then distributed multiple times.
----
---
