Addison Higham created FLINK-9612:
-------------------------------------
Summary: Add option for minimal artifacts being pulled in meseos
Key: FLINK-9612
URL: https://issues.apache.org/jira/browse/FLINK-9612
Project: Flink
Issue Type: Improvement
Components: Docker, Mesos
Reporter: Addison Higham
NOTE: this assumes mesos, but this improvement could also be useful for future
container deployments.
Currently, in mesos, the FlinkDistributionOverlay copies the entire `conf`,
`bin`, and `lib` folders from the running JobManager/ResourceManager. When
using docker with a pre-installed flink distribution, this is relatively
inefficient as it pulls jars that are already baked into the container image.
A new option that disables pulling most (if not all?) of the
FlinkDistributionOverlay could allow for much faster and more scalable
provisions of TaskManagers. As it currently stands, trying to run a few hundred
TaskManagers is likely to result in poor performance in pulling all the
artifacts from the MesosArtifactServer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
