The directory referred to by `blob.storage.directory` is best described as
a local cache. For recovery purposes the JARs are also stored in `
high-availability.storageDir`.At least that's my reading of the code in
1.2. Maybe there's some YARN specific behavior too, sorry if this
information
Stephan,
Regarding your last reply to
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/blob-store-defaults-to-tmp-and-files-get-deleted-td11720.html
You mention "Flink (via the user code class loader) actually holds a reference
to the JAR files in "/tmp", so even if "/tmp" ge
Hi Shannon!
Looking into this a bit more, I think it is something all together
different:
- Flink (via the user code class loader) actually holds a reference to
the JAR files in "/tmp", so even if "/tmp" get wiped, the JAR file remains
usable by the class loader (at least under Linux/Unix/Mac).
Hi Shannon!
In the latest HA and BlobStore changes (1.3) it uses "/tmp" only for
caching and will re-obtain the files from the persistent storage.
I think we should make this a bigger point, even:
- Flink should not use "/tmp" at all (except for mini cluster mode)
- Yarn and Mesos should alwa
Hey Shannon,
good idea! We currently have this:
https://ci.apache.org/projects/flink/flink-docs-release-1.3/ops/production_ready.html
It has a strong focus on managed state and not the points you mentioned.
Would you like to create an issue for adding this to the production
check list? I think i
