Thank you Chesnay and Timo for your answers.
Il 17/05/2017 16:49, Chesnay Schepler ha scritto:
I don't know why we delete it either, but my guess is that at one
point this was a temporary directory that we properly cleaned up, and
later allowed it to be configurable.
Currently, this directory must be in the local file system. We could
change it to also allow non-local paths, which should be fairly easy
to do. Add a condition that the directory is only cleaned up when it
wasn't explicitly configured and we should've covered everything.
In regards to the HistoryServer, i mean of course could we include the
user-jar, but I'm wondering about the user-cases for it.
It's not like you could "just download it and give it to a
JobManager"; you would be missing any additional libraries put under
/lib, as well as the original configuration of the cluster. It would
also require extra steps (and tools) to identify which dependencies
are required.
And there are downsides to this, like a significant increase in the
size of the job archives (along with a duplication of jars for every
job submission), and the archiving also becomes more complex.
On 17.05.2017 15:52, Timo Walther wrote:
Hey Mauro,
I'm not aware of any reason for that. I loop in Chesnay, maybe he
knows why. @Chesnay wouldn't it be helpful to also archive the jars
using the HistoryServer?
Timo
Am 17.05.17 um 12:31 schrieb Mauro Cortellazzi:
Hi Flink comunity,
is there a particular reason to delete the jobmanager.web.upload.dir
content when the JobManager restart? Could it be interesting have
the jar previously uploaded when JM restart? Could the jars be saved
to HDFS for HA mode?
Thank you for your support.
Mauro
