Hi there-
How do people track which version of a samza job is running in yarn? The job
name and job id can’t be used, as they are used to create the checkpoint topic,
etc. I’m looking for a way of determining if the current job running in yarn
is the latest version, and if not, kill it and la
I suppose it would be possible to add a custom ‘job.version’ field to the samza
job properties file, and then query for it via the REST /config endpoint on the
ApplicationMananger, but I’m unclear how I find the RPC port for the
ApplicationManager from the ResourceManager. The ResourceManager s
Hey Richard,
The ApplicationReport returned by YarnClient.getApplications() or
getApplicationReport(appId) includes the AM host and rpc port.
https://hadoop.apache.org/docs/r2.7.0/api/org/apache/hadoop/yarn/client/api/YarnClient.html#getApplications()
https://hadoop.apache.org/docs/r2.7.0/api/org
Hmm.. what if you aren’t using Java? I don’t see the RPC port in the REST
ResourceManager application endpoint… just the proxied tracking URL.
It appears that other ApplicationManagers (such as MapReduce) put the REST
endpoint on the same port as the proxied Tracking UI (under a /ws/v1/… path).
As far as I know there is no notion of job version, and you should not run two
instances of a job with the same (job name , job id) pair since it will mess up
the checkpoint and etc. The job id is used to run the same job with different
instances at the same time.
However I think it might be u
> On Sep 19, 2015, at 4:44 PM, Abdollahian Noghabi, Shadi
> wrote:
>
> As far as I know there is no notion of job version, and you should not run
> two instances of a job with the same (job name , job id) pair since it will
> mess up the checkpoint and etc. The job id is used to run the same
The more I look into this, the more I think that the Samza ApplicationManager
is making things difficult by putting its REST interface on the RPC port rather
than somewhere under the tracking URL. From reading the Hadoop ResourceManager
webapp code, it does not look like they expect to expose t
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37817/
---
(Updated Sept. 20, 2015, 4:45 a.m.)
Review request for samza, Yan Fang, Chinmay
