It is your best way to do this right now and this hasn't changed in a while (region was added to project and job ids in the past 6 years).
On Mon, Oct 12, 2020 at 10:53 AM Peter Littig <plit...@nianticlabs.com> wrote: > Thanks for the reply, Kyle. > > The DataflowClient::getJob method uses a Dataflow instance that's provided > at construction time (via DataflowPipelineOptions::getDataflowClient). If > that Dataflow instance can be obtained from a minimal instance of the > options (i.e., containing only the project ID and region) then it looks > like everything should work. > > I suppose a secondary question here is whether or not this approach is the > recommended way to solve my problem (but I don't know of any alternatives). > > On Mon, Oct 12, 2020 at 9:55 AM Kyle Weaver <kcwea...@google.com> wrote: > >> > I think the answer is to use a DataflowClient in the second service, >> but creating one requires DataflowPipelineOptions. Are these options >> supposed to be exactly the same as those used by the first service? Or do >> only some of the fields have to be the same? >> >> Most options are not necessary for retrieving a job. In general, Dataflow >> jobs can always be uniquely identified by the project, region and job ID. >> https://github.com/apache/beam/blob/ecedd3e654352f1b51ab2caae0fd4665403bd0eb/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowClient.java#L100 >> >> On Mon, Oct 12, 2020 at 9:31 AM Peter Littig <plit...@nianticlabs.com> >> wrote: >> >>> Hello, Beam users! >>> >>> Suppose I want to build two (Java) services, one that launches >>> (long-running) dataflow jobs, and the other that monitors the status of >>> dataflow jobs. Within a single service, I could simply track a >>> PipelineResult for each dataflow run and periodically call getState. How >>> can I monitor job status like this from a second, independent service? >>> >>> I think the answer is to use a DataflowClient in the second service, but >>> creating one requires DataflowPipelineOptions. Are these options supposed >>> to be exactly the same as those used by the first service? Or do only some >>> of the fields have to be the same? >>> >>> Or maybe there's a better alternative than DataflowClient? >>> >>> Thanks in advance! >>> >>> Peter >>> >>
