Hi everyone, I am writing a test [1] for the support of @RequiresStableInput annotation in Java SDK [2]. In the test I need to have a ParDo make some side effects and then fail. Currently, the only way to persist the side effect during retries without depending on external states is using attempted metrics. However, attempted metrics cannot be accessed from within a DoFn. So I am wondering if we should make an improvement on this by, for example, exposing a job handler to DoFn.
As Luke mentioned in another email thread, this improvement will also be useful since: * cancelling a pipeline from within the pipeline is useful * starting a new job against the existing runner from in a pipeline is useful * accessing attempted metrics to test DoFn's with side effects is useful for error handling testing What do you think? All suggestions are appreciated. Best, Robin [1] https://github.com/apache/beam/pull/6220 [2] https://docs.google.com/document/d/117yRKbbcEdm3eIKB_26BHOJGmHSZl1YNoF0RqWGtqAM/edit#
