It looks like something failed within your job and the error your getting is from your driver program (not the remote execution that is happening within Google Cloud). You'll want to look at the Stackdriver logs for details, you can get more details about how to see your Stackdriver logs here[1]. You can also reach out to Google Cloud support with a job id for help as well.
1: https://cloud.google.com/dataflow/pipelines/logging On Thu, May 10, 2018 at 12:02 AM Prabeesh K. <[email protected]> wrote: > please somebody have a look. > > On 7 May 2018 at 21:31, Driesprong, Fokko <[email protected]> wrote: > >> I'm sorry, on too many mailinglists >> >> 2018-05-07 19:28 GMT+02:00 Prabeesh K. <[email protected]>: >> >>> This is regarding Apache beam not regarding Airflow. I am using Apache >>> Beam 2.4 version. >>> >>> On 7 May 2018 at 21:22, Driesprong, Fokko <[email protected]> wrote: >>> >>>> Hi Prabeesh, >>>> >>>> Thanks for reaching out. What version are you running and do you have >>>> the full stacktrace? It looks like there is missing a part (I don't see any >>>> references to Airflow). >>>> >>>> Cheers, Fokko >>>> >>>> 2018-05-07 6:42 GMT+02:00 Prabeesh K. <[email protected]>: >>>> >>>>> Hi, >>>>> >>>>> Please help me to fix this error. The job seems running in the >>>>> dataflow console. But getting following assertion error. >>>>> >>>>> >>>>> /local/lib/python2.7/site-packages/apache_beam/io/gcp/gcsio.py:166: >>>>> DeprecationWarning: object() takes no parameters >>>>> super(GcsIO, cls).__new__(cls, storage_client)) >>>>> Traceback (most recent call last): >>>>> File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main >>>>> "__main__", fname, loader, pkg_name) >>>>> File "/usr/lib/python2.7/runpy.py", line 72, in _run_code >>>>> exec code in run_globals >>>>> File "/home/gw/src/legacy/dataflow/cma/report/build.py", line 96, in >>>>> <module> >>>>> run() >>>>> File "/home/gw/src/legacy/dataflow/cma/report/build.py", line 91, in run >>>>> return p.run().wait_until_finish() >>>>> File >>>>> "/home/gw/venvs/dfvenv-0.2.0/local/lib/python2.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", >>>>> line 989, in wait_until_finish >>>>> 'Job did not reach to a terminal state after waiting indefinitely.') >>>>> AssertionError: Job did not reach to a terminal state after waiting >>>>> indefinitely. >>>>> >>>>> >>>>> >>>>> Regards, >>>>> Prabeesh K. >>>>> >>>> >>>> >>> >> >
