[
https://issues.apache.org/jira/browse/AIRFLOW-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16714671#comment-16714671
]
Iuliia Volkova edited comment on AIRFLOW-2774 at 12/10/18 12:45 PM:
--------------------------------------------------------------------
but DirectRunner is not a 'local' version of DataflowRunner, it's just a
different engine that supported by Apache Beam, as SparkRunner or ApexRunner,
and others.
was (Author: xnuinside):
but DirectRunner is not a 'local' version of DataflowRunner, it's just a
different engine that supported by Apache Beam, as SparkRunner or ApexRunner,
or others.
> DataFlowPythonOperator needs to support DirectRunner to speed up end-to-end
> testing of Airflow dag
> --------------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-2774
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2774
> Project: Apache Airflow
> Issue Type: Improvement
> Components: Dataflow
> Affects Versions: 1.9.0
> Reporter: Evgeny Podlepaev
> Priority: Minor
>
> **DataFlowPythonOperator needs to support DirectRunner as a runner option to
> facilitate local end-to-end testing of the entire Airflow dag. Right now if
> DirectRunner is set via job options, the DataFlowHook will wait infinitely
> trying to get status of the remote job which does not exist:
> _DataflowJob(self.get_conn(), variables['project'], name,
> variables['region'], self.poll_sleep).wait_for_done()
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
