Luis Angel Vicente Sanchez created AIRFLOW-1409:
---------------------------------------------------
Summary: SparkSubmitHook doesn't generate a valid master URL
Key: AIRFLOW-1409
URL: https://issues.apache.org/jira/browse/AIRFLOW-1409
Project: Apache Airflow
Issue Type: Bug
Reporter: Luis Angel Vicente Sanchez
If you configure the connection using an ENV variable (and possibly through the
UI), SparkSubmitHook is ignoring the scheme and doesn't produce valid master
URL. Example, this ENV variable,
AIRFLOW_CONN_SPARK_CLUSTER=spark://spark-master:7077
will produce the following master URL:
spark-master:7077
This URL is not valid and the job submission fails (the same happens with a
mesos master).
I will create a PR on github that fixes this.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
