[
https://issues.apache.org/jira/browse/AIRFLOW-3410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16702139#comment-16702139
]
ASF GitHub Bot commented on AIRFLOW-3410:
-----------------------------------------
kaxil closed pull request #4249: [AIRFLOW-3410] Add feature to allow Host Key
Change for SSH Op
URL: https://github.com/apache/incubator-airflow/pull/4249
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/airflow/contrib/hooks/ssh_hook.py
b/airflow/contrib/hooks/ssh_hook.py
index c496762187..97e93bbc60 100755
--- a/airflow/contrib/hooks/ssh_hook.py
+++ b/airflow/contrib/hooks/ssh_hook.py
@@ -80,6 +80,7 @@ def __init__(self,
# Default values, overridable from Connection
self.compress = True
self.no_host_key_check = True
+ self.allow_host_key_change = False
self.host_proxy = None
# Placeholder for deprecated __enter__
@@ -110,6 +111,10 @@ def __init__(self,
and\
str(extra_options["no_host_key_check"]).lower() ==
'false':
self.no_host_key_check = False
+ if "allow_host_key_change" in extra_options\
+ and\
+ str(extra_options["allow_host_key_change"]).lower() ==
'true':
+ self.allow_host_key_change = True
if not self.remote_host:
raise AirflowException("Missing required param: remote_host")
@@ -146,8 +151,13 @@ def get_conn(self):
self.log.debug('Creating SSH client for conn_id: %s', self.ssh_conn_id)
client = paramiko.SSHClient()
- client.load_system_host_keys()
+ if not self.allow_host_key_change:
+ self.log.warning('Remote Identification Change is not verified. '
+ 'This wont protect against Man-In-The-Middle
attacks')
+ client.load_system_host_keys()
if self.no_host_key_check:
+ self.log.warning('No Host Key Verification. This wont protect '
+ 'against Man-In-The-Middle attacks')
# Default is RejectPolicy
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
diff --git a/airflow/www/static/connection_form.js
b/airflow/www/static/connection_form.js
index 5a74813d5b..8ed42c2dbe 100644
--- a/airflow/www/static/connection_form.js
+++ b/airflow/www/static/connection_form.js
@@ -61,6 +61,12 @@
'password': 'Auth Token',
}
},
+ ssh: {
+ hidden_fields: ['schema'],
+ relabeling: {
+ 'login': 'Username',
+ }
+ },
}
function connTypeChange(connectionType) {
$("div.form-group").removeClass("hide");
diff --git a/airflow/www_rbac/static/js/connection_form.js
b/airflow/www_rbac/static/js/connection_form.js
index dabd1f7359..09034f8785 100644
--- a/airflow/www_rbac/static/js/connection_form.js
+++ b/airflow/www_rbac/static/js/connection_form.js
@@ -53,6 +53,12 @@ $(document).ready(function () {
'password': 'Auth Token',
}
},
+ ssh: {
+ hidden_fields: ['schema'],
+ relabeling: {
+ 'login': 'Username',
+ }
+ },
};
function connTypeChange(connectionType) {
diff --git a/docs/howto/manage-connections.rst
b/docs/howto/manage-connections.rst
index ec5cce189c..35b5807c9e 100644
--- a/docs/howto/manage-connections.rst
+++ b/docs/howto/manage-connections.rst
@@ -216,7 +216,7 @@ Extra (optional)
}
When specifying the connection as URI (in AIRFLOW_CONN_* variable) you
should specify it
- following the standard syntax of DB connections, where extras as passed as
parameters
+ following the standard syntax of DB connections, where extras are passed
as parameters
of the URI (note that all components of the URI should be URL-encoded).
For example:
@@ -248,7 +248,7 @@ Password (required)
Specify the password to connect.
Extra (optional)
- Specify the extra parameters (as json dictionary) that can be used in mysql
+ Specify the extra parameters (as json dictionary) that can be used in
postgres
connection. The following parameters out of the standard python parameters
are supported:
@@ -278,12 +278,12 @@ Extra (optional)
{
"sslmode": "verify-ca",
"sslcert": "/tmp/client-cert.pem",
- "sslca": "/tmp/server-ca.pem'",
+ "sslca": "/tmp/server-ca.pem",
"sslkey": "/tmp/client-key.pem"
}
When specifying the connection as URI (in AIRFLOW_CONN_* variable) you
should specify it
- following the standard syntax of DB connections, where extras as passed as
parameters
+ following the standard syntax of DB connections, where extras are passed
as parameters
of the URI (note that all components of the URI should be URL-encoded).
For example:
@@ -321,7 +321,7 @@ Password (required)
Specify the password to connect.
Extra (optional)
- Specify the extra parameters (as json dictionary) that can be used in mysql
+ Specify the extra parameters (as json dictionary) that can be used in
gcpcloudsql
connection.
Details of all the parameters supported in extra field can be found in
@@ -341,7 +341,7 @@ Extra (optional)
}
When specifying the connection as URI (in AIRFLOW_CONN_* variable) you
should specify it
- following the standard syntax of DB connections, where extras as passed as
parameters
+ following the standard syntax of DB connections, where extras are passed
as parameters
of the URI (note that all components of the URI should be URL-encoded).
For example:
@@ -350,3 +350,51 @@ Extra (optional)
gcpcloudsql://user:[email protected]:3306/mydb?database_type=mysql&project_id=example-project&location=europe-west1&instance=testinstance&use_proxy=True&sql_proxy_use_tcp=False
+SSH
+~~~
+The SSH connection type provides connection to use
:class:`~airflow.contrib.hooks.ssh_hook.SSHHook` to run commands on a remote
server using :class:`~airflow.contrib.operators.ssh_operator.SSHOperator` or
transfer file from/to the remote server using
:class:`~airflow.contrib.operators.ssh_operator.SFTPOperator`.
+
+Configuring the Connection
+''''''''''''''''''''''''''
+Host (required)
+ The Remote host to connect.
+
+Username (optional)
+ The Username to connect to the remote_host.
+
+Password (optional)
+ Specify the password of the username to connect to the remote_host.
+
+Port (optional)
+ Port of remote host to connect. Default is 22.
+
+Extra (optional)
+ Specify the extra parameters (as json dictionary) that can be used in ssh
+ connection. The following parameters out of the standard python parameters
+ are supported:
+
+ * **timeout** - An optional timeout (in seconds) for the TCP connect.
Default is ``10``.
+ * **compress** - ``true`` to ask the remote client/server to compress
traffic; `false` to refuse compression. Default is ``true``.
+ * **no_host_key_check** - Set to ``false`` to restrict connecting to hosts
with no entries in ``~/.ssh/known_hosts`` (Hosts file). This provides maximum
protection against trojan horse attacks, but can be troublesome when the
``/etc/ssh/ssh_known_hosts`` file is poorly maintained or connections to new
hosts are frequently made. This option forces the user to manually add all new
hosts. Default is ``true``, ssh will automatically add new host keys to the
user known hosts files.
+ * **allow_host_key_change** - Set to ``true`` if you want to allow
connecting to hosts that has host key changed or when you get 'REMOTE HOST
IDENTIFICATION HAS CHANGED' error. This wont protect against Man-In-The-Middle
attacks. Other possible solution is to remove the host entry from
``~/.ssh/known_hosts`` file. Default is ``false``.
+
+ Example "extras" field:
+
+ .. code-block:: json
+
+ {
+ "timeout": "10",
+ "compress": "false",
+ "no_host_key_check": "false",
+ "allow_host_key_change": "false"
+ }
+
+ When specifying the connection as URI (in AIRFLOW_CONN_* variable) you
should specify it
+ following the standard syntax of connections, where extras are passed as
parameters
+ of the URI (note that all components of the URI should be URL-encoded).
+
+ For example:
+
+ .. code-block:: bash
+
+
ssh://user:pass@localhost:22?timeout=10&compress=false&no_host_key_check=false&allow_host_key_change=true
diff --git a/tests/contrib/hooks/test_ssh_hook.py
b/tests/contrib/hooks/test_ssh_hook.py
index ad5621fe92..64b015d09e 100644
--- a/tests/contrib/hooks/test_ssh_hook.py
+++ b/tests/contrib/hooks/test_ssh_hook.py
@@ -132,15 +132,18 @@ def test_tunnel_without_password(self, ssh_mock):
def test_conn_with_extra_parameters(self):
db.merge_conn(
- models.Connection(conn_id='ssh_with_extra',
- host='localhost',
- conn_type='ssh',
- extra='{"compress" : true, "no_host_key_check" :
"true"}'
- )
+ models.Connection(
+ conn_id='ssh_with_extra',
+ host='localhost',
+ conn_type='ssh',
+ extra='{"compress" : true, "no_host_key_check" : "true", '
+ '"allow_host_key_change": false}'
+ )
)
ssh_hook = SSHHook(ssh_conn_id='ssh_with_extra')
self.assertEqual(ssh_hook.compress, True)
self.assertEqual(ssh_hook.no_host_key_check, True)
+ self.assertEqual(ssh_hook.allow_host_key_change, False)
def test_ssh_connection(self):
hook = SSHHook(ssh_conn_id='ssh_default')
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Remote Host Identification Has Changed Error while using SSHOperator
> --------------------------------------------------------------------
>
> Key: AIRFLOW-3410
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3410
> Project: Apache Airflow
> Issue Type: New Feature
> Components: contrib
> Reporter: Kaxil Naik
> Assignee: Kaxil Naik
> Priority: Minor
> Fix For: 1.10.2
>
>
> Currently, there is no provision in Airflow's implementation of SSHOperator
> to disable Host Checking and use a Null Known Host file to allow connecting
> to host that has a new IP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
