(impala) branch master updated: IMPALA-12530: Make Postgres server to accept remote connections from same subnet

[wzhou]

This is an automated email from the ASF dual-hosted git repository.

wzhou pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git


The following commit(s) were added to refs/heads/master by this push:
     new d9f1271c9 IMPALA-12530: Make Postgres server to accept remote 
connections from same subnet
d9f1271c9 is described below

commit d9f1271c96393245c3022129603d1080b206addc
Author: wzhou-code <wz...@cloudera.com>
AuthorDate: Sun Oct 29 14:35:34 2023 -0700

    IMPALA-12530: Make Postgres server to accept remote connections from same 
subnet
    
    Currently Postgres server in Impala mini-cluster only accepts local
    connections. In dockerised-tests, Impala daemons and Postgres server
    are running on different hosts so Postgres server is not accessible
    for Impala coordinators.
    
    This patch changes the configurations of Postgres server to make it
    accept remote connections from hosts in the same sub network.
    Enables query_test/test_ext_data_sources.py for dockerised-tests.
    
    Testing:
     - Passed dockerised-tests.
     - Passed regular core-tests.
    
    Change-Id: I7dfaf38bf9178cb2ec7ef15c79c17a5ab1e1c6dc
    Reviewed-on: http://gerrit.cloudera.org:8080/20634
    Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
    Reviewed-by: Quanlong Huang <huangquanl...@gmail.com>
    Reviewed-by: Michael Smith <michael.sm...@cloudera.com>
---
 bin/bootstrap_system.sh                   | 10 ++++++++++
 tests/query_test/test_ext_data_sources.py |  8 +-------
 2 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/bin/bootstrap_system.sh b/bin/bootstrap_system.sh
index 2c645c788..ee1427277 100755
--- a/bin/bootstrap_system.sh
+++ b/bin/bootstrap_system.sh
@@ -386,10 +386,20 @@ ubuntu sudo service postgresql stop
 # widely.
 ubuntu sudo sed -ri 's/local +all +all +peer/local all all trust/g' \
   /etc/postgresql/*/main/pg_hba.conf
+# Accept remote connections from the hosts in the same subnet.
+ubuntu sudo sed -ri "s/#listen_addresses = 'localhost'/listen_addresses = 
'0.0.0.0'/g" \
+  /etc/postgresql/*/main/postgresql.conf
+ubuntu sudo sed -ri 's/host +all +all +127.0.0.1\/32/host all all samenet/g' \
+  /etc/postgresql/*/main/pg_hba.conf
 redhat sudo sed -ri 's/local +all +all +(ident|peer)/local all all trust/g' \
   /var/lib/pgsql/data/pg_hba.conf
 # Accept md5 passwords from localhost
 redhat sudo sed -i -e 's,\(host.*\)ident,\1md5,' 
/var/lib/pgsql/data/pg_hba.conf
+# Accept remote connections from the hosts in the same subnet.
+redhat sudo sed -ri "s/#listen_addresses = 'localhost'/listen_addresses = 
'0.0.0.0'/g" \
+  /var/lib/pgsql/data/postgresql.conf
+redhat sudo sed -ri 's/host +all +all +127.0.0.1\/32/host all all samenet/g' \
+  /var/lib/pgsql/data/pg_hba.conf
 
 ubuntu sudo service postgresql start
 redhat notindocker sudo service postgresql start
diff --git a/tests/query_test/test_ext_data_sources.py 
b/tests/query_test/test_ext_data_sources.py
index b3a3a4260..985e07cb2 100644
--- a/tests/query_test/test_ext_data_sources.py
+++ b/tests/query_test/test_ext_data_sources.py
@@ -20,7 +20,7 @@ from __future__ import absolute_import, division, 
print_function
 import re
 
 from tests.common.impala_test_suite import ImpalaTestSuite
-from tests.common.skip import SkipIf, SkipIfDockerizedCluster
+from tests.common.skip import SkipIf
 from tests.common.test_dimensions import create_uncompressed_text_dimension
 from tests.util.filesystem_utils import FILESYSTEM_PREFIX
 
@@ -82,12 +82,6 @@ class TestExtDataSources(ImpalaTestSuite):
   def test_data_source_tables(self, vector, unique_database):
     self.run_test_case('QueryTest/data-source-tables', vector, 
use_db=unique_database)
 
-  # The test uses pre-written jdbc external tables where the jdbc.url refers 
to Postgres
-  # server via full URI, i.e. url starts with 
'jdbc:postgresql://hostname:5432/'. In the
-  # dockerised environment, the Postgres server is running on a different 
host. It is
-  # configured to accept only local connection. Have to skip this test for 
dockerised
-  # cluster since Postgres server is not accessible from impalad.
-  @SkipIfDockerizedCluster.internal_hostname
   @SkipIf.not_hdfs
   def test_jdbc_data_source(self, vector, unique_database):
     self.run_test_case('QueryTest/jdbc-data-source', vector, 
use_db=unique_database)