On Wed, 19 Oct 2016 12:04:27 -0400
Robert Haas <robertmh...@gmail.com> wrote:
> On Thu, Oct 13, 2016 at 5:53 AM, Victor Wagner <vi...@wagner.pp.ru>
> wrote:
> > On Thu, 13 Oct 2016 12:30:59 +0530
> > Mithun Cy <mithun...@enterprisedb.com> wrote:
> >> On Fri, Sep 30, 2016 at 2:14 PM, Victor Wagner <vi...@wagner.pp.ru>
> >> wrote:
> >> Okay but for me consistency is also important. Since we agree to
> >> disagree on some of the comments and others have not expressed any
> >> problem I am moving it to committer.
> >
> > Thank you for your efforts improving my patch
>
> I haven't deeply reviewed this patch, but on a quick read-through it
> certainly seems to need a lot of cleanup work.
>
I've did some cleanup, i.e. running pgindent, spell-checking comments
and indenting sgml and attaching cleaned up version here.
I've also re-added test file t/001-multihost.pl which get lost from
previous post.
Thanks for your suggestions.
--
Victor Wagner <vi...@wagner.pp.ru>
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 4e34f00..b7c62d2 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -792,7 +792,7 @@ host=localhost port=5432 dbname=mydb connect_timeout=10
<para>
The general form for a connection <acronym>URI</acronym> is:
<synopsis>
-postgresql://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]
+postgresql://[user[:password]@][netloc][:port][,netloc[:port]...][/dbname][?param1=value1&...]
</synopsis>
</para>
@@ -809,6 +809,7 @@ postgresql://localhost/mydb
postgresql://user@localhost
postgresql://user:secret@localhost
postgresql://other@localhost/otherdb?connect_timeout=10&application_name=myapp
+postgresql://node1,node2:5433,node3:4432,node4/mydb?hostorder=random&target_server_type=any
</programlisting>
Components of the hierarchical part of the <acronym>URI</acronym> can also
be given as parameters. For example:
@@ -831,7 +832,9 @@ postgresql:///mydb?host=localhost&port=5433
<para>
For improved compatibility with JDBC connection <acronym>URI</acronym>s,
instances of parameter <literal>ssl=true</literal> are translated into
- <literal>sslmode=require</literal>.
+ <literal>sslmode=require</literal> and
+ <literal>loadBalanceHosts=true</literal> into
+ <literal>hostorder=random</literal>.
</para>
<para>
@@ -841,6 +844,10 @@ postgresql:///mydb?host=localhost&port=5433
postgresql://[2001:db8::1234]/database
</synopsis>
</para>
+ <para>
+ There can be several host specifications, optionally accompanied
+ with port, separated by comma.
+ </para>
<para>
The host component is interpreted as described for the parameter <xref
@@ -881,6 +888,21 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
when <productname>PostgreSQL</> was built). On machines without
Unix-domain sockets, the default is to connect to <literal>localhost</>.
</para>
+ <para>
+ There can be more than one <literal>host</literal> parameter in
+ the connection string. In this case these hosts would be considered
+ alternate entries into same database and if connection to first one
+ fails, the second would be attemped, and so on. This can be used
+ for high availability clusters or for load balancing. See the
+ <xref linkend="libpq-connect-hostorder"> parameter. This feature
+ works for TCP/IP host names only.
+ </para>
+ <para>
+ The network host name can be accompanied by a port number, separated by
+ colon. This port number is used only when connected to
+ this host. If there is no port number, the port specified in the
+ <xref linkend="libpq-connect-port"> parameter would be used instead.
+ </para>
</listitem>
</varlistentry>
@@ -933,7 +955,6 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
is used to identify the connection in <filename>~/.pgpass</> (see
<xref linkend="libpq-pgpass">).
</para>
-
<para>
Without either a host name or host address,
<application>libpq</application> will connect using a
@@ -943,6 +964,43 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
</listitem>
</varlistentry>
+ <varlistentry id="libpq-connect-hostorder" xreflabel="hostorder">
+ <term><literal>hostorder</literal></term>
+ <listitem>
+ <para>
+ Specifies how to choose the host from the list of alternate hosts,
+ specified in the <xref linkend="libpq-connect-host"> parameter.
+ </para>
+ <para>
+ If the value of this argument is <literal>sequential</literal> (the
+ default) connections to the hosts will be attempted in the order in
+ which they are specified.
+ </para>
+ <para>
+ If the value is <literal>random</literal>, the host to connect to
+ will be randomly picked from the list. It allows load balacing between
+ several cluster nodes. However, PostgreSQL doesn't currently support
+ multimaster clusters. So, without the use of third-party products,
+ only read-only connections can take advantage from
+ load-balancing. See <xref linkend="libpq-connect-target-server-type">
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="libpq-connect-target-server-type" xreflabel="target_server_type">
+ <term><literal>target_server_type</literal></term>
+ <listitem>
+ <para>
+ If this parameter is <literal>master</literal>, upon
+ successful connection the host is checked to determine whether
+ it is in a recovery state. If it is, it then tries next host
+ in the connection string. If this parameter is
+ <literaL>any</literal>, connection to standby nodes are
+ considered successful.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-connect-port" xreflabel="port">
<term><literal>port</literal></term>
<listitem>
@@ -985,7 +1043,6 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
</para>
</listitem>
</varlistentry>
-
<varlistentry id="libpq-connect-connect-timeout" xreflabel="connect_timeout">
<term><literal>connect_timeout</literal></term>
<listitem>
@@ -996,7 +1053,28 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
</para>
</listitem>
</varlistentry>
-
+ <varlistentry id="libpq-connect-falover-timeout" xreflabel="failover_timeout">
+ <term><literal>failover_timeout</literal></term>
+ <listitem>
+ <para>
+ Maximum time to cyclically retry all the hosts in the connection string.
+ (as decimal integer number of seconds). If not specified, then
+ hosts are tried just once.
+ </para>
+ <para>
+ If we have replicating cluster, and master node fails, it might
+ take some time to promote one of the standby nodes to the new master.
+ So clients which detect failure to connect to the master might
+ abandon attempts to reestablish a connection before the new master
+ becomes available.
+ </para>
+ <para>
+ Setting this parameter to a value that takes into account the amount of
+ time needed for failover to complete will ensure attempts to connect
+ to hosts continue to be made until the new master becomes available.
+ </para>
+ </listitem>
+ </varlistentry>
<varlistentry id="libpq-connect-client-encoding" xreflabel="client_encoding">
<term><literal>client_encoding</literal></term>
<listitem>
@@ -7227,6 +7305,19 @@ user=admin
An example file is provided at
<filename>share/pg_service.conf.sample</filename>.
</para>
+ <para>
+ If more than one <literal>host</literal> option is present in the same
+ section of the service file, it is interpeted as alternate servers for
+ failover or load-balancing. See <xref linkend="libpq-connect-host">
+ option in the connection string.
+ </para>
+ <para>
+ For all other options, the first value takes precedence over later ones.
+ </para>
+ <para>
+ Options specified in the connection string, along with the service option,
+ have precedence over values from the service file.
+ </para>
</sect1>
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index b1789eb..4c14cc5 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -36,7 +36,7 @@ OBJS= fe-auth.o fe-connect.o fe-exec.o fe-misc.o fe-print.o fe-lobj.o \
libpq-events.o
# libpgport C files we always use
OBJS += chklocale.o inet_net_ntop.o noblock.o pgstrcasecmp.o pqsignal.o \
- thread.o
+ thread.o pgsleep.o
# libpgport C files that are needed if identified by configure
OBJS += $(filter crypt.o getaddrinfo.o getpeereid.o inet_aton.o open.o system.o snprintf.o strerror.o strlcpy.o win32error.o win32setlocale.o, $(LIBOBJS))
# src/backend/utils/mb
@@ -129,6 +129,9 @@ install: all installdirs install-lib
installcheck:
$(MAKE) -C test $@
+check:
+ $(prove_check)
+
installdirs: installdirs-lib
$(MKDIR_P) '$(DESTDIR)$(includedir)' '$(DESTDIR)$(includedir_internal)' '$(DESTDIR)$(datadir)'
@@ -142,6 +145,7 @@ uninstall: uninstall-lib
clean distclean: clean-lib
$(MAKE) -C test $@
rm -f $(OBJS) pthread.h libpq.rc
+ rm -rf tmp_check
# Might be left over from a Win32 client-only build
rm -f pg_config_paths.h
rm -f inet_net_ntop.c noblock.c pgstrcasecmp.c pqsignal.c thread.c
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index f3a9e5a..67080c5 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -299,7 +299,16 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
{"replication", NULL, NULL, NULL,
"Replication", "D", 5,
offsetof(struct pg_conn, replication)},
-
+ /* Parameters added by failover patch */
+ {"hostorder", NULL, "sequential", NULL,
+ "Host order", "", 10,
+ offsetof(struct pg_conn, hostorder)},
+ {"target_server_type", NULL, NULL, NULL,
+ "Target server type", "", 6,
+ offsetof(struct pg_conn, target_server_type)},
+ {"failover_timeout", NULL, NULL, NULL,
+ "Failover Timeout", "", 10,
+ offsetof(struct pg_conn, failover_timeout)},
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -336,6 +345,7 @@ static PGconn *makeEmptyPGconn(void);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
+static void pqTerminateConn(PGconn *conn);
static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage);
static PQconninfoOption *parse_connection_string(const char *conninfo,
PQExpBuffer errorMessage, bool use_defaults);
@@ -380,6 +390,7 @@ static bool getPgPassFilename(char *pgpassfile);
static void dot_pg_pass_warning(PGconn *conn);
static void default_threadlock(int acquire);
+static int try_next_address(PGconn *conn);
/* global variable because fe-auth.c needs to access it */
pgthreadlock_t pg_g_threadlock = default_threadlock;
@@ -806,7 +817,9 @@ connectOptions2(PGconn *conn)
{
if (conn->pgpass)
free(conn->pgpass);
- conn->pgpass = PasswordFromFile(conn->pghost, conn->pgport,
+ conn->pgpass = PasswordFromFile(
+ conn->actualhost ? conn->actualhost : conn->pghost,
+ conn->actualport ? conn->actualport : conn->pgport,
conn->dbName, conn->pguser);
if (conn->pgpass == NULL)
{
@@ -890,6 +903,22 @@ connectOptions2(PGconn *conn)
}
/*
+ * Validate target_server_mode option.
+ */
+ if (conn->target_server_type)
+ {
+ if (strcmp(conn->target_server_type, "any") != 0
+ && strcmp(conn->target_server_type, "master") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("invalid target_server_type value: \"%s\" should be \"any\" or \"master\"\n"),
+ conn->target_server_type);
+ return false;
+ }
+ }
+
+ /*
* Only if we get this far is it appropriate to try to connect. (We need a
* state flag, rather than just the boolean result of this function, in
* case someone tries to PQreset() the PGconn.)
@@ -1173,6 +1202,8 @@ connectFailureMessage(PGconn *conn, int errorno)
if (conn->pghostaddr && conn->pghostaddr[0] != '\0')
displayed_host = conn->pghostaddr;
+ else if (conn->actualhost && conn->actualhost[0] != '\0')
+ displayed_host = conn->actualhost;
else if (conn->pghost && conn->pghost[0] != '\0')
displayed_host = conn->pghost;
else
@@ -1390,11 +1421,17 @@ setKeepalivesWin32(PGconn *conn)
static int
connectDBStart(PGconn *conn)
{
+ struct nodeinfo
+ {
+ char *host;
+ char *port;
+ };
int portnum;
char portstr[MAXPGPATH];
struct addrinfo *addrs = NULL;
struct addrinfo hint;
- const char *node;
+ struct nodeinfo *nodes,
+ *node;
int ret;
if (!conn)
@@ -1436,21 +1473,183 @@ connectDBStart(PGconn *conn)
if (conn->pghostaddr != NULL && conn->pghostaddr[0] != '\0')
{
/* Using pghostaddr avoids a hostname lookup */
- node = conn->pghostaddr;
+
+ nodes = calloc(sizeof(struct nodeinfo), 2);
+ if (nodes == NULL)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("Out of memory\n"));
+ conn->options_valid = false;
+ goto connect_errReturn;
+ }
+
+ nodes->host = strdup(conn->pghostaddr);
+ if (nodes->host == NULL)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("Out of memory\n"));
+ conn->options_valid = false;
+ goto connect_errReturn;
+ }
+
hint.ai_family = AF_UNSPEC;
hint.ai_flags = AI_NUMERICHOST;
}
else if (conn->pghost != NULL && conn->pghost[0] != '\0')
{
/* Using pghost, so we have to look-up the hostname */
- node = conn->pghost;
+ char *p = conn->pghost,
+ *q,
+ *r;
+ int nodecount = 0,
+ nodesallocated = 4;
+
+ /*
+ * Parse comma-separated list of host-port pairs into function-local
+ * array of records.
+ */
+ nodes = malloc(sizeof(struct nodeinfo) * 4);
+ if (nodes == NULL)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("Out of memory\n"));
+ conn->options_valid = false;
+ goto connect_errReturn;
+ }
+
+ while (*p)
+ {
+ q = p;
+ r = NULL;
+
+ /* Scan for the comma or end of string */
+ while (*q != ',' && *q != 0)
+ {
+ if (*q == ':')
+ r = q;
+ if (*q == ']')
+ r = NULL; /* if there is IPv6, colons before close
+ * bracket are part of address */
+ q++;
+ }
+ if (r)
+ {
+ /* Host has explicitly specified port */
+ char *nptr;
+
+ /* Check if port is numeric */
+ for (nptr = r + 1; nptr < q; nptr++)
+ {
+ if (*nptr < '0' || *nptr > '9')
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("Port is not numeric"));
+ conn->options_valid = false;
+ goto connect_errReturn;
+ }
+ }
+
+ /* Allocate memory for port string */
+ nodes[nodecount].port = malloc(q - r);
+ if (nodes[nodecount].port == NULL)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("Out of memory\n"));
+ conn->options_valid = false;
+ goto connect_errReturn;
+ }
+
+ strncpy(nodes[nodecount].port, r + 1, q - r);
+ nodes[nodecount].port[q - r - 1] = 0;
+ }
+ else
+ {
+ r = q;
+ nodes[nodecount].port = NULL;
+ }
+
+ if ((*p) == '[' && *(r - 1) == ']')
+ {
+ /* IPv6 address found. Strip brackets */
+ p++;
+ r--;
+ }
+
+ /* Fill node record */
+ nodes[nodecount].host = malloc(r - p + 1);
+ if (nodes[nodecount].host == NULL)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("Out of memory\n"));
+ conn->options_valid = false;
+ goto connect_errReturn;
+ }
+
+ strncpy(nodes[nodecount].host, p, r - p);
+ nodes[nodecount].host[r - p] = 0;
+
+ /* skip a comma */
+ if (*q)
+ q++;
+
+ nodecount++;
+ if (nodecount == nodesallocated)
+ nodes = realloc(nodes, sizeof(struct nodeinfo) * (nodesallocated += 4));
+ if (nodes == NULL)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("Out of memory\n"));
+ conn->options_valid = false;
+ goto connect_errReturn;
+ }
+
+ p = q;
+ }
+
+ /* Fill end-of-host list marker */
+ nodes[nodecount].host = NULL;
+ nodes[nodecount].port = NULL;
hint.ai_family = AF_UNSPEC;
+ if (nodecount > 1 && conn->target_server_type == NULL)
+ {
+ /*
+ * if there is more than one host in the connect string and
+ * target_server_type is not specified explicitly, set
+ * target_server_type to "master", because default mode of
+ * operation is failover, and so, we need to connect to RW host,
+ * and keep other nodes of the cluster in the connect string just
+ * in case master would fail and one of standbys would be promoted
+ * to master.
+ *
+ * If we want to loadbalance readonly queries, set
+ * target_server_type = "any" explicitly
+ *
+ * But global default is "any" because if there is only one host
+ * in the connect string, we want backward-compatible behavior.
+ */
+ conn->target_server_type = strdup("master");
+ if (conn->target_server_type == NULL)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("Out of memory\n"));
+ conn->options_valid = false;
+ goto connect_errReturn;
+ }
+ }
}
else
{
#ifdef HAVE_UNIX_SOCKETS
/* pghostaddr and pghost are NULL, so use Unix domain socket */
- node = NULL;
+ nodes = calloc(sizeof(struct nodeinfo), 2);
+ if (nodes == NULL)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("Out of memory\n"));
+ conn->options_valid = false;
+ goto connect_errReturn;
+ }
+
hint.ai_family = AF_UNIX;
UNIXSOCK_PATH(portstr, portnum, conn->pgunixsocket);
if (strlen(portstr) >= UNIXSOCK_PATH_BUFLEN)
@@ -1460,33 +1659,113 @@ connectDBStart(PGconn *conn)
portstr,
(int) (UNIXSOCK_PATH_BUFLEN - 1));
conn->options_valid = false;
+ free(nodes);
+ goto connect_errReturn;
+ }
+
+ nodes->port = strdup(portstr);
+ if (nodes->port == NULL)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("Out of memory\n"));
+ conn->options_valid = false;
goto connect_errReturn;
}
#else
/* Without Unix sockets, default to localhost instead */
- node = DefaultHost;
+ nodes = calloc(sizeof(struct nodeinfo), 2);
+ if (nodes == NULL)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("Out of memory\n"));
+ conn->options_valid = false;
+ goto connect_errReturn;
+ }
+
hint.ai_family = AF_UNSPEC;
+ nodes->host = strdup(DefaultHost);
+ if (nodes->host == NULL)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("Out of memory\n"));
+ conn->options_valid = false;
+ goto connect_errReturn;
+ }
#endif /* HAVE_UNIX_SOCKETS */
}
/* Use pg_getaddrinfo_all() to resolve the address */
- ret = pg_getaddrinfo_all(node, portstr, &hint, &addrs);
- if (ret || !addrs)
+ /* loop over all the host specs in the node variable */
+ for (node = nodes; node->host != NULL || node->port != NULL; node++)
{
- if (node)
- appendPQExpBuffer(&conn->errorMessage,
- libpq_gettext("could not translate host name \"%s\" to address: %s\n"),
- node, gai_strerror(ret));
+ struct addrinfo *this_node_addrs;
+
+ /* Resolve each hostname into list of addrinfo structures */
+ ret = pg_getaddrinfo_all(node->host, (node->port ? node->port : portstr),
+ &hint, &this_node_addrs);
+ if (ret || !this_node_addrs)
+ {
+ if (node->host)
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not translate host name \"%s\" to address: %s\n"),
+ node->host, gai_strerror(ret));
+ else
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not translate Unix-domain socket path \"%s\" to address: %s\n"),
+ node->port, gai_strerror(ret));
+ if (this_node_addrs)
+ pg_freeaddrinfo_all(hint.ai_family, this_node_addrs);
+
+ /*
+ * We shouldn't fail here unless there is no valid addrinfos left
+ */
+ continue;
+ }
+
+ if (node->host)
+ {
+ struct addrinfo *n;
+
+ for (n = this_node_addrs; n != NULL; n = n->ai_next)
+ {
+ n->ai_canonname = strdup(node->host);
+ }
+ }
+
+ /* add this host addrs to addrs field of PGconn structure */
+ if (!addrs)
+ {
+ addrs = this_node_addrs;
+ }
else
- appendPQExpBuffer(&conn->errorMessage,
- libpq_gettext("could not translate Unix-domain socket path \"%s\" to address: %s\n"),
- portstr, gai_strerror(ret));
- if (addrs)
- pg_freeaddrinfo_all(hint.ai_family, addrs);
+ {
+ struct addrinfo *p;
+
+ /* This loop finds pointer to the last element of the list */
+ for (p = addrs; p->ai_next != NULL; p = p->ai_next)
+ {
+ }
+ p->ai_next = this_node_addrs;
+ }
+ }
+
+ /* Free nodes array */
+ for (node = nodes; node->host != NULL || node->port != NULL; node++)
+ {
+ if (node->host)
+ free(node->host);
+ if (node->port)
+ free(node->port);
+ }
+
+ free(nodes);
+
+ /* Check if we've found at least one usable address */
+ if (!addrs)
+ {
conn->options_valid = false;
goto connect_errReturn;
}
-
#ifdef USE_SSL
/* setup values based on SSL mode */
if (conn->sslmode[0] == 'd') /* "disable" */
@@ -1499,11 +1778,26 @@ connectDBStart(PGconn *conn)
* Set up to try to connect, with protocol 3.0 as the first attempt.
*/
conn->addrlist = addrs;
- conn->addr_cur = addrs;
+
+ /*
+ * We cannot just assign first addrs record to addr_cur, because host
+ * order may be random. So, use try_next_address
+ */
+ conn->addr_cur = NULL;
+ try_next_address(conn);
conn->addrlist_family = hint.ai_family;
conn->pversion = PG_PROTOCOL(3, 0);
conn->send_appname = true;
conn->status = CONNECTION_NEEDED;
+ if (conn->failover_timeout)
+ {
+ conn->failover_finish_time = time(NULL) + atoi(conn->failover_timeout);
+ }
+ else
+ {
+ conn->failover_finish_time = (time_t) 0; /* it is in past, so its
+ * ok */
+ }
/*
* The code for processing CONNECTION_NEEDED state is in PQconnectPoll(),
@@ -1521,6 +1815,23 @@ connect_errReturn:
return 0;
}
+/*
+ * This function is used to convert integer port number from the
+ * addrinfo structure back into string representation, because function
+ * PQport needs to return string representation of port.
+ */
+
+static char *
+get_port_from_addrinfo(struct addrinfo * ai)
+{
+ char port[6];
+
+ sprintf(port, "%d", htons(((struct sockaddr_in *) ai->ai_addr)->sin_port));
+
+ /* allocation failure must be checked by caller */
+ return strdup(port);
+}
+
/*
* connectDBComplete
@@ -1603,6 +1914,118 @@ connectDBComplete(PGconn *conn)
}
}
+/*
+ * Gets address of pointer to the list of addrinfo structures.
+ * If order is random, rearranges the list by moving random element to
+ * the beginning (and putting its address into given pointer.
+ * Returns address of first list element
+ */
+static struct addrinfo *
+get_next_element(struct addrinfo ** list, char *order)
+{
+ struct addrinfo *choice = NULL,
+ *prev,
+ *current,
+ *prechoice = NULL;
+ int count = 0;
+
+ if (*list == NULL)
+ return NULL;
+ if (strcmp(order, "random") == 0)
+ {
+ /* Peek random element from the list. */
+ for (current = *list, prev = NULL; current != NULL;
+ prev = current, current = current->ai_next)
+ {
+ count++;
+ if ((rand() & 0xffff) < 0x10000 / count)
+ {
+ choice = current;
+ prechoice = prev;
+ }
+ }
+
+ /*
+ * If prechoice is not NULL, selected element is not first in the
+ * list. We have to move it to he head
+ */
+ if (prechoice != NULL)
+ {
+ prechoice->ai_next = choice->ai_next;
+ choice->ai_next = *list;
+ *list = choice;
+ }
+ }
+
+ /* We always return first element of the list */
+ return *list;
+}
+
+/* -------------
+ * try_next_address
+ * Attempts to set next address from the list of known ones.
+ * Returns 1 if address is chosen and 0 if there are no more addresses
+ * to try
+ * Takes into account hostorder parameter
+ * ------------
+ */
+
+static int
+try_next_address(PGconn *conn)
+{
+ if (strcmp(conn->hostorder, "random") == 0)
+ {
+ /*
+ * Initialize random number generator for it to be initialized for
+ * certain. Use value from rand along with time in case random number
+ * have been initialized by application. Use address of conn structure
+ * to load-balance different connections in the same app
+ */
+ srand((unsigned int) ((long int) conn ^ (long int) time(NULL) ^
+ (long int) rand()));
+ }
+
+ if (conn->addr_cur == NULL)
+ {
+
+ conn->addr_cur = get_next_element(&(conn->addrlist),
+ conn->hostorder);
+ conn->actualhost = conn->addr_cur->ai_canonname;
+
+ return 1;
+ }
+ else
+ {
+ conn->addr_cur = get_next_element(&(conn->addr_cur->ai_next),
+ conn->hostorder);
+ }
+
+ if (conn->addr_cur == NULL && time(NULL) < conn->failover_finish_time)
+ {
+ /*
+ * If failover timeout is set, retry list of hosts from the beginning
+ */
+ pg_usleep(1000000);
+ conn->addr_cur = get_next_element(&(conn->addrlist),
+ conn->hostorder);
+ }
+
+ if (conn->addr_cur != NULL)
+ {
+ /*
+ * Clean up error message buffer.
+ */
+ resetPQExpBuffer(&conn->errorMessage);
+ conn->actualhost = conn->addr_cur->ai_canonname;
+
+ return 1;
+ }
+ else
+ {
+ return 0;
+ }
+}
+
/* ----------------
* PQconnectPoll
*
@@ -1681,6 +2104,9 @@ PQconnectPoll(PGconn *conn)
case CONNECTION_NEEDED:
break;
+ case CONNECTION_CHECK_RW:
+ break;
+
default:
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext(
@@ -1718,11 +2144,11 @@ keep_going: /* We will come back to here until there is
* ignore socket() failure if we have more addresses
* to try
*/
- if (addr_cur->ai_next != NULL)
+ if (try_next_address(conn))
{
- conn->addr_cur = addr_cur->ai_next;
continue;
}
+
appendPQExpBuffer(&conn->errorMessage,
libpq_gettext("could not create socket: %s\n"),
SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
@@ -1739,7 +2165,7 @@ keep_going: /* We will come back to here until there is
if (!connectNoDelay(conn))
{
pqDropConnection(conn, true);
- conn->addr_cur = addr_cur->ai_next;
+ try_next_address(conn);
continue;
}
}
@@ -1749,7 +2175,7 @@ keep_going: /* We will come back to here until there is
libpq_gettext("could not set socket to nonblocking mode: %s\n"),
SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
pqDropConnection(conn, true);
- conn->addr_cur = addr_cur->ai_next;
+ try_next_address(conn);
continue;
}
@@ -1760,7 +2186,7 @@ keep_going: /* We will come back to here until there is
libpq_gettext("could not set socket to close-on-exec mode: %s\n"),
SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf)));
pqDropConnection(conn, true);
- conn->addr_cur = addr_cur->ai_next;
+ try_next_address(conn);
continue;
}
#endif /* F_SETFD */
@@ -1807,7 +2233,7 @@ keep_going: /* We will come back to here until there is
if (err)
{
pqDropConnection(conn, true);
- conn->addr_cur = addr_cur->ai_next;
+ try_next_address(conn);
continue;
}
}
@@ -1883,6 +2309,10 @@ keep_going: /* We will come back to here until there is
* go do the next stuff.
*/
conn->status = CONNECTION_STARTED;
+
+ /*
+ * Save the name of the current host
+ */
goto keep_going;
}
@@ -1898,7 +2328,7 @@ keep_going: /* We will come back to here until there is
/*
* Try the next address, if any.
*/
- conn->addr_cur = addr_cur->ai_next;
+ try_next_address(conn);
} /* loop over addresses */
/*
@@ -1944,9 +2374,8 @@ keep_going: /* We will come back to here until there is
* If more addresses remain, keep trying, just as in the
* case where connect() returned failure immediately.
*/
- if (conn->addr_cur->ai_next != NULL)
+ if (try_next_address(conn))
{
- conn->addr_cur = conn->addr_cur->ai_next;
conn->status = CONNECTION_NEEDED;
goto keep_going;
}
@@ -2596,13 +3025,22 @@ keep_going: /* We will come back to here until there is
conn->errorMessage.data[conn->errorMessage.len - 1] != '\n')
appendPQExpBufferChar(&conn->errorMessage, '\n');
PQclear(res);
+
+ /*
+ * If we have more than one host in the connect string,
+ * fatal message from one of them is not really fatal
+ */
+ if (try_next_address(conn))
+ {
+ /* Must drop the old connection */
+ pqDropConnection(conn, true);
+ conn->status = CONNECTION_NEEDED;
+ goto keep_going;
+ }
+
goto error_return;
}
- /* We can release the address list now. */
- pg_freeaddrinfo_all(conn->addrlist_family, conn->addrlist);
- conn->addrlist = NULL;
- conn->addr_cur = NULL;
/* Fire up post-connection housekeeping if needed */
if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
@@ -2614,8 +3052,8 @@ keep_going: /* We will come back to here until there is
}
/* Otherwise, we are open for business! */
- conn->status = CONNECTION_OK;
- return PGRES_POLLING_OK;
+ conn->status = CONNECTION_CHECK_RO;
+ goto keep_going;
}
case CONNECTION_SETENV:
@@ -2645,9 +3083,141 @@ keep_going: /* We will come back to here until there is
goto error_return;
}
- /* We are open for business! */
+ /*
+ * check if connection is readonly if we need readwrite one
+ */
+ conn->status = CONNECTION_CHECK_RO;
+ goto keep_going;
+
+ case CONNECTION_CHECK_RO:
+
+ /*
+ * Connection to readonly host is allowed if taget_server_type is
+ * set to 'any' or is not exlicitely
+ */
+ if (conn->pghost == NULL || !conn->target_server_type ||
+ strcmp(conn->target_server_type, "any") == 0)
+
+ {
+ /*
+ * We can release the address list now but first make a copy
+ * of the name of host we are connected to or it would be
+ * freed with list
+ */
+ if (conn->actualhost)
+ {
+ conn->actualhost = strdup(conn->actualhost);
+ conn->actualport = get_port_from_addrinfo(conn->addr_cur);
+ if (!conn->actualhost || !conn->actualport)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("Out of memory"));
+ goto error_return;
+ }
+ }
+
+ pg_freeaddrinfo_all(conn->addrlist_family, conn->addrlist);
+ conn->addrlist = NULL;
+ conn->addr_cur = NULL;
+
+ conn->status = CONNECTION_OK;
+ return PGRES_POLLING_OK;
+ }
+
+ /* Otherwise request result pg_is_in_recovery() */
+ /* pretend that status is OK for time of sending query */
conn->status = CONNECTION_OK;
- return PGRES_POLLING_OK;
+ PQsendQuery(conn, "SELECT pg_catalog.pg_is_in_recovery()");
+ conn->status = CONNECTION_CHECK_RW;
+ return PGRES_POLLING_READING;
+
+ case CONNECTION_CHECK_RW:
+ {
+ char *value;
+ PGresult *res;
+
+ conn->status = CONNECTION_OK;
+ if (!PQconsumeInput(conn))
+ {
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+
+ if (PQisBusy(conn))
+ {
+ /* Result is not ready yet */
+ conn->status = CONNECTION_CHECK_RW;
+ return PGRES_POLLING_READING;
+ }
+
+ res = PQgetResult(conn);
+
+ /*
+ * Call PQgetResult second time to clear connection state.
+ * Should return NULL, so result is ignored
+ */
+ PQgetResult(conn);
+ if (!res || PQresultStatus(res) != PGRES_TUPLES_OK ||
+ PQntuples(res) != 1)
+ {
+ /*
+ * Something wrong happened with this host. Skip to next
+ * one
+ */
+ conn->status = CONNECTION_NEEDED;
+ }
+ else
+ {
+ value = PQgetvalue(res, 0, 0);
+ if (value[0] == 't')
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("cannot make RW connection to hot "
+ "standby node %s"), conn->actualhost);
+ conn->status = CONNECTION_NEEDED;
+ }
+ }
+
+ if (res)
+ PQclear(res);
+ if (conn->status != CONNECTION_OK)
+ {
+ ConnStatusType save_status = conn->status;
+
+ conn->status = CONNECTION_OK;
+ pqTerminateConn(conn);
+ pqDropConnection(conn, true);
+ conn->sock = PGINVALID_SOCKET;
+ if (try_next_address(conn))
+ {
+ conn->status = save_status;
+ goto keep_going;
+ }
+ else
+ {
+ conn->status = CONNECTION_BAD;
+ return PGRES_POLLING_FAILED;
+ }
+ }
+
+ /* We can release the address list now. */
+ if (conn->actualhost)
+ {
+ conn->actualhost = strdup(conn->actualhost);
+ conn->actualport = get_port_from_addrinfo(conn->addr_cur);
+ if (!conn->actualhost || !conn->actualport)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("Out of memory"));
+ goto error_return;
+ }
+ }
+
+ pg_freeaddrinfo_all(conn->addrlist_family, conn->addrlist);
+ conn->addrlist = NULL;
+ conn->addr_cur = NULL;
+ return PGRES_POLLING_OK;
+ }
default:
appendPQExpBuffer(&conn->errorMessage,
@@ -2938,19 +3508,16 @@ freePGconn(PGconn *conn)
}
/*
- * closePGconn
- * - properly close a connection to the backend
+ * pqTerminateConn
+ *
+ * - send terminate message to the backend, but do not free any
+ * transient state of PGconn object, which can be needed to reconnect
*
- * This should reset or release all transient state, but NOT the connection
- * parameters. On exit, the PGconn should be in condition to start a fresh
- * connection with the same parameters (see PQreset()).
*/
+
static void
-closePGconn(PGconn *conn)
+pqTerminateConn(PGconn *conn)
{
- PGnotify *notify;
- pgParameterStatus *pstatus;
-
/*
* Note that the protocol doesn't allow us to send Terminate messages
* during the startup phase.
@@ -2966,6 +3533,26 @@ closePGconn(PGconn *conn)
(void) pqFlush(conn);
}
+
+}
+
+/*
+ * closePGconn
+ * - properly close a connection to the backend
+ *
+ * This should reset or release all transient state, but NOT the connection
+ * parameters. On exit, the PGconn should be in condition to start a fresh
+ * connection with the same parameters (see PQreset()).
+ */
+static void
+closePGconn(PGconn *conn)
+{
+ PGnotify *notify;
+ pgParameterStatus *pstatus;
+
+ /* Send terminate request to backend */
+ pqTerminateConn(conn);
+
/*
* Must reset the blocking status so a possible reconnect will work.
*
@@ -2978,11 +3565,29 @@ closePGconn(PGconn *conn)
* Close the connection, reset all transient state, flush I/O buffers.
*/
pqDropConnection(conn, true);
+
conn->status = CONNECTION_BAD; /* Well, not really _bad_ - just
* absent */
conn->asyncStatus = PGASYNC_IDLE;
pqClearAsyncResult(conn); /* deallocate result */
resetPQExpBuffer(&conn->errorMessage);
+
+ /*
+ * If addrlist is not freed, actualhost points in there. Otherwice it is
+ * allocated and should be freed
+ */
+ if (conn->addrlist == NULL && conn->actualhost != NULL)
+ {
+ free(conn->actualhost);
+ conn->actualhost = NULL;
+ }
+
+ if (conn->actualport)
+ {
+ free(conn->actualport);
+ conn->actualport = NULL;
+ }
+
pg_freeaddrinfo_all(conn->addrlist_family, conn->addrlist);
conn->addrlist = NULL;
conn->addr_cur = NULL;
@@ -3969,6 +4574,8 @@ parseServiceFile(const char *serviceFile,
{
int linenr = 0,
i;
+ bool hostflag = false; /* true if we already have seen 'host'
+ * parameter in the service file, and */
FILE *f;
char buf[MAXBUFSIZE],
*line;
@@ -4088,7 +4695,38 @@ parseServiceFile(const char *serviceFile,
if (strcmp(options[i].keyword, key) == 0)
{
if (options[i].val == NULL)
+ {
options[i].val = strdup(val);
+
+ /*
+ * Set flag that we get value of host option from
+ * this service file, so subsequent host lines
+ * should be appended to it, not ignored
+ */
+ if (strcmp(key, "host") == 0)
+ hostflag = true;
+ }
+ else if (strcmp(key, "host") == 0 && hostflag)
+ {
+ /*
+ * Old host value is from same service file, so
+ * append new one to it
+ */
+ char *old = options[i].val;
+ int oldlen = strlen(old);
+
+ options[i].val = malloc(oldlen + 1 + strlen(val) + 1);
+
+ if (options[i].val)
+ {
+ strncpy(options[i].val, old, oldlen);
+ options[i].val[oldlen] = ',';
+ strcpy(options[i].val + oldlen + 1, val);
+ }
+
+ free(old);
+ }
+
if (!options[i].val)
{
printfPQExpBuffer(errorMessage,
@@ -4809,86 +5447,127 @@ conninfo_uri_parse_options(PQconninfoOption *options, const char *uri,
p = start;
}
- /*
- * "p" has been incremented past optional URI credential information at
- * this point and now points at the "netloc" part of the URI.
- *
- * Look for IPv6 address.
- */
- if (*p == '[')
+ host = p;
+ if (*p == ':')
{
- host = ++p;
- while (*p && *p != ']')
- ++p;
- if (!*p)
+ int portnum;
+ char *portstr;
+
+ *(p++) = '\0';
+ portstr = p;
+ portnum = 0;
+ while (*p >= '0' && *p <= '9')
{
- printfPQExpBuffer(errorMessage,
- libpq_gettext("end of string reached when looking for matching \"]\" in IPv6 host address in URI: \"%s\"\n"),
- uri);
- goto cleanup;
+ portnum = portnum * 10 + (*(p++) - '0');
}
- if (p == host)
+ if (portnum > 65535 || portnum < 1)
{
printfPQExpBuffer(errorMessage,
- libpq_gettext("IPv6 host address may not be empty in URI: \"%s\"\n"),
- uri);
+ libpq_gettext("invalid port number: \"%d\"\n"),
+ portnum);
goto cleanup;
}
-
- /* Cut off the bracket and advance */
- *(p++) = '\0';
-
- /*
- * The address may be followed by a port specifier or a slash or a
- * query.
- */
- if (*p && *p != ':' && *p != '/' && *p != '?')
+ prevchar = *p;
+ *p = '\0';
+ if (*portstr &&
+ !conninfo_storeval(options, "port", portstr,
+ errorMessage, false, true))
{
- printfPQExpBuffer(errorMessage,
- libpq_gettext("unexpected character \"%c\" at position %d in URI (expected \":\" or \"/\"): \"%s\"\n"),
- *p, (int) (p - buf + 1), uri);
goto cleanup;
}
}
else
{
- /* not an IPv6 address: DNS-named or IPv4 netloc */
- host = p;
+ do
+ {
+ if (*p == ',')
+ p++;
- /*
- * Look for port specifier (colon) or end of host specifier (slash),
- * or query (question mark).
- */
- while (*p && *p != ':' && *p != '/' && *p != '?')
- ++p;
- }
+ /*
+ * "p" has been incremented past optional URI credential
+ * information at this point and now points at the "netloc" part
+ * of the URI.
+ *
+ * Look for IPv6 address.
+ */
+ if (*p == '[')
+ {
+ char *ipv6start = ++p;
- /* Save the hostname terminator before we null it */
- prevchar = *p;
- *p = '\0';
+ while (*p && *p != ']')
+ ++p;
+ if (!*p)
+ {
+ printfPQExpBuffer(errorMessage,
+ libpq_gettext("end of string reached when looking for matching \"]\" in IPv6 host address in URI: \"%s\"\n"),
+ uri);
+ goto cleanup;
+ }
+ if (p == ipv6start)
+ {
+ printfPQExpBuffer(errorMessage,
+ libpq_gettext("IPv6 host address may not be empty in URI: \"%s\"\n"),
+ uri);
+ goto cleanup;
+ }
- if (*host &&
- !conninfo_storeval(options, "host", host,
- errorMessage, false, true))
- goto cleanup;
+ p++;
+ /*
+ * The address may be followed by a port specifier, a comma or
+ * a slash or a query.
+ */
+ if (*p && *p != ',' && *p != ':' && *p != '/' && *p != '?')
+ {
+ printfPQExpBuffer(errorMessage,
+ libpq_gettext("unexpected character \"%c\" at position %d in URI (expected \":\" or \"/\"): \"%s\"\n"),
+ *p, (int) (p - buf + 1), uri);
+ goto cleanup;
+ }
- if (prevchar == ':')
- {
- const char *port = ++p; /* advance past host terminator */
+ }
+ else
+ {
+ /* not an IPv6 address: DNS-named or IPv4 netloc */
- while (*p && *p != '/' && *p != '?')
- ++p;
+ /*
+ * Look for port specifier (colon) or end of host specifier
+ * (slash), or query (question mark).
+ */
+ while (*p && *p != ',' && *p != ':' && *p != '/' && *p != '?')
+ ++p;
+ }
+ /* Skip port specifier */
+ if (*p == ':')
+ {
+ int portnum;
+
+ p++;
+ portnum = 0;
+ while (*p >= '0' && *p <= '9')
+ {
+ portnum = portnum * 10 + (*(p++) - '0');
+ }
+ if (portnum > 65535 || portnum < 1)
+ {
+ printfPQExpBuffer(errorMessage,
+ libpq_gettext("invalid port number: \"%d\"\n"),
+ portnum);
+ goto cleanup;
+ }
+ }
+ } while (*p == ',');
+ /* Save the hostname terminator before we null it */
prevchar = *p;
*p = '\0';
- if (*port &&
- !conninfo_storeval(options, "port", port,
+ if (*host &&
+ !conninfo_storeval(options, "host", host,
errorMessage, false, true))
goto cleanup;
- }
+ }
if (prevchar && prevchar != '?')
{
const char *dbname = ++p; /* advance past host terminator */
@@ -5018,6 +5697,21 @@ conninfo_uri_parse_params(char *params,
keyword = "sslmode";
value = "require";
}
+ if ((strcmp(keyword, "loadBalanceHosts") == 0 ||
+ strcmp(keyword, "load_balance_hosts") == 0) &&
+ strcmp(value, "true") == 0)
+ {
+ free(keyword);
+ free(value);
+ malloced = false;
+ keyword = "hostorder";
+ value = "random";
+ }
+ if (strcmp(keyword, "targetServerType") == 0)
+ {
+ free(keyword);
+ keyword = strdup("target_server_type");
+ }
/*
* Store the value if the corresponding option exists; ignore
@@ -5232,7 +5926,21 @@ conninfo_storeval(PQconninfoOption *connOptions,
}
if (option->val)
+ {
+ if (strcmp(option->keyword, "host") == 0)
+ {
+ /* Accumulate multiple hosts in the single string */
+ int val_len = strlen(option->val),
+ new_len = strlen(value);
+
+ free(value_copy);
+ value_copy = malloc(val_len + 1 + new_len + 1);
+ strncpy(value_copy, option->val, val_len + 1);
+ value_copy[val_len] = ',';
+ strncpy(value_copy + val_len + 1, value, new_len + 1);
+ }
free(option->val);
+ }
option->val = value_copy;
return option;
@@ -5352,7 +6060,9 @@ PQhost(const PGconn *conn)
{
if (!conn)
return NULL;
- if (conn->pghost != NULL && conn->pghost[0] != '\0')
+ if (conn->actualhost != NULL && conn->actualhost[0] != '\0')
+ return conn->actualhost;
+ else if (conn->pghost != NULL && conn->pghost[0] != '\0')
return conn->pghost;
else
{
@@ -5372,6 +6082,8 @@ PQport(const PGconn *conn)
{
if (!conn)
return NULL;
+ if (conn->actualport)
+ return conn->actualport;
return conn->pgport;
}
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 9ca0756..23560f4 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -62,7 +62,11 @@ typedef enum
* backend startup. */
CONNECTION_SETENV, /* Negotiating environment. */
CONNECTION_SSL_STARTUP, /* Negotiating SSL. */
- CONNECTION_NEEDED /* Internal state: connect() needed */
+ CONNECTION_NEEDED, /* Internal state: connect() needed */
+ CONNECTION_CHECK_RO, /* Internal state: need to check is RO
+ * connection acceptable */
+ CONNECTION_CHECK_RW, /* Internal state: waiting that server replies
+ * if it is in recovery */
} ConnStatusType;
typedef enum
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 7007692..3b0fb64 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -334,7 +334,12 @@ struct pg_conn
#if defined(ENABLE_GSS) || defined(ENABLE_SSPI)
char *krbsrvname; /* Kerberos service name */
#endif
-
+ char *hostorder; /* How to handle multiple hosts */
+ char *target_server_type; /* If "any" could work with readonly
+ * standby server. Otherwise should be
+ * "master" */
+ char *failover_timeout; /* If no usable server found, how long
+ * to wait before retry */
/* Optional file to write trace info to */
FILE *Pfdebug;
@@ -381,6 +386,11 @@ struct pg_conn
struct addrinfo *addrlist; /* list of possible backend addresses */
struct addrinfo *addr_cur; /* the one currently being tried */
int addrlist_family; /* needed to know how to free addrlist */
+ time_t failover_finish_time; /* how long to retry host list waiting
+ * for new master to appear */
+ char *actualhost; /* Name of host we are actually connected to
+ * (if there is list in the pghost) */
+ char *actualport; /* Port number we are actually connected to */
PGSetenvStatusType setenv_state; /* for 2.0 protocol only */
const PQEnvironmentOption *next_eo;
bool send_appname; /* okay to send application_name? */
@@ -466,6 +476,7 @@ struct pg_conn
/* Buffer for receiving various parts of messages */
PQExpBufferData workBuffer; /* expansible string */
+
};
/* PGcancel stores all data necessary to cancel a connection. A copy of this
diff --git a/src/interfaces/libpq/t/001-multihost.pl b/src/interfaces/libpq/t/001-multihost.pl
new file mode 100644
index 0000000..7748aea
--- /dev/null
+++ b/src/interfaces/libpq/t/001-multihost.pl
@@ -0,0 +1,529 @@
+# Minimal test testing streaming replication
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 28;
+
+# Initialize master node
+
+my $node_master = get_new_node('master');
+$node_master->init(allows_streaming => 1, use_tcp => 1);
+$node_master->start;
+my $backup_name = 'my_backup';
+
+# Take backup
+$node_master->backup($backup_name);
+
+# Create streaming standby linking to master
+my $node_standby_1 = get_new_node('standby_1');
+
+$node_standby_1->init_from_backup(
+ $node_master, $backup_name,
+ has_streaming => 1,
+ use_tcp => 1);
+$node_standby_1->start;
+
+# Take backup of standby 1 (not mandatory, but useful to check if
+# pg_basebackup works on a standby).
+$node_standby_1->backup($backup_name);
+
+# Create second standby node linking to standby 1
+my $node_standby_2 = get_new_node('standby_2');
+$node_standby_2->init_from_backup(
+ $node_standby_1, $backup_name,
+ has_streaming => 1,
+ use_tcp => 1);
+$node_standby_2->start;
+
+
+sub get_host_port
+{
+ my $node = shift;
+ return "$PostgresNode::test_localhost:" . $node->port;
+}
+
+sub multiconnstring
+{
+ my $nodes = shift;
+ my $database = shift || "postgres";
+ my $params = shift;
+ my $extra = "";
+ if ($params)
+ {
+ my @cs;
+ while (my ($key, $val) = each %$params)
+ {
+ push @cs, $key . "=" . $val;
+ }
+ $extra = "?" . join("&", @cs);
+ }
+ my $str =
+ "postgresql://"
+ . join(",", map({ get_host_port($_) } @$nodes))
+ . "/$database$extra";
+ return $str;
+
+}
+
+sub connstring2
+{
+ my $nodes = shift;
+ my $database = shift;
+ my $params = shift;
+ my @args = ();
+ for my $n (@$nodes)
+ {
+ push @args, "host=" . get_host_port($n);
+ }
+ push @args, "dbname=$database" if defined($database);
+ while (my ($key, $val) = each %$params)
+ {
+ push @args, "$key=$val";
+ }
+ return join(" ", @args);
+}
+
+#
+# Copied from PosgresNode.pm passing explicit connect-string instead of
+# constructed from object
+#
+
+sub psql
+{
+ # We expect dbname to be part of connstr
+ my ($connstr, $sql, %params) = @_;
+
+ my $stdout = $params{stdout};
+ my $stderr = $params{stderr};
+ my $timeout = undef;
+ my $timeout_exception = 'psql timed out';
+ my @psql_params = ('psql', '-XAtq', '-d', $connstr, '-f', '-');
+
+ # If the caller wants an array and hasn't passed stdout/stderr
+ # references, allocate temporary ones to capture them so we
+ # can return them. Otherwise we won't redirect them at all.
+ if (wantarray)
+ {
+ if (!defined($stdout))
+ {
+ my $temp_stdout = "";
+ $stdout = \$temp_stdout;
+ }
+ if (!defined($stderr))
+ {
+ my $temp_stderr = "";
+ $stderr = \$temp_stderr;
+ }
+ }
+
+ $params{on_error_stop} = 1 unless defined $params{on_error_stop};
+ $params{on_error_die} = 0 unless defined $params{on_error_die};
+
+ push @psql_params, '-v', 'ON_ERROR_STOP=1' if $params{on_error_stop};
+ push @psql_params, @{ $params{extra_params} }
+ if defined $params{extra_params};
+
+ $timeout =
+ IPC::Run::timeout($params{timeout}, exception => $timeout_exception)
+ if (defined($params{timeout}));
+
+ ${ $params{timed_out} } = 0 if defined $params{timed_out};
+
+ # IPC::Run would otherwise append to existing contents:
+ $$stdout = "" if ref($stdout);
+ $$stderr = "" if ref($stderr);
+
+ my $ret;
+
+ # Run psql and capture any possible exceptions. If the exception is
+ # because of a timeout and the caller requested to handle that, just return
+ # and set the flag. Otherwise, and for any other exception, rethrow.
+ #
+ # For background, see
+ # http://search.cpan.org/~ether/Try-Tiny-0.24/lib/Try/Tiny.pm
+ do
+ {
+ local $@;
+ eval {
+ my @ipcrun_opts = (\@psql_params, '<', \$sql);
+ push @ipcrun_opts, '>', $stdout if defined $stdout;
+ push @ipcrun_opts, '2>', $stderr if defined $stderr;
+ push @ipcrun_opts, $timeout if defined $timeout;
+
+ IPC::Run::run @ipcrun_opts;
+ $ret = $?;
+ };
+ my $exc_save = $@;
+ if ($exc_save)
+ {
+
+ # IPC::Run::run threw an exception. re-throw unless it's a
+ # timeout, which we'll handle by testing is_expired
+ die $exc_save
+ if (blessed($exc_save) || $exc_save ne $timeout_exception);
+
+ $ret = undef;
+
+ die "Got timeout exception '$exc_save' but timer not expired?!"
+ unless $timeout->is_expired;
+
+ if (defined($params{timed_out}))
+ {
+ ${ $params{timed_out} } = 1;
+ }
+ else
+ {
+ die "psql timed out: stderr: '$$stderr'\n"
+ . "while running '@psql_params'";
+ }
+ }
+ };
+
+ if (defined $$stdout)
+ {
+ chomp $$stdout;
+ $$stdout =~ s/\r//g if $TestLib::windows_os;
+ }
+
+ if (defined $$stderr)
+ {
+ chomp $$stderr;
+ $$stderr =~ s/\r//g if $TestLib::windows_os;
+ }
+
+ # See http://perldoc.perl.org/perlvar.html#%24CHILD_ERROR
+ # We don't use IPC::Run::Simple to limit dependencies.
+ #
+ # We always die on signal.
+ my $core = $ret & 128 ? " (core dumped)" : "";
+ die "psql exited with signal "
+ . ($ret & 127)
+ . "$core: '$$stderr' while running '@psql_params'"
+ if $ret & 127;
+ $ret = $ret >> 8;
+
+ if ($ret && $params{on_error_die})
+ {
+ die "psql error: stderr: '$$stderr'\nwhile running '@psql_params'"
+ if $ret == 1;
+ die "connection error: '$$stderr'\nwhile running '@psql_params'"
+ if $ret == 2;
+ die "error running SQL: '$$stderr'\nwhile running '@psql_params'"
+ if $ret == 3;
+ die "psql returns $ret: '$$stderr'\nwhile running '@psql_params'";
+ }
+
+ if (wantarray)
+ {
+ return ($ret, $$stdout, $$stderr);
+ }
+ else
+ {
+ return $ret;
+ }
+}
+
+sub psql_conninfo
+{
+ my ($connstr) = shift;
+ my ($timed_out);
+ my ($retcode, $stdout, $stderr) =
+ psql($connstr, '\conninfo', timed_out => \$timed_out);
+ if ($retcode == 0 && $stdout =~ /on host "([^"]*)" at port "([^"]*)"/s)
+ {
+ return "$1:$2";
+ }
+ else
+ {
+ return "STDOUT:$stdout\nSTDERR:$stderr";
+ }
+}
+
+sub psql_server_addr
+{
+ my ($connstr) = shift;
+ my ($timed_out);
+ my $sql =
+ "select abbrev(inet_server_addr()) ||':'||inet_server_port();\n";
+ my ($retcode, $stdout, $stderr) =
+ psql($connstr, $sql, timed_out => \$timed_out);
+ if ($retcode == 0)
+ {
+ return $stdout;
+ }
+ else
+ {
+ return "STDOUT:$stdout\nSTDERR:$stderr";
+ }
+}
+my $conninfo;
+
+# Test 1.1 - all hosts available, master first, readwrite requested
+$conninfo =
+ psql_conninfo(
+ multiconnstring([ $node_master, $node_standby_1, $node_standby_2 ]));
+is($conninfo, get_host_port($node_master), "master first, rw, conninfo");
+
+# Test 1.2
+$conninfo =
+ psql_server_addr(
+ multiconnstring([ $node_master, $node_standby_1, $node_standby_2 ]));
+is($conninfo, get_host_port($node_master), "master first, rw, server funcs");
+
+# Test 2.1 - use symbolic name for master and IP for slave
+$conninfo =
+ psql_conninfo("postgresql://localhost:"
+ . $node_master->port
+ . ",127.0.0.1:"
+ . $node_standby_1->port
+ . "/postgres");
+
+is( $conninfo,
+ "localhost:" . $node_master->port,
+ "master symbolic, rw, conninfo");
+
+# Test 2.2 - check server-side connect info (would return numeric IP)
+$conninfo =
+ psql_server_addr("postgresql://localhost:"
+ . $node_master->port
+ . ",127.0.0.1:"
+ . $node_standby_1->port
+ . "/postgres");
+is( $conninfo,
+ "127.0.0.1:" . $node_master->port,
+ 'master symbolic, rw server funcs');
+
+# Test 3.1 - all nodes available, master second, readwrite requested
+$conninfo =
+ psql_conninfo(
+ multiconnstring([ $node_standby_1, $node_master, $node_standby_2 ]));
+
+is($conninfo, get_host_port($node_master), "master second,rw, conninfo");
+
+# Test 3.2 Check server-side connection info
+$conninfo =
+ psql_server_addr(
+ multiconnstring([ $node_standby_1, $node_master, $node_standby_2 ]));
+
+is($conninfo, get_host_port($node_master), "master second, rw, server funcs");
+
+# Test 4.1 - use symbolic name for slave and IP for smaster
+$conninfo =
+ psql_conninfo("postgresql://localhost:"
+ . $node_standby_1->port
+ . ",127.0.0.1:"
+ . $node_master->port
+ . "/postgres");
+is( $conninfo,
+ "127.0.0.1:" . $node_master->port,
+ "slave symbolic, rw,conninfo");
+
+# Test 4.2 - check server-side connect info
+$conninfo =
+ psql_server_addr("postgresql://localhost:"
+ . $node_standby_1->port
+ . ",127.0.0.1:"
+ . $node_master->port
+ . "/postgres");
+is( $conninfo,
+ "127.0.0.1:" . $node_master->port,
+ "slave symbolic rw, server funcs");
+
+# Test 5 - all nodes available, master first, readonly requested
+
+$conninfo = psql_conninfo(
+ multiconnstring(
+ [ $node_master, $node_standby_1, $node_standby_2 ],
+ undef, { target_server_type => 'any' }));
+
+is($conninfo, get_host_port($node_master), "master first, ro, conninfo");
+
+# Test 6 - all nodes available, master second, readonly requested
+$conninfo = psql_conninfo(
+ multiconnstring(
+ [ $node_standby_1, $node_master, $node_standby_2 ],
+ undef, { target_server_type => 'any' }));
+
+is($conninfo, get_host_port($node_standby_1), "master second, ro conninfo");
+
+# Test 7.1 - all nodes available, random order, readonly.
+# Expect that during six attempts any of three nodes would be collected
+# at least once
+
+my %conncount = ();
+for (my $i = 0; $i < 9; $i++)
+{
+ my $conn = psql_conninfo(
+ multiconnstring(
+ [ $node_master, $node_standby_1, $node_standby_2 ],
+ undef,
+ { target_server_type => 'any', hostorder => 'random' }));
+ $conncount{$conn}++;
+}
+is(scalar(keys(%conncount)), 3, 'random order, readonly connect');
+
+# Test 7.2 - alternate (jdbc compatible) syntax for randomized hosts
+
+for (my $i = 0; $i < 6; $i++)
+{
+ my $conn = psql_conninfo(
+ multiconnstring(
+ [ $node_master, $node_standby_1, $node_standby_2 ],
+ undef,
+ { targetServerType => 'any', loadBalanceHosts => "true" }));
+ $conncount{$conn}++;
+}
+
+#diag(join(",",keys %conncount));
+is(scalar(keys %conncount),
+ 3, "alternate JDBC-compatible syntax for random order");
+
+# Test 8 - all nodes available, random order, readwrite
+# Expect all six connections go to the master
+
+%conncount = ();
+for (my $i = 0; $i < 6; $i++)
+{
+ my $conn = psql_conninfo(
+ multiconnstring(
+ [ $node_master, $node_standby_1, $node_standby_2 ],
+ undef, { hostorder => 'random' }));
+ $conncount{$conn}++;
+}
+
+is(length(keys %conncount), 1, 'random order, rw connect only one node');
+ok(exists $conncount{ get_host_port($node_master) },
+ 'random order, rw connects master');
+
+# Test 8.1 one host in URL, master
+$conninfo = psql_conninfo(multiconnstring([$node_master]));
+is($conninfo, get_host_port($node_master), "old behavoir compat - master");
+
+
+# Test 8.2 one host in URL, slave
+$conninfo = psql_conninfo(multiconnstring([$node_standby_1]));
+is($conninfo, get_host_port($node_standby_1), "old behavoir compat - slave");
+
+# Test 9 - try to connect only slaves in rw mode
+
+$conninfo =
+ psql_conninfo(multiconnstring([ $node_standby_1, $node_standby_2 ]));
+is( $conninfo,
+"STDOUT:\nSTDERR:psql: cannot make RW connection to hot standby node 127.0.0.1",
+ "cannot connect just slaves in RW mode");
+
+
+
+# Test 10 - one of slaves is not available
+$node_standby_1->stop();
+
+# Test 10.1
+
+$conninfo =
+ psql_conninfo(
+ multiconnstring([ $node_standby_1, $node_master, $node_standby_2 ]));
+
+is($conninfo, get_host_port($node_master), "first node is unavailable");
+
+# Test 10.2
+
+$conninfo =
+ psql_conninfo(
+ multiconnstring([ $node_standby_2, $node_standby_1, $node_master ]));
+
+is( $conninfo,
+ get_host_port($node_master),
+ "first node standby, second unavailable");
+
+# Test 10.3
+
+$conninfo = psql_conninfo(
+ multiconnstring(
+ [ $node_standby_1, $node_standby_2, $node_master ],
+ undef, { target_server_type => 'any' }));
+is( $conninfo,
+ get_host_port($node_standby_2),
+ "first node unavailable, second standmby, readonly mode");
+
+$node_standby_1->start();
+
+$node_master->stop();
+
+$conninfo =
+ psql_conninfo(
+ multiconnstring([ $node_standby_1, $node_master, $node_standby_2 ]));
+
+is( $conninfo,
+"STDOUT:\nSTDERR:psql: cannot make RW connection to hot standby node 127.0.0.1",
+ "master unavialble, cannot connect just slaves in RW mode");
+
+$conninfo = psql_conninfo(
+ multiconnstring(
+ [ $node_master, $node_standby_1, $node_standby_2 ],
+ undef, { target_server_type => 'any' }));
+
+is( $conninfo,
+ get_host_port($node_standby_1),
+ "Master unavailable, read only ");
+
+$node_master->start();
+
+# Test 11 Alternate syntax
+
+$conninfo =
+ psql_conninfo(
+ connstring2([ $node_standby_1, $node_standby_2, $node_master ]));
+
+is( $conninfo,
+ get_host_port($node_master),
+ "Alternate syntax, master third, rw");
+
+
+
+$conninfo =
+ psql_conninfo(
+ connstring2([ $node_master, $node_standby_1, $node_standby_2 ]));
+
+is( $conninfo,
+ get_host_port($node_master),
+ "Alternate syntax, master first, rw");
+
+
+
+$conninfo = psql_conninfo(
+ connstring2(
+ [ $node_standby_1, $node_standby_2, $node_master ],
+ undef, { target_server_type => 'any' }));
+
+is( $conninfo,
+ get_host_port($node_standby_1),
+ "Alternate syntax, master third, ro");
+
+
+
+$conninfo = psql_conninfo(
+ connstring2(
+ [ $node_master, $node_standby_1, $node_standby_2 ],
+ undef, { target_server_type => 'any' }));
+
+is( $conninfo,
+ get_host_port($node_master),
+ "Alternate syntax, master first, ro");
+
+
+# Test 11.5 one host in URL, master
+$conninfo = psql_conninfo(connstring2([$node_master]));
+is( $conninfo,
+ get_host_port($node_master),
+ "alt syntax old behavoir compat - master");
+
+
+# Test 11.6 one host in URL, slave
+$conninfo = psql_conninfo(connstring2([$node_standby_1]));
+is( $conninfo,
+ get_host_port($node_standby_1),
+ "alt syntax old behavoir compat - slave");
+
+
+
diff --git a/src/interfaces/libpq/test/expected.out b/src/interfaces/libpq/test/expected.out
index d375e82..4832bdd 100644
--- a/src/interfaces/libpq/test/expected.out
+++ b/src/interfaces/libpq/test/expected.out
@@ -1,20 +1,20 @@
trying postgresql://uri-user:secret@host:12345/db
-user='uri-user' password='secret' dbname='db' host='host' port='12345' (inet)
+user='uri-user' password='secret' dbname='db' host='host:12345' (inet)
trying postgresql://uri-user@host:12345/db
-user='uri-user' dbname='db' host='host' port='12345' (inet)
+user='uri-user' dbname='db' host='host:12345' (inet)
trying postgresql://uri-user@host/db
user='uri-user' dbname='db' host='host' (inet)
trying postgresql://host:12345/db
-dbname='db' host='host' port='12345' (inet)
+dbname='db' host='host:12345' (inet)
trying postgresql://host/db
dbname='db' host='host' (inet)
trying postgresql://uri-user@host:12345/
-user='uri-user' host='host' port='12345' (inet)
+user='uri-user' host='host:12345' (inet)
trying postgresql://uri-user@host/
user='uri-user' host='host' (inet)
@@ -23,10 +23,10 @@ trying postgresql://uri-user@
user='uri-user' (local)
trying postgresql://host:12345/
-host='host' port='12345' (inet)
+host='host:12345' (inet)
trying postgresql://host:12345
-host='host' port='12345' (inet)
+host='host:12345' (inet)
trying postgresql://host/db
dbname='db' host='host' (inet)
@@ -62,7 +62,7 @@ trying postgresql://host/db?u%7aer=someotheruser&port=12345
uri-regress: invalid URI query parameter: "uzer"
trying postgresql://host:12345?user=uri-user
-user='uri-user' host='host' port='12345' (inet)
+user='uri-user' host='host:12345' (inet)
trying postgresql://host?user=uri-user
user='uri-user' host='host' (inet)
@@ -71,19 +71,19 @@ trying postgresql://host?
host='host' (inet)
trying postgresql://[::1]:12345/db
-dbname='db' host='::1' port='12345' (inet)
+dbname='db' host='[::1]:12345' (inet)
trying postgresql://[::1]/db
-dbname='db' host='::1' (inet)
+dbname='db' host='[::1]' (inet)
trying postgresql://[2001:db8::1234]/
-host='2001:db8::1234' (inet)
+host='[2001:db8::1234]' (inet)
trying postgresql://[200z:db8::1234]/
-host='200z:db8::1234' (inet)
+host='[200z:db8::1234]' (inet)
trying postgresql://[::1]
-host='::1' (inet)
+host='[::1]' (inet)
trying postgres://
(local)
@@ -143,7 +143,7 @@ trying postgres://@host
host='host' (inet)
trying postgres://host:/
-host='host' (inet)
+uri-regress: invalid port number: "0"
trying postgres://:12345/
port='12345' (local)
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index c1b16ca..43d079d 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -398,6 +398,7 @@ sub init
unless defined $params{hba_permit_replication};
$params{allows_streaming} = 0 unless defined $params{allows_streaming};
$params{has_archiving} = 0 unless defined $params{has_archiving};
+ $params{use_tcp} = 0 unless defined $params{use_tcp};
mkdir $self->backup_dir;
mkdir $self->archive_dir;
@@ -432,7 +433,14 @@ sub init
else
{
print $conf "unix_socket_directories = '$host'\n";
- print $conf "listen_addresses = ''\n";
+ if ($params{use_tcp})
+ {
+ print $conf "listen_addresses = '$test_localhost'\n";
+ }
+ else
+ {
+ print $conf "listen_addresses = ''\n";
+ }
}
close $conf;
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
