I'm interested in abstracting out features of replication from libpq too. It would be nice if we could implement different communication bus modules.
For example if you have dozens of replicas you may want to use something like spread to distribute the records using multicast. Sorry for top posting -- I haven't yet figured out how not to in this client. On 16 Dec 2009 09:54, "Heikki Linnakangas" < heikki.linnakan...@enterprisedb.com> wrote: Fujii Masao wrote: > On Tue, Dec 15, 2009 at 3:47 AM, Heikki Linnakangas > <heikki.linnakan...@enter... Great! The logical next step is move the handling of TimelineID and system identifier out of libpq as well. I'm thinking of refactoring the protocol along these lines: 0. Begin by connecting to the master just like a normal backend does. We don't necessarily need the new ProtocolVersion code either, though it's probably still a good idea to reject connections to older server versions. 1. Get the system identifier of the master. Slave -> Master: Query message, with a query string like "GET_SYSTEM_IDENTIFIER" Master -> Slave: RowDescription, DataRow CommandComplete, and ReadyForQuery messages. The system identifier is returned in the DataRow message. This is identical to what happens when a query is executed against a normal backend using the simple query protocol, so walsender can use PQexec() for this. 2. Another query exchange like above, for timeline ID. (or these two steps can be joined into one query, to eliminate one round-trip). 3. Request a backup history file, if needed: Slave -> Master: Query message, with a query string like "GET_BACKUP_HISTORY_FILE XXX" where XXX is XLogRecPtr or file name. Master -> Slave: RowDescription, DataRow CommandComplete and ReadyForQuery messages as usual. The file contents are returned in the DataRow message. 4. Start replication Slave -> Master: Query message, with query string "START REPLICATION: XXXX", where XXXX is the RecPtr of the starting point. Master -> Slave: CopyOutResponse followed by a continuous stream of CopyData messages with WAL contents. This minimizes the changes to the protocol and libpq, with a clear way of extending by adding new commands. Similar to what you did a long time ago, connecting as an actual backend at first and then switching to walsender mode after running a few queries, but this would all be handled in a separate loop in walsender instead of running as a full-blown backend. We'll still need small changes to libpq to allow sending messages back to the server in COPY_IN mode (maybe add a new COPY_IN_OUT mode for that). Thoughts? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscript...
