Hi, In PGCon 2008, I proposed synchronous log shipping replication. Sorry for late posting, but I'd like to start the discussion about its implementation from now. http://www.pgcon.org/2008/schedule/track/Horizontal%20Scaling/76.en.html
First of all, I'm not planning to put the prototype which I demoed in PGCon into core directly. - Portability issues (using message queue, multi-threaded ...) - Have too much dependency on Heartbeat Yes, since the prototype is useful reference of implementation, I plan to open it ASAP. But, I'm sorry - it still takes a month to open it. Pavan re-designed the sync replication based on the prototype and I posted that design doc on wiki. Please check it if you are interested in it. http://wiki.postgresql.org/wiki/NTT%27s_Development_Projects This design is too huge. In order to enhance the extensibility of postgres, I'd like to divide the sync replication into minimum hooks and some plugins and to develop it, respectively. Plugins for the sync replication plan to be available at the time of 8.4 release. In my design, WAL sending is achieved as follow by WALSender. WALSender is a new process which I introduce. 1) On COMMIT, backend requests WALSender to send WAL. 2) WALSender reads WAL from walbuffers and send it to slave. 3) WALSender waits for the response from slave and replies backend. I propose two hooks for WAL sending. WAL-writing hook ---------------- This hook is for backend to communicate with WALSender. WAL-writing hook intercepts write system call in XLogWrite. That is, backend requests WAL sending whenever write is called. WAL-writing hook is available also for other uses e.g. Software RAID (writes WAL into two files for durability). Hook for WALSender ------------------ This hook is for introducing WALSender. There are the following three ideas of how to introduce WALSender. A required hook differs by which idea is adopted. a) Use WALWriter as WALSender This idea needs WALWriter hook which intercepts WALWriter literally. WALWriter stops the local WAL write and focuses on WAL sending. This idea is very simple, but I don't think of the use of WALWriter hook other than WAL sending. b) Use new background process as WALSender This idea needs background-process hook which enables users to define new background processes. I think the design of this hook resembles that of rmgr hook proposed by Simon. I define the table like RmgrTable. It's for registering some functions (e.g. main function and exit...) for operating a background process. Postmaster calls the function from the table suitably, and manages a start and end of background process. ISTM that there are many uses in this hook, e.g. performance monitoring process like statspack. c) Use one backend as WALSender In this idea, slave calls the user-defined function which takes charge of WAL sending via SQL e.g. "SELECT pg_walsender()". Compared with other ideas, it's easy to implement WALSender because postmater handles the establishment and authentication of connection. But, this SQL causes a long transaction which prevents vacuum. So, this idea needs idle-state hook which executes plugin before transaction starts. I don't think of the use of this hook other than WAL sending either. Which idea should we adopt? Comments welcome. -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
