Markus Wanner wrote: > Besides the communication between the replication manager and the > backends, which is currently done by using these imessages, the > replication manager also needs to communicate with the postmaster: it > needs to be able to request new helper backends and it wants to be > notified upon termination (or crash) of such a helper backend (and other > backends as well...). I'm currently doing this with imessages as well, > which violates the rule that the postmaster may not to touch shared > memory. I didn't look into ripping that out, yet. I'm not sure it can be > done with the existing signaling of the postmaster.
In Replicator we avoided the need for postmaster to read/write backend's shmem data by using it as a signal forwarder. When a backend wants to inform a special process (i.e. queue monitor) about replication-related event (such as commit) it sends SIGUSR1 to Postmaster with a related "reason" flag and the postmaster upon receiving this signal forwards it to the destination process. Termination of backends and special processes are handled by the postmaster itself. > > Let's have a simple example: consider a local transaction which changes > some tuples. Those are being collected into a change set, which gets > written to the shared memory area as an imessage for the replication > manager. The backend then also signals the manager, which then awakes > from its select(), checks its imessages queue and processes the message, > delivering it to the GCS. It then removes the imessage from the shared > memory area again. Hm...what would happen with the new data under heavy load when the queue would eventually be filled with messages, the relevant transactions would be aborted or they would wait for the manager to release the queue space occupied by already processed messages? ISTM that having a fixed size buffer limits the maximum transaction rate. > > My initial design features only a single doubly linked list as the > message queue, holding all messages for all processes. An imessages lock > blocks concurrent writing acces. That's still what's in there, but I > realize that's not enough. Each process should better have its own > queue, and the single lock needs to vanish to avoid contention on that > lock. However, that would require dynamically allocatable shared > memory... What about keeping the per-process message queue in the local memory of the process, and exporting only the queue head to the shmem, thus having only one message per-process there. When the queue manager gets a message from the process it may signal that process to copy the next message from the process local memory into the shmem. To keep a correct ordering of queue messages an additional shared memory queue of pid_t can be maintained, containing one pid per each message. -- Alexey Klyukin http://www.commandprompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
