Hi, I wanted to float another idea to solve these tablesync/apply worker problems.
This idea may or may not have merit. Please consider it. ~ Basically, I was wondering why can't the "tablesync" worker just gather messages in a similar way to how the current streaming feature gathers messages into a "changes" file, so that they can be replayed later. e.g. Imagine if A) The "tablesync" worker (after the COPY) does not ever apply any of the incoming messages, but instead it just gobbles them into a "changes" file until it decides it has reached SYNCDONE state and exits. B) Then, when the "apply" worker proceeds, if it detects the existence of the "changes" file it will replay/apply_dispatch all those gobbled messages before just continuing as normal. So - IIUC this kind of replay is like how the current code stream commit applies the streamed "changes" file. - "tablesync" worker would only be doing table sync (COPY) as its name suggests. Any detected "changes" are recorded and left for the "apply" worker to handle. - "tablesync" worker would just operate in single tx with a temporary slot as per current code - Then the "apply" worker would be the *only* worker that actually applies anything. (as its name suggests) Thoughts? --- Kind Regards, Peter Smith. Fujitsu Australia
