Hello Andres, >> The xReader design was based on the idea that it would be nice not >> to cause load on the master machine, and that by proxying the WAL >> stream to the HS, using synchronous replication style to write from >> xReader to the HS, you could use the HS for a source for that data >> with it being at exactly the right point in time to query it. >Yes, that does make sense for some workloads. I don't think its viable for >everything though, thats why were not aiming for that ourselves atm.
Regarding the above, what would be a case where querying the HS will not suffice? On Sat, Apr 28, 2012 at 4:02 AM, Andres Freund <and...@2ndquadrant.com>wrote: > Hi Kevin, Hi Aakash, > > On Saturday, April 28, 2012 12:18:38 AM Kevin Grittner wrote: > > Andres Freund <and...@2ndquadrant.com> wrote: > > > In the current, prototypal, state there is one component thats > > > integrated into the server (because it needs information thats > > > only available there). > > The xReader design was based on the idea that it would be nice not > > to cause load on the master machine, and that by proxying the WAL > > stream to the HS, using synchronous replication style to write from > > xReader to the HS, you could use the HS for a source for that data > > with it being at exactly the right point in time to query it. > Yes, that does make sense for some workloads. I don't think its viable for > everything though, thats why were not aiming for that ourselves atm. > > > I'm not convinced that I would rather see the logic fixed inside the > > master as opposed to being deployable on the master's machine, the > > slave machine, or even on its own machine in between. > I don't think that you can do everything apart from the master. We > currently > need shared memory for coordination between the moving parts, thats why we > have it inside the master. > It also have the advantage of being easier to setup. > > > > That component is layered ontop of a totally generic xlog > > > reading/parsing library that doesn't care at all where its > > > running. > > That's cool. > > > > Its also used in another cluster to read the received (filtered) > > > stream. > > I don't quite follow what you're saying there. > To interpret the xlog back into something that can be used for replication > you > need to read it again. After filtering we again write valid WAL, so we can > use > the same library on the sending|filtering side and on the receiving side. > But thats actually off topic for this thread ;) > > > > > I took a *very* short glance over the current wiki description of > > > xReader and from that it seems to me it would benefit from trying > > > to make it architecturally more similar to the rest of pg. > > We're planning on using existing protocol to talk between pieces. > > Other than breaking it out so that it can run somewhere other than > > inside the server, and allowing clients to connect to xReader to > > listen to WAL events of interest, are you referring to anything > > else? > It sounds like the xReader is designed to be one multiplexing process. > While > this definitely has some advantages resource-usage-wise it doesn't seem to > be > fitting the rest of the design that well. The advantages might outweigh > everything else, but I am not sure about that. > Something like registering/deregistering also doesn't fit that well with > the > way walsender works as far as I understand it. > > Greetings, > > Andres > -- > Andres Freund http://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Training & Services >
