Re: Strategy for materialisation and centralisation of data

2020-05-21 Thread Rory Campbell-Lange
On 21/05/20, Michael Stephenson (domehead...@gmail.com) wrote: > On Thu, May 21, 2020 at 10:36 AM Rory Campbell-Lange wrote > > Presently I've been thinking of using triggers or materialized views in > > each database to materialise data into a "matview" schema which is then > > shipped via logica

Re: Strategy for materialisation and centralisation of data

2020-05-21 Thread Michael Stephenson
You might find Materialize interesting: https://materialize.io/ https://youtu.be/zWSdkGq1XWk On Thu, May 21, 2020 at 10:36 AM Rory Campbell-Lange < r...@campbell-lange.net> wrote: > We have quite a few databases of type a and many of type b in a cluster. > Both a and b types are fairly complex

Re: Strategy for materialisation and centralisation of data

2020-05-21 Thread Karsten Hilbert
On Thu, May 21, 2020 at 09:52:02PM +0100, Rory Campbell-Lange wrote: > > Might postgres_fdw help in any way ? > > Thanks for the suggestion. As I noted we are already using pl/proxy and > it works well, although we are soaking up a lot of connections with it. > >From my reading of the postgres_fdw

Re: Strategy for materialisation and centralisation of data

2020-05-21 Thread Rory Campbell-Lange
On 21/05/20, Karsten Hilbert (karsten.hilb...@gmx.net) wrote: > On Thu, May 21, 2020 at 03:35:59PM +0100, Rory Campbell-Lange wrote: > > > We have quite a few databases of type a and many of type b in a cluster. > > Both a and b types are fairly complex and are different solutions to a > > similar

Re: Strategy for materialisation and centralisation of data

2020-05-21 Thread Karsten Hilbert
On Thu, May 21, 2020 at 03:35:59PM +0100, Rory Campbell-Lange wrote: > We have quite a few databases of type a and many of type b in a cluster. > Both a and b types are fairly complex and are different solutions to a > similar problem domain. All the databases are very read-centric, and all > dat

Strategy for materialisation and centralisation of data

2020-05-21 Thread Rory Campbell-Lange
We have quite a few databases of type a and many of type b in a cluster. Both a and b types are fairly complex and are different solutions to a similar problem domain. All the databases are very read-centric, and all database interaction is currently through plpgsql with no materialised data. Some