Re: PG Sharding

2018-02-02 Thread Matej
I thought that this kind of solution had high latency and bad OLTP capabilities (low trans/second)? Analytics is not a high priority. BR 2018-02-01 19:01 GMT+01:00 Dan Wierenga : > > > On Wed, Jan 31, 2018 at 7:48 PM, Steven Lembark > wrote: > >> On Mon, 29 Jan 2018 15:34:18 +0100 >> Matej wro

Re: PG Sharding

2018-02-01 Thread Dan Wierenga
On Wed, Jan 31, 2018 at 7:48 PM, Steven Lembark wrote: > On Mon, 29 Jan 2018 15:34:18 +0100 > Matej wrote: > > > Hi Everyone. > > > > We are looking at a rather large fin-tech installation. But as > > scalability requirements are high we look at sharding of-course. > > > > I have looked at many

Re: PG Sharding

2018-01-31 Thread Steven Lembark
On Mon, 29 Jan 2018 15:34:18 +0100 Matej wrote: > Hi Everyone. > > We are looking at a rather large fin-tech installation. But as > scalability requirements are high we look at sharding of-course. > > I have looked at many sources for Postgresql sharding, but we are a > little confused as to sh

Re: PG Sharding

2018-01-31 Thread Matej
Thanks Thomas. Still fancying the manual approach litlle bit more. Will probably go with 8 database and 32 schema per machine. This way, will keep in limits for administration tools as well as autovacuum, also will be ready for connection pooling, as 8 databases is not to much. This will give us

Re: PG Sharding

2018-01-30 Thread Thomas Boussekey
Using citusdb enterprise, you can replicate the table shards. Here is the link to the documentation: https://docs.citusdata.com/en/v7.2/reference/user_defined_functions.html#replicate-table-shards Regards, Thomas 2018-01-30 12:18 GMT+01:00 Matej : > As already said. It's missing 2 level shardi

Re: PG Sharding

2018-01-30 Thread Matej
As already said. It's missing 2 level sharding and is restricted with SPOF. BR Matej 2018-01-30 12:05 GMT+01:00 Rakesh Kumar : > > > > >We are looking for multi tenancy but at scale. That's why the sharding > and partitioning. It depends how you look at the distributed part. > > Citusdb. >

Re: PG Sharding

2018-01-30 Thread Rakesh Kumar
>We are looking for multi tenancy but at scale. That's why the sharding and >partitioning. It depends how you look at the distributed part. Citusdb.

Re: PG Sharding

2018-01-30 Thread Matej
We are looking for multi tenancy but at scale. That's why the sharding and partitioning. It depends how you look at the distributed part. BR Matej 29. jan. 2018 17.50 je oseba "Rakesh Kumar" napisala: > > > > On Jan 29, 2018, at 09:34 , Matej wrote: > > > > Hi Everyone. > > > > We are looking

Re: PG Sharding

2018-01-29 Thread Rakesh Kumar
> On Jan 29, 2018, at 09:34 , Matej wrote: > > Hi Everyone. > > We are looking at a rather large fin-tech installation. But as scalability > requirements are high we look at sharding of-course. > > I have looked at many sources for Postgresql sharding, but we are a little > confused as to

Re: PG Sharding

2018-01-29 Thread Konstantin Gredeskoul
ter/simple-feed Best, Konstantin __ Konstantin Gredeskoul https://kig.re/ https://reinvent.one/ (415) 265-1054 From: Matej Sent: Monday, January 29, 2018 7:49:19 AM To: Thomas Boussekey Cc: Melvin Davidson; pgsql-general@lists.postgresql.org Subject: Re: PG Shar

Re: PG Sharding

2018-01-29 Thread Thomas Boussekey
Hello Matej, I found some interesting implementation of postgres_XL at TenCent(WeChat) and Javelin. You can find video capture of conferences of IT people from these companies. Moreover, I attended to PgConf.eu at Warsaw in October, and The ESA (European Space Agency) made a lightning talk on thei

Re: PG Sharding

2018-01-29 Thread Matej
Hi Thomas. Thanks. Also looked at those solutions: - PGXL Am a ltille afraid we would be the test dummies. Did not hear of many production installs. - Citus seems heavily limited scalability vise, because of the master node design. Regarding partitioning we are considering ourselves pg_pathman.

Re: PG Sharding

2018-01-29 Thread Thomas Boussekey
Hello, Facing the same situation, I'm considering 3 solutions: - Sharding with postgres_xl (waiting for a Pg10 release) - Sharding with citusdata (Release 7.2, compatible with Pg10 and pg_partman, seems interesting) - Partitioning with PG 10 native partitioning or pg_partman With colleagues, we h

Re: PG Sharding

2018-01-29 Thread Melvin Davidson
On Mon, Jan 29, 2018 at 9:34 AM, Matej wrote: > Hi Everyone. > > We are looking at a rather large fin-tech installation. But as scalability > requirements are high we look at sharding of-course. > > I have looked at many sources for Postgresql sharding, but we are a little > confused as to shared

PG Sharding

2018-01-29 Thread Matej
Hi Everyone. We are looking at a rather large fin-tech installation. But as scalability requirements are high we look at sharding of-course. I have looked at many sources for Postgresql sharding, but we are a little confused as to shared with schema or databases or both. So far our understandin