On Tue, Dec 19, 2017 at 3:07 PM, Kellner Thiemo <thiemo.kell...@usb.ch> wrote:
> Hi > > We are developing a data warehouse of which the integration layer will > start with over 100 TB of data. There are not many entities though we > probably can partition and foremost we should use inheritance for the lab > results. I just was wondering if PostgreSQL was able to cope with. In case > it depends on the modelling kind, we have not yet decided between classic > erd, anchor modelling and data vault. > > Does someone have experience with such a set up? > There are a significant number of issues that happen when you try to put that much data in PostgrSQL. Remember there is a hard limit of 32TB per table. I currently help administer an analytics environment where 400TB of data is pre-aggregated into 32TB of ready-to-serve metrics. We generally try to keep our initial point of entry databases to under 20TB of possible. Nonetheless it is quite possible either using distributed add-ons like Citus, forks like Postgres-XL, or (as we did at Adjust) a mixture of data consolidation and application-level sharding. As a plug, I expect to deliver talks about this in various places. Stay tuned ;-) > > Kind regards > > Thiemo > > -- Best Wishes, Chris Travers Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in. http://www.efficito.com/learn_more
