On Sun, Jan 8, 2012 at 2:25 PM, Simon Riggs <si...@2ndquadrant.com> wrote:
> I've taken that idea and used it to build a second Clog cache, known > as ClogHistory which allows access to the read-only tail of pages in > the clog. Once a page has been written to for the last time, it will > be accessed via the ClogHistory Slru in preference to the normal Clog > Slru. This separates historical accesses by readers from current write > access by committers. Historical access doesn't force dirty writes, > nor are commits made to wait when historical access occurs. Why do we need this in 9.2? We now have clog_buffers = 32 and we have write rates ~16,000 tps. At those write rates we fill a clog buffer every 2 seconds, so the clog cache completely churns every 64 seconds. If we wish to achieve those rates in the real world, any access to data that was written by a transaction more than a minute ago will cause clog cache page faults, leading to stalls in new transactions. To avoid those problems we need * background writing of the clog LRU (already posted as a separate patch) * a way of separating access to historical data from the main commit path (this patch) And to evaluate such situations, we need a way to simulate data that contains many transactions. 32 buffers can hold just over 1 million transaction ids, so benchmarks against databases containing > 10 million separate transactions are recommended (remembering that this is just 10 mins of data on high TPS systems). A pgbench patch is provided separately to aid in the evaluation. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
