On Mon, May 31, 2021 at 12:33:44PM +0500, Andrey Borodin wrote: > Would it make sense to run our own benchmarks?
Yes, I think that it could be a good idea to run some custom-made benchmarks as that could mean different bottlenecks found when it comes to PG. There are a couple of factors that matter here: - Is the algo available across a maximum of platforms? ZLIB and LZ4 are everywhere and popular, for one. And we already plug with them in the builds. No idea about the others but I can see quickly that Zstd has support across many systems, and has a compatible license. - Speed and CPU usage. We should worry about that for CPU-bounded environments. - Compression ratio, which is just monitoring the difference in WAL. - Effect of the level of compression perhaps? - Use a fixed amount of WAL generated, meaning a set of repeatable SQL queries, with one backend, no benchmarks like pgbench. - Avoid any I/O bottleneck, so run tests on a tmpfs or ramfs. - Avoid any extra WAL interference, like checkpoints, no autovacuum running in parallel. It is not easy to draw a straight line here, but one could easily say that an algo that reduces a FPI by 90% costing two times more CPU cycles is worse than something doing only a 70%~75% compression for two times less CPU cycles if environments are easily constrained on CPU. As mentioned upthread, I'd recomment to design tests like this one, or just reuse this one: https://www.postgresql.org/message-id/cab7npqsc97o-ue5paxfmukwcxe_jioyxo1m4a0pmnmyqane...@mail.gmail.com In terms of CPU usage, we should also monitor the user and system times of the backend, and compare the various situations. See patch 0003 posted here that we used for wal_compression: https://www.postgresql.org/message-idCAB7nPqRC20=mKgu6d2st-e11_QqqbreZg-=sf+_uysmvwnu...@mail.gmail.com This just uses getrusage() to get more stats. -- Michael
signature.asc
Description: PGP signature
