On 30.10.2023 22:51, Andres Freund wrote:
There's really no point in comparing peformance with assertions enabled (leaving aside assertions that cause extreme performance difference, making development harder). We very well might have added assertions making things more expensive, without affecting performance in optimized/non-assert builds.
Thanks for advice! I repeated measurements on my pc without asserts and CFLAGS="-O2". Also i reduced the number of clients to -c6 to leave a reserve of two cores from my 8-core cpu and used -j6 accordingly. The results were similar: on my pc REL_10_STABLE(c18c12c9) was faster than REL_16_STABLE(07494a0d) but the effect became weaker: REL_10_STABLE gives ~965+-15 TPS(+-2%) while REL_16_STABLE gives ~920+-30 TPS(+-3%) in the test: pgbench -s8 -c6 -T20 -j6 So 10 is faster than 16 by ~5%. (see raw-my-pc.txt attached for the raw data) Then, thanks to my colleagues, i carried out similar measurements on the more powerful 24-core standalone server. The REL_10_STABLE gives 8260+-100 TPS(+-1%) while REL_16_STABLE gives 8580+-90 TPS(+-1%) in the same test: pgbench -s8 -c6 -T20 -j6 The test gave an opposite result! On that server the 16 is faster than 10 by ~4%. When i scaled the test on server to get the same reserve of two cores, the results became like this: REL_10_STABLE gives ~16000+-300 TPS(+-2%) while REL_16_STABLE gives ~18500+-200 TPS(+-1%) in the scaled test: pgbench -s24 -c22 -T20 -j22 Here the difference is more noticeable: 16 is faster than 10 by ~15%. (raw-server.txt) The configure options and test scripts on my pc and server were the same: export CFLAGS="-O2" ./configure --enable-debug --with-perl --with-icu --enable-depend --enable-tap-tests #reinstall #reinitdb #create database bench for ((i=0; i<100; i++)); do pgbench -U postgres -i -s8 bench> /dev/null 2>&1; psql -U postgres -d bench -c "checkpoint"; RES=$(pgbench -U postgres -c6 -T20 -j6 bench; Configurations: my pc: 8-core AMD Ryzen 7 4700U @ 1.4GHz, 64GB RAM, NVMe M.2 SSD drive. Linux 5.15.0-88-generic #98~20.04.1-Ubuntu SMP Mon Oct 9 16:43:45 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux server: 2x 12-hyperthreaded cores Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 24GB RAM, RAID from SSD drives. Linux 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux I can't understand why i get the opposite results on my pc and on the server. It is clear that the absolute TPS values will be different for various configurations. This is normal. But differences? Is it unlikely that some kind of reference configuration is needed to accurately measure the difference in performance. Probably something wrong with my pc, but now i can not figure out what's wrong. Would be very grateful for any advice or comments to clarify this problem. With the best wishes! -- Anton A. Melnikov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
REL_10_STABLE c18c12c983a84d55e58b176969782c7ffac3272b pgbench -s8 -c6 -T20 -j6 940.47198 954.621585 902.319686 965.387517 959.44536 970.882218 922.012141 969.642272 964.549628 935.639076 958.835093 976.912892 975.618375 960.599515 981.900039 973.34447 964.563699 960.321335 962.643262 975.631214 971.78315 965.226256 961.106572 968.520002 973.825485 978.49579 963.863368 973.906058 966.676175 965.186708 954.572371 977.620229 981.419347 962.751969 963.676599 967.966257 974.68403 955.342462 957.832817 984.065968 972.364891 977.489316 957.352265 969.463156 974.320994 949.679765 969.081674 963.190554 962.394456 966.84177 975.840044 954.471689 977.764019 968.67597 963.203923 964.752374 965.957151 979.17749 915.412491 975.120789 962.105916 980.343235 957.180492 953.552183 979.783099 967.906392 966.926945 962.962301 965.53471 971.030289 954.21045 961.266889 973.367193 956.736464 980.317352 911.188865 979.274233 980.267316 982.029926 977.731543 937.327052 978.161778 978.575841 962.661776 914.896072 966.902901 973.539272 980.418576 966.073472 963.196341 962.718863 977.062467 958.303102 959.937588 959.52382 934.876632 971.899844 979.716666 964.154208 960.051284 REL_16_STABLE 07494a0df9a66219e5f1029de47ecabc34c9cb55 pgbench -s8 -c6 -T20 -j6 952.061203 905.964458 921.009294 921.970342 924.810464 935.988344 917.110599 925.62075 933.423024 923.445651 932.740532 927.994569 913.773152 922.955946 917.680486 923.145074 925.133017 922.36253 907.656249 927.980182 924.249294 933.355461 923.359649 919.694726 923.178731 929.250348 921.643735 916.546247 930.960814 913.333819 773.157522 945.293028 924.839608 932.228796 912.846096 924.01411 924.341422 909.823505 882.105606 920.337305 930.297982 909.238148 880.839643 939.582115 927.263785 927.921499 932.897521 931.77316 943.261293 853.433421 921.813303 916.463003 919.652647 914.662188 912.137913 923.279822 922.967526 936.344334 946.281347 801.718759 950.571673 928.845848 888.181388 885.603875 939.763546 896.841216 934.904546 929.369005 884.065433 874.953048 933.411683 930.654935 952.833611 942.193108 930.491705 931.869631 946.252479 934.395527 928.942777 946.957957 935.907976 931.704551 937.455759 936.721128 877.313871 919.203148 919.044372 917.102309 935.013909 952.88992 865.529703 965.71241 944.114434 966.89352 904.204782 960.964733 940.664823 951.03435 952.043972 859.82228
REL_10_STABLE c18c12c983a84d55e58b176969782c7ffac3272b pgbench -s8 -c6 -T20 -j6 7840.693706 8147.071773 8233.186916 8250.830768 8184.034761 8372.01559 8056.35878 8335.101362 8203.906394 8412.365372 8169.353655 8276.23181 8245.978862 8238.438713 8053.687042 8319.898895 8331.894327 8298.678374 8301.808427 8312.989865 8346.792722 8382.794572 8171.537854 8240.419933 8136.62466 8403.803267 8245.406916 8410.019814 8347.167928 8355.415348 8298.360081 8209.591048 8366.338451 8201.620847 8134.579489 8390.476163 8090.260512 8353.933054 8319.527375 8343.414127 8131.089684 8061.514678 8321.30137 8318.344408 8304.265262 8223.272819 8229.860196 8225.85561 8266.017087 8213.108465 8404.505357 8215.726178 8190.075622 8230.62093 8294.530674 8379.980778 8216.212552 8217.986697 8257.644883 8355.518235 8227.872894 8345.789811 8334.769139 8230.523975 8233.4346 8371.661881 8262.587211 8312.084021 8079.924087 8244.781066 8176.863467 8337.782794 8348.035689 8349.712662 8172.758098 8322.627133 8213.904947 8372.890373 8240.018958 8130.357649 8311.177862 8341.409614 8123.134534 8182.18376 8158.525092 8188.105123 8273.306547 8191.176745 8255.705237 8123.987312 8102.479847 8308.55259 8276.491599 8264.646608 8278.179617 8227.334952 8384.06559 8244.820361 8310.870179 8375.361794 pgbench -s24 -c22 -T20 -j22 14670.97732 16280.58069 15993.26409 15965.93638 16180.65584 15997.61952 15979.49376 15779.96428 16604.66307 15370.58819 16055.27501 16080.55212 16061.197 15950.12869 16016.25484 16107.18508 16081.52684 16140.69095 15999.86662 16135.87569 15835.69345 16198.561 15557.45313 16675.87414 15527.75386 16033.38508 16203.52902 16227.36199 16077.63614 16027.29064 16039.09341 16122.17816 16261.08805 16124.3244 16036.03852 15738.36594 16104.93231 15481.08531 16757.70292 15376.84798 16245.11218 16047.52162 16203.21809 16172.47835 16211.98256 16029.27521 16042.29896 16182.66251 15999.55122 15951.99975 15705.90974 16309.73973 15515.73408 16937.98549 15522.88782 16205.6509 16030.3237 16071.44969 16039.10342 16181.45884 16052.78653 16058.99637 16121.16257 16067.46719 16234.85053 15766.12082 16559.75986 15359.00212 16063.79464 15831.93905 15860.46265 16093.73441 15984.63738 15921.44079 16194.0051 16163.17878 16084.58289 16227.65736 16055.95145 16251.74886 15778.80999 16454.83073 15360.7492 16232.04935 15771.64502 16007.08101 16089.4596 16180.18315 16064.22501 16141.3966 15995.38539 15958.60448 16168.70552 15974.17096 16044.83537 15837.72296 16410.33025 15339.01219 16088.30394 15823.30683 REL_16_STABLE 07494a0df9a66219e5f1029de47ecabc34c9cb55 pgbench -s8 -c6 -T20 -j6 8323.140992 8453.126802 8675.329445 8725.157925 8572.174643 8694.175461 8448.748794 8714.783903 8644.842848 8690.417992 8553.739161 8588.304348 8586.922898 8651.634989 8727.839552 8565.391338 8701.208809 8503.500521 8618.269499 8439.424117 8355.826108 8696.132649 8476.596036 8685.45175 8550.58631 8507.20698 8600.125207 8508.551657 8708.530877 8642.310593 8598.300883 8506.088833 8562.622651 8554.061049 8597.458751 8579.958381 8559.983846 8653.547563 8612.277298 8603.577499 8702.12649 8612.316781 8495.42618 8641.637982 8596.358783 8640.419798 8657.906263 8515.134297 8692.207802 8666.020161 8515.183983 8628.556475 8574.995822 8626.0342 8579.35097 8688.545273 8619.156718 8749.13384 8487.756379 8495.88 8578.618896 8489.395544 8645.100089 8626.326069 8584.79179 8701.603466 8576.109238 8440.848751 8405.201458 8507.882038 8604.625392 8682.466924 8514.519804 8630.914742 8527.881986 8595.126406 8678.12195 8437.196193 8434.612811 8670.784858 8703.195124 8677.139521 8462.17992 8537.011829 8523.817027 8643.500882 8550.982508 8607.800358 8641.190713 8499.817025 8551.580038 8618.370846 8564.797787 8567.042181 8448.692604 8572.953214 8433.159559 8711.516107 8516.455862 8648.265466 pgbench -s24 -c22 -T20 -j22 17799.74432 18414.42326 18538.7228 18502.45134 18353.97147 18397.70913 18632.98643 18172.07613 18508.24122 18426.30082 18549.3725 18559.39782 18517.50255 18574.20969 18452.69943 18552.12662 18545.99002 18610.7133 18544.61722 18502.4523 18648.06364 18516.80614 18607.7642 18587.10349 18329.07479 18596.39038 17977.47212 18595.01766 18459.95889 18528.75469 18499.40727 18486.30486 18649.35122 18484.083 17987.46908 18523.41745 18552.45687 18588.554 18402.85749 18533.71559 18587.0482 18480.97158 18550.65806 18582.43771 18621.81726 17986.45714 18598.61691 18446.63074 18539.75579 18497.40008 18483.05235 18550.32183 18430.77558 18560.73804 18622.42222 18580.30253 18547.15016 18514.79711 18384.98024 18554.39619 18550.49649 17940.61537 18420.79578 18523.89656 17847.73404 18467.42391 18573.31865 18547.98826 18563.02676 18377.2113 18558.15092 18407.73887 18574.10229 18457.18218 18510.97421 18492.4137 18538.4266 18533.83723 18617.23778 18602.23771 18485.08766 18638.45303 18531.64026 18117.9283 18301.18523 18598.41985 18622.51447 18595.8512 18491.64099 18574.22862 18258.11008 18641.8935 18529.34347 18489.35226 18428.49614 18521.3419 18531.61554 18550.74378 18540.10764 18534.75483
