On 5/31/22 16:36, Ranier Vilela wrote: > Em dom., 29 de mai. de 2022 às 17:10, Ranier Vilela <ranier...@gmail.com > <mailto:ranier...@gmail.com>> escreveu: > > Em dom., 29 de mai. de 2022 às 15:21, Andres Freund > <and...@anarazel.de <mailto:and...@anarazel.de>> escreveu: > > On 2022-05-29 18:00:14 +0200, Tomas Vondra wrote: > > Also, there's the question of correctness, and I'd bet Andres > is right > > getting snapshot without holding ProcArrayLock is busted. > > Unless there's some actual analysis of this by Rainier, I'm just > going to > ignore this thread going forward. It's pointless to invest time when > everything we say is just ignored. > > Sorry, just not my intention to ignore this important point. > Of course, any performance gain is good, but robustness comes first. > > As soon as I have some time. > > I redid the benchmarks, with getting a snapshot with holding ProcArrayLock. > > Average Results > > > > > Connections: > tps head tps patch diff > 1 39196,3088985 39858,0207936 661,711895100008 101,69% > 2 65050,8643819 65245,9852367 195,1208548 100,30% > 5 91486,0298359 91862,9026528 376,872816899995 100,41% > 10 131318,0774955 131547,1404573 229,062961799995 100,17% > 50 116531,2334144 116687,0325522 155,799137800001 100,13% > 100 98969,4650449 98808,6778717 -160,787173199991 99,84% > 200 89514,5238649 89463,6196075 -50,904257400005 99,94% > 300 88426,3612183 88457,2695151 30,9082968000002 100,03% > 400 88078,1686912 88338,2859163 260,117225099995 100,30% > 500 87791,1620039 88074,3418504 283,179846500003 100,32% > 600 87552,3343394 87930,8645184 378,530178999994 100,43% > 1000 86538,3772895 86771,1946099 232,817320400005 100,27% > avg 89204,4088731917 89420,444631825 1981,0816042 100,24% > > > For clients with 1 connections, the results are good.
Isn't that a bit strange, considering the aim of this patch was scalability? Which should improve higher client counts in the first place. > But for clients with 100 and 200 connections, the results are not good. > I can't say why these two tests were so bad. > Because, 100 and 200 results, I'm not sure if this should go ahead, if > it's worth the effort. > I'd argue this is either just noise, and there's no actual difference. This could be verified by some sort of statistical testing (e.g. the well known t-test). Another option is that this is simply due to differences in binary layout - this can result in small differences (easily 1-2%) that are completely unrelated to what the patch does. This is exactly what the "stabilizer" talk I mentioned a couple days ago was about. FWIW, when a patch improves scalability, the improvement usually increases with the number of clients. So you'd see no/small improvement for 10 clients, 100 clients would be improved more, 200 more, etc. We see nothing like that here. So either the patch does not really improve anything, or perhaps the benchmark doesn't even hit the bottleneck the patch is meant to improve (which was already suggested in this thread repeatedly). regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
