Hi Peter, Peter Smith <smithpb2...@gmail.com>, 11 Tem 2023 Sal, 05:59 tarihinde şunu yazdı: > Even if patches 0003 and 0002 are to be combined, I think that should > not happen until after the "reuse" design is confirmed which way is > best. > > e.g. IMO it might be easier to compare the different PoC designs for > patch 0002 if there is no extra logic involved. > > PoC design#1 -- each tablesync decides for itself what to do next > after it finishes > PoC design#2 -- reuse tablesync using a "pool" of available workers
Right. I made a patch 0003 to change 0002 so that tables will be assigned to sync workers by apply worker. It's a rough POC and ignores some edge cases. But this is what I think how apply worker would take the responsibility of table assignments. Hope the implementation makes sense and I'm not missing anything that may cause degraded perforrmance. PoC design#1 --> apply only patch 0001 and 0002 PoC design#2 --> apply all patches, 0001, 0002 and 0003 Here are some quick numbers with 100 empty tables. +--------------+----------------+----------------+----------------+ | | 2 sync workers | 4 sync workers | 8 sync workers | +--------------+----------------+----------------+----------------+ | POC design#1 | 1909.873 ms | 986.261 ms | 552.404 ms | +--------------+----------------+----------------+----------------+ | POC design#2 | 4962.208 ms | 1240.503 ms | 1165.405 ms | +--------------+----------------+----------------+----------------+ | master | 2666.008 ms | 1462.012 ms | 986.848 ms | +--------------+----------------+----------------+----------------+ Seems like design#1 is better than both design#2 and master overall. It's surprising to see that even master beats design#2 in some cases though. Not sure if that is expected or there are some places to improve design#2 even more. What do you think? PS: I only attached the related patches and not the whole patch set. 0001 and 0002 may contain some of your earlier reviews, but I'll send a proper updated set soon. Thanks, -- Melih Mutlu Microsoft
v18-0002-Reuse-Tablesync-Workers.patch
Description: Binary data
v18-0001-Refactor-to-split-Apply-and-Tablesync-Workers.patch
Description: Binary data
v18-0003-apply-worker-assigns-tables.patch
Description: Binary data
