Unfortunately, there's > 175K files under //third_party/blink/web_tests, which would increase the size of the build files *a lot* and I have no idea what that would do to GN and Ninja run times ... But, it is theoretically possible to do so.
-- Dirk On Tue, Aug 17, 2021 at 10:58 AM Ken Rockot <[email protected]> wrote: > Would it be a bad idea to add build rules covering all web tests, and > manually maintain lists of test sources like we do for all other source > files in the tree? > > On Tue, Aug 17, 2021 at 10:48 AM Ian Kilpatrick <[email protected]> > wrote: > >> So I was a little surprised by this last night. >> >> 1. This was relatively slow on my macbook, running locally takes ~40s. >> From just now: >> >> ikilpatrick-macbookpro:src ikilpatrick$ touch >> third_party/blink/web_tests/external/wpt/css/css-tables/test.html >> >> ikilpatrick-macbookpro:src ikilpatrick$ time >> third_party/blink/tools/collect_web_tests.py external/wpt >> >> >> real 0m40.492s >> >> user 0m23.007s >> >> sys 0m41.715s >> >> 2. I had a bunch of uncommitted tests in various wpt directories that I >> needed to move out to somewhere else (rather than the script just adding >> committed tests). >> >> (I'd also note that uploading patches is already relatively slow, do we >> have metrics for how long developers are waiting for upload scripts to run? >> - I suspect this is due to WPT presubmit scripts but unsure) >> >> Ian >> >> >> >> On Tue, Aug 17, 2021 at 10:20 AM 'Dirk Pranke' via blink-dev < >> [email protected]> wrote: >> >>> >>> >>> On Tue, Aug 17, 2021 at 10:08 AM 'Weizhong Xia' via blink-dev < >>> [email protected]> wrote: >>> >>>> tl;dr: We introduced a mechanism that pre-collects test cases to >>>> //third_party/blink/web_tests/AllTestsByDirectories.json, and uses a >>>> presubmit check to ensure this file is up to date. You can stop here if you >>>> don't write web test cases. >>>> >>>> Dear blink-devs, >>>> >>>> You may have noticed that when you try to upload a CL, you are prompted >>>> to update AllTestsByDirectories.json if your CL changes web test cases. >>>> Yes, today we landed a CL >>>> <https://chromium-review.googlesource.com/c/chromium/src/+/3067372> >>>> that pre-collects test cases and saves that to the json file. The reason to >>>> do this is that rwt tries to collect tests on every swarming bots, and that >>>> takes about 30 seconds (the best case when running on SSD). This is not a >>>> small amount of time as we are targeting a 10-min CQ. After this CL lands, >>>> collecting tests now takes less than 2 seconds for all platforms. >>>> >>>> Impact to you: if your CL adds/deletes/renames web test cases, you will >>>> be prompted. Follow the instructions there should be enough. But if you >>>> still have an issue, (even though we have tested many different scenarios >>>> we can think about), feel free to ping/email me. >>>> >>>> There is no change to how you run rwt locally. We added a new switch to >>>> tell the swarming bots to load tests from the json file. This switch is >>>> turned off by default. Developers usually don't need to turn this on, >>>> unless all of your tests are saved to the json file and you are running a >>>> full test so want to save about 30 seconds. >>>> >>> >>> FWIW, I think this is an important and probably worthwhile change >>> (disclaimer: I did review it and was involved in the design). The number of >>> tests we have now is substantial and the startup delay is a significant >>> cost. We need to be looking into ways to reduce this. >>> >>> I think it's possible we should turn this on by default and/or do some >>> hooking to make things aware of when you've added new tests (e.g., via git >>> status) to try and make this a little more seamless, but I also think >>> landing it off by default was a good first step. >>> >>> Some may ask if we can just generate this at compile time. >>> Unfortunately, no, we can't, because there's no way to keep it correctly >>> up-to-date when you do add tests after having generated the file at least >>> once [ we asked this ourselves and actually tried this approach and broke >>> deterministic builds :( ]. >>> >>> I'm happy to hear other thoughts. Perhaps there are other ways we can >>> optimize things to be fast enough ... >>> >>> -- Dirk >>> >>> >>>> >>>> Thanks for your time! >>>> >>>> Cheers, Weizhong >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "blink-dev" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADXrSiprRWy_UCC0w2oohbSGGTbeP%3DdkyYnBOx7j8Y-abDSe2A%40mail.gmail.com >>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADXrSiprRWy_UCC0w2oohbSGGTbeP%3DdkyYnBOx7j8Y-abDSe2A%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "blink-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAEoffTAc389E6wteVXUAuKX8ZBFWBa2UEMwAbd9oCxJ9pxNCKQ%40mail.gmail.com >>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAEoffTAc389E6wteVXUAuKX8ZBFWBa2UEMwAbd9oCxJ9pxNCKQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "blink-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJL3UpS64fRvbJLf%2B3uFA_6UDgXoLWgUFGZSN7mAsVFOrP8syA%40mail.gmail.com >> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJL3UpS64fRvbJLf%2B3uFA_6UDgXoLWgUFGZSN7mAsVFOrP8syA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAEoffTCi7yqKgYic0F7kZbuyYcmhaE5wz2ZvUucjsgp8hyiCww%40mail.gmail.com.
