>> I recently found a test that was TIMEOUT in TestExpectations, and still timed out in Slow, so I left it in TestExpectations under the assumption that it will timeout faster there. What do you mean "timed out in Slow"? A test did not mark as slow, but timeout after 60 seconds? Worth mentioning some slow wpt tests are given the same timeout value as slow tests in the code.
But I did see some tests timed out at some abnormal value. For example tests below timeouts at 22 second. Not sure how this could happen. 19:13:50.493 20120 [3522/4178] external/wpt/webrtc-identity/idlharness.https.window.html failed unexpectedly (test timed out) 22.3300s On Wed, Aug 18, 2021 at 11:26 AM Stephen Chenney <[email protected]> wrote: > > > On Wed, Aug 18, 2021 at 12:01 PM 'Weizhong Xia' via blink-dev < > [email protected]> wrote: > >> Dear blink-devs, >> >> As another effort trying to speed up blink_web_tests on CQ, We are >> looking at slow tests that also timeouts. From the timing stats(example >> <https://test-results.appspot.com/data/layout_results/linux-rel/773524/blink_web_tests%20%28with%20patch%29/layout-test-results/results.html>) >> and some research, there are about 0.25% such tests, but running such tests >> takes about 4% of the total run time. So we are thinking of >> doing something about it. >> >> We would like to propose as below, and get feedback on this: >> 1. We add Skip to test expectations for slow tests that also timeouts. >> This will effectively skip all such tests on swarming bots, and save ~50s >> real time on linux-rel. >> 2. We add a presubmit check to make any unskipped Slow+Timeout an error. >> (We can also make this a warning, but such warning will persist, so kind of >> annoying) >> 3. Devs are encouraged to take actions to either fix the timeout, or >> break the slow tests into pieces to make it not slow. >> >> > I recently found a test that was TIMEOUT in TestExpectations, and still > timed out in Slow, so I left it in TestExpectations under the assumption > that it will timeout faster there. > > The underlying issue is not a slow test but a test that hangs and we may > or may not want to skip such a test. Would identifying these cases and > moving them from Slow to TestExpectations have any noticeable impact? > > >> Please let us know your opinions on this. >> >> Thanks, Weizhong >> >> -- >> You received this message because you are subscribed to the Google Groups >> "blink-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADXrSipU3NDSp7tWd%3DqA%3DrZWMrmXjfT4NzTG7U8EoEAgsQ1QXw%40mail.gmail.com >> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADXrSipU3NDSp7tWd%3DqA%3DrZWMrmXjfT4NzTG7U8EoEAgsQ1QXw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADXrSip%3DRz%3D46ZfZ5nyg10ORBwKg2tcFAX%2Bh0di%3D98Q50R5%3DrA%40mail.gmail.com.
