Hi all, I'm sharing an update with results of the experiments over 21 days on Chrome Beta M102.
The results below are mostly small percentages changes, and the finch guidelines <https://groups.google.com/a/google.com/g/chrome-catan/c/ZL-FCrq-Q40/m/ntQk88UFBgAJ?utm_medium=email&utm_source=footer> says to trust results on Stable. Given no issue have been raised AFAIK I would like to roll out a 1% finch experiment on Stable and I'm wondering what would be the required steps to do this (besides flipping boxes on the launch crbug.com/1298967 <http://crbug.com/1298967>) without amending what the spec <https://html.spec.whatwg.org/multipage/timers-and-user-prompts.html> says just yet. The experiment MaxUnthrottledTimeoutNestingLevel <https://docs.google.com/document/u/1/d/1boT0k8BQjl7mXXzvI9SdN4XJPSza27vE8T0CNxmMhCI/edit> has been running on canary/dev/beta M101 and M102. Another experiment SetTimeoutZeroWithoutClamping <http://crbug.com/1263190> which removes setTimeout(,0) clamping to 1 ms is running at Enabled(25%) and Default(50%) on Beta. Analyzing the results for both experiments together makes sense because our goal is to avoid delayed tasks for immediate setTimeout()s. MaxUnthrottledTimeoutNestingLevel (50% enabled / 50% control) + SetTimeoutZeroWithoutClamping (Default + Enabled is 75%) Finch Win <https://uma.googleplex.com/p/chrome/variations?sid=d5a6fc8df0a156c8cf12f5271ee633ac> , No impact on Guiding Metrics Finch Mac <https://uma.googleplex.com/p/chrome/variations?sid=4720de277607785af52a737fcf5dad82> -15.85% Time To Paint For Each Ad Frame @ 95%ile 2-diamonds Finch Android <https://uma.googleplex.com/p/chrome/variations?sid=61617c1a3a8be13ab86afd5efd47eac7> -1.78% First Input Delay @ 99%ile 1-diamonds -0.74% Time to Largest Contentful Paint @ 99%ile 1-diamonds -0.79% Cumulative Layout Shift @ 95%ile 2-diamonds -1.45 / 1.26% Startup Time @ 95 / 99%ile 1 / 2-diamonds +1.93% Scroll Latency Count 1-diamonds +0.32% Memory: Renderer@ 50%ile 1-diamonds MaxUnthrottledTimeoutNestingLevel ignoring SetTimeoutZeroWithoutClamping Note: Results here are less interesting since our goal is to avoid delayed tasks for immediate setTimeout()s, which is only achieved under SetTimeoutZeroWithoutClamping. Finch Win <https://uma.googleplex.com/p/chrome/variations?sid=a67092052bb30ce06b7d71914f174a68> +1.73% Time to First Contentful Paint @ 95%ile 2-diamonds -1.28% Frame Drawing Interval @ 95%ile 1-diamonds -1.44% Startup Time @ 99%ile 1-diamonds Note: the downside of experimenting on Stable is that the intersection of both 1% experiments is very small and we probably won't be able to analyze results alongside before SetTimeoutZeroWithoutClamping ships (this should happen soon though). Maybe a console warning message if we hit the nesting level that we plan to > increase? I'm not sure how effective that would be in helping get folks to > try it. > I'm trying this out here <https://chromium-review.googlesource.com/c/chromium/src/+/3668605>. On Wed, Mar 30, 2022 at 2:19 PM Scott Haseley <[email protected]> wrote: > On Wed, Mar 30, 2022 at 10:41 AM Etienne Pierre-doray < > [email protected]> wrote: > >> Thanks! >>> >>> What's the plan for finding sites this breaks? Monitor bug reports? Or >>> is there something more proactive we could do? >>> >> I'm thinking about bug reports and guardian metrics. shaseley@ might >> have additional insight from running a similar experiment >> crbug.com/1263190 >> > > Yeah that's what I was thinking; that's the approach we're taking for > removing setTimeout(0) 1 ms clamping. It would be helpful if devs could try > this out and report issues (it's enabled behind a flag > <https://groups.google.com/a/chromium.org/g/blink-dev/c/-TjeYs7shTQ/m/th-1ACvYDAAJ>). > Maybe a console warning message if we hit the nesting level that we plan to > increase? I'm not sure how effective that would be in helping get folks to > try it. > > On Wed, Mar 30, 2022 at 6:00 AM Yoav Weiss <[email protected]> wrote: >> >>> LGTM to experiment M101-102 inclusive >>> Thanks for working on this!! >>> What's the plan for finding sites this breaks? Monitor bug reports? Or >>> is there something more proactive we could do? >>> >>> On Monday, March 28, 2022 at 9:36:30 PM UTC+2 Etienne Pierre-doray wrote: >>> >>>> Contact [email protected] >>>> >>>> ExplainerNone >>>> >>>> Specification >>>> https://html.spec.whatwg.org/multipage/timers-and-user-prompts.html >>>> >>>> Design docs >>>> >>>> https://docs.google.com/document/d/1boT0k8BQjl7mXXzvI9SdN4XJPSza27vE8T0CNxmMhCI >>>> >>>> Summary >>>> >>>> Increase the nesting threshold before which setTimeout(..., <4ms) start >>>> being clamped, from 5 to 100. setTimeout(..., 0) is commonly used to break >>>> down long Javascript tasks and let other internal tasks run, which prevents >>>> the browser from hanging. setTimeouts and setIntervals with an interval < >>>> 4ms are not clamped as aggressively as they were before. This improves >>>> short horizon performance, but websites abusing the API will still >>>> eventually have their set setTimeouts clamped >>>> >>>> >>>> Blink componentBlink >>>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink> >>>> >>>> TAG review >>>> >>>> TAG review statusNot applicable >>>> >>>> Risks >>>> >>>> >>>> Interoperability and Compatibility >>>> >>>> setTimeout is a well established and mature API. This change poses a >>>> risk of breaking websites and tests that rely on the current timing caused >>>> by clamping and the subtle task ordering that it entails. As an example, >>>> this change breaks assumptions about the ordering between setTimeout(0) and >>>> unrelated tasks in at least one case in Chrome tests (crbug.com/1302309). >>>> On the flip side, the implementation in Chrome is already non compliant ( >>>> crbug.com/1108877). There's also a similar experiment on beta that is >>>> ongoing (crbug.com/1263190). Devs can use >>>> chrome://flags#unthrottled-nested-timeout to test their sites for >>>> compatibility issues. >>>> >>>> >>>> Gecko: No signal >>>> >>>> WebKit: No signal >>>> >>>> Web developers: No signals >>>> >>>> Other signals: >>>> >>>> WebView Application Risks >>>> >>>> Does this intent deprecate or change behavior of existing APIs, such >>>> that it has potentially high risk for Android WebView-based applications? >>>> >>>> >>>> >>>> Goals for experimentation >>>> >>>> Gain insight on potential compatibility issues and evaluate impact on >>>> guardian metrics (page load, latency). >>>> >>>> >>>> >>>> Ongoing technical constraints >>>> >>>> None >>>> >>>> >>>> Debuggability >>>> >>>> setTimeout() and setInterval() have an associated trace event in >>>> DevTools. >>>> https://developer.chrome.com/docs/devtools/evaluate-performance/performance-reference/ >>>> >>>> >>>> Will this feature be supported on all six Blink platforms (Windows, >>>> Mac, Linux, Chrome OS, Android, and Android WebView)?Yes >>>> >>>> Is this feature fully tested by web-platform-tests >>>> <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_platform_tests.md> >>>> ?No >>>> >>>> Flag nameunthrottled-nested-timeout >>>> >>>> Requires code in //chrome?False >>>> >>>> Tracking bughttps://crbug.com/1108877 >>>> >>>> Estimated milestones >>>> OriginTrial desktop last 102 >>>> OriginTrial desktop first 101 >>>> DevTrial on desktop 101 >>>> OriginTrial android last 102 >>>> OriginTrial android first 101 >>>> DevTrial on android 101 >>>> >>>> Targeting Beta 50% for at least 8 weeks for more chance of teasing out >>>> breakage. I'll send a follow-up with what we learn prior to experimenting >>>> on Stable. >>>> >>>> Link to entry on the Chrome Platform Status >>>> https://chromestatus.com/feature/5710690097561600 >>>> >>>> Links to previous Intent discussionsReady for Trial: >>>> https://groups.google.com/a/chromium.org/g/blink-dev/c/-TjeYs7shTQ/m/FhJq0mQyDAAJ >>>> >>>> >>>> This intent message was generated by Chrome Platform Status >>>> <https://chromestatus.com/>. >>>> >>> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CALoDvsb23Mezq6Ehdt0uN-YLVSq8_R8NDDFaECEv8cK7W%2BGL6A%40mail.gmail.com.
