Hi Gavin, I initially responded to Gijs' e-mail a while ago, before it got through dev-platform moderation. Since then I've discovered one bug which I believe is related. This one is tracked in bug 1017298. In particular this occurs when a user first has more tabs open than the window can fit, i.e. the tab strip becomes 'scrollable', and then starts closing them, without interruption, to the point where they do fit, and they start 'growing'. This causes layers to rapidly change size (every animation frame), since the tab strip in this scenario 'remains actively layerized' for a little while (unlike when it was never overflowing in the first place, and therefor never got layerized).
There are reasons to belief the new architecture performs somewhat worse than the old architecture when resizing layers. Usually this is a rare situation but I believe the situation described above -might- be exactly what happens during the TART tab close tests. In general I don't think many users will run into this, but it might explain part of that. Matt is looking into whether something can be done here. I'll continue looking into what might affect our performance numbers. If either people from the performance teams or the desktop teams are interested in investigating our tests and how well they measure practical performance, I think that would be very valuable, and it would help us a lot in identifying for what sort of interactions we need to investigate the performance. In addition to that if the performance or desktop teams have good ideas of other interactions (than tab-close) which seem to have regressed a lot in particular, that will also help a lot in our investigations. My knowledge of TART is limited. Bug 1013262 is the general tracking bug but there's not too much info there to be honest. Thanks! Bas ----- Original Message ----- From: "Gavin Sharp" <ga...@gavinsharp.com> To: "Vladimir Vukicevic" <vladi...@mozilla.com> Cc: "Gijs Kruitbosch" <gijskruitbo...@gmail.com>, "Bas Schouten" <bschou...@mozilla.com>, dev-tech-...@lists.mozilla.org, "mozilla.dev.platform group" <dev-platform@lists.mozilla.org>, "release-drivers" <release-driv...@mozilla.org> Sent: Wednesday, May 28, 2014 7:15:09 PM Subject: Re: OMTC on Windows Who's responsible for looking into the test/regression? Bas? Does the person looking into it need help from the performance or desktop teams? What bug is tracking that work? Gavin On Wed, May 28, 2014 at 12:12 PM, Vladimir Vukicevic <vladi...@mozilla.com>wrote: > (Note: I have not looked into the details of CART/TART and their > interaction with OMTC) > > It's entirely possible that (b) is true *now* -- the test may have been > good and proper for the previous environment, but now the environment > characteristics were changed such that the test needs tweaks. Empirically, > I have not seen any regressions on any of my Windows machines (which is > basically all of them); things like tab animations and the like have > started feeling smoother even after a long-running browser session with > many tabs. I realize this is not the same as cold hard numbers, but it > does suggest to me that we need to take another look at the tests now. > > - Vlad > > ----- Original Message ----- > > From: "Gijs Kruitbosch" <gijskruitbo...@gmail.com> > > To: "Bas Schouten" <bschou...@mozilla.com>, "Gavin Sharp" < > ga...@gavinsharp.com> > > Cc: dev-tech-...@lists.mozilla.org, "mozilla.dev.platform group" < > dev-platform@lists.mozilla.org>, "release-drivers" > > <release-driv...@mozilla.org> > > Sent: Thursday, May 22, 2014 4:46:29 AM > > Subject: Re: OMTC on Windows > > > > Looking on from m.d.tree-management, on Fx-Team, the merge from this > > change caused a >40% CART regression, too, which wasn't listed in the > > original email. Was this unforeseeen, and if not, why was this > > considered acceptable? > > > > As gavin noted, considering how hard we fought for 2% improvements (one > > of the Australis folks said yesterday "1% was like Christmas!") despite > > our reasons of why things were really OK because of some of the same > > reasons you gave (e.g. running in ASAP mode isn't realistic, "TART is > > complicated", ...), this hurts - it makes it seem like (a) our > > (sometimes extremely hacky) work was done for no good reason, or (b) the > > test is fundamentally flawed and we're better off without it, or (c) > > when the gfx team decides it's OK to regress it, it's fine, but not when > > it happens to other people, quite irrespective of reasons given. > > > > All/any of those being true would give me the sad feelings. Certainly it > > feels to me like (b) is true if this is really meant to be a net > > perceived improvement despite causing a 40% performance regression in > > our automated tests. > > > > ~ Gijs > > > > On 18/05/2014 19:47, Bas Schouten wrote: > > > Hi Gavin, > > > > > > There have been several e-mails on different lists, and some > communication > > > on some bugs. Sadly the story is at this point not anywhere in a > condensed > > > form, but I will try to highlight a couple of core points, some of > these > > > will be updated further as the investigation continues. The official > bug > > > is bug 946567 but the numbers and the discussion there are far outdated > > > (there's no 400% regression ;)): > > > > > > - What OMTC does to tart scores differs wildly per machine, on some > > > machines we saw up to 10% improvements, on others up to 20% > regressions. > > > There also seems to be somewhat more of a regression on Win7 than > there is > > > on Win8. What the average is for our users is very hard to say, > frankly I > > > have no idea. > > > - One core cause of the regression is that we're now dealing with two > D3D > > > devices when using Direct2D since we're doing D2D drawing on one > thread, > > > and D3D11 composition on the other. This means we have DXGI locking > > > overhead to synchronize the two. This is unavoidable. > > > - Another cause is that we're now having two surfaces in order to do > double > > > buffering, this means we need to initialize more resources when new > layers > > > come into play. This again, is unavoidable. > > > - Yet another cause is that for some tests we composite 'ASAP' to get > > > interesting numbers, but this causes some contention scenario's which > are > > > less likely to occur in real-life usage. Since the double buffer might > > > copy the area validated in the last frame from the front buffer to the > > > backbuffer in order to prevent having to redraw much more. If the > > > compositor is compositing all the time this can block the main thread's > > > rasterization. I have some ideas on how to improve this, but I don't > know > > > how much they'll help TART, in any case, some cost here will be > > > unavoidable as a natural additional consequence of double buffering. > > > - The TART number story is complicated, sometimes it's hard to know > what > > > exactly they do, and don't measure (which might be different with and > > > without OMTC) and how that affects practical performance. I've been > told > > > this by Avi and it matches my practical experience with the numbers. I > > > don't know the exact reasons and Avi is probably a better person to > talk > > > about this than I am :-). > > > > > > These are the core reasons that we were able to identify from > profiling. > > > Other than that the things I said in my previous e-mail still apply. We > > > believe we're offering significant UX improvements with async video and > > > are enabling more significant improvements in the future. Once we've > fixed > > > the obvious problems we will continue to see if there's something that > can > > > be done, either through tiling or through other improvements, > particularly > > > in the last point I mentioned there might be some, not 'too' complex > > > things we can do to offer some small improvement. > > > > > > If we want to have a more detailed discussion we should probably pick a > > > list to have this on and try not to spam people too much :-). > > > > > > Bas > > > > > > ----- Original Message ----- > > > From: "Gavin Sharp" <ga...@gavinsharp.com> > > > To: "Bas Schouten" <bschou...@mozilla.com> > > > Cc: "dev-tree-management" <dev-tree-managem...@lists.mozilla.org>, > > > dev-tech-...@lists.mozilla.org, "release-drivers" > > > <release-driv...@mozilla.org>, "mozilla.dev.platform group" > > > <dev-platform@lists.mozilla.org> > > > Sent: Sunday, May 18, 2014 6:23:58 PM > > > Subject: Re: OMTC on Windows > > > > > >> but tart will regress by ~20%, and several other suites will regress > as > > >> well. > > >> We've investigated this extensively and we believe the majority of > these > > >> regressions are due to the nature of OMTC and the fact that we have > to do > > >> more work. > > > > > > Where can I read more about the TART investigations? I'd like to > > > understand why it is seen as inevitable, and get some of the details > > > of the regression. OMTC is important, and I'm excited to see it land > > > on Windows, but the Firefox and Performance teams have just come off a > > > months-long effort to make significant wins in TART, and the thought > > > of taking a 20% regression (huge compared to some of the improvements > > > we fought for) is pretty disheartening. > > > > > > Gavin > > > > > > On Sun, May 18, 2014 at 12:16 AM, Bas Schouten <bschou...@mozilla.com> > > > wrote: > > >> Hey all, > > >> > > >> After quite a lot of waiting we've switched on OMTC on Windows by > default > > >> today (bug 899785). This is a great move towards moving all our > platforms > > >> onto OMTC (only linux is left now), and will allow us to remove a lot > of > > >> code that we've currently been duplicating. Furthermore it puts us on > > >> track for enabling other features on desktop like APZ, off main thread > > >> animations and other improvements. > > >> > > >> Having said that we realize that what we've currently landed and > turned on > > >> is not completely bug free. There's several bugs still open (some more > > >> serious than others) which we will be addressing in the coming weeks, > > >> hopefully before the merge to Aurora. The main reason we've switched > it > > >> on now is that we want to get as much data as possible from the > nightly > > >> channel and our nightly user base before the aurora merge, as well as > > >> wanting to prevent any new regressions from creeping in while we fix > the > > >> remaining problems. This was extensively discussed both internally in > the > > >> graphics team and externally with other people and we believe we're > at a > > >> point now where things are sufficiently stabilized for our nightly > > >> audience. OMTC is enabled and disabled with a single pref so if > > >> unforeseen, serious consequences occur we can disable it quickly at > any > > >> stage. We will inevitably find new bugs in the coming weeks, please > link > > >> any bugs you happen to come across to bug 899785, if anything > > se > > >> ems very serious, please let us know, we'll attempt to come up with > a > > >> solution on the short-term rather than disabling OMTC and reducing > the > > >> amount of feedback we get. > > >> > > >> There's also some important notes to make on performance, which we > expect > > >> to be reported by our automated systems: > > >> > > >> - Bug 1000640 is about WebGL. Currently OMTC regresses WebGL > performance > > >> considerably, patches to fix this are underway and this should be > fixed > > >> on the very short term. > > >> > > >> - Several of the Talos test suite numbers will change considerably > > >> (especially with Direct2D enabled), this means Tscroll for example > will > > >> improve by ~25%, but tart will regress by ~20%, and several other > suites > > >> will regress as well. We've investigated this extensively and we > believe > > >> the majority of these regressions are due to the nature of OMTC and > the > > >> fact that we have to do more work. We see no value in holding off OMTC > > >> because of these regressions as we'll have to go there anyway. Once > the > > >> last correctness and stability problems are all solved we will go > back to > > >> trying to find ways to get back some of the performance regressions. > > >> We're also planning to move to a system more like tiling in desktop, > > >> which will change the performance characteristics significantly > again, so > > >> we don't want to sink too much time into optimizing the current > > >> situation. > > >> > > >> - Memory numbers will increase somewhat, this is unavoidable, there's > > >> several steps which have to be taken when doing off main thread > > >> compositing (like double-buffering), which inherently use more memory. > > >> > > >> - On a brighter note: Async video is also enabled by these patches. > This > > >> means that when the main thread is busy churning JavaScript, instead > of > > >> stuttering your video should now happily continue playing! > > >> > > >> - Also there's some indications that there's a subjective increase in > > >> scrolling performance as well. > > >> > > >> > > >> If you have any questions please feel free to reach out to myself or > other > > >> members of the graphics team! > > >> > > >> > > >> Bas > > >> _______________________________________________ > > >> dev-platform mailing list > > >> dev-platform@lists.mozilla.org > > >> https://lists.mozilla.org/listinfo/dev-platform > > > > _______________________________________________ > > dev-tech-gfx mailing list > > dev-tech-...@lists.mozilla.org > > https://lists.mozilla.org/listinfo/dev-tech-gfx > > > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
