On Mon, Mar 14, 2022 at 4:40 AM David Faure <fa...@kde.org> wrote: > After the recent discussions on state of CI, I fixed the last unittest > failures (kio, purpose... + apol fixed ECM) so that > https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/ > is all green^H^Hblue again. > Please keep it that way! >
Thanks for looking into and fixing all of these David. > > Note however that > > * kwayland has a flaky test: > > > https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/job/kwayland/job/kf5-qt5%20SUSEQt5.15/171/testReport/junit/projectroot.autotests/client/kwayland_testDataDevice/ > > FAIL! : TestDataDevice::testReplaceSource() Compared values are not the > same > Actual (selectionOfferedSpy.count()): 1 > Expected (2) : 2 > Loc: [autotests/client/test_datadevice.cpp(557)] > > Who can look at this one? git log mostly shows Martin Flöser < > mgraess...@kde.org> > who I think isn't active anymore? > Not sure if it applies to KWayland as well, but I know that KWin has load sensitive tests (which is why the Gitlab .kde-ci.yml files support the flag tests-load-sensitive) If this test appears to be flaky, then it is quite possible that it is load sensitive as well. > > * krunner has a flaky test [2] because it measures time spent and expects > small values like 65ms > (I changed that one to 100ms), 250ms, 300ms. With only 10% safety margins. > On a busy CI system, > this is bound to fail regularly, even with bigger safety margins. In my > experience this kind of test > is just not possible (we're not running on a real time OS), I vote for > removing the test. > CC'ing Eduardo. > > > https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/job/krunner/job/kf5-qt5%20SUSEQt5.15/325/testReport/junit/projectroot/autotests/runnermanagertest/ Yes, that will definitely fail more often than not - your only way to make sure tests like this pass on our CI system is to set tests-load-sensitive=True (in Gitlab CI) Note however that option should be avoided where possible as it means your build will stop and wait for load to fall to low levels before proceeding with running tests - which blocks a CI worker slot from being used by another project. I'd also be in favour of removing this test. > > -- > David Faure, fa...@kde.org, http://www.davidfaure.fr > Working on KDE Frameworks 5 > > > > Cheers, Ben
