Thank you, it will help a lot. How/Where are we going to document this? Maybe in the "Testing" web page? https://core.dpdk.org/testing/
25/08/2023 16:51, Adam Hassick: > Hello DPDK developers, > > Currently, various testing labs perform CI testing on new patch series sent > to dev@dpdk.org and report their results to > https://patchwork.dpdk.org/project/dpdk/list/. On each series on the patch > list, the results appear in the test category contexts for IOL (community > lab), GitHub, and LoongSon. > > If a reported failure on a series seems suspicious to the patch submitter > or maintainer, then there may be an interest in requesting a retest on the > series for the failing label(s) in order to verify the failure is not > spurious or a false positive. This retest demonstrates to the submitter or > maintainer that the failure can be reliably reproduced. Unfortunately, at > present, the best way to accomplish this is to reach out to lab maintainers > via email or Slack. This is not ideal for developers in need of quick test > results. > > Going forward, CI testing labs will be implementing the option to request > retest for their respective test labels on patchwork via emails sent to the > dev mailing list. This feature is ready today for labels reported by the > UNH-IOL Community Lab, and will soon also be an option for the Github Robot > at least. > > In order to request a retest on your patch series, send an email reply to > one of your series’s patch or cover letter emails with email content of the > format used below: > > Recheck-request: <test names> > > The valid delimiter is a comma optionally followed by a space: “,” “, “ > > Valid examples: > > Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance, > iol-compile-arm64-testing, > > Recheck-request: iol-compile-amd64-testing,iol-broadcom-Performance, > iol-compile-arm64-testing, > > Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance, > iol-compile-arm64-testing > > Invalid examples: > > Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance > > Recheck-request: iol-compile-amd64-testing > iol-broadcom-Performance,iol-compile-arm64-testing, > > Some important notes: > > 1. > > At present, there is only support for retesting the series as it existed > when the lab received it. As in, if the lab applied the series on DPDK > mainline when the head was commit X, and a retest is requested, then > retests will be run using those same sources applied on top of commit X. > This is important to note because this means retest requests will not > provide a solution to your patch being submitted when the tree is in a “bad > state.” Getting test results with your patch applied on the current DPDK > mainline could be achieved by re-submitting the patch to the mailing list > as a workaround. > 2. > > For any series submitted earlier than August 2023, you must submit a > retest request in reply to a patch email, NOT in reply to a cover letter > email. > 3. > > The initial policy is to accept no more than one retest request per > patch series version per lab. > 4. > > Your patch should begin to retest within 15 minutes of your request, but > wait time is subject to the testing queue just like any other series. As a > result, retesting will be slower during peak submission time. > > > Improvements we are considering for v2 of the email retesting framework: > > 1. > > Add in an option to re-apply on the latest commit on DPDK mainline. So, > if your patch was originally applied on commit X, and you want to retest, > but have it be applied to commit Y (latest), you could specify that. Under > these circumstances, we would have to do a retest of all labels, since it > would be inappropriate to mix reports for results from different commits. > 2. > > Add a policy for vetting retest requesters - so maybe only maintainers, > or maybe only maintainers and the submitter, or another set of people. > 3. > > Add in an option to request a retest for next-* branches and/or LTS > branches. >
